FINAL REPORT MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND FINAL EVALUATION OF THE CAPACITY-BUILDING PORTION OF THE WA-WASH PROGRAM March 2019 This document was prepared for review by the United States Agency for International Development (USAID). It was prepared by the Analytical Support Services and Evaluations for Sustainable Systems (ASSESS) activity, a partnership of the United States Department of Agriculture/Foreign Agriculture Service (USDA/FAS), the University of Rhode Island (URI), and the Kwame Nkrumah University of Science and Technology (KNUST) | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV ii April - 2019 Disclaimer: The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Cooperative Agreement #: AID -624-P-14-00004 | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV iii TABLE OF CONTENTS LIST OF TABLES................................................................................................................................................. IV LIST OF FIGURES ............................................................................................................................................... IV ACRONYMS .................................................................................................................................................... V EXECUTIVE SUMMARY.................................................................................................................................... 1 Evaluation Purpose ...........................................................................................................................................................................1 Evaluation Methodology ..................................................................................................................................................................1 Evaluation Findings and Recommendations................................................................................................................................2 I. FINDINGS, CONCLUSIONS, & RECOMMENDATIONS .................................................................................... 8 1.1. Findings Question 1: Did the combined activities implemented by Florida International University and AfWA contribute to an improvement in the capacity of: (1) personnel working within AfWA in West Africa; and (2) AfWA as an association?............................................................................................................................................8 1.2. Findings Question 2: Did efforts to build regional organizational capacity result in improved coordination in the sector at a regional level and did these lead to a more effective and efficient delivery of sustainable WASH service delivery models by the targeted organizations?...................................................................................22 1.3. Findings Question 3: How is AfriCAP contributing to AfWA’s regional coordination, advocacy and dissemination, and harmonization of policies that are informed by broad-based understanding of best practices and promising models for WASH service delivery?......................................................................................34 1.4. Findings Question 4: How have investments focused on strengthening AfWA Secretariat’s strategic financial management capacity impacted on its ability to provide strategic financial and planning support to the organization’s programs and leadership decision-making?.............................................................................................39 1.5. Findings Question 5: What are the gender-specific and differential effects of AfriCAP and WA-WASH activities?.....................................................................................................................................................................................47 2. SUMMARY CONCLUSIONS......................................................................................................................... 56 2.1. Was USAID strategy effective to support AfWA?.........................................................................................................56 2.2. Was the support provided by AfriCAP and WA-WASH sufficient to build the capacity of AfWA?...............56 2.3. What is the current capacity of AfWA to deliver services to its members and carry out its own mandate? .......................................................................................................................................................................................................57 ANNEX I: EVALUATION SCOPE OF WORK ................................................................................................. 58 ANNEX II: EVALUATION METHODS AND LIMITATIONS................................................................................ 70 ANNEX III: DATA COLLECTION INSTRUMENTS............................................................................................ 73 ANNEX IV: BIBLIOGRAPHICAL REFERENCES................................................................................................. 76 ANNEX V: LIST OF PERSONS INTERVIEWED.................................................................................................. 77 ANNEX VI: FIELD DATA COLLECTION SCHEDULE......................................................................................... 79 ANNEX VII: DISCLOSURE OF ANY CONFLICTS OF INTEREST........................................................................ 82 ANNEX VIII: INFORMATION ABOUT TEAM MEMBERS.................................................................................... 85 ANNEX IX: CURRENT AND SUGGESTED ORGANIZATION CHARTS FOR AFWA ............................................ 90 ANNEX X: EXCERPT FROM THE STRATEGIC BUSINESS PLAN....................................................................... 91 ANNEX XI: SERVICE DELIVERY MODELS ...................................................................................................... 91 | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV iv ANNEX XII: LOP SUMMARY ........................................................................................................................ 92 ANNEX XIII: DETAILED RECOMMENDATIONS AND TIMING ......................................................................... 95 ANNEX XIV: FINANCIAL METRICS............................................................................................................. 100 LIST OF TABLES TABLE 1. HISTORIC AFWA DUES COLLECTED BY REGION ..............................................................................................................16 TABLE 2. QUESTION 1 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS...............................................................19 TABLE 3. QUESTION 2 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS...............................................................31 TABLE 4. QUESTION 3 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS...............................................................38 TABLE 5. QUESTION 4 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS...............................................................46 TABLE 6. GENDER IMBALANCE WITHIN AFWA’S BODIES ....................................................................................................................49 TABLE 7. NETWORKS OF PROFESSIONAL WOMEN IN THE WASH SECTOR........................................................................................49 TABLE 8. QUESTION 5 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS...............................................................54 LIST OF FIGURES FIGURE 1. EVOLUTION OF AFWA MEMBERSHIP SINCE 2015..............................................................................................................15 FIGURE 2. DUES PAYMENT BY AFWA MEMBERS BETWEEN 2015 AND 2018 ..................................................................................17 FIGURE 3. GEOGRAPHICAL REPRESENTATION OF AFWA MEMBERS (ALL TYPES BETWEEN 2015 AND 2018)............................17 FIGURE 4. THEORY OF CHANGE BEHIND AFRICAP.............................................................................................................................23 FIGURE 5. ACTIVITIES PLANNED UNDER AFRICAP’S COMPONENT A ..............................................................................................23 FIGURE 6. SCREENSHOT OF THE “SUMMARY OF AFWA 2018-2022 STRATEGIC BUSINESS PLAN” ...............................................35 FIGURE 7. AFWA’S EFFECTIVE (2012-2017) AND EXPECTED (2018-2022) FINANCIAL RESOURCES PLAN IN USD .................43 FIGURE 8. STRUCTURE OF AFWA’S 2018-2022 STRATEGIC BUSINESS PLAN...................................................................................45 FIGURE 9. EXAMPLE OF THE CONTENT OF A BUSINESS PLAN...........................................................................................................45 | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV v ACRONYMS ACWUA Arab Countries Water Utilities Association ADB Asian Development Bank ADS Automated Directives System AFPEEC Cameroon Association of Professional Women in Water and Environment AfriCAP AfWA Regional Institutional Capacity Building Program AfWA African Water Association AMCOW African Ministers Council on Water AMEP Activity Monitoring and Evaluation Plan ASSESS Analytical Support Services and Evaluations for Sustainable Systems AWWA American Water Works Association BMGF Bill & Melinda Gates Foundation CEO Chief Executive Officer CICF International Center for Counseling and Training EAWAG Swiss Federal Institute of Aquatic Science and Technology ECOWAS Economic Community of West African States ED Executive Director FABRI Further Advancing the Blue Revolution Initiative FGD Focus Group Discussion FIU Florida International University FSM Fecal Sludge Management GCR Global Credit Rating GD General Director GOWPA Global Water Operators’ Partnership Alliance HR Human Resources HQ Headquarters IAG Inter-Agency Group IB-NET International Benchmarking Network ICT Information and Communication Technology IME Mediterranean Institute for Water IR Intermediate Result IRC International Research Center IWA International Water Association IWMI International Water Management Institute KM Knowledge Management LOP Laboratory Operators Partnership M&E Monitoring and Evaluation MD Managing Director MDG Millennium Development Goal ME/CLA Monitoring and Evaluation/Collaborating, Learning, and Adapting MOU Memorandum of Understanding MTE Mid-Term Evaluation NGO Non-Governmental Organization NRW Non-Revenue Water NUPAS Non-US Pre-Award Survey NWASCO National Water Supply and Sanitation Council NWSC National Water and Sewerage Corporation OHADA Organizations for African Harmonization of Business Rights ONEA L'Office National de l'Eau et de l'Assainissement | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV vi PIP Project Implementation Plan POE Post-Occupancy Evaluation PPP Public-Private Partnership PSI Population Services International PW4WATSAN Patron of Professional Women for Water and Sanitation RASOP Reinforcing African Sanitation Operators Partnerships RBFPEA Réseau Bénin des Femmes Professionnelles de l'eau et de l'Assainissement REDFEPEA Réseau Professionnel des Femmes Djiboutiennes de l'Eau et de l'Assainissement REGO USAID/West Africa – Regional Economic Growth Office REMAFPEA Réseau des Femmes Maliennes Professionnelles du Secteur Eau et Assainissement RIFPEA Réseau Ivoirien des Femmes Professionelles de l’Eau et de l’Assainissement RSFPEA Réseau Sénégalaise des Femmes Professionnelles de l’Eau et de l’Assainissement SDG Sustainable Development Goal SODECI SOP Société de Distribution d'Eau de la Côte d'Ivoire Sanitation Operators Partnership SOW Scope of Work SSD Sanitation Service Delivery STC Scientific and Technical Council SuSanA Sustainable Sanitation Alliance SUWASA Sustainable Water and Sanitation in Africa SWB State Water Boards (Nigeria) SWOT Strengths, Weaknesses, Opportunities and Threats TOR Terms of Reference UNICEF United Nations Children’s Fund USEPA US Environmental Protection Agency USAID United States Agency for International Development USAID/WA United State Agency for International Development / West Africa WALIS Water for Africa Through Leadership and Institutional Support WAN Wide Area Network WASH Water, Sanitation, and Hygiene WA-WASH West Africa Water Supply, Sanitation, and Hygiene Program WEF Water and Environment Federation WHO World Health Organization WIA Women International Association WiWASH Women in WASH, Ghana WOP Water Operators Partnership for Africa WOWASH Women in WASH, Nigeria WQTF Water Quality Task Force WRCC Water Resources Coordinating Center WSA Water and Sanitation for Africa WSRB Water Services Regulatory Board of Kenya WUP Water Utility Partnership | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 1 EXECUTIVE SUMMARY EVALUATION PURPOSE This report constitutes the Mid-term Evaluation (MTE) of the African Water Association (AfWA) Regional Institutional Capacity-Building Program (AfriCAP) and Final Evaluation of the capacity-building portion of the West Africa Water Supply, Sanitation, and Hygiene Program (WA-WASH) to AfWA. This evaluation covers the final two years of WA-WASH implementation (from January 2016 to October 2017) and the first three years of AfriCAP implementation (from December 2015 to December 2018). The Analytical Support Services and Evaluations for Sustainable Systems (ASSESS) Project was commissioned by the United States Agency for International Development/West Africa (USAID/WA) Regional Mission to conduct this evaluation to identify and document any successes, failures, and lessons learned of the capacity-building support to AfWA, provided by the combined efforts of WA-WASH and AfriCAP. The development hypothesis underlying USAID’s support to AfWA is based on the assumption that strong and effective regional organizations are necessary to address water, sanitation, and hygiene (WASH)-related challenges by implementing strategies and regional initiatives across stakeholder communities. This is the basis of the AfriCAP Theory of Change. Evaluation questions formulated in the Scope of Work (SOW) for this study are directly linked to the theory of change as they target: • The relevance and effectiveness of capacity-building interventions (Question 1); • The impact of USAID-funded programs’ efforts to strengthen AfWA’s capacity to provide sectoral coordination at a regional level and WASH service delivery models (Question 2); • The impact of interventions on AfWA’s capacity to provide advocacy, dissemination, and harmonization of WASH policies (Question 3); • The impact of interventions on AfWA’s strategic financial-management capacity (Question 4); • Gender-specific effects of USAID programs on AfWA’s products/services (Question 5); and • Favorable and hindering factors - addressed for each of the evaluation questions. EVALUATION METHODOLOGY1 The Assessment Team consisted of a team leader and two international consultants with expertise in water and sanitation utility management, WASH service delivery, capacity-building, gender, and program evaluation. The evaluation was supported by the ASSESS Activity and Evaluation Manager, who provided methodological and logistical tools and facilitated the organization of interviews, meetings, and group discussions. The desktop review was undertaken from October to December 2018 and the Assessment Team conducted field data collection in Ghana and Côte d’Ivoire in January 2019. Further interviews and data analyses were conducted in February 2019. The team used a mixed methodology, aimed at collecting 1 Refer to the Inception Report for the detailed evaluation methodology. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 2 qualitative and quantitative data through focus group discussions (FGDs), individual interviews, and a Survey Monkey survey of 31 AfWA utility members. Data collection was based on an evaluation matrix containing six evaluation questions as well as corresponding sub-questions, indicators and variables, data sources, and collection and analysis methods. Based on the evaluation matrix and the stakeholder sampling frame, the Assessment Team developed interview and FGD guides, as well as a comprehensive questionnaire to ensure representativeness of key stakeholder institutions (i.e. member utilities) that could not be interviewed directly due to geographical constraints. Overall, data collection was systematic and comprehensive and was facilitated by appropriate allocation of time for desktop review, field data collection, and analysis. The Assessment Team faced a significant limitation in the data-collection process related to inaccessibility of key stakeholders. For example, in spite of repeated requests over four months, the Assessment Team was unable to talk to the Chairman of the AfWA Board. Additionally, the survey instruments sent to 31 utilities obtained only six responses, even after significant prodding. EVALUATION FINDINGS AND RECOMMENDATIONS This section contains a summary of the key findings and recommendations elaborated in Section 1 of the report and its corresponding Annexes. When recommendations are ascribed to AfWA, they also implicitly apply to AfWA donor-funded programs (e.g., AfriCAP) if and when the recommendations are within the scope of those programs. QUESTION 1: DID THE COMBINED ACTIVITIES IMPLEMENTED BY FLORIDA INTERNATIONAL UNIVERSITY AND AFWA CONTRIBUTE TO AN IMPROVEMENT IN THE CAPACITY OF: (1) PERSONNEL WORKING WITHIN AFWA IN WEST AFRICA AND (2) AFWA AS AN ASSOCIATION? Capacity-building activities delivered by WA-WASH and AfriCAP were founded on a Non-US Pre￾Award Survey (NUPAS) assessment of AfWA conducted in 2015 by USAID. The Assessment Team observed improvements in terms of the AfWA Secretariat’s staffing, financial, procurement, and HR management capacities, communication and knowledge management skills, and better organization and follow up of Scientific and Technical Council (STC) meetings and Congresses. Additionally, improved capacity and support from USAID paved the way for further support from other donors, which in turn provides an opportunity for AfWA to continue “learning by doing” and have a financial safety net for the near future. Favorable factors include: a) the dedication of the Secretariat staff and the Executive Director to improve AfWA’s performance, and; b) the NUPAS assessment, followed by effective interventions from WA-WASH and AfriCAP to address the gaps identified. The Assessment Team identified five areas where improvements are still needed to allow AfWA to fulfill its vision and carry out its missions: 1. Human Resources (HR) organization and management – lack of transparency, modification of organogram pending since 2013; | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 3 2. Governance – lack of transparency and accountability, low representation of utilities from English-speaking countries, the Board often acts as a “rubber stamp” of the Secretariat rather than a driving force; 3. Strategic and financial planning – current Strategic Business Plan is not likely to make AfWA a financially sustainable organization and does not provide necessary details to reach its objectives; 4. Membership – lack of a database and effective consultation of members, disengagement of English-speaking organizations, only 30 percent of members pay their dues; and 5. Approach – AfWA focuses on researching donor-funded programs rather than developing internal high added-value services for its members. The main hindering factors to capacity improvement are: a) the lack of comprehensive and holistic baseline assessments at staff and organizational levels; b) the composition of the Board of Directors (made up only of member-utility Managing Directors) with limited time and commitment to support the building of a pan-African WASH institution; c) the persistent lack of capacity of AfWA to address organizational weaknesses that have been on the agenda for several years. PRIORITY RECOMMENDATIONS FOR QUESTION 1 PER SUB-QUESTION: 2 • Sub-Question 1.1 Donors should require a comprehensive capacity and institutional-gap analysis of staff at the Secretariat and a comprehensive organizational assessment of AfWA; • Sub-Question 1.1 Donors should formalize coordination of AfWA capacity-building with each other; • Sub-Question 1.3 The AfWA Board should strengthen its oversight role to increase effectiveness toward AfWA’s mandate and goals – particularly financial sustainability; • Sub-Question 1.3 AfWA should conduct a comprehensive analysis of its HR/staffing/appointments, starting with Board membership; • Sub-Question 1.8 AfWA, as a matter of urgency, should conduct a members’ survey/needs assessment; • Sub-Question 1.8 AfWA’s Secretariat and Board should identify and implement activities to improve AfWA’s rapport with members; and • Sub-Question 1.8 AfWA should review and revise its membership policy. QUESTION 2: DID EFFORTS TO BUILD REGIONAL ORGANIZATIONAL CAPACITY RESULT IN IMPROVED COORDINATION IN THE SECTOR AT A REGIONAL LEVEL AND DID THESE LEAD TO A MORE EFFECTIVE AND EFFICIENT DELIVERY OF SUSTAINABLE WASH SERVICE DELIVERY MODELS BY THE TARGETED ORGANIZATIONS? Regional coordination capacity and improvement of service delivery models were addressed by WA￾WASH and AfriCAP through a set of activities. These included establishment of the Coordinating Secretariat and the Laboratory Operators Partnership (LOP) – modeled after previous successful peer￾to-peer partnerships. While these activities tackled both coordination and service delivery models, the Assessment Team found they did not build capacity of the organization in a persistent manner, as AfWA is not able to sustain the outcomes or results of these activities when donor funding ends. 2 For a comprehensive set of recommendations, refer toTable 2 in the main text and Annex XIII, which provides detailed recommendations for all questions, including general recommendations. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 4 AfWA has a strong potential to provide coordination services and ensure efficient service delivery models with its capacity to convene STC meetings and Congresses because of its unique position as the main organization established by WASH services operators. One of the main barriers to assessing “organizational capacity resulting in improved coordination” and increased “effectiveness and efficiency in delivering sustainable WASH service delivery models” was the lack of indicators to follow and quantify AfWA’s progress in achieving these objectives. For example, the Activity Monitoring and Evaluation Plan (AMEP) of the donor-funded programs did not include coordination and service delivery model performance indicators. Other hindering factors include the lack of internal and external resources to address member utility needs, the lack of objectives and activities related to coordination and service delivery model performance in the Strategic Business Plan, the lack of benchmarking of utility performance and coordination needs, the language barrier, and the limited legitimacy and representation of AfWA in Southern and Eastern Africa. PRIORITY RECOMMENDATIONS FOR QUESTION 2 PER SUB-QUESTION: 3 • Sub-Question 2.1 AfWA should design a larger role for itself in supporting member utilities to adopt best practices in the different service delivery models; • Sub-Question 2.1 At the higher levels of government where policy is discussed, AfWA should devise and follow a plan to make itself better known and relevant; • Sub-Question 2.2 Regional offices of AfWA should be strongly considered, weighing the pros and cons and mitigating noted obstacles; • Sub-Question 2.2 AfWA should develop a WASH coordination “master plan” and manage/execute that plan; • Sub-Question 2.2 Indicators used by AfWA should be more outcome and performance based rather than activity based; and • Sub-Question 2.8 The LOP design and impact should be improved in the following ways: o Increase the number of mentors; o Develop and coordinate an integrated approach toward water quality testing in laboratories with all the components of the water quality value chain; o Prepare a road map of post-AfriCAP LOP activities with stakeholders to sustain the program; o Investigate regional purchase of lab consumables and maintenance of lab equipment; and o Determine required levels of water quality policy and advocacy in member utilities. QUESTION 3: HOW IS AFRICAP CONTRIBUTING TO AFWA’S REGIONAL COORDINATION, ADVOCACY AND DISSEMINATION, AND HARMONIZATION OF POLICIES THAT ARE INFORMED BY BROAD-BASED UNDERSTANDING OF BEST PRACTICES AND PROMISING MODELS FOR WASH SERVICE DELIVERY? Despite being one of the objectives of AfriCAP, the Assessment Team could not identify capacity￾building activities targeting policy coordination, advocacy, and dissemination. There was neither a baseline assessment of AfWA’s initial capacity in policy advocacy nor an action plan to leverage AfWA’s capacity to influence governmental policy makers. The Assessment Team also observed that there is confusion in AfriCAP’s Terms of Reference (TOR) relating to policy “formulation,” “adaptation,” and 3 For a comprehensive set of recommendations, refer to Table 3 in the main text. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 5 “implementation” – which are governmental mandates – rather than relating to technical leadership and policy advice, i.e. policy advocacy. As a result, efforts made by AfWA (e.g., drafting a white paper on sanitation, attending international meetings, and strengthening relations with African Ministers Council on Water [AMCOW]) were not as effective as they could have been, and no specific policies have been influenced or enacted to date. The Assessment Team observed a number of weaknesses in terms of policy advocacy that could be harnessed by AfWA in the future, possibly with support from external donors. Weaknesses include: • Weak rationale or purpose for AfWA's involvement in policy advocacy in the Strategic Business Plan – “increasing AfWA's reputation” rather than serving members; • Policy Advocacy – is policy advocacy a priority for members, and if so, on what topic?; • Lack of quality products for advocacy – e.g., a white paper on sanitation is a Knowledge Management (KM) document not a policy document; and • Challenges met by AfWA to coordinate with AMCOW– memorandum of understanding (MOU) awaiting signature over a year. PRIORITY RECOMMENDATIONS FOR QUESTION 3 PER SUB-QUESTION: 4 • Sub-Question 3.1 AfWA should develop and follow an internal (AfWA) Total Quality Management (TQM) protocol that covers all aspects of the organization from correspondence to publications to congresses and internal deliverables. AfWA’s management and staff should continuously improve the quality of its systems, processes, and outputs to ensure an increase in its membership retention rate; • Sub-Question 3.1 In order to address advocacy for all future capacity-building programs, AfWA secretariat should put the following questions before the Board: o Decide if advocacy remains a priority for its members; and o Decide who should be responsible for advocacy within AfWA; • Sub-Question 3.2 AfWA should take action on the afore-mentioned member survey regarding KM requests, especially related to policy; determine what sort of database is needed and where data is missing; then develop a plan for collecting that data with regular progress updates; • Sub-Question 3.3 AfWA should identify who has the best leverage to conduct advocacy and ensure their input; and • Sub-Question 3.3 AfWA should implement a structured policy formulation, harmonization, and adoption strategy. QUESTION 4: HOW HAVE INVESTMENTS FOCUSED ON STRENGTHENING AFWA SECRETARIAT’S STRATEGIC FINANCIAL MANAGEMENT CAPACITY IMPACTED ON ITS ABILITY TO PROVIDE STRATEGIC FINANCIAL AND PLANNING SUPPORT TO THE ORGANIZATION’S PROGRAMS AND LEADERSHIP DECISION-MAKING? AfriCAP’s capacity-building activities have supported AfWA’s procurement and budgeting processes (as also elaborated in Question 1), and WA-WASH provided support by funding an income-generating study carried out by an independent company. This support was complemented by the Bill & Melinda Gates Foundation (BMGF) in funding the Finance and Strategy Director’s position for two years (for the 4 For a comprehensive set of recommendations, refer to Table 4 in the main text. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 6 duration of the Reinforcing African Sanitation Operators Partnerships [RASOP] program). The combined support contributed to AfWA’s development of a Strategic Business Plan and at least an awareness of existing challenges such as “internal governance,” “sustainable financing model (…) capable of stimulating new sources of funding for the Association and making the mechanism for mobilizing contributions effective,” and “broadening and strengthening of the Association's membership.5 ” In spite of this awareness, the latest Strategic Business Plan lacks a comprehensive financial analysis and an evidence-and client-based action plan, including a cost-benefit analysis to ensure financial sustainability and future development of AfWA based on the identification of services and products for which members are willing to pay. As a result, financial planning until 2022 relies on external grants and does not provide the necessary framework to enable AfWA to fulfill its vision and deliver its mission. The main hindering factors identified are: a) the lack of a robust software to generate financial data necessary to make financial and strategic decisions; b) the lack of key financial indicators to monitor the financial health of the organization; c) the Board’s decision to approve the Strategic Business Plan without crucial information; d) the lack of benchmarking with business plans of similar organizations; and e) membership and income-generation projections not based on market analysis. PRIORITY RECOMMENDATIONS FOR QUESTION 4 PER SUB-QUESTION: 6 • Sub-Question 4.1 AfWA should identify relevant financial metrics and include these in reporting; • Sub-Question 4.2 AfWA should consistently use financial monitoring to implement measures that address negative trends in key financial metrics; and • Sub-Question 4.3 AfWA should incorporate financial analysis as basis for strategic choices in the next review of the Strategic Business Plan. QUESTION 5: WHAT ARE THE GENDER-SPECIFIC AND DIFFERENTIAL EFFECTS OF AFRICAP AND WA￾WASH ACTIVITIES? WA-WASH and AfriCAP have generally been successful in strengthening female staff capacity at both Secretariat and STC levels. AfWA’s Secretariat has strongly influenced utilities’ managing directors (MDs) to increase women’s participation in STC meetings and has effectively sponsored women’s participation in international events organized by AfWA and other organizations (Women’s Alliance, AfricaSan, and Fecal Sludge Management [FSM] conferences among others). AfWA is on track to exceed AfriCAP’s target to create 12 national networks of African women professionals. Positive effects of female involvement are empirically observed in WASH service delivery models (e.g. higher fee collection) and on utilities (improvement of customer relations, increased capacities of staff, rigor). Favorable factors were identified, such as strong engagement of AfWA’s Executive Director, existence of a pool of qualified female professionals eager to participate in the strengthening and coordination of the WASH sector, and stakeholders’ relatively strong awareness of the benefits of female involvement in leadership positions. The Assessment Team identified a number of areas for improvement. In terms of capacity-building, training did not focus on female-specific needs (such as leadership management) and there is still 5 AfWA Strategic Business Plan 2018-2022 (Foundation and Strategic Bases) 6 For a comprehensive set of recommendations, refer to Table 5 in the main text. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 7 significant imbalance at the level of leadership positions in all AfWA bodies (Secretariat, STCs, and Board’s senior positions are largely male dominated). Furthermore, gender-sensitive budgeting, planning, and financial strategies are not integrated into the Strategic Business Plan or at program-management level. Hence, the national networks of African women professionals in the WASH sector have limited means to implement the activities established in their national and regional road maps. Awareness of the positive impact of women’s empowerment in the WASH sector remains empirical, but there is a lack of evidence-based research and benchmarking on the topic in Africa. Hindering factors are related to: a) the lack of a gender strategy at AfWA’s level; b) the lack of designation of a focal point (at program-management and administration/finance levels) to define and enforce a gender policy and deal with gender-based issues; and c) the lack of indicators to track gender differentiation both internally and externally (benchmarking of female leadership in utilities). AfWA tends to hold further gender support in abeyance until the appointment of a gender specialist. Priority recommendations for question 5 per sub-Question: 7 • Sub-Question 5.1 In future programs, women’s empowerment should be specifically targeted by integrating women’s needs assessment in the baseline capacity assessment. AfWA should draft a gender strategy to formalize and ensure implementation of gender empowerment measures; • Sub-Question 5.2 Designate a focal point, at AfWA’s level, to ensure coordination of women’s related activities, capitalization, and coherence; • Sub-Question 5.3 AfWA should sustain current efforts to reach statutory representation of women in AfWA’s decision-making bodies as part of a larger effort to increase representativeness within AfWA (gender, languages, geographical scope, and youth); • Sub-Question 5.5 AfWA should include definition and enforcement of a gender policy as the responsibility of the Program Director and ensure capacity to lead this task; • Sub-Question 5.5 In case the AfWA organogram is modified, AfWA should train the head of administration/finance in gender mainstreaming; and • Sub-Question 5.6 AfWA should conduct research on gender impacts in utilities and promising gender-sensitive models in Africa (benchmarking). Develop policy advocacy on gender management in utilities. 7 For a comprehensive set of recommendations, refer to Table 6 in the main text. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 8 I. FINDINGS, CONCLUSIONS, & RECOMMENDATIONS The Assessment Team sought to answer evaluation questions, with a number of sub-questions, in order to reach logical conclusions regarding the West Africa Water Supply, Sanitation, and Hygiene Program (WA￾WASH) and African Water Association (AfWA) Regional Institutional Capacity-Building Program (AfriCAP). In addition, the Assessment Team has provided recommendations to address identified weaknesses in the two programs. The aim is to provide information and analysis that is useful for decision-making within AfWA and by donors potentially providing support to AfWA in the future. Findings were mixed. Some concrete improvements within AfWA were a direct result of WA-WASH capacity-building efforts and AfriCAP; however, significant opportunities for improvement remain. A summary of the conclusions is provided in Section 2. 1.1. FINDINGS QUESTION 1: DID THE COMBINED ACTIVITIES IMPLEMENTED BY FLORIDA INTERNATIONAL UNIVERSITY AND AFWA CONTRIBUTE TO AN IMPROVEMENT IN THE CAPACITY OF: (1) PERSONNEL WORKING WITHIN AFWA IN WEST AFRICA; AND (2) AFWA AS AN ASSOCIATION? This evaluation question seeks to assess the effectiveness of combined programs on capacity-building of the personnel working within AfWA and of AfWA as an organization. SUB-QUESTION 1.1: WAS THE AFRICAP AND WA-WASH APPROACH TO CAPACITY￾BUILDING RELEVANT? The focus on institutional and human capacity-building efforts to enable regional Water, Sanitation, and Hygiene (WASH) organizations to accomplish their strategic objectives corresponds to Objective 2 of the WA-WASH program and to Component A of AfriCAP. These objectives included training staff and strengthening different institutional components from procurement, to financial management, program implementation, governance, and knowledge management. It also included development of a women’s network and the development of a new AfWA Strategic Business Plan. The approach was relevant insofar as this was and remains a critical area requiring attention. The last two components will be reviewed under the evaluation questions specifically addressing this topic - Question 4 (strategic business plan) and Question 5 (women’s network). Both programs designed capacity-building activities based on a Non-US Pre Award Survey (NUPAS) assessment conducted by USAID in 2015 to determine whether AfWA, as an organization, had sufficient capability to manage USAID funding and achieve the objectives of the program. The NUPAS identified institutional and capacity gaps such as the lack of comprehensive systems, manuals, and procedures in key functional areas (procurement, human resources, financial management, and monitoring and evaluation [M&E]) and formulated 18 concrete recommendations targeting: • Governance - especially developing ethical standards for Board members (two recommendations); Favorable factor No. 1 The NUPAS assessment was relevant and identified goals (18 concrete recommendations) Completeness – partial | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 9 • Accounting and financial management (five recommendations); • Procurement (three recommendations); • Human resources (HR) management (three recommendations); • Program performance management (four recommendations); and • Cash management called “organizational sustainability” (one recommendation). WA-WASH and AfriCAP did support AfWA to address a number of recommendations from the NUPAS assessment as well as other identified needs during the inception phase of the programs. Interventions were conducted as follows: • Recruitment of five additional staff (fully or partially supported by AfriCAP) including: Program Coordinator, Knowledge Sharing Specialist, a Procurement and Grants Officer, an Assistant Accountant, a Program Assistant, and a M&E Officer; • Following an action plan, WA-WASH hired the Malian-based International Center for Counseling and Training (CICF) to deliver eight training sessions between August 2016 and January 2017; • Technical support for the purchase and functionality of a computer server; • Support to water and sanitation stakeholders’ coordination through the establishment of a WASH Coordinating Secretariat with core members including AfWA, WA￾WASH, US Environmental Protection Agency (USEPA), Population Services International/Sanitation Service Delivery (PSI/SSD), and USAID/West Africa (USAID/WA); and • Support for elaboration of four procedure manuals (financial, human resources, program management, and procurement). However, the Assessment Team found that there was no comprehensive institutional assessment. Institutional strengthening should not be based only on improving processes; and, there exists a significant analysis gap in what the French call “Projet Associatif” (which includes vision, mission, and strategy), and AfWA’s assets (financial resources, human resources, governance, organization’s culture, leadership etc.). For example, as discussed in detail in Question 4, the financial sustainability recommendation in the NUPAS was limited to cash flow analysis and did not cover a comprehensive assessment of income-generation and attractiveness of AfWA’s products and services to its members. Governance covered by-laws and an ethics chart but did not tackle effective governance mechanisms (such as accountability for decision-making, role of the Board, and relationships with the STC and the Secretariat). Program performance management was limited to an assessment of processes and did not address AfWA’s capacity to reach program outcomes and to plan and implement programs by themselves (outside of those initiated and funded by donors). Additionally, there was no baseline capacity survey of key AfWA staff when the assignment started. Hence, while recommendations and capacity-building activities conducted were in line with the objective of increasing capacity to manage USAID funds, it did not comprehensively address other aspects related to Hindering factor No. 1 Lack of comprehensive organizational diagnosis and baseline survey Hindering factor No. 2 Insufficient emphasis on sustainability/durability of AfWA, particularly financially Hindering factor No. 3 Lack of AfWA understanding of the needs of member utilities and focus to address those needs Hindering factor No. 4 Absence of metrics for measuring progress against goals and accountability Hindering factor No. 5 Lack of comprehensive AfWA staff baseline capacity assessment Hindering factor No. 6 Lack of strong coordination between donors and programs aimed at strengthening AfWA’s capacity | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 10 staff and institution strengthening. Thorough organizational diagnosis covering the institution’s strengths and weakness to effectively conduct coordination, and improve WASH models or harmonize policies was lacking. The Assessment Team found there were ad hoc efforts between the Bill & Melinda Gates Foundation (BMGF) and USAID to coordinate their AfWA capacity-building programs with quarterly informal meetings. Additionally, the Reinforcing African Sanitation Operators Partnerships (RASOP) program was designed based on the USAID-funded Further Advancing the Blue Revolution Initiative’s (FABRI’s) lessons learned and a number of capacity-building efforts targeting components not covered by AfriCAP, such as financial support to recruit a financial strategic manager. However, AfWA’s capacity-building could have been much more effective and relevant if donors’ coordination was formalized and if they increased the sharing of their findings and approaches. A similar lack of coordination was observed at the level of USAID’s programs. For example, the USAID Water for Africa Through Leadership and Institutional Support (WALIS) program “places greater focus on the challenge of building the capacity of national and regional leaders to capture and apply evidence in the development of policies, strategies, programs, and investments. 8 ” However, there were no formal articulation mechanisms between WALIS, AfriCAP, and RASOP despite the three programs having common goals in terms of building AfWA’s leadership capacity. SUB-QUESTION 1.2. WHAT WERE THE EFFECTIVE IMPROVEMENTS IN TERMS OF STAFF CAPACITY-BUILDING OBSERVED AT THE TIME OF THE EVALUATION? WHAT WERE THE LIMITS? Interviews with the staff from the Secretariat showed an overall satisfaction with the support provided by AfriCAP and WA-WASH. A number of external stakeholders interviewed also mentioned improvements in AfWA related to staffing, such as: • Increased skills and confidence of staff in general, including female staff who had been able to take on new tasks and responsibilities (HR, procurement, M&E); • Better aligned roles to staff capabilities and, where staff lacked professional qualifications, they were given additional training. For instance, WA-WASH sent the M&E specialist of AfWA to Burkina Faso for practical training; • Reduced outsourcing (e.g. several information and communication technology [ICT] functions that were previously outsourced to service providers are now being performed internally), thereby reducing costs to the organization; • Improvement of staff availability and thus capacity to respond (e.g. communicate information); and • Increased attractiveness of AfWA to major donors based on USAID’s sustained support (e.g., from the BMGF and the German Water Partnership). Beyond the benefits for AfWA as an organization, the Assessment Team observed that this was beneficial for the water and sanitation sector as a whole, as it helped to reinforce AfWA’s position as a focal point. 8 Accessed from https://www.dai.com/our-work/projects/worldwide-water-africa-through-leadership-and￾institutional-support-walis in March 2019. Completeness – significant Favorable factor No. 2 AfWA’s Secretariat has dedicated staff committed to make the organization better Favorable factor No. 3 Improved alignment of staff capabilities towards their responsibilities Favorable factor No. 4 Increased attractivity of AfWA to present potential development partners | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 11 Despite the successes of AfWA, staff capacity-building under AfriCAP and WA-WASH encountered significant challenges. For AfWA to derive full benefit from the increased staff capacity, leadership will need to address the challenges below: • Attitudes and capability of staff are both a strength and a weakness. AfWA has a dedicated staff looking to make the organization better. AfWA’ Secretariat is very supportive in increasing the capacity of the staff. However, staff do not have the opportunity to leverage their full potential because the hierarchical organizational structure does not sufficiently encourage staff to take responsibilities and decisions. • Lack of transparency in HR management. Stakeholders reported patronage in staff recruitment, absence of career roadmaps, and incoherent salary grids. This results in lost opportunities to recruit top talent, relatively high attrition rates, and demotivation of staff. No training delivered in project management and coordination. Despite expansion of the number and complexity of programs, there is a divide between the strong knowledge and experience of coordination staff in the sector and the capacity to effectively manage and coordinate programs. One example is the Laboratory Operators Partnership (LOP), which was delayed to such an extent that results will be a challenge to achieve by the end of AfriCAP; and there will be a lack of continuous supervision in its implementation. The team also observed lack of coordination between programs and other departments. For example, at the programmatic level, membership is considered to be a strength of AfWA, while marketing data clearly shows this to be a weak link of the organization. Program operations are run in such a way as to fulfill the donors’ requirements and justify successes rather than with a mindset to provide services to AfWA’s members. • Slow responsiveness in reviewing documents (e.g., work plans, TORs, draft reports) resulted in delays in implementing planned activities, which, in some cases, led to congestion of activities within a short period of time in an attempt to get back on track. The absence of a comprehensive capacity-needs assessment prior to the programs’ start did not allow customized capacity-building interventions tailored to the needs of individual staff; and the NUPAS focused more on management processes. Hence, there is a clear impact on staff’s capacity to mainstream the use of systematic processes (in accounting, procurement, and HR management). The lack of individual capacity assessments and of AfWA’s capacity to deliver services to its members and achieve results within the sector was compounded by insufficient effort to strengthen management and coordination skills at a programmatic level. SUB-QUESTION 1.3. WHAT WERE THE EFFECTIVE IMPROVEMENTS IN TERMS OF INSTITUTIONAL STRENGTH OBSERVED AT THE TIME OF THE EVALUATION? WHAT WERE THE LIMITS? As with many of the WA-WASH and AfriCAP goals and objectives, institutional strengthening had both its clear achievements as well as missed opportunities. GOVERNANCE Completeness – partial Favorable factor No. 5 AfWA’s Secretariat has been supportive of capacity￾building interventions Hindering factor No. 7 Organizational structure tends to impede staff motivation and taking of initiative | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 12 AfWA has a workable legal framework with clear by-laws and statutes and regular convocations of general assemblies. Decision-making processes and election of STC’s presidents, vice–presidents, and rapporteurs follow clear rules and are reported to be transparent. By-laws, statutes, and ethical standards for the Executive Board and removal policy documents were developed and approved by both the Board and general assembly in 2016 and 2017, respectively. Additionally, systems and procedures at the Secretariat have strengthened in the last two years. This has largely moved AfWA from widespread, ad hoc decision-making, toward a more formal approach and established procedures. The STC and Board have grown in competency, but much is still required to make them truly effective. The Board has shown growing engagement with the Secretariat, and the Chairman is systematic in attending meetings. However, there is an overall lack of continuity in participation, and the delegation of junior staff without decision-making authority to participate in STC and Board meetings remains an impeding factor. AfWA needs to consider a major analysis of its staffing operations, starting with Board and STCs. In its STCs, there should be a requirement that working groups be comprised of experts. All senior staff need specific, measurable, acceptable, realistic, and time bound (SMART) goals in order for the organization to become more impactful (see Hindering Factor 14 below). The current restriction of Board membership to only Managing Directors (MDs) of member utilities constitutes a weak point in the overall oversight role of the Board. Their time and commitment to AfWA is periodic as they often consider AfWA a lower priority; and a stronger focus for them is executing the political mandates of their governments, and not the building of a pan-African WASH institution. This is exacerbated when these MDs either cannot attend, delegate their attendance to others, or are replaced within their utility organizations - as is often the case in utility leadership. This situation breaks up the continuity of leadership and participation in the Board’s activities, diminishing its effectiveness. Another challenge is finding the right point of contact with member utilities. The norm is to work at the senior management level of utilities when sometimes decisions are made at their board level or at ministerial levels. Three sub-committees were formed to address “ethical governance and HR,” finance, and programs. Yet, on several occasions, the Assessment Team found that the Board did not demonstrate strategic guidance or provide support regarding finance and programming. The Board of Directors is not a driving force in seeking funding, developing new programs, or enhancing AfWA's ability to provide services to all of its members. Nor does the Board act as a counter-weight to challenge the Secretariat and thereby strengthen the quality of services, the long-term vision, and sustainability of the organization. The Board is known to act as a “rubber stamp” of the propositions of the Secretariat and is not sufficiently engaged in guiding AfWA’s programming to attract members and ensure that all members are being served. Furthermore, multiple stakeholders pointed out that AfWA can be overly political, which impedes governance, including among member utilities. A proxy indicator of the communication between the Secretariat and the Board is that despite several attempts between November 2018 and February 2019, including assistance from the Secretariat, the Assessment Team was not able to conduct an interview with the Chairman of the Board. While this limited some of the information sought by the Assessment Team, it also reflects serious internal challenges Favorable factor No. 6 AfWA’s workable legal framework, by-laws and statutes as well as improved systems and procedures Hindering factor No. 8 Untapped strength and leadership of the Board in order to effectuate improvements | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 13 in AfWA’s executive-level governance. In addition, a few stakeholders reported a lack of transparency in terms of recruitment and renewal of executive positions. PROCUREMENT The AfWA Procurement Officer was adequately trained and has the capacity to follow new procurement procedures described in a procurement manual. AfWA has also realized gains in procurement efficiencies, including better documentation and transparency. AfWA, however, needs to work on a more systematic use of procurement processes and to streamline these processes into operational processes across all programs. HR MANAGEMENT Notwithstanding the existing need to recruit an HR Manager, HR functions have slightly improved. The Administrative Assistant to the Executive Director was trained on HR functions and is executing some of the HR tasks. This is a good strategy to enhance capacity of existing staff to take on added tasks to limit functional gaps while programs are effectively growing. An HR management manual was also drafted. Despite these improvements, the HR organization and AfWA’s organogram is not adapted to functional and operational effectiveness. Notable is the fact that the aforementioned NUPAS document contained an annex with what appeared to be a valuable proposal for reorganizing the HR structure; but this document was detached from the rest of the NUPAS document and it was not clear if the document was supported by any analysis. The organogram of AfWA remains hierarchical, with eight managers reporting directly to the Executive Director. Organizational effectiveness remains weak and still needs to evolve to address AfWA’s new ambitions (see comparative organizational charts in Annex IX). A similar recommendation was formulated in the post-occupancy evaluation (POE) conducted by the BMGF in 2018.9 As indicated above (see Hindering Factor 7), the Assessment Team found the hierarchical organization and AfWA culture was not ideal for empowerment, autonomy, and initiative of staff. For example, stakeholders interviewed indicated it was not uncommon for staff to wait for instructions from senior management to act – a proxy indicator of limited initiative. This is aggravated by the fact that AfWA offices in Abidjan are geographically scattered and senior staff are often out of office - resulting in insufficient levels of supervision as well as stalled actions. The hierarchical structure results in “siloed” departments working in parallel (program, finance, communication, marketing, etc.) rather than together. The proposed organization chart (Annex IX) could be further improved by highlighting the relationships between programs and functional departments as they currently all report to the top. More integration is needed between programs and policy, communication, finance etc. The current HR structure is not in line with AfWA’s vision. Most Secretariat staff are Francophone from West Africa (exceptions are two Central Africans and one anglophone staff based in Nigeria), and there are no women at senior management positions. This is not coherent with AfWA’s ambitions to be a leader and role model for water and sanitation 9 In general, observations of the Assessment Team were triangulated with those of stakeholders (utility members, partners, donors). Hindering factor No. 9 Capacity-building efforts and institutional strengthening are hampered by lack of proper office space for the Secretariat Favorable factor No. 7 Strong procurement capabilities | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 14 utilities, to increase engagement from English-speaking members, and to promote women’s leadership. AfWA lacks the physical space for its own headquarters (HQ). Currently, they have a HQ agreement with the Société de Distribution d'Eau de la Côte d'Ivoire (SODECI), a private contractor of the government of Cote d’Ivoire, with the 23 staff of the Secretariat located in different buildings. This negatively impacts staff’s daily routines and coordination between departments, causing significant time loss, ineffectiveness, and inefficiency. PROGRAM PERFORMANCE MANAGEMENT (INCLUDING ICT AND KNOWLEDGE MANAGEMENT [KM] CAPACITY) The NUPAS’s approach to program performance management focused on existing tools and processes within AfWA to ensure effective program management such as ICT tools and M&E tools and frameworks. The ICT Manager and KM Coordinator received instructional, guided and independent practice training. AfWA was equipped with a computer server in 2017, which opened a new and valuable opportunity for the ICT Manager. These interventions resulted in improvement of knowledge sharing on the AfWA website, with the regular publication of articles related to AfWA’s projects, reports on STC meetings and Congresses, as well as articles from AfWA’s members and partners. A central program management manual was drafted and an M&E Officer was recruited and trained as per the recommendations of the NUPAS. Moreover, an M&E system was adopted to follow the key achievements in implementation of the recommendations based on the gaps identified in the NUPAS report. At the program level (AfriCAP, WA-WASH, and LOP), the monitoring framework is well established and informed but focused essentially on activities rather than outcomes. All indicators track the achievement of project activities but, for example, do not include the impact on utilities’ performance and at the sectoral level. In addition, the LOP, which aims at improving the quality of water to users, does not include a water quality indicator, baseline, and endline. An assessment of outcomes, impact on the beneficiaries (users, utilities), and value for money of the programs are not streamlined. The use of ICT for KM improved with the establishment of the new computer server and website, which is regularly updated. However, there is a need for wider use of ICT in knowledge sharing and networking, which still relies completely on the systematic use of face-to￾face meetings and training, involving significant logistics and costs for AfWA and its members (flight, hotel, venue for training sessions). Stakeholders interviewed and respondents to the Assessment Team’s questionnaire noted that travelling was a significant financial barrier to wider participation by small utilities and stakeholders in the sector. The Assessment Team found the method of delivering courses, in person for one week (such as Master Classes), is not adapted to the constraints of professionals who find it challenging to mobilize for such a period of time. Modern capacity-building methods increasingly rely on online tools and are delivered over time, allowing professionals to digest and implement information. Professional discussion forums are also powerful tools for sharing knowledge and experience over the long term. Knowledge sharing with utilities Hindering factor No. 10 Too high a focus on activities and not enough on outcomes and results Hindering factor No. 11 Too high a focus on face-to-face learning/sharing and not enough on (virtual) use of the Internet | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 15 (e.g., Master Classes and technical aspects) is ongoing but constrained by the lack of some ICT equipment and unfamiliarity with the use of some ICT tools to meet or train members remotely (e.g., video teleconferencing equipment or use of webinars for training and connecting with people distributed across different geographical locations). Not only is this a weakness in terms of training delivery and networking, but this is not financially sound. AFWA MEMBERSHIP AfWA membership and members’ satisfaction were not included in the NUPAS assessment. Yet, the Assessment Team considers that increasing the number of members and membership related income to be the cornerstone of AfWA’s development and financial sustainability (see Hindering Factor 2 and Question 4). It serves the vision of AfWA to become a leader in capacity-building of water and sanitation utilities in Africa and is vital to ensure the generation of income and limited dependency on outside donor support. AfWA has four types of membership10 - regular, affiliate, individual, and honorary. Despite repeated statements and projections from the Secretariat in the two Strategic Business Plans to enhance AfWA’s attractiveness to its members and on growth of fee-based income, membership remains well below its potential. The Assessment Team observed that AfWA could not provide an analysis of membership or, in other words, an analysis of its client base (see Hindering Factor 3). There has been no formal, comprehensive, and systematic consultation of AfWA members. Not only is there a lack of an updated database of members, but there is no concrete strategy to increase the membership base, payment of dues, and associated data analysis. The argument put forward by the Secretariat, that the representativeness of the organization is ensured by the fact that the participants of the STC meeting are the member organizations, is not sufficient. Participation in the STC meetings, which is largely dominated by members from West and North Africa, does not reflect the aspirations of all members. In addition, affiliated members limit their participation to the exhibitions. This results in a range of relevant industry stakeholders not adding value to important decision making and debates. The Assessment Team observed an increase in the number of members between 2015 and 2018 from 36 to 43 for regular members (+19 percent) and from 20 to 33 for affiliate members (+65 percent) as shown in Figure 1. While the percentage increase is meaningful, it must be put into proper context. Starting with a relatively low number (36) makes it easier to achieve more significant percentage increases; and, membership increases did not correlate, one-to-one, with increases in dues payment. Figure 1. Evolution of AfWA Membership Since 2015 10 Regular members are water and sanitation companies. Annual dues for water operators depend on their water production capacity (from 0 to 200 million m3 per year) and range from 4,200 € to 8,000€ per year and are set to 2,000€ for sanitation utilities; Affiliate members are training and research centers (838€), businesses and consultant agencies (2,100€) and international holdings (4,200 €); Individual members’ fee is 153 €, but many individual members are honorary and are exempted from dues. [Source: AfWA’s website, pages on membership, numbers have been rounded.] | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 16 Source: AfriCAP (2019, based on 2015-2018 M&E data) For example, Table 1 shows a four-year 70 percent reduction (between target and actual) in dues . Regular and affiliate members accumulated gross defaults of €1.2 million as of December 31, 2018 because only 29 percent of members are paying their dues yearly (Figure 2), though those delinquent on dues still receive the same level of services as regular dues-paying members. This absence of a link between non-payment of membership dues and access to the benefits that members can gain from AfWA is a hindering factor. Historically, it has not been uncommon for a utility that does not pay its dues to participate in STC meetings and be a beneficiary of a peer-to-peer partnership, in the same way as a member that is up to date with its dues. This approach is not conducive to settlement of dues by members. Source: AfriCAP/WA-WASH Evaluation (2019) An example of the membership and dues challenges is AfWA’s struggle to get Nigeria’s State Water Boards (SWBs) to join AfWA. Even for those that once joined, according to one stakeholder, “They [SWBs] come and go, and never pay any dues.” AfWA has even overlooked dues-paying members and launched programs in Nigeria to attract these SWBs to join, to no avail. AfWA needs to examine what went wrong in 2018 and take measures to correct it. 36 39 41 43 20 23 29 33 0 21 24 25 0 20 40 60 80 100 120 2015 2016 2017 2018 Active Associate Individual TABLE 1. HISTORIC AFWA DUES COLLECTED BY REGION YEAR DUES COLLECTED, IN EUROS NORTH AFRICA WEST AFRICA CENTRAL AFRICA SOUTH AFRICA EAST AFRICA TOTAL COLLECTED TARGET AMOUNT GROSS LOSS 2015 17,333 71,015 26,183 11,662 10,519 136,712 478,492 (341,780) 2016 18,663 62,618 18,065 17,074 24,011 140,431 446,826 (306,395) 2017 15,778 65,591 18,180 16,236 13,492 129,277 399,238 (269,961) 2018 18,278 77,013 29,346 4,497 8,994 138,128 414,384 (276,256) Total 70,052 276,237 91,774 49,469 57,016 544,548 1,738,940 (1,194,392) Hindering factor No. 12 Members who do not pay their dues receive the same services as members who are up to date. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 17 FIGURE 2. DUES PAYMENT BY AFWA MEMBERS BETWEEN 2015 AND 2018 Source: AfriCAP/WA-WASH Evaluation (2019) Membership and payment from South and East Africa have decreased (Table 1). Increases in membership were mainly from West African utilities (Figure 3). AfWA has lost paying utility members (e.g. Tanzania, Rwanda, Kenya, Zambia, South Africa); and not expanding across Africa is the unfortunate trajectory of AfWA, unless something fundamental changes. Some members previously benefited from AfWA’s past projects (e.g., FABRI) but no longer see this specific benefit when the project ends. Particularly the Anglophone utilities, unable to see any concrete value in AfWA, are unwilling to be paying members. Most AfWA private-sector members are foreign companies that joined AfWA with the expectation of access to the African markets to sell their products and services. After a while, when this market opportunity was not realized, their interest waned with a corresponding end to their participation in AfWA or dues payment. AfWA is also unable to attract a meaningful collection of individual members. FIGURE 3. GEOGRAPHICAL REPRESENTATION OF AFWA MEMBERS (ALL TYPES BETWEEN 2015 AND 2018) Source: AfRICAP/WA-WASH Evaluation (2019) The Assessment Team conducted a survey to assess, among other questions, utility members’ satisfaction with the services provided by AfWA. During this exercise, the Assessment Team found that AfWA did not have an updated list of its members and their current contact details. The survey was sent to 31 29% 20% 51% Members who have paid dues every year Members who have paid dues 1 to 3 times Members who have never paid dues 0 10 20 30 40 50 2015 2016 2017 2018 North Africa West Africa Central Africa Southern Africa East Africa | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 18 autilities with only a 19 percent response rate after significant prodding (i.e., six utilities completed the survey). As such, the survey responses could not be considered representative of all utility members. But the lack of responses could be an indicator of a lack of confidence, trust, or interest in AfWA. In spite of the limited responses, some of the answers (below) supported findings from other sources, such as: • There is overall satisfaction with participation in Congresses and STC meetings, but room for improvement (e.g., lack of continuity in participation, only face-to-face meetings, high costs); • AfWA was focusing on West African countries, and non-West African utilities lamented that programs (RASOP, AfriCAP) were unavailable to them; • Those who had been involved in programs (FABRI, RASOP, Water Operators Partnership for Africa [WOP]) were largely satisfied. However, it should be noted that in general, satisfaction for services provided for free (such as participation in the aforementioned programs) was higher than levels of satisfaction from services that required financial contributions; • Some utilities commented that they did not get the value for money/dues, and that membership dues and participation in STC meetings and Congresses were too high compared to the benefits. As a comparison, membership dues to the Arab Countries Water Utilities Association (ACWUA) and the Mediterranean Institute for Water (IME) were much cheaper and provided more value; • AfWA governance was cited several times as a challenge - pointing to a lack of transparency in the selection of beneficiary utilities to participate in programs and the lack of effective control of the Board on the activities of the Secretariat; • Some utilities also pointed out AfWA’s real services were organization of STC meetings and Congresses, but yet there are no “services” outside of donor-funded projects; • There was a demand for a list of member utilities as well as their characteristics; and for a large survey of members’ need, in order to gain awareness of their learning priorities and needs; • One surveyed utility pointed out there was a wide range of WASH stakeholders that were not part of AfWA’s network, yet they played an important role at the sectoral level; and • Efforts in KM were significant but there was room to increase the quality of communication. For instance, AfWA’s newsletter was only rated 3 out of 5 by all of the questionnaire respondents. Within AfWA staff, there was not enough focus on membership. The Marketing Manager, in charge of promoting AfWA to members, had not received training since he joined AfWA in 1996. His first formal training started under AfriCAP, where he learned how to manage memberships, the type of articles to produce, and how to approach other players (e.g., manufacturers to participate in and sponsor AfWA meetings). AfriCAP could be useful in linking the AfWA Marketing Manager with the high performing marketing department of the American Water Works Association (AWWA). While waiting for a full-fledged member needs assessment in 2019, some efforts were made in the past to gain a general understanding of the needs of AfWA’s members, including talking to members about their needs during STC meetings and the Congresses. This, however, was inadequate and the Assessment Team’s surveys and interviews yielded a consistent message of a disconnect between what members needed and a) AfWA’s understanding of these needs, as well as b) AfWA’s inability to meet these needs. The following table summarizes the conclusions and recommendations based on the above findings. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 19 TABLE 2. QUESTION 1 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS DID THE COMBINED ACTIVITIES IMPLEMENTED BY FLORIDA INTERNATIONAL UNIVERSITY AND AFWA CONTRIBUTE TO AN IMPROVEMENT IN THE CAPACITY OF: (1) PERSONNEL WORKING WITHIN AFWA IN WEST AFRICA AND (2) OF AFWA AS AN ASSOCIATION? SUB-QUESTION KEY FINDINGS/CONCLUSIONS SPECIFIC RECOMMENDATIONS HOW WILL THIS HELP? WHO ACTIONS? SQ. 1.1: -The NUPAS assessment analyzed a number of staff and institutional gaps. WA-WASH and AfriCAP delivered capacity-building in line with the 18 recommendations formulated by NUPAS. However, NUPAS was limited to assessing organizational processes and there was no comprehensive organizational diagnosis and no comprehensive baseline survey of staff capacity, limiting the scope of support to mostly processes. -The Assessment Team also found that there is no formal coordination between USAID’s programs addressing capacity-building and USAID and other development partners supporting AfWA. R1.1.1. In future capacity-building programs, the donors should conduct a comprehensive capacity gap analysis of staff at the Secretariat and a comprehensive organizational assessment to ensure that more than processes are addressed. R1.1.2. Capacity-building programs should focus on AfWA’s capacity to deliver services to its members and to ensure financial sustainability (which includes more than cash flow management). R1.1.3. More formalized development partners’ coordination [e.g., USAID, BMGF, and the German Water Partnership] on AfWA’s capacity-building. Articulation is also needed between programs of the same donor (e.g. between AfriCAP and WALIS). 1.1.1. Expanded and targeted staff capacity-building helps ensure they adequately meet individual and organizational requirements. 1.1.2. Focusing on capacity to deliver services to members makes AfWA more relevant and attractive to its members. 1.1.3. This avoids duplication and ensures USAID’s efforts complement, or are being complemented by, other AfWA capacity-building interventions. 1.1.1. AfWA Secretariat and USAID 1.1.2. AfWA Secretariat and USAID 1.1.3. AfWA Secretariat and development partners SQ. 1.2: -The capacity-building provided by WA-WASH and AfriCAP increased the skills and capacity of staff to follow systematic process and reinforced AfWA’s attractiveness to external donors. Capacity-building of staff was limited by the lack of training on project coordination and management. The lack of transparency and HR hierarchical structure limit the recruitment of top talent, does not permit unleashing of staff’s full potential, and limits interactions between program management and other departments of the Secretariat. R1.2.1. AfWA Secretariat should reform HR management by improving transparency of recruitment processes and harmonizing salaries. R.1.2.2. Donors should integrate the strengthening of program management and coordination into future projects. R.1.2.3. AfWA should review/reform its organizational chart, including shifting from a hierarchical model to a delegation with distributed control model, creating links between programs and support functions to facilitate staff autonomy and foster links between the program and other departments (e.g. communication, membership, and training). 1.2.1. This will ensure the attraction of best available talent into AfWA. 1.2.2. This ensures the best possible results from AfWA’s limited resources. 1.2.3. This improves supervision, accountability, and initiative among staff. 1.2.1. Executive Director (ED)/Head of HR 1.2.2. USAID and AfWA Secretariat 1.2.3. AfWA Secretariat SQ. 1.3: -Overall, governance has improved and AfWA largely moved from widespread ad hoc decision making towards a more formal approach and laid down new and helpful procedures. Yet, the Board of Directors is not a driving force in seeking funding, developing new programs, and enhancing AfWA's ability to provide R1.3.1. The Board should demonstrate the importance of its strategic role by strengthening its oversight role to increase its effectiveness for the Secretariat. In doing this, the Board should select a strong Chairperson willing and able to hold the Board accountable to measurable success. 1.3.1. This will improve strategic guidance to the Secretariat and increase AfWA accountability. 1.3.2. This will improve the stability and continuity of 1.3.1. The Board of AfWA 1.3.2. The Board of AfWA | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 20 TABLE 2. QUESTION 1 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS DID THE COMBINED ACTIVITIES IMPLEMENTED BY FLORIDA INTERNATIONAL UNIVERSITY AND AFWA CONTRIBUTE TO AN IMPROVEMENT IN THE CAPACITY OF: (1) PERSONNEL WORKING WITHIN AFWA IN WEST AFRICA AND (2) OF AFWA AS AN ASSOCIATION? SUB-QUESTION KEY FINDINGS/CONCLUSIONS SPECIFIC RECOMMENDATIONS HOW WILL THIS HELP? WHO ACTIONS? services to all of its members; the Board too often “rubber stamps” the Secretariat’s ideas and direction. Decision making is unduly influenced by political motives rather than simply focused on delivering services. The lack of full transparency on decisions is perceived negatively by utility members. Board membership, currently limited to only MDs of utilities, constitutes a weakness in the oversight role of the Board. MDs focus on executing the political mandates of their governments, and supporting the building of a pan-African WASH institution is a low priority. R1.3.2. Board membership should not be limited to MDs of member utilities who have no control over whether they will complete a term in office or not as a result of political exposure. AfWA should review and revise the criteria for membership to include other professionals who are technically competent but not MDs since they have more predictable tenure of office compared to MDs. oversight functions of the Board. -Improvements were observed in accounting processes, but financial management remains a weak link for AfWA as an institution - with current efforts targeting fund- raising from donors rather than internal income generation by selling membership, services, and products. The financial strategy in the Strategic Business Plan does not serve AfWA’s sustainable financial development. R1.3.3. Question 4 addresses financial strategy and sustainability. Precise recommendations will be found there. See Q4 -The procurement function has improved with the development of a procurement manual and the recruitment of a procurement manager. R1.3.4. AfWA should continue to strictly adhere to the procurement processes systematically as prescribed by the manual. 1.3.4. This will increase value for money, and make AfWA more efficient. 1.3.4. Director of Finance and ED. -WA-WASH and AfriCAP contributed to improvement of HR processes with the elaboration of an HR management manual and the training of the Administrative Assistant to the Executive Director on HR functions. However, NUPAS recommendations to review the HR organizational chart (2015) were not followed through. The current organization is hierarchical and partitioned. Executive staff are West African and largely males. This hampers the representativeness (and attractiveness) of a pan-African organization, the empowerment of women, and the effectiveness of the organization as a whole by untapping staff skills. Finally, the geographical location R1.3.5. AfWA should review its’ organizational chart and shift from a hierarchical model to a delegation with distributed11control model creating links between programs and support functions (same recommendations as R2.3) R1.3.6. AfWA should diversify the gender, language, and regional balance of its executive staff. R1.3.7. While pursuing larger office space to accommodate all staff, AfWA should consider using Wide Area Networks (WAN) to improve coordination of daily work and encourage more exchanges, interaction, and 1.3.5. Improves supervision and increases productivity of staff. 1.3.6 Increases diversity and establishes AfWA as a regional institution. 1.3.7 Reduces transaction costs, improves coordination and team work. 1.3.5. AfWA secretariat 1.3.6. The Board and Secretariat 1.3.7. The Board and Secretariat 11See for more details on this model: https://www.management-issues.com/opinion/7072/delegation-and-distributed-control/ | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 21 TABLE 2. QUESTION 1 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS DID THE COMBINED ACTIVITIES IMPLEMENTED BY FLORIDA INTERNATIONAL UNIVERSITY AND AFWA CONTRIBUTE TO AN IMPROVEMENT IN THE CAPACITY OF: (1) PERSONNEL WORKING WITHIN AFWA IN WEST AFRICA AND (2) OF AFWA AS AN ASSOCIATION? SUB-QUESTION KEY FINDINGS/CONCLUSIONS SPECIFIC RECOMMENDATIONS HOW WILL THIS HELP? WHO ACTIONS? of the 23 Secretariat staff in two different buildings is a major challenge for the organization. communication between departments. Ultimately AfWA should have one HQ location to house all its staff. -The NUPAS assessment partially addressed program performance management by focusing on the use of ICT tools and the existence of program management processes (e.g. the program manual). AfriCAP and WA- WASH were successful in supporting the development of the program management manual, the existence of a constantly updated knowledge management platform (website), and in providing training in ICT, KM, and M&E. However, the use of ICT tools is not yet mainstreamed in AfWA outreach as most of the networking, knowledge exchange, and training require members’ mobility. The M&E function is now organized but indicators used are based on activities rather than on outcomes. Hence, the impact of AfWA’s interventions cannot be quantified at a sectoral level (e.g. improvement of utility performance, water quality improvement, etc.). R1.3.8. AfWA should move towards online communication and virtual meetings (e.g. through the use of webinars and virtual meeting platforms, video conferencing technologies). R1.3.9. AfWA should reinforce the links between programs and M&E and include indicators that can track the impact of the programs and services on the utilities, affiliates, members, and on the WASH sector. 1.3.8. This will reduce the cost of delivering capacity- building services to members and reach more beneficiaries. 1.3.9. This will improve consistency of goals, objectives, and activities and overall coherence of projects with the Strategic Business Plan of AfWA. 1.3.8. AfWA Secretariat 1.3.9. Programs Director and M&E Manager -Membership management is AfWA’s weakness. There is a lack of database and effective consultation of members on their needs and expectations. Data collected showed fewer than 30 percent of members pay their dues regularly, and that members who do not pay still benefit from the same level of services (and even get the opportunity to participate in programs that paying members do not have access to). -Members appreciate opportunities to exchange information within STCs and Congresses as well as participate in programs; but also hoped for improvements in governance, opportunity to access programs and geographical representativeness. Information gathered from multiple sources indicates that AfWA is perceived as a West African organization focusing on West African members. R1.3.10. AfWA, as a matter of urgency, should conduct a members’ needs assessment to understand the specific services that members need. R1.3.11. AfWA’s Secretariat and Board should identify and implement activities that improve rapport between members and the Secretariat. AfWA should increase its attention on members and allocate sufficient resources (human resources and money) to members’ management. R1.3.12. AfWA should revise its membership policy. Utilities that do not pay dues should not be considered members. R1.3.13. AfWA should formalize diversity of employees in terms of language and nationality in future recruitments. This includes special consideration to qualified candidates originating from less represented geographical blocks. 1.3.10. Enables AfWA activities that are customized and better target the needs of members. Members will be motivated to consistently pay their dues. 1.3.11. This will increase membership retention and dues payment. 1.3.12. Provides incentives to continue paying, and attracts new members to pay. 1.3.13. Increases diversity and establishes AfWA as a regional institution. 1.3.10. AfWA Secretariat, led by Program Director 1.3.11. AfWA Board and Secretariat 1.3.12. AfWA Board and Secretariat 1.3.13. AfWA HR Source: AfRICAP/WA-WASH Evaluation (2019) | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 22 1.2. FINDINGS QUESTION 2: DID EFFORTS TO BUILD REGIONAL ORGANIZATIONAL CAPACITY RESULT IN IMPROVED COORDINATION IN THE SECTOR AT A REGIONAL LEVEL AND DID THESE LEAD TO A MORE EFFECTIVE AND EFFICIENT DELIVERY OF SUSTAINABLE WASH SERVICE DELIVERY MODELS BY THE TARGETED ORGANIZATIONS? SUB-QUESTION 2.1: WHAT ARE THE DIFFERENT SERVICE DELIVERY MODELS BEING IMPLEMENTED BY AFWA MEMBER UTILITIES PRIOR TO AFRICAP? HOW HAVE THE COMBINED EFFORTS OF WA-WASH AND AFRICAP IMPROVED SOME OR ALL OF THESE MODELS, OR MADE THEM MORE SUSTAINABLE? AfWA member utilities (from approximately 47 countries) implement a variety of service delivery models from centralized and decentralized to public-private partnership models (described in Annex XI). While traditional centralized public models predominate, there exists a potential and significant collective body of knowledge for all models, when one includes AfWA’s 101 members and potential future members. However, currently, that body of knowledge, in terms of the lessons learned from the different models in the water sector, is neither assembled by AfWA nor directly disseminated to its members. EFFORTS TO IMPROVE SERVICE DELIVERY MODELS As part of the WA-WASH contract extension (Objective 3), there was an intent to focus on building the capacity of AfWA for better WASH service delivery across the African region. One of the goals for AfriCAP (Objective 3) is to strengthen the effectiveness and sustainability of: a) all AfWA staff; and b) members and participants in the implemented activities (trainings, WOPs, research innovation through small grants, etc.) for improved service delivery by the use of existing structures in project implementation and institutionalized learning. These above listed WA-WASH and AfriCAP objectives were largely unmet. AfWA/AfriCAP does not have a highly focused and/or clear, goal-driven approach (e.g., using specific, measurable, acceptable, realistic, and time bound [SMART] goals), including benchmarking, follow-up, and improving unique service delivery models for its utility members. Utilities, perhaps due to lack of knowledge and/or confidence in AfWA, have not requested assistance from AfWA/AfriCAP on best practices and/or lessons learned on service delivery models; and AfWA does not offer this support (e.g., via webinars). In addition, case studies on innovative approaches from Sustainable Water and Sanitation in West Africa (SUWASA), Water Utility Partnership (WUP), WOP, Sanitation Operators Partnership (SOP), or donor-produced reports highlighting different service delivery models were not made available to member utilities. There have not been WA-WASH or AfriCAP-directed efforts to improve these models or to make them more sustainable. There have been some indirect activity related to service delivery models, including: Completeness – minimal Hindering factor No. 13 Lack of AfWA database of information important to member utilities, including from other utilities in Africa Favorable factor No. 8 A rich (potential) body of knowledge from the many utilities in Africa Hindering factor No. 14 Lack of AfWA use of SMART goals and accountability to those goals to improve WASH service delivery models/operational efficiency of utilities Hindering factor No. 15 Lack of confidence by existing and potential members and stakeholders in AfWA | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 23 • AfWA – through WUPs, WOPs, SOPs, the peer-to-peer partnership assistance (e.g., past benchmarking with some utilities), STCs, and Congresses – has the ability to be aware of the different service delivery models used by its member utilities and has informally shared some lessons learned from these different utilities; • In 2018, AfWA convened a meeting of utility communication officers in Ghana to gain information on their KM needs, offering member utilities the opportunity to express needs and questions unique to their service delivery model. Unfortunately, the utilities did not request from AfWA any of the needs they identified; and • Indirectly, through the above, as well as the KM function of AfWA (e.g., ShareWater), utility members have some information on members’ needs to help improve their service delivery models. THEORY OF CHANGE As stated above, the combined efforts of both WA-WASH and AfriCAP only indirectly contributed to improving WASH service delivery models. This may be indirectly related to WASH service delivery models not being a part of the “if statement” theory of change (Figure 4), while efficiency of utility operations was clearly stated in the “then statement.” FIGURE 4. THEORY OF CHANGE BEHIND AFRICAP FIGURE 5. ACTIVITIES PLANNED UNDER AFRICAP’S COMPONENT A IF… • Capability of AfWA to coordinate, formulate, adapt, harmonize, and implement WASH policies across Africa is strengthened; and • Knowledge sharing of best practice is enhanced and helps create awareness and advocacy in the WASH sector across Africa; and • Capacity of drinking water quality laboratories in West Africa is improved for effective water quality services; THEN… • Regional coordination of sector policies and efficiency of utility operations will be improved; • Access to sustainable and quality drinking water supply and sanitation services will be increased; and • The health and well-being of Africa dwellers will be improved. Source: AfRICAP Activity Monitoring and Evaluation Plan (AMEP) (May, 2016) • Staff training on financial/procurement procedures • Procurement of key program products • Development of internal management manuals • Review and adoption of statutes/by-laws • Promotion of a network of African Women professional of the WASH sector • Implementation of research projects in the WASH sector based on small grant approach using the network of Young Africans water professional • Review of AfWA business plan and development of new program/project proposals • Elaboration of a new AfWA Strategic Business Plan (2018-2022) Source: African Water Association (AfWA) Capacity Building Program (AfRICAP), Information Sheet (undated) The only reference to WASH service delivery models in the Activity Monitoring and Evaluation Plan (AMEP) of both programs is related to USAID’s WA’s investment strategy: “The overall aim of these investments is to produce effective regional coordination and harmonization of practices and policies Hindering factor No. 16 Improvement of WASH models is not part of the “if” statement of the theory of change while it is an expected result in the “then” statement. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 24 that are informed by broad-based understanding of best practices and promising models for WASH service delivery. 12” At AfWA’s level, there is no mention of WASH service delivery models in the Strategic Business Plan. The only reference to WASH service delivery models on AfWA’s website is in a non-governmental organizations’ (NGO’s) presentation and the “Models for replication in the sanitation sectors” of the white paper on sanitation, which are examples of fecal sludge management (FSM) technologies. To effectively build the capacity of AfWA to improve the WASH service delivery models of their members, preparatory work needs to be conducted to define a successful model and what indicators should be chosen to monitor models’ improvements and compare their respective performance (e.g., using AguaRating criteria). 13 SUB-QUESTION 2.2: WHAT SPECIFIC COORDINATION CHALLENGES IN THE WASH SECTOR WERE IDENTIFIED TO BE ADDRESSED BY AFRICAP? TO WHAT EXTENT WERE THE INTENDED ACTIONS TO ADDRESS THESE GAPS IMPLEMENTED? WHAT CAPACITY NEEDS OF AFWA WERE IDENTIFIED TO BE STRENGTHENED AS A BASIS FOR IMPROVING COORDINATION IN THE WASH SECTOR? A two-year cost extension was given to WA-WASH (from its previous end date of December 2015) associated with the WA-WASH activity (Objective 2) to coordinate USAID/WA WASH activities in the region, targeting capacity-building to regional organizations such as AfWA. AfriCAP’s vision was based on the premise that sustainable development in WASH required regional coordination (e.g., sector policies via its Objective 1) through the involvement of local institutions and organizations and also improving their functioning. In fact, AfriCAP called for “transformational changes”. The coordination challenges proved to be significant and only partially addressed through AfriCAP. Challenges ranged from language, cultural, and geographical barriers, to lack of workable strategic plans for coordination, and the questionable authority of AfWA to coordinate its members. MEMBER FOCUS In all the successes that AfWA is making under AfriCAP and other projects, a lingering question is how AfWA can remain relevant to its members and other stakeholders to merit continued patronage. The Secretariat should develop the ability to provide and sell services to its members, as discussed above in Question 1. Instead it often acts like an NGO – always looking for funding, according to one stakeholder. The question has to be asked of AfWA senior management, “does the 12 Cooperative Agreement No: AID-624-A-16-00003 – WASH - African Water Association (AfWA) Capacity building Program (AfRICap) (November, 2015) 13 Whatever indicators are selected should be what the theory of change calls “efficiency of operation.” In other words, performance indicators of water and sanitation utilities such as coverage, non-revenue water (NRW), tariff recovery, water quality, affordability for users, cost-benefit analysis, percentage of safely managed sludge, etc. “AfWA’s secretariat does not provide services to their members and rather behaves like an NGO trying to raise funds to do programs. Does the Secretariat serve the members or do the members serve the secretariat? The core question is: Do you understand your members? Do you know what they need?” – External Stakeholder Completeness – partial Hindering factor No. 17 Significant language and geographic barriers largely unaddressed | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 25 Secretariat serve members or are members there to serve the Secretariat?” As a member-based organization, AfWA should know what members want and attend to those needs (see Hindering Factor 3). COORDINATION PLANNING AND LEADERSHIP AfriCAP was neither able to create nor follow (hold itself accountable to) a guiding “organizational master plan” for regional WASH coordination, harmonization by AfWA or other regional stakeholders (e.g., development partners). In this context, the master plan is required to substantially provide more detail than a strategic business plan. A more detailed master plan supports the broad strategic business plan by providing executable or actionable direction. As a “master” plan, it also requires definition of interactions and interfaces with others external to AfWA. For example, one of the consequences of the absence of such a master plan is the inadequate defining and universal acceptance of roles and responsibilities or authority (e.g., between AfWA and Africa Ministers Council on Water [AMCOW]) of stakeholders, which can lead to confusion, finger pointing, and regional coordination action items going unaddressed. Another ramification was and remains the deleterious lack of consistency of participation in group activities (e.g., STC meetings), which ordinarily require multiple convocations of the same individuals to achieve consistency and results. The changing of participants often requires almost starting anew on topics, initiatives, action items, etc. Similarly, there is a frequent change of critical stakeholders over time (e.g., changing of utility MDs). The suggested “master plan” would have been the ideal structure to put into place many of the WA-WASH recommendations associated with WASH coordination. Another outcome of the absence of an agreed upon plan, by key (and perhaps influential) stakeholders, is WASH coordination was given a low priority, often exacerbated by perceived or real lack of funds (e.g., no utility budgets for regional activities and conferences) by both AfWA and its members. Finally, without a guiding plan, AfWA became focused on apparent disconnected programmatic activities, resulting in limited benefits to its membership base and further eroding their confidence in AfWA. Language was and remains an acute obstacle, largely between French- and English-speaking staff and members. English and Portuguese speaking members largely feel left out and remain disengaged. The “master plan” could analyze the language barriers and challenges to help in the formulation of the steps needed to mitigate this important issue. The AfWA Board, which has a significant potential to effectuate change, was either unaware of this regional WASH coordination objective of AfWA or was not able to actively push management to achieve the objective. From the Board down the ladder of authority at AfWA, there was effectively little accountability. IMPLEMENTATION OF MITIGATIVE ACTIONS TO ADDRESS GAPS One of the WASH coordination “gaps” occurred when one program or initiative, after having reached much of its potential, ended – effectively curtailing, if not completely ending, its momentum and benefits. AfWA has tried to continue successful programs (e.g., FABRI, WUP), through Favorable factor No. 9 Past, successful programs which could be scaled or replicated in the future Favorable factor No. 10 AfWA already has a number of effective “tools” it can use for WASH coordination Hindering factor No. 18 Lack of good, strategic plans and, more importantly, regular and effective use of the plans Hindering factor No. 19 Inconsistent participation in key organs of AfWA | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 26 the development of project implementation plans (PIPs), but the impacts have been severely muted due to lack of funding for their execution. AfWA has a number of tools, including additional staff, to help with regional WASH coordination. These tools have been sharpened with the help of AfriCAP, including STCs and Congresses, newsletters, and the KM team. In this regard, AfWA was aided by AfriCAP (Objective 2) support in KM and ICT. These are all effective tools for regional coordination, but they need to be well planned and organized for increased effectiveness. For example, while AfWA consults the Sustainable Sanitation Alliance (SuSanA), World Health Organization (WHO), USEPA, the Akvo Foundation, etc. for recent publications, the selection of publications is not associated with a coordinated KM plan or the needs of AfWA’s members. 14 To compound this problem, selection of publications is often based on their cost, not their content. IDENTIFIED AFWA CAPACITY NEEDS A recurring theme in all AfWA evaluations has been the absence of a detailed and complete capacity assessment in all areas, including those that would or could be engaged in WASH regional coordination (see Hindering Factor 5). As stated above, without a thorough capacity assessment of the AfWA Secretariat, there can be no comprehensive capacity-building plan and the ability for individuals to improve will be limited. CONTINENTAL AND NATIONAL LEVEL COORDINATION As stated in AfWA’s 2018-2022 Strategic Business Plan, weak coordination of actions at the levels of planning, implementation, and M&E is a challenge for most African countries. Coordination can be considered from two perspectives: • At the continental level, regional coordination between the water and sanitation utilities as described in AfWA’s 2018-2022 Strategic Business Plan, which highlights that effectiveness of utilities deserves to be strengthened, especially in terms of integration and technical cooperation; and • At the national level, coordination between the different stakeholders in the WASH sector such as utilities, governmental institutions, regulating bodies, private entrepreneurs, academia, NGOs, and civil society organizations. At the continental level, the main favorable factor toward improvement of coordination is related to the unique position of AfWA - the main organization set up by the operators of WASH services. Additionally, AfWA potentially gathers other WASH stakeholders through its affiliate members, which can be private entrepreneurs, academia, and NGOs. One of the strong points is the capacity of AfWA to organize STC meetings three times a year and Congresses every two years. Regardless of the need for further AfWA improvements, a number of stakeholders acknowledged the improvement: a) of the organization of 14 One of the interviewed stakeholders shared that the most important constraints to utilities are: their own government, their capacity, and willingness and commitment of leadership to change. This type of information should be in an AfWA database on the needs and challenges of its member utilities. Hindering factor No. 20 Insufficient leveraging and continuation of past donor￾funded programs Favorable factor No. 11 Unique position of AfWA that remains the main organization set up by the operators of WASH services. Favorable factor No. 12 Capacity of AfWA to organize STCs and Congresses | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 27 STC meetings and Congresses in terms of reporting and monitoring implementation decisions, and b) in communication and knowledge sharing. By building up AfWA’s capacity and KM channels such as the website, and by conducting peer-to-peer programs, WA-WASH and AfriCAP have indirectly contributed to improved coordination. To improve sectoral coordination by AfWA, all stakeholders interacting with utilities should be involved in a formal and comprehensive consultation process. This requires the survey discussed earlier as well as interviews and group discussions conducted by AfWA with all utilities that have been active in the past and those that have withdrawn. Future capacity-building programs should support AfWA in designing and conducting the consultation process, analyzing the results in a participatory manner (with the members), and presenting results to ensure ownership of the conclusions and action plan. The capacity-building support should aim at supporting a paradigm shift in AfWA, from members serving the Secretariat to the Secretariat serving the members. Future capacity-building programs should also draw lessons learned from other development partners. For example, the evaluation workshop of the RASOP program, which attempted to build capacity of AfWA through the implementation of sanitation partnerships, provides valuable guidance. RASOP designed the SOP as an investment to support AfWA’s delivery of service to its members. Implementation of future donor-funded programs must focus as much on building capacity of AfWA as achieving outcomes. Peer-to-peer programs are one of the most effective approaches to sectoral coordination across utilities. Yet, the number of utilities benefiting from peer-to-peer programs is limited and there is a lack of sustainability and replication of these programs after completion. It appears AfWA is running peer-to-peer partnerships like donor-based funding programs and does not yet own the partnership as a mainstream approach of its own. Therefore, it lacks the capacity to replicate this support without donors. Future capacity-building should focus on supporting AfWA to fully embrace its coordinating role by being an interface between utilities and other members by providing a framework for members to express both what they need and what they have to offer. At the organizational level, the limited coordination capacity compared to the existing potential (as well as the limited policy and advocacy capacity) is related to the institutional capacity (see Question I) that still needs to be targeted regardless of improvements. At national and continental levels, WA-WASH has conducted activities to address regional coordination capacity improvements and organization of numerous meetings with sectoral stakeholders in different countries (Côte d’Ivoire, Benin, Ghana, Burkina Faso). However, these meetings ended with the WA-WASH program, and AfWA has not been able to sustain them. There is an opportunity for AfWA to strengthen its coordination role and to increase KM through the integration of affiliate members – academia, the private sector, and NGOs – in the KM framework and working groups. The organization of STC meetings in different countries is a good opportunity to invite national stakeholders to participate in ad hoc sectoral dialogue. According to some stakeholders, STCs operate behind closed doors while Hindering factor No. 21 Lack of formal and comprehensive consultation of all members. Hindering factor No. 22 Partnership programs are still dependent on external funding when they should be part of AfWA's regular and fee-based services. Hindering factor No. 23 AfWA has overlooked national coordination activities in spite of good practice established by WA-WASH | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 28 NGOs and the private sector are mostly participating in exhibitions (and sometimes in technical presentations) but are excluded from working groups and discussion where they could bring added value. SUB-QUESTION 2.3: HAS AFWA’S CAPACITY TO COORDINATE WASH SERVICES' DELIVERY IN WEST AFRICA IMPROVED? IF YES, IN WHAT SPECIFIC WAYS? As a general statement, AfWA is not, nor expected to be, capable of impacting the WASH sector writ large. This is because AfWA is largely working with utilities in the urban space (unless, in the rare case, when a utility incorporates a semi-rural component such as the Uganda cluster15 approach) – and many WASH challenges exist in the rural space. 16 AfWA is stretched thin even to meet urban needs. Extending its mandate to the rural space would represent an enormous challenge and waste of resources for the organization. HAS AFWA’S CAPACITY TO COORDINATE WASH ACTIVITIES IMPROVED AND HOW? AfWA’s capacity to coordinate WASH has seen improvement with the addition of staff, staff training, ICT capabilities, KM platform, Congresses, and STC meetings. For example, during AfWA’s STC meeting held from February 13th to 15th, 2017 in Rabat, Morocco, a high-level panel was organized on the theme of "Water and Sanitation in Africa: Challenges and Prospects." Moreover, as a side event, AfWA initiated a round table with funding institutions. SUB-QUESTION 2.4: HOW APPROPRIATE WERE THE INDICATORS USED TO MEASURE INCREASED CAPACITY OF AFWA AS A RESULT OF THE PROJECT? There were no indicators used to measure AfWA’s capacity - neither baseline capacity nor changes in capacity over time. SUB-QUESTION 2.5: WERE ACTIVITIES IMPLEMENTED ACCORDING TO THE PROPOSED SCHEDULE AND WAS THIS SCHEDULEAPPROPRIATE? IF YES, HOW FAR HAS THE INCREASED INSTITUTIONAL CAPACITY OF AFWA PRODUCED THE EXPECTED IMPROVEMENTS IN SERVICE DELIVERY? WERE ACTIVITIES IMPLEMENTED PER SCHEDULE? A number of the activities, such as capacity-building in terms of bringing on additional staff, were implemented, largely on schedule; 17 but not all, including AfriCAP Objective 3 on water quality and the 15 In the rural space and in sanitation, predominantly urban focused utilities should engage with NGOs, keeping in mind that there is a lack of sectoral coordination at national level, between government, utilities, the private sector and NGOs. To some extent WA-WASH had a cluster approach – at the regional level – but perhaps AfWA could pilot this in Côte d’Ivoire. NGOs have complained about the lack of clustering. For example, in the RASOP in Côte d’Ivoire some activities (FSM in Yamoussoukro) can be undertaken by NGOs (e.g., PSI) that have funding for FSM from USAID, but because they lacked the sector coordination, it took them months before they started participating in the project. Time and opportunities are lost due to lack of sectoral coordination. In a number of countries the United Nations Children’s Fund (UNICEF) takes the cluster lead. 16 A continent-wide organization to fill the rural space does not exist. Donors could assist to re-organizing Water and Sanitation for Africa (WSA) or creating another regional organization. 17 The recruitment of two additional employees who started on May 2, 2018 were an M&E Assistant at the Program Office and a Procurement Assistant at Finance and Strategy Department. The trainings included the Accounting Director on the European Union's H20 / 20 financial audit procedures and four staff on M&E/Collaborating, Learning and Adapting (ME/CLA) and the planning of additional trainings for staff. Completeness – minimal Completeness – partial Completeness – partial | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 29 Objective 1 element on advocacy. In a similar theme to Hindering Factor 4 related to accountability and metrics, the Assessment Team could not find any demonstrated evidence, such as a chart or schedule at AfWA of progress toward its many goals and objectives. WAS THE SCHEDULE APPROPRIATE? The Assessment Team viewed the schedule as both clear and achievable as originally defined. That is, if the schedule and tasks had been efficiently managed, the targets could have been met. HAS INCREASED AFWA CAPACITY PRODUCED IMPROVEMENTS IN SERVICE DELIVERY? One of the challenges of determining the efficacy of institutional capacity-building to produce results (e.g., service delivery) was the dearth of data and metrics (see Hindering Factor 4). While some capacity-building was appropriately scheduled and implemented, there was no obvious data linking AfWA’s institutional capacity-building and improved service delivery. It is important to note that not all data needs to be furnished by AfWA. For example, Burkina Faso data is available - showing access to urban water and sanitation in 2015 as 97 and 50 percent, respectively. 18 For Ghana, access to urban water and sanitation in 2015 was 93 and 20 percent, respectively.19 AfWA, after implementation of its targeted assistance in service delivery, could have collected post-intervention data to find at least proxy indicators for its success. SUB-QUESTION 2.6: TO WHAT EXTENT ARE THE OBJECTIVES AND EXPECTED RESULTS OF AFRICAP BEING ACHIEVED/ARE LIKELY TO BE ACHIEVED AFRICAP OBJECTIVES 1, 2, AND 3 With the exception of policy: a) formulation, b) coordination, c) adaptation and d) harmonization of WASH policies across West Africa, AfriCAP Objective 1 is being met. Because of the exceptions noted, it is unlikely that Objective 1 will be fully met within the lifetime of AfriCAP. Excluding: a) dissemination of best practice documents, white papers, and knowledge pieces to a wide audience; b) dissemination of 20 knowledge products on WAS; c) eight virtual learning events; d) six professional trainings; and e) 20 research products related to WASH by young water professionals, Objective 2 is being met. Significantly more progress could be made in all these areas as the impediments are not insurmountable for AfWA. As with many of the AfriCAP achievements, they are mixed. For example, various manuals have been developed; however, they are only valuable if regularly and effectively used. AfriCAP Objective 3 has gotten off to a late start with essentially only pilot activities/audits/manuals and a limited number of peer-to-peer milestones being reached. Furthermore, the more regional-wide aspirations of the objective have not been met. This objective will not likely be met within the timespan of AfriCAP. 18 WHO/UNICEF (2015) Joint Monitoring Programme for Water Supply and Sanitation: Progress on Sanitation and Drinking Water Update (2015). Geneva Switzerland. 19 WHO/IWA/USEPA Water Safety Planning Baseline Assessments: West Africa Country Summary Report: Ghana 6 May 2016 Completeness – partial Hindering factor No. 24 Lack of AfWA demonstration of progress toward goals (e.g., graphs and schedules) | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 30 SUB-QUESTION 2.7: IF AFWA IS ACHIEVING ITS REGIONAL COORDINATION ROLE, HOW DID AFRICAP CONTRIBUTE TO THESE ACHIEVEMENTS? With existing and new talented AfWA staff, it is well positioned in the initial stages for progress on this regional coordination goal. In addition, AfWA’s newsletter, Congresses, STCs, and KM are strong elements that can continue to be deployed in AfWA’s coordination role. However, AfWA’s coordination role is achieving only moderate success – its potential has fallen short of expectations. SUB-QUESTION 2.8: HAVE WEST AFRICAN UTILITY MEMBERS IMPROVED THEIR WATER QUALITY MANAGEMENT WITHIN THEIR ORGANIZATIONS? West African utility members have improved their water quality management, but primarily those few that participated in the peer-to￾peer activities, and to levels that fall short of what is required. The Assessment Team’s findings are discussed in more detail in Annex XII. In general, the labs do contribute to the water quality process because they are a component of the whole quality process, but water quality can only improve if all other segments are integrated in the program (i.e., water treatment, pipe maintenance, households’ management of water at the point of use) and the program does not address these issues. If the goal is to improve water quality, then a holistic approach of the water value chain is required. Another lesson learned was there were positive impacts and capacity-building due to informal exchanges between stakeholders as a result of this program. SUB-QUESTION 2.9: AFWA’S CAPACITY IN IMPROVING WATER QUALITY MANAGEMENT IN THE REGION AfWA’s capacity and that of the Water Quality Task Force (WQTF) exist to impact water quality in the region. This capacity is in its initial stages and a lot more can and should be done. An Academy with regional hubs has been discussed to, among other objectives, address water quality. Included in AfWA’s budget to be launched in 2020, it is considered an important intervention. AfWA has not yet signed the memorandum of understanding (MOU) with the National Water and Sewerage Corporation (NWSC) in Uganda to host the Academy. Comments from one stakeholder have been used to reinforce some of the general findings listed above: Favorable factor No. 13 Peer-to-peer exchanges improve quality for the utilities involved and they also lead to informal sharing of best practices between utilities Hindering factor No. 25 Insufficient attention to West Africa water quality writ large and how to leverage the LOP to help address this larger issue AfWA could develop laboratory training modules on what needs to be improved in the Priority Action Plans (PAPs). AfWA should have a trained water quality talent pool. This pool of experts should be formalized and used by AfWA to build capacity and sustain the activity (peer-to-peer capacity-building). Within the region or sub-region AfWA should be able to send auditors and facilitate implementation of water quality monitoring for any country experiencing water quality difficulties; and AfWA must not let this network of experts dwindle and die. Completeness – partial Completeness – partial Completeness – partial | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 31 SUB-QUESTION KEY FINDINGS/CONCLUSIONS SPECIFIC RECOMMENDATIONS HOW WILL THIS HELP? WHO ACTIONS? SQ. 2.1. -AfWA does not yet have sufficient internal and external technical resources to help member utilities with their unique service delivery models’ questions and needs. R2.1.1. AfWA should establish and regularly update a database of specialists who satisfy pre-determined criteria related to their areas of competency with select service delivery models and would be willing to help member utilities with unique questions about their service delivery models. Member utilities will better perform in their unique service delivery models if they can tap into experts’ knowledge. AfWA, Marketing -AfWA does not use indicators to monitor changes in member utility performance making it hard to quantify its contributions to utility improvement. R2.1.2. AfWA should develop member utility performance indicators (e.g. using AguaRating), and appoint a staff member to assemble/update non-confidential utility statistics (perhaps re-starting the statistics previously found in the AfWA Yearbook, a.k.a. Annuaire AAE). Applying the adage “that which is measured gets done,” utility performance that is measured and known by its peers will improve. AfWA, M&E -AfWA is not sufficiently able to support member utilities’ unique service delivery models at high levels of government and AfWA/AfriCAP does not have leverage to advocate for best practices in different service delivery models. R2.1.3. AfWA should set targets for itself to play a larger role in supporting member utilities to adopt best practices in the different service delivery models. At the higher levels of government where policy is discussed, AfWA must make itself better known. It must also help utility general directors (GDs) to better understand what AfWA is about and its potential to help. In some cases, such as with potential high-value utility memberships, the AfWA Executive Director should establish direct relations with utility GDs and at Ministerial levels. AfWA/AfriCAP should increase its leverage to advocate for best practices in different service delivery models and how to increase the uptake of this information from its utility managers. Greater respect for and influence from AfWA will increase its leverage and ability to effectuate positive changes for its member utilities. AfWA, Marketing SQ. 2.2: -AfWA is not yet fully equipped to offer best practices and tools in sanitation to its members. R2.2.1. AfWA should continue providing the SOPs peer-to-peer approach to: • Improve/build on RASOP to have a package of sanitation models with best practices’ tools and institutional approaches and to replicate and scale these for city sanitation; and • Build capacity of base level of sanitation (keeping in mind most utilities spend their time thinking about water and need to understand key sanitation issues and tools to analyze cases of what works/does not work; perhaps a sub-grant to a training institute where AfWA is supporting its members by recruiting participants [not design/deliver curriculum]). For example, AfWA UNESCO Inter-Agency Group Improves member utilities’ ability to expand sanitation coverage. AfWA, Programs Director with help from BMGF if this collaboration continues TABLE 3. QUESTION 2 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS DID EFFORTS TO BUILD REGIONAL ORGANIZATIONAL CAPACITY RESULT IN IMPROVED COORDINATION IN THE SECTOR AT A REGIONAL LEVEL AND DID THESE LEAD TO A MORE EFFECTIVE AND EFFICIENT DELIVERY OF SUSTAINABLE WASH SERVICE DELIVERY MODELS BY THE TARGETED ORGANIZATIONS? | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 32 SUB-QUESTION KEY FINDINGS/CONCLUSIONS SPECIFIC RECOMMENDATIONS HOW WILL THIS HELP? WHO ACTIONS? (IAG) network sanitation course to 30 universities and the Swiss Federal Institute of Aquatic Science and Technology (EAWAG) using consultants and contractors. AfWA can play a role in building up a baseline of understanding. SQ. 2.2., 2.4., 2.7., & 2.8: -WA WASH coordination master plan does not exist, leaving coordination to be inefficiently defined and executed by the more general AfWA business strategy and the “push and pull” of development partners. Not enough attention is given to the role of Ministers and Ministries under which member utilities operate. R 2.2.1. Develop and have approved by critical stakeholders, a WASH coordination master plan and effectively manage/execute that plan. The master plan should, at a minimum, include: • Specific action item to push WASH coordination; • Specific action item to improve AfWA relevance to its members; • Clearly defined roles and responsibilities, including with Board; • Action items with due dates and who is responsible for what; and • Results of the members survey and how to address them. R2.2.2. An analysis to help in the formulation of the steps needed to improve the coordination functions within the AfWA organization. R2.2.3. Depending on the results of the analysis, preliminary recommendations include using the results of a suggested member survey as well as ad hoc information from Congresses, STCs, and communications to put together a list of areas needing coordination (e.g., best practices such as with NRW, regulations and regulators, new technologies, development partner funding, and peer-to-peer contacts). Preparing a 4- column table with the 1st column listing the areas to coordinate, the 2nd listing the stakeholders requiring the coordination, the 3rd listing the type and date of most recent coordination for a particular area, and the 4th column listing when the next coordination(s) should occur (type, who at AfWA is responsible, and date to implement). Maintenance of this table should be assigned to one individual, who is also responsible for overseeing the implementation of the actions required from the table (e.g. the next coordination on new FSM technologies) Improves uptake, efficiency and effectiveness of WASH coordination by engaging influential stakeholders. AfWA, Marketing/ Communication SQ. 2.2: -AfWA does not have sufficient pan-Africa representation, membership and influence. R2.2.4. Regional offices of AfWA should be considered, but, as AfWA’s financial situation is tenuous, it must overcome its own financial and capacity deficits to enable sufficient support to direct and hold a regional branch accountable. Accesses the full strength of pan- Africa, not just West Africa, presence. AfWA Executive Director and development partners -AfWA staff indicators are activity-based and not quantifiable; roles and responsibilities are unclear. R2.2.5. Indicators should be part of a larger capacity-building plan and one that is more output and performance based (e.g. certification) vs. activity based. R2.2.6. Capacity needs to be monitored for progress, from qualifiable evaluation to more quantifiable evaluation. Those charged with M&E within AfWA, and individuals supporting the M&E efforts, must be evaluated annually on how well they qualify, but more importantly quantify, the impact of AfWA’s assistance and outreach activities. R2.2.7. AfWA needs to clearly define roles and responsibilities including for coordination. Focus on human resources strengthening in terms of definition of roles Increases performance of senior management, which will trickle down to mid-level management and have an overall positive impact on AfWA Executive Director and HR | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 33 SUB-QUESTION KEY FINDINGS/CONCLUSIONS SPECIFIC RECOMMENDATIONS HOW WILL THIS HELP? WHO ACTIONS? and responsibilities, restructuring the organogram, and increasing the responsibilities at program management level. A focal point should be designated for cross-sectoral and coordination activities. AfWA achieving its goals. SQ. 2.5: -Capacity-building is more focused on input and activities than on output and outcomes. R2.5.1. Senior leadership of AfWA should insist staff regularly use and display graphical information on progress of their objectives such as scheduled activities, showing baseline schedule vs. actual progress (on a monthly basis). A monthly basis is recommended in the short-run to help establish momentum, consistency of efforts, and a greater emphasis on results. Once the momentum and consistency are established, quarterly reporting should be adequate. Increased adherence to scheduled activities and goals. AfWA Executive Director SQ, 2.8. & 2.9: - Pilot LOP activities were successful, but moving water quality towards pan-Africa scale has not happened as quickly as desired. R2.8.1. The LOP program’s design could be improved in the following ways: • Mainstream Life cycle cost analysis in Performance Action Plans; • Consider the possibility of having several mentors to increase impact and avoid political competition; • Ensure a formalized support of mentors to mentees and creating communication channels for knowledge sharing (mailing list, discussion forum); • Provide comprehensive information on USEPA and International Organization for Standardization (ISO) standards to enable laboratories to make an informed choice; • Adopt an integrated approach of laboratory process by coordinating improvement of water quality testing in laboratories with the other components of the value chain to ensure that better water testing effectively leads to better drinking water treatment and quality at the point of use; • Make a road map of future activities with stakeholders to sustain the program and encourage replication, as AfriCAP will end before the completion of the LOP; • Coordinate the maintenance of lab equipment (labs/utilities can fund) at regional level to decrease the price of maintenance and coordinate the purchase of lab consumables (reagents etc.); • Increase water quality work at the policy level, advocacy to systematize water quality monitoring in all projects in Africa; • Ensure coordination by encouraging utilities to standardize analytical equipment and facilitate equipment maintenance. This is because it is extremely expensive for them and leads to a reluctance on the part of the authorities to invest; and • Bring in a maintenance person to maintain all the laboratories, funded by laboratories or water companies. Increased pan- Africa water quality. AfWA, Programs Director with significant assistance from development partners (e.g., USEPA) Source: AfRICAP/WA-WASH Evaluation (2019) | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 34 1.3. FINDINGS QUESTION 3: HOW IS AFRICAP CONTRIBUTING TO AFWA’S REGIONAL COORDINATION, ADVOCACY AND DISSEMINATION, AND HARMONIZATION OF POLICIES THAT ARE INFORMED BY BROAD-BASED UNDERSTANDING OF BEST PRACTICES AND PROMISING MODELS FOR WASH SERVICE DELIVERY? SUB-QUESTION 3.1: HAS AFWA ESTABLISHED A LEADING ROLE IN WASH POLICY COORDINATION AND HARMONIZATION IN WEST AFRICA? To “build the capacity of AfWA to formulate, coordinate, adapt and harmonize WASH policies across West Africa in order to improve performance of the sector” is part of the first component of the AfriCAP program. Findings on sub-question 2.6, that were focused on service delivery, indicated that the objectives in terms of policy were largely unmet – this is also true for AfWA WASH policy writ large. The Assessment Team found the following obstacles to policy-related objectives: • Acknowledging that this is a challenging objective, the actions listed under this component do not address capacity improvement in formulation, coordination, and harmonization of policies. There are no activities targeting the “policy cycle” such as setting up an agenda, undertaking policy analysis, going for public consultation, decision-making, setting up policy instruments, etc. There was also no assessment of the initial capacity of AfWA in the field of policy; and • The “If statement” related to policy in the theory of change - “If the capability of AfWA to coordinate, formulate, adapt, harmonize, and implement WASH policies across Africa is strengthened” - is not relevant. AfWA does not have the legitimacy nor the mandate to formulate policies and even less to implement them. This is the mandate of governmental bodies. The publications of AfWA so far have not demonstrated capacity in policy formulation. In fact, there exists confusion between policy and advocacy. When AfWA works through AMCOW, the aim is not to formulate policy but to provide technical leadership and policy advice, or policy advocacy. Intermediate Results (IR) 4.1 of AfWA’s 2018-2022 Strategic Business Plan does mention advocacy as one of the intermediate results (Figure 6). The IR4.1 states: “The strategy and the communication & advocacy plan in place are implemented” under the strategic result “The reputation of AfWA has increased” and the Strategic Objective of “Improving visibility of the Association”. This confirms the finding under sub￾question 2.2, that AfWA needs a shift in approach from members serving the Secretariat to the Secretariat serving the members. The purpose of advocacy is to support members to address their needs. Completeness – minimal Hindering factor No. 26 WA-WASH and AfriCAP did not conduct activities related to policy capacity-building Hindering factor No. 27 The policy objective of the TOC is not relevant and should be replaced by an advocacy | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 35 FIGURE 6. SCREENSHOT OF THE “SUMMARY OF AFWA 2018-2022 STRATEGIC BUSINESS PLAN” Source: Summary of AfWA 2018-2022 Strategic Business Plan, p. 32 Currently, to strengthen its policy advocacy position and leadership role in the sector as an institution for regional coordination, KM, information sharing, and capacity-building of WASH stakeholders, AfWA prepares white papers for distribution to stakeholders and decision-makers in the sector and undertakes a number of advocacy actions to raise awareness on issues of the WASH sector. During the first year of AfriCAP, a draft white paper on sanitation, more specifically on FSM, was developed, published, and disseminated. A shorter version was produced for advocacy purposes. AfWA, however, needs significantly more leverage and a strategic approach to effectuate any meaningful changes and impact with regards to policy. One interviewed stakeholder stated, “AfWA may be effective in sharing knowledge, but where is their leverage in facilitating the uptake of such knowledge among members?” One leverage point could be the desire of members to belong to the association. In addition, three WA-WASH analyses on WASH (Burkina Faso, Côte d’Ivoire, and Ghana) were developed and shared with AfWA. They were to be used to assess the needs for strengthening national institutions: government, service providers, regulators, universities, and NGOs, with an emphasis on regional organizations for advocacy work with AMCOW. The advocacy activity, also mentioned earlier, was not as targeted as it could have been. The random activities included: • The previously noted white paper on sanitation; • Regular attendance of AfWA’s management to AMCOW’s meetings, and other stakeholder meetings; • Advocacy sessions during the 18th & 19th Congresses; • Seven high-level panels during STC meetings; • World Water Day and National Water Week advocacy sessions; • AMCOW Water Week advocacy session; • Malaysia advocacy session; and • Utility staff trained on SMART Operation and Maintenance approaches and KM. AfWA is a monopoly, and the only association of utilities in Africa, bringing together many key urban WASH players through its Board and STC Committees (see Favorable Factor 11). Consequently, its STRATEGIC OBJECTIVES uilding of water and sanitation operators in Africa Diversifying partnerships and strengthening synergies Improving visibility of the Association STRATEGIC RESULTS mances of African Water and n Companies are enhanced Diversified Partnerships Increase the Effectiveness of the Association and Ensure a Greater Impact of Actions The reputation of AfWA has increased INTERMEDIATE RESULTS ional rial e of pliant onal orms IR.2.1: Peer￾to-peer partnerships are implemented IR.2.2: Innovative approaches to water and sanitation service management are developed IR.3.1: The African Water and Sanitation Academy is operational IR.3.2: Scientific Technical, & financial partners are established IR.4.1: The strategy and the communication & advocacy plan in place are implemented IR.4.2: Technical and scientific publications are disseminated IR.4.3: AfWA organizes professional meetings in Africa and participate in | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 36 potential is significant. For example, its Bi-Annual Water Congress, which is relatively well organized, brings together key industry players with good advocacy and knowledge sharing opportunities. To some extent AfWA is aware of this and taps into its own potential such as “shining a light” on urban sanitation, including stimulating member utilities to focus on urban sanitation. SUB-QUESTION 3.2: HAS AFWA DEVELOPED ANY KNOWLEDGE MANAGEMENT PRODUCTS USING ITS OWN LEARNING OR LEARNINGS FROM OTHER USAID SUPPORTED PROGRAMS, E.G. WA-WASH, SSD, USEPA ETC? KNOWLEDGE MANAGEMENT PRODUCTS KM products are available such as the Annual Yearbook of Statistics (see Favorable Factor 10); however, there is room for expansion of these services. AfriCAP needs to focus products much more closely on what the members (especially utilities) need (addressing Hindering Factor 3). Fortunately, the AfWA website is comprehensive and updated. It is easy to search for documents, articles are well referenced on the Internet, and it is appreciated by most interviewed stakeholders. In the future, AfWA should work on the content of the website information to increase its quality. Currently, the content is informative, but it should gradually become more analytical and technical. AfWA feels that it needs a statistics manager to establish/maintain a database of key performance indicators in utilities in Africa. It wants a dedicated staff member to be doing this at the STC as well - to collect key performance indicators such as non-revenue water (NRW), tariffs, etc. This is an important task, but it remains to be determined if an additional person needs to be hired or if an existing AfWA team member can be trained to do this. Per Favorable Factors 2 and 3, this should be an easy analysis to accomplish. According to AfriCAP’s Year Two Annual Report, a number of knowledge products were produced, including ‘AfWA News:’ AfWA NEWS magazine Nos. 116, 117, and 118, and SHAREWATER Nos. 4 and 5 are available. The draft version of AfWA News No. 119 is available. These products continue to be developed and/or made available. For example, a film on the annual evaluation workshop of RASOP￾Africa, the AfWA project on sanitation, was published on YouTube to further promote sharing of knowledge and experience in the sector. The production process for a second film on best practices in sanitation has started. MEMBER-FOCUSED KM As repeated above, there has never been a member survey completed by AfWA to help guide its KM function (see Hindering Factor 3) 20. Such a survey will likely yield the need, of members, for AfWA 20 WA-WASH pushed hard for KM and information sharing by AfWA with questionable success on the sharing front. AfWA received a $50K grant to get started thinking more about KM, and a direct result was that some member utilities opened up positions on their staff for Knowledge Managers. WA-WASH put together a KM platform in Morocco and trained the AfWA IT Manager in the U.S. and Cote D’Ivoire. According to the WA￾WASH Year 6 Final Report, capacity-building was conducted, but there was no indication of the efficacy of the training – which has also not been registered by AfriCAP. The WA-WASH report listed activities – not results. In Indicator #9, it did indicate that 4 of the targeted 6 capacity assessments were completed but the Assessment Team was not able to obtain them. Item 4.1, Activity 2.1 indicated that an inventory and assessment of regional organizations’ capacity and impact was completed – but again, the Assessment Team has not seen them. It also Completeness – partial Favorable factor No. 14 AfWA has started the development of KM products and can expand upon this base | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 37 to serve as a collective voice of all the utilities (whereas just one utility’s voice may not be strong enough). In stakeholder interviews with the Assessment Team, they expressed the need for AfWA to also share best practices, lessons learned, and to facilitate exchange of ideas. Finally, the members need AfWA for networking and, through this function, to grow professionally. This is not to suggest the complete absence of KM for AfWA’s members. AfWA to some degree has served as a platform for KM and sharing (in collaboration with WA-WASH and others) such as in the organization of the learning workshop on knowledge sharing on July 15 to 16, 2017 and of the Master Class on sanitation in Luanda, from August 17 to 21, 2017. In addition, three white papers on WASH (Burkina Faso, Côte d’Ivoire, and Ghana) were developed by WA-WASH and shared with AfWA. AfWA indicates that two staff in the Communication Department effected a smooth implementation of the knowledge sharing strategy. The question that remains is: Are these the products requested/needed by its members? It is well documented that engagement, support, and revenue from members is a severe problem – to some extent because they are not receiving the knowledge that they require from AfWA. AfWA has sent out a members’ satisfaction survey, which will hopefully yield information about how better AfWA can serve its clients. However, during the Assessment Team’s visit, AfWA appeared reluctant to communicate the results of this survey, as the study was ongoing. As mentioned above, when the Assessment Team sent its own survey to member utilities, based on contacts given by AfWA, not only was it discovered that the AfWA database of members was outdated but the response rate was unacceptably low at less than 20%. AfWA has the potential to become a member service organization with the right strategic planning and focus of resources. For example, utilities want help to share knowledge and learn from one another better than they currently are and this is a request AfWA can easily address. SUB-QUESTION 3.3: WHAT ROLE HAS AFWA PLAYED IN POLICY FORMULATION AND HARMONIZATION TO DATE? WHICH SPECIFIC POLICIES WERE REVIEWED, FORMULATED, ADAPTED OR HARMONIZED? DID ANY SUCH POLICY INTERVENTION IMPROVE WASH SERVICES DELIVERY IN WEST AFRICA? This is an important and useful role for AfWA. However, no specific policies have been influenced or enacted to date as a result of AfriCAP. By proxy, AfWA considers the white papers it published on sanitation and water services as having achieved these targets. The impact of these white papers, is questionable. If AfWA/AfriCAP’s role continues to be in policy formulation and harmonization, it could be substantially improved as, to-date, it has not been an area of focus. Besides the two aforementioned papers, AfWA/AfriCAP has only indirectly supported or introduced policies through its Congresses and STC meetings. Policy formulation in the WASH sector is currently led by AMCOW, although AfWA could and should assist AMCOW at least from a technical perspective. AfWA has, without success, spent about a year trying to finalize an MOU with AMCOW, which is important if they are to be a key player in WASH. indicated that “A post training M&E has been recommended and conducted in three phases to assess the impact of the training on AfWA’s institutional and human capacity.” It is not clear if this was done. Completeness – minimal | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 38 Source: AfRICAP/WA-WASH Evaluation (2019) TABLE 4. QUESTION 3 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS HOW IS AFRICAP CONTRIBUTING TO AFWA’S REGIONAL COORDINATION, ADVOCACY AND DISSEMINATION, AND HARMONIZATION OF POLICIES THAT ARE INFORMED BY BROAD-BASED UNDERSTANDING OF BEST PRACTICES AND PROMISING MODELS FOR WASH SERVICE DELIVERY? SUB-QUESTION KEY FINDINGS/CONCLUSIONS SPECIFIC RECOMMENDATIONS HOW WILL THIS HELP? WHO ACTIONS? SQ. 3.1: - Lack of quality of AfWA products and documents. R3.1.1. AfWA should develop and strictly follow a quality management protocol within AfWA for all external documentation and communication. Improvement of respect and understanding by members. AfWA Executive Director SQ. 3.2: - Lack of knowledge of what members need in terms of policy knowledge, coordination, and advocacy. R3.2.1. As recommended in Question 1, collect and act on a member survey – including regarding KM requests, especially related to policy. Based on this, as well as an internal survey, determine the type of database needed, investigate what is available, and where data is missing, develop a plan for collecting that data, and regularly updating it. The plan should include the resources to perform this task, any costs, and any training required. Member utilities having more appropriate and needed information to help them improve their performance. AfWA, M&E SQ. 3.3: - Lack of an advocacy component within AfWA. -AfWA does not have a structured WASH policy formulation, harmonization, and adoption strategy. R3.3.1. In order to address the advocacy component rather than policy, future capacity- building programs should support AfWA in addressing the following: • Decide if advocacy remains a priority for AfWA’s members. If yes, determine members’ priority topics. Determine the strategic results expected, knowing that the strategic results mentioned in the Strategic Business Plan – the reputation of AfWA has increased – is inadequate. Consult members directly to answer this question. Advocate for streamlining sanitation under a single organization – e.g. the Ministry of Water and Sanitation – and giving the mandate for sanitation to the water utilities, along with necessary resources; • Decide who should be responsible for advocacy within AfWA, including setting the agenda, conducting the research, and implementation. Clarify the respective roles of the STC and Board and how these roles articulate with that of the Secretariat. In parallel, identifying who has experience and skills in advocacy within AfWA; and • Identify who has the best leverage to conduct policy advocacy and whether it should be done by utilities themselves with AfWA’s support or at AMCOW’s level. Consider advocacy at donor’s level given that country studies show water and sanitation budgets sometimes rely 95 percent on external donors. R3.3.2. AfWA should propose, gain approval, and implement a structured policy formulation, harmonization, and adoption strategy. A subset to this strategy could include white papers (e.g., on gender, NRW, and water quality). Also, AfWA can package its white papers for AMCOW. If current and recent programs (e.g. AfriCAP and RASOP) result in policy recommendations, AfWA must ensure these are taken up beyond their member utilities, but also to the Ministers and Ministries that work in parallel and above the utilities’ organization. Well planned (with buy-in) and executed advocacy will result in transformational changes (e.g., reducing the sanitation access gap – leading to improved health). Consolidating the institutional framework (e.g., sanitation) under a single organization such as the Ministry of Water and Sanitation – and giving the mandate for sanitation to water utilities, along with necessary resources, will improve the attainment of Sustainable Development Goal (SDG) 6, which is to ensure availability and sustainable management of water and sanitation for all. AfWA, Marketing | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 39 1.4. FINDINGS QUESTION 4: HOW HAVE INVESTMENTS FOCUSED ON STRENGTHENING AFWA SECRETARIAT’S STRATEGIC FINANCIAL MANAGEMENT CAPACITY IMPACTED ON ITS ABILITY TO PROVIDE STRATEGIC FINANCIAL AND PLANNING SUPPORT TO THE ORGANIZATION’S PROGRAMS AND LEADERSHIP DECISION-MAKING? SUB QUESTION 4.1: WHAT WERE THE BASELINE (ORIGINAL) ASSESSMENTS OF THE SECRETARIAT’S STRATEGIC FINANCIAL MANAGEMENT CAPACITY, SPECIFICALLY RELATED TO: FINANCIAL AND PLANNING SUPPORT TO ITS PROGRAMS AND LEADERSHIP DECISION MAKING? There was no comprehensive baseline data on AfWA’s strategic financial management performance prior to AfriCAP, especially in the areas of budgeting capacity and budget quality, risk management, and ongoing review and evaluation. A summary of the status of AfWA financial management and internal controls prior to AfriCAP include the following: • AfWA has a manual – Manuel de Politiques et des Procedures Comptables Financières et Administratives [Financial and Administrative Manual of Policies and Procedures] – to guide financial management. The accounting staff of AfWA have the requisite qualifications and experience to effectively manage the accounting functions; • A control structure had been established to ensure all accounts are free of personal funds of all employees and Board members; • Bank reconciliations are performed monthly and prepared, reviewed, and approved by the Finance Director; • Major improvements in financial management after the launch of AfriCAP include the following: - The organization currently has two accounting systems in place, one for the general funds and another for the WOP project funds. The general funds accounts are maintained at Société de Distribution d'Eau de la Côte d'Ivoire (SODECI) using Afrique Compta as its financial management system. The system has the ability to account for financial data by departments and activities. The WOP project accounting system is maintained by the staff of AfWA, who previously did not have the ability to manage funds; - The chart of accounts follows the standards of Organizations pour l’Harmonisation en Afrique des Droits d’Affaires (OHADA) [Organizations for African Harmonization of Business Rights]; and - The annual financial reports are presented at the General Assembly and discussions are made on the financial position and performance of the organization. An annual audit is performed currently by the auditors Ernst & Young. Audit reports for the years 2011 to 2013 were approved by external auditors without any conditions. SUB QUESTION 4.2: WHAT IS THE CURRENT ASSESSMENT OF THE SECRETARIAT’S STRATEGIC FINANCIAL MANAGEMENT CAPACITY, SPECIFICALLY RELATED TO: FINANCIAL AND PLANNING SUPPORT TO ITS PROGRAMS AND LEADERSHIP DECISION MAKING? The development of a procurement and program management manual, along with some capacity￾building, contributed to improve financial management capacity. USAID’s efforts were complemented Completeness – partial Completeness – partial | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 40 with the recruitment of a Finance/Strategy Director using BMGF funding. This resulted in timely delivery of improved quality financial statements and, for the first time, the validation of 100% of AfWA’s expenditures in the 2016’s external financial audit conducted by USAID. In general, however, AfWA/AfriCAP objectives and results are not being fully achieved nor are they likely to be achieved within the time period of the AfriCAP project. As stated above, a glaring case in point is AfWA’s financial sustainability (see Hindering Factor 2), as expected dues to support the organization (e.g., from members, Congresses and publications) are inadequate and have been given insufficient attention. As described in Question 1, AfWA is not gaining individual and private-sector members, has lost important utility members (e.g., Tanzania, Rwanda, Kenya, Zambia, South Africa), and is not expanding across Africa as expected (see Hindering Factor 17). Potential Anglophone utilities, that do not see the value of AfWA, are not joining and, if they have joined, many are not consistently paying their membership dues as discussed in Question 1. Splitting AfWA into regional affiliates (similar to how the USEPA has regional offices) with, at the very least, full Francophone and Anglophone Africa representation should be given serious consideration21. In parallel, AfWA must address the fundamental issue that financial management remains a weak link for AfWA as an institution. The organization still relies heavily (up to 90%) on donor funding and in the latest Strategic Business Plan was unable to set up a strong and evidence-based strategy to increase membership (as discussed in Question 1) and design income-generating activities. The AfWA strategy appears to be to look for donors to fund programs rather than surveying members’ needs and estimating their willingness to pay for the services to help generate more income. In order to dig deeper into the current situation of AfWA’s strategic financial management capacity with regard to financial and planning support to its programs and leadership, the Assessment Team met with senior management of AfWA to solicit their views on specific elements of strategic financial management. In the absence of a baseline of the key indicators, the Team solicited responses for 2015 and 2018 to make a reasonable comparison between the baseline and the current situation. Budgeting: Consolidated budgets for all the organization’s activities are prepared, in addition to project specific budgets. However, AfWA indicated these were more comprehensive since 2017, mid-way into implementation of AfriCAP. Forecasting is part of its budgeting. Risk Assessment and Management: According to AfWA, only the budget burn rate and cash reserves are monitored regularly, but none of the three most commonly used ratios are used. 22 21 Establishing regional sub offices of AfWA would bring the Secretariat closer to its members and increase AfWA’s capacity to relate with its members at decentralized levels. Having regional staff as citizens of the host country will reduce the cost of labor as remuneration will be based on local, rather than expatriate expectations. The existing Secretariat in Abidjan will use these sub offices as vehicles to regularly reach out to members who are delinquent on paying their dues, as well as coordinate the delivery of client services to its members. 22Some of the financial ratios commonly used to assess an organization's financial risk level and overall financial health include the debt-to-capital ratio, debt-to-equity ratio, interest-coverage ratio, and degree of combined leverage. Hindering factor No. 28 Lack of strategy, focus and implementation on financial sustainability | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 41 Ongoing Review and Evaluation: These are a routine part of AfWA’s strategic financial management. Prior to AfriCAP, AfWA held three budget review meetings per annum with the Board and financial management sub-committee. This increased to nine times per annum in 2017. It also reviewed the budget aspect of the 2013-2017 Strategic Business Plan at its mid-term to reflect changes over the previous two years. SUB QUESTION 4.3: WHAT IS THE CURRENT (AND FUTURE) ABILITY OF THE AFWA BOARD TO INCORPORATE STRONG FINANCIAL PLANNING INTO ITS STRATEGIC DECISION-MAKING? While AfWA’s STC and Board members may include individuals with finance and accounting experience, their knowledge and expertise in the use of financial information for strategic decisions has not been brought to bear on the organization. AfWA’s Board focuses on only two financial metrics for strategic decision-making to the neglect of many other important financial metrics. For instance, following a trend analysis of their cash reserves over the past five years, the Board has tasked the Secretariat to improve the cash flow by reversing the current overdraft situation to positive cash flow by 2021. AfWA’s finance department would play an important role guiding the overall financial goals for the organization in line with the broad mission and goals of the organization and then monitor progress toward those goals. This kind of financial planning is crucial to helping determine if the organization can afford to hire new positions, give raises or bonuses, offer new employee benefits, or offer customer sales on products. See Annex XIV for a discussion of financial metrics. SUB QUESTION 4.4: WHAT IS THE CURRENT (AND FUTURE) ABILITY OF AFWA COMMITTEES TO INCORPORATE STRONG FINANCIAL PLANNING INTO ITS STRATEGIC DECISION-MAKING? The various AfWA Committees of the STC present a great opportunity for group interaction and decision-making.23 AfWA’s Finance Committee does have individual members with the capacity to incorporate strong financial management into strategic decision-making. However, despite the availability of individual competencies, the absence of adequate financial data is a hindrance to its ability to incorporate strong financial planning into strategic decision-making.24 The head of the Finance Committee is a senior Finance Director of SODECI with strong and relevant capacity. Other members include the Mali Deputy MD and a senior finance specialist with L'Office National de l'Eau et de l'Assainissement (ONEA) of Burkina Faso. Despite the individual qualifications, the ability of the Committee to incorporate strong financial planning into strategic decision-making remains constrained by the inadequacy of financial indicators, inconsistencies in Committee meeting 23 The capability for analyzing alternative strategies and better screening of options depends on the expertise and perspectives of group members. The best alternatives are thus likely to be chosen and acted upon if decision makers are given adequate financial and program data upon which financial plans and reports and other requirements can be made. 24 Organizations that possess strong financial management capabilities and incorporate financial planning into their strategic decision making are able to make resource‐related decisions intelligently, all in the pursuit of maximizing impact and promoting their mission. Financial management practices include accounting information systems, financial reporting, working capital management, financial structure management, financial planning and control, as well as financial advice and financial management expertise. Completeness – partial Completeness – partial | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 42 participation, frequent use of junior staff to represent superiors in most meetings, and insufficient time during meetings to address issues adequately. Besides the Finance Committee, other committees such as the Programs Committee do not have financial planning capacity and do not appear to factor finances into their strategic decision making. SUB QUESTION 4.5: WHAT ARE EXAMPLES OF OTHER REGIONAL INSTITUTIONS THAT INCORPORATE STRONG FINANCIAL PLANNING INTO THEIR STRATEGIC DECISION-MAKING? AfWA is not conversant with similar organizations that incorporate strong financial planning into strategic decision-making. However, during the interviews, staff did express that the International Water Association (IWA) could be such an organization. WA-WASH also tried to introduce AfWA to the AWWA and its 10,000 members but was unsuccessful; AfWA should try again via AfriCAP. SUB-QUESTION 4.6: WHAT ARE THE FINDINGS REGARDING IMPROVEMENT OF FINANCIAL AND PLANNING CAPACITY OF AFWA? CHANGING CUSTOMER BASE AND INCOME GENERATION It is difficult to establish historical trends in membership and dues payment-growth due to the rising and falling pattern of membership over the years. As noted earlier, the Affiliate member category, which comprises largely foreign private vendors that seek to enter the African market through AfWA, easily fade away after a few years when they are unable to realize the market opportunities expected. Some regular members that regularly pay dues may suddenly become dormant if their senior representative is replaced (e.g., for political reasons). This is further compounded by some members defaulting payment for a couple of years, and suddenly paying off all arrears outstanding in one year. This lack of consistency in membership makes it difficult for AfWA to generate and analyze credible data for planning and decision-making. AfWA needs to invest more in its marketing department to both retain current members and attract new ones. WA-WASH contracted an external provider to develop an income-generating strategy, but the output was disappointing as none of the solutions proposed were costed. There was no analysis of demand for products and services, no market analysis, and no financial analysis (return on investment). The reasons are not known. Furthermore, the study was too general; it was more a collection of general intentions and ideas on income generation than a client- and cost-based plan. There was no member consultation or market study of AfWA’s current and potential client base regarding the services they expect and the amount they are ready to pay for them, no business environment analysis, and no cost-benefit analysis. FINANCIAL RATIOS TO MONITOR THE FINANCIAL HEALTH OF AFWA Ideally, AfWA should be reviewing at least two of four key categories of financial ratios on a monthly basis to keep on top of changing trends in the organization. The absence of historical data on these key ratios significantly limits AfWA’s ability to incorporate financial planning into strategic decision-making. AfriCAP’s capacity-building targeted accounting but not financial management, which could address this challenge. Nor did it address the crucial need for accounting software. Completeness – partial Hindering factor No. 29 The finance team lacks crucial equipment such as professional financial software Completeness – partial | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 43 FINANCE LEADERSHIP The absence of a full-time AfWA Finance Director in the past was in some ways responsible for the lack of attention to analytics. With the position filled, emphasis has now been placed on analytics as the basis for generating relevant data to input into strategic decision-making and planning, improvements are expected. With a qualified team of finance experts in the Secretariat and at the STC level, the use of relevant financial information for strategic decision-making has been enhanced. BUDGET ANALYSIS Regular budget reviews to compare against actual financial results is critical. AfWA reported it had undertaken up to nine budget reviews per annum since 2017. While this may seem unnecessarily frequent, it offers management the opportunity to rapidly re-align budget execution and program activities with changing circumstances. BY-LAW AND OPERATIONAL FLEXIBILITY The AfWA Secretariat has the power to propose amendments in activities if there is strong reason to do so. The Board has shown sufficient flexibility in approving such amendments. DONOR ASSISTANCE/DONOR DEPENDENCE USAID support through AfriCAP, WA-WASH, and past program, such as FABRI has contributed to AfWA becoming an attractive organization for key WASH development partners, namely BMGF and the German Water Partnership. As a result, AfWA strategic and financial planning for the next five years remains dependent on external support (Figure 7). FIGURE 7. AFWA’S EFFECTIVE (2012-2017) AND EXPECTED (2018-2022) FINANCIAL RESOURCES PLAN IN USD25 Source: AfRICAP/WA-WASH Evaluation (2019) 25 Based on data received from AfWA (2013 – 2017) and from the 2018-2022 Strategic Business Plan. Data from 2013 – 2017 were converted from CFA to USD using 03/2019 currency exchange rate, hence this graph does not pretend being accurate. The aim is to show the trend in financing. Favorable factor No. 15 USAID support has been a critical factor in AfWA's attractiveness to leading WASH donors 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 Membership fees 392,700 156,174 216,978 206,450 211,011 195,341 257,091 290,364 323,636 373,273 406,545 Additional Incomes 56,500 53,062 278,797 87,156 133,994 179,637 345,227 719,020 999,247 724,701 1,017,4 Grants 166,920 6,835 1,416,9 114,021 1,131,6 2,325,9 1,987,7 4,203,6 3,703,6 3,703,6 3,703,6 (500,000) - 500,000 1,000,000 1,500,000 2,000,000 2,500,000 3,000,000 3,500,000 4,000,000 4,500,000 Membership fees Additional Incomes Grants Linear (Membership fees) Linear (Additional Incomes) Linear (Grants) | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 44 With regards to financial sustainability, AfWA is not maximizing the benefit of current donor funding to develop its own income-generating activities during this five-year period. For instance: • In Year 5, dues from members and income-generating activities will remain almost insignificant compared to donor funding (Figure 7); • There is a lack of financial figures in the section related to income-generating activities; and • The analysis of membership dues’ effectiveness and potential collection is minimal, and there is no information on how AfWA will improve on this. STRATEGIC PLANNING AfWA is working within its second Strategic Business Plan (2018-2022) following the 2013-2017 Plan, which ended in December 2017. The second Strategic Business Plan was established with some support from AfriCAP and was approved by the Board. AfriCAP indirectly contributed to the establishment of the Strategic Business Plan by supporting the financial and procurement processes; and WA-WASH provided support by funding the income-generating study, which was conducted by an independent company. Another result of the support provided by WA-WASH and AfriCAP is the Strategic Business Plan contains observations on AfWA’s SWOT. The combined support has contributed to raising AfWA’s awareness of its challenges and strengths to establish relevant goals such as “assessing internal governance,” “progressing toward a sustainable financing model (…) capable of stimulating new sources of funding for the Association and making the mechanism for mobilizing contributions effective,” and “broadening and strengthening of the Association's membership.” In spite of these positive aspects, a number of stakeholders found weaknesses in the Strategic Business Plan as noted in the textbox below: The Assessment Team reviewed the content of the Strategic Business Plan and, together with other feedback, concluded that the assertions made by the respondents were founded. “The new business plan is not impressive” “The new financial plan is still weak” “It is not clear how they will achieve some of those objectives” “First of all, we need to know the needs of the members, we don’t make a business plan on donor’s money, so the BP I am not sure it’s sustainable” “[M]odern ICT tools can be used to conduct remote meetings” “The finance committee must validate the Business Plan” “Some board members and other experts raised very relevant observations, but the Board validated the Business Plan eyes wide shut” “The STC did not have the opportunity to give their opinion on the BP” Hindering factor No. 31 The Board struggles in its function to ensure the financial viability of AfWA Favorable factor No. 16 The business plan contains an analysis of AfWA’s SWOT Hindering factor No. 30 Strategic and financial planning is still weak | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 45 The Strategic Business Plan lacks clarity and rigorousness even with the title. It is not clear whether the document is a business “plan” or a document about the foundation and strategic bases of the Plan (Figure 8). The content of the Strategic Business Plan also lacks clarity and rigorous in each section. For example, in Section 2.2 there is confusion regarding what should be the “way forward”. In particular, with the goals, the needs, and the challenges (Annex X). The Plan lacks crucial components such as the description of products and services of AfWA along with detailed costs and benefits, a marketing plan, etc. While there is no standard structure for a business plan, it normally at least aims to explain how the organization will make money (Figure 9). The comparative structures presented in Figures 8 and 9 clearly show AfWA’s Strategic Business Plan is neither structured around the development of products and services, nor includes a strategy to develop the client base and does not include detailed financial calculations – something that should have concerned the Board. The Plan contains expected outputs, strategic results, and objectives but there are no details on how these are going to be achieved. The financial plan does not articulate necessary details of a strategy. A number of financial figures are missing to enable a thorough understanding, and the financial plan is not in line with some intended goals. For example, what are the financial resources needed to implement income-generating activities, what will be the price of AfWA services and products, what is the client base, how much are the members willing to pay, and what are the forecasted benefits? What are the necessary activities to increase membership, what are the figures (current, potential, forecast) based on? As elaborated in Question 5, there was the absence of gender/youth sensitive-budgeting and financial strategy. Strategic and financial planning remains weak. In spite of effective progress, the Program and Finance teams require more capacity to draft a quality business plan. FIGURE 8. STRUCTURE OF AFWA’S 2018-2022 STRATEGIC BUSINESS PLAN Source: AfWA Strategic Business Plan, 2018- 2022 (Foundation and Strategic Bases) (2019) FIGURE 9. EXAMPLE OF THE CONTENT OF A BUSINESS PLAN Source: Generic Content of a Business Plan Hindering factor No. 32 The approach of AfriCAP and WA￾WASH was not effective enough to ensure the development of a quality business plan | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 46 SUB-QUESTION KEY FINDINGS/CONCLUSIONS SPECIFIC RECOMMENDATIONS HOW WILL THIS HELP? WHO ACTIONS? SQ. 4.2: -The Board and relevant committees of the STC do not receive the full complement of financial reports and information to enable them to make strategic decisions. R4.2.1. AfWA should consistently use information from its financial monitoring to implement measures that address negative trends in key financial metrics. Include financial monitoring reporting in STC and Board meetings, highlighting progress and negative trends. The Board Chairman and Executive Director SQ. 4.3: -AfWA does not adequately use key relevant financial ratios to monitor the financial health of AfWA. R4.3.1. AfWA Secretariat should identify up to four relevant financial metrics and include these in routine reporting, in addition to the budget execution rate and cash reserves rate that it currently monitors. Use of financial ratios is one way to analyze the financial health of an organization. By looking closely at these key financial ratios, AfWA would be able to identify how it might be able to improve its operations or make comparisons with other industry players. Executive Director and Director of Finance -In the past, AfWA was challenged with absorbing staff after a project had ended with no financial plan for it. Later, it was faced with a shortage of staff, but could not recruit until the onset of a donor project. R4.3.2. Employment of new staff should not only be based on requirements for donor-funded projects now, but also based on a systematic HR assessment, AfWA’ s projected cash flows, and operational needs. This will increase staff retention and ensure proper long-term planning of HR requirements and the associated financial resources to pay. Executive Director and Board Chairman -The Strategic Business Plan lacks a comprehensive financial analysis, including sustainability scenarios for different strategic choices. R4.3.3. In the next review of the Strategic Business Plan, AfWA should incorporate strong financial analysis as basis for strategic choices in the plan. AfWA will be able to further refine the activities and programs envisaged in the Strategic Business Plan. From a wish list, it can settle on key priority actions. Executive Director/Pro grams Director SQ. 4.5: -AfWA is not aware of the best practice case in the application of strong financial planning into its strategic decision-making. R4.5.1. AfWA should research/identify organizations with mandates similar to its, such as IWA or AWWA, and identify how they incorporate strong financial planning into their strategic decision-making. AfWA will understand the importance of and how relevant financial ratios are applied in financial planning. Executive Director and Director of Finance SQ. 4.6: -AfWA’s current financial management software is not robust enough to generate all the requisite financial reports for financial and strategic decision-making. R4.6.1. AfWA should migrate to a more robust financial management software, a suggestion that was made three years ago. AfWA should ensure that the selected vendor provides adequate training on the full functionality of the software. Initially, AfWA should sign a one-year maintenance contract with the software vendor to provide on-the-job support in the first year of its roll out. This will simplify the accounting processes, improve the credibility of financial system, and can offer automation of financial reports that are critical for strategic decision-making. The Director of Finance Source: AfRICAP/WA-WASH Evaluation (2019) TABLE 5. QUESTION 4 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS HOW HAVE INVESTMENTS FOCUSED ON STRENGTHENING AFWA SECRETARIAT’S STRATEGIC FINANCIAL MANAGEMENT CAPACITY IMPACTED ON ITS ABILITY TO PROVIDE STRATEGIC FINANCIAL AND PLANNING SUPPORT TO THE ORGANIZATION’S PROGRAMS AND LEADERSHIP DECISION-MAKING? | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 47 1.5. FINDINGS QUESTION 5: WHAT ARE THE GENDER-SPECIFIC AND DIFFERENTIAL EFFECTS OF AFRICAP AND WA-WASH ACTIVITIES? The Assessment Team examined the extent to which gender equality and empowerment were achieved as a result of the combined WA-WASH and AfriCAP programs. Data was gathered to assess the extent to which the programs achieved positive results for women and reduced gender gaps to appreciate women professionals’ sustainable empowerment in the WASH sector. Throughout the interview process, the Assessment Team applied gender-sensitive methods and systematically included questions focusing on gender mainstreaming. While collecting and analyzing data, the Assessment Team looked at sex disaggregation, gender-focused activities, and gender-sensitive M&E indicators. The team applied a gender lens to each of the five evaluation questions described above. This is reflected in the Evaluation Matrix26 and the findings are summarized as follows. SUB-QUESTION 5.1: TO WHAT EXTENT DID THE CAPACITY-BUILDING ACTIVITIES AT THE INSTITUTIONAL AND STAFF LEVEL CONSIDER FEMALE EMPOWERMENT? CAPACITY-BUILDING The capacity-building activities did not specifically consider female equality and empowerment. As there was no detailed baseline evaluation of AfWA’s capacity and training needs assessment, women’s empowerment was neither a planned nor executed component of the capacity-building program of AfWA as an institution nor of AfWA’s staff. Despite this, women still retained the opportunity to participate in training courses (e.g., M&E, accounting, project management, etc.) on an equivalent basis to other staff members, including to solicit AfWA support for other trainings at their own initiative. Women demonstrated general satisfaction, and indicated positive impacts, from their training. Additionally, AfWA’s gender-focused activities gave women opportunities to participate in STC meetings and Congresses as well as parallel events (such as the International Women Association forum in Marrakech). All AfWA female staff also have the opportunity to become members of the Ivorian Network of Professional Women in the WASH sector (Réseau Ivoirien des Femmes Professionelles de l’Eau et de l’Assainissement, RIFPEA) and as such, some of them participated in training on women's leadership. Overall, the AfWA environment is conducive to gender promotion. HUMAN RESOURCES POLICY The AfWA HR management manual abides by legal requirements of the Côte d’Ivoire labor code relative to women’s specific needs such as maternity leave and breastfeeding. But there is no specific AfWA strategy to promote gender equality and women’s empowerment. More broadly, the AfWA HR management manual, and common practice at AfWA, indicates a lack of transparency in terms of appointments, promotions, and salary pay scale. These shortcomings negatively 26 See the Inception Report. Completeness – significant Hindering factor No. 33 Lack of baseline on AfWA (female) staff capacity Hindering factor No. 34 Gender imbalance in leadership positions | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 48 impact HR management, and existing research27 points out that this directly contributes to gender gaps and the low number of women in organizational leadership positions. PROGRAM MANAGEMENT The AfWA Program Management manual does not include a gender strategy, lacks sex segregation in the M&E section, and only refers to gender as an opportunity to capture funds. The AMEP of AfriCAP contains 18 indicators out of which two directly address gender (IN 05 and 06 – on the number of women’s networks created and the number of their members), but none is disaggregated. Yet, due to the nature of its activities and purpose, AfWA undertakes many activities and has used indicators that are ideal for segregation, such as training, learning events, scholarships, and knowledge products. The institution has the opportunity to gain in-depth knowledge by tracking disaggregated indicators such as gender, youth, geographical area, or language. This knowledge could lead to reflection, provide evidence, and support decision-making on future strategy and activities to ensure AfWA as an institution appropriately focused on: • Women in utilities (to fill the gender gap); • Youth (to increase capacity of future leaders); • The five geographical areas covered by AfWA; and • French, English, and Portuguese-speaking members. The last two bullet points support AfWA in gaining legitimacy in regions other than West Africa and to ensure coherence with AfWA’s vision to become a recognized “pan-Africa leader.” There exists a reasonable level of awareness of shortcomings in terms of gender-tracking within AfWA and the will to strengthen these components, especially since USAID provided gender training. Yet, despite this awareness, there is a tendency to hold back reinforcement of gender improvements until the appointment of a Gender Specialist. Roles and responsibilities should be clarified and program management and coordination should be capacitated to ensure AfWA’s vision for gender is effectively enforced within the organization. STAFF RECRUITMENT AND GENDER BALANCE The more recent recruitments have significantly improved gender balance within the organization. Since 2015, a number of women have been recruited in AfWA in various program and support positions including: M&E officer, Communication Manager, Human Resources Assistant, Procurement Manager, Accountant, etc. However, in positions of leadership, there exists a striking gender imbalance with (Table 6): 27 African Development Bank (ADB) (2014) Hindering factor No. 35 There is not enough focus on representativeness (reflected in the M&E indicators which are not segregated) | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 49 • No women in senior leadership position at Secretariat level; • Only two women as Board members, out of 14 members; • 30 percent of the staff of the Executive Secretariat are women but none in a leadership position; and • The STC’s specialized committees have a good balance, with five women rapporteurs out of six but only one woman president out of six. Again, within AfWA, there is a strong awareness of the importance of gender equality and women’s empowerment under the vision, impetus, facilitation, and personal involvement of its Executive Director. Therefore, the next step consists of institutionalizing this vision; a step that is currently ongoing as the Executive Secretariat is preparing a modification of the statutes to increase women’s representation in AfWA’s bodies. It is interesting to note that if AfWA’s bodies were more representative of utilities in general, the number of women may increase as English-speaking countries have more women at executive levels. SUB-QUESTION 5.2: WAS THE CREATION OF THE AFWA NETWORK OF AFRICAN WOMEN PROFESSIONALS EFFECTIVE AND WHAT ARE THE IMPACTS ON WOMEN IN THE WASH SECTOR? SUCCESSFUL CREATION OF NETWORKS AfriCAP placed significant importance on AfWA fostering gender mainstreaming in the WASH sector under its cooperation agreement with USAID. The promotion, establishment, and support of women’s networks is an integrated component of AfriCAP Objective 1 on capacity-building and policy harmonization, with a target of creating 12 networks, including 400 members during the lifetime of the project. This activity was effectively implemented and, at the time of the evaluation, 10 networks had officially been created and three were in the process of being finalized. Two Regional Coordinators are in charge of the network, respectively for French and English-speaking Communities of Practice (Table 7). TABLE 6. GENDER IMBALANCE WITHIN AFWA’S BODIES POSITION MALE FEMALE Board President 1 0 Board Members 14 2 STC Presidents 1 1 (vice president) STC Committees’ President 5 1 STC Committees 7 8 AfWA’s Executive Secretariat 15 8 AfWA’s Senior Management Team 4 0 Source: AfRICAP/WA-WASH Evaluation (2019) TABLE 7. NETWORKS OF PROFESSIONAL WOMEN IN THE WASH SECTOR REGIONAL COORDINATION NETWORKS OF PROFESSIONAL WOMEN IN THE WASH SECTOR Géraldine Mpouma, Regional Coordinator for Networks of Professional Women from Francophone countries • RIFPEA, Côte d’Ivoire • Réseau des Femmes Maliennes Professionnelles du Secteur Eau et Assainissement (REMAFPEA), Mali • Cameroon Association of Professional Women in Water and Environment (AFPEEC), Cameroon Completeness – partial | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 50 Source: AfRICAP/WA-WASH Evaluation (2019) COORDINATION, DEVELOPMENT OF ROAD MAPS, AND IMPLEMENTATION OF TARGETED ACTIVITIES Both networks are coordinated by dynamic female utility managers who have coordinated the establishment of regional roadmaps that are translated at a national level to address local contextual needs. The networks focus on female leadership in WASH utilities, advocating for more females graduating in water and sanitation, and socially-oriented activities. The female leadership component targets advocacy for parity and equity, since women, despite their diplomas, seldom hold management positions in water and sanitation utilities. This implies that AfWA strongly encourages utility Executive Directors to implement an affirmative action policy. It also involves capacity-building and personal development, as women often underestimate themselves and their capacity to integrate within senior management. As such, women from the networks have organized capacity-building trainings (e.g. in Côte d’Ivoire and Cameroon) and have pushed toward the organization of a Master Class on female leadership, which is expected to take place in 2019. Women from the national networks also conduct activities to show women are as proficient as men in leadership positions. Different actions have been carried out such as proactive reporting on and implementation of new knowledge acquired during the STC meetings. In Cameroon, the president of the women’s network – who is also the President of the STC Specialized Committee on Management – has pushed for the creation of an AfWA/Community of Practice Committee within CamWater. The objective is to increase mutual knowledge dissemination into the utility and to link up utility technicians with the communication department of AfWA to raise the technical content of AfWA’s communication material. In Cameroon, the network is currently working on a mapping project in partnership with a student from AfWA’s network to create a database on women’s positions in utilities and their professional evolution.28 The female leadership component also targets female students at high school and university levels, to encourage them to engage in water and sanitation studies – especially in sanitation, which is likely to develop in the future. Concrete actions conducted by network members include the organization of career days, conferences to attract women in the sector, and visiting schools. Networks also advocate 28 Statistics on existing positions and the presence of women at these positions. • Réseau Bénin des Femmes Professionnelles de l'eau et de l'Assainissement (RBFPEA), Benin • Réseau Professionnel des Femmes Djiboutiennes de l'Eau et de l'Assainissement (REDFEPEA), Djibouti • Réseau Sénégalaise des Femmes Professionnelles de l’Eau et de l’Assainissement (RSFPEA), Sénégal Faustina Boachie, Regional Coordinator for Networks of Professional Women from Anglophone countries • Women in WASH (WOWASH), Nigeria, Ogun State • Women in Malawi Chapter, Malawi • Women in WASH (WiWASH), Ghana • Patron of Professional Women for Water and Sanitation (PW4WATSAN), Uganda Currently being formalized Women’s networks in Burkina Faso, Niger, and Central African Republic. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 51 for deliberate policy to include equal number of men and women in scholarship programs at different levels (AfWA and AMCOW) and to facilitate internships of all students in utilities. The socially-oriented activities address access to water and sanitation and hygiene in vulnerable areas, such as facilitating access to drinking water in primary schools in low-income districts, sensitizing girls on menstrual hygiene management, and sensitizing communities on water leaks and pipe breaks to improve utilities’ performance (Nigeria). These actions are undertaken with the support of national and local utilities. Women have mixed opinions on these types of activities. While some point to the benefits for the utilities’ customer relationship management, etc., others would like to move beyond the social approach of women's involvement to focus more on the issue of female leadership representation. BENEFITS FOR WOMEN AfWA’s proactive policy has benefited professional women in the WASH sector in many ways, for example: • The number of female participants in STCs and Congresses has increased significantly since 2015; • Increased participation has provided professional women with more visibility and legitimacy; • Women are empowered to express their views on sector development as a whole, not only from their job description, which has provided them with opportunities to network; • Women took this opportunity to strengthen their professional role with the support of AfWA. For example, they created a sub-group for women and young people in the Specialized Committee on Management, which is chaired by a woman (from CamWater Utility, President of the Cameroon Network of Women). This group formulated several proposals in terms of capacity￾building (Master Class), ethics (gender chart), and governance (affirmative action); and • Participation in thematic seminars (NRW, Water Quality) has increased their knowledge and they have gained respect from their peers. BENEFITS FOR AFWA AND UTILITIES Women’s increased participation has a trickledown effect on AfWA and utilities as observed not only by women themselves but also by other utility professionals. Women bring their own knowledge at the STC level and disseminate what they have learned from the STCs into their own work and utility colleagues. As the networks undertake social activities (hand washing stations, menstrual hygiene management, promotion of mobile money for the payment of water bills) in partnership with utilities, they benefit the utilities by building customer relations, improving utility’s revenues, and increasing customers’ good will. Now that the foundations of the women’s network are laid, AfWA needs to carry on supporting activities to ensure the networks conduct activities in line with AfWA’s vision and contribute to the development of a gender-balanced WASH sector. A number of stakeholders raised the issue of the lack of effective achievements from women’s network and as a tokenized “folkloric topic.” In fact, activities such as providing information to students on “I have seen a number of women grow professionally (…) the seminars are an opportunity to learn and network. Since I participate in AfWA’s events, my position in the company has changed, I am more respected because of the knowledge I have on many topics of the sector, even if I am not a technician”. (Utility staff and member of WASH professional women’s network) | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 52 sectoral studies and careers and advocacy toward accountability in recruiting processes, career paths, and equal salaries will benefit utilities and the WASH sector as a whole. However, the results can only be visible after a few years, provided that they are scaled up and monitored. SUB-QUESTION 5.3: WERE THERE ANY PROMISING GENDER SENSITIVE MODELS IDENTIFIED AND IS THERE ANY EVIDENCE OF GENDER MAINSTREAMING IN NATIONAL POLICIES? The Assessment Team sought to understand whether sector stakeholders were delivering gender￾sensitive WASH service delivery models and how this potentially impacted their effectiveness. The respondents were not aware of any formalized studies on gender sensitiveness of WASH service delivery models and the potential impact of gender differentiation in the performance of utilities or service providers. However, in the WASH context, stakeholders observed and reported on the benefits of mobilizing women at grassroots and middle management levels. Some stakeholders envisioned women's networks should be levers for achieving water and sanitation objectives. For example, various stakeholders noted benefits of gender differentiation, such as: • Improved performance of rural water projects when women were involved in the management of water boards; • Improved management of community assets such as standpipes, along with improved maintenance and allocation of resources when women were in charge; • Increase in user’s fee collection rates when women were involved; • Improvement of processes related to water quality due to the presence of female laboratory technicians; • More rigorousness in assignment management; and • Improved client relations and accounting processes. Conducting studies on gender impact on utilities is likely to be a valuable research topic for AfWA in the framework of a comprehensive study of utilities’ performance by introducing gender indicators, among others, to determine its influences. Research conducted by Ernst and Young in 2016 on 1,500 firms demonstrated that increasing of the number of women in top management positions improved innovation, profitability, as well as non-financial performance. AfWA is in an ideal position to access data from its members and conduct a comparative study since some countries stand out (Djibouti, Rwanda) and others are taking positive steps (Côte d’Ivoire) in this area. SUB-QUESTION 5.4: WHAT WERE THE OBSTACLES AND FAVORABLE FACTORS TOWARD IMPROVEMENT OF GENDER EQUALITY AND WOMEN EMPOWERMENT IN THE WASH SECTOR THROUGHOUT THIS PROJECT? “I fully support the strengthening of women's place in the field of water quality because female laboratory technicians are much more thorough (“minutieuses”). We need support at the level of training institutes so that there are more women in water chemistry.” (Laboratory manager) Completeness – partial | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 53 A number of factors contributed to the successes of the gender equality and women’s empowerment strategy, such as clear targets, strong engagement from AfWA’s Executive Secretariat, buy-in of professional women from members, and coordination of activities under regional roadmaps. This buy-in can be explained by the growing presence of qualified female employees within utilities and existing frustration due to limited career perspectives. AfWA inspired the creation of the network in each member country and, according to all the stakeholders interviewed, the support was significant and consistent, including: • Creating awareness of AfWA members and encouraging the creation of networking during STC meetings and Congresses; • Directly advocating and urging utility Managing Directors to select women representatives at events and to support the creation of the networks at the utility level; • Funding the participation of women to attend AfWA’s events and other events such as the Women International Association (WIA) congress; and • A personal engagement of AfWA’s Executive Director in promoting gender-sensitive programming. The women’s national networks have managed to coordinate to establish roadmaps and action plans, but these cannot be scaled up without further financial support. To date, AfWA’s financial support has enabled the development of the networks and empowerment of women through attendance at international events. Women managed to conduct a small number of activities through private sponsors (football clubs, hygiene products corporates, for example) and through financial support of their respective utilities. However, AfriCAP did not integrate planning, budgeting, or follow through of high-impact activities by the women’s network. Additionally, AfWA’s 2018-2022 Strategic Business Plan does not include a gender-sensitive budgeting and financial strategy. This needs to be planned together with a clear strategy on how AfWA wants the networks to evolve and how this aligns with AfWA’s mission holistically. As stated by one of the networks’ leaders: “It is not enough to create; you should sustain it to grow.” “AfWA’s executive management has been very supportive and has given us a forum to raise awareness among the leaders. We were able to do this up to Executive Committee level. This has spurred the acceptance of the creation of committees by the Managing Directors of utilities”. (Utility staff and member of WASH professional women’s network) Favorable factor No. 17 The existence of a pool of women professionals in the sector who are eager to participate in the strengthening and coordination of the WASH sector Favorable factor No. 18 Strong engagement of AfWA’s Executive Direction Hindering factor No. 36 AfriCAP did not integrate planning, budgeting and follow through activities implemented by Women’s network Hindering factor No. 37 AfWA 2018-2022 business plan does not include gender sensitive budgeting and financial strategy | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 54 TABLE 8. QUESTION 5 SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS WHAT ARE THE GENDER-SPECIFIC AND DIFFERENTIAL EFFECTS OF AFRICAP AND WA-WASH ACTIVITIES? SUB-QUESTION KEY FINDINGS/CONCLUSIONS SPECIFIC RECOMMENDATIONS HOW WILL THIS HELP? WHO ACTIONS? SQ. 5.1: -AfriCAP and WA-WASH objectives and activities have contributed to women’s capacity-building within AfWA. R5.1.1. Future programs should specifically target women’s empowerment by integrating women’s needs assessment in the baseline capacity assessment. This would support the delivery of training addressing specific gender needs (such as leadership). Donors, AfWA -AfWA’s program management manual lacks a gender strategy. -There is a high level of awareness of the benefits of gender differentiation. -Gender vision is guided by personal involvement/convictions of AfWA’s Executive Director. R5.1.2. AfWA should draft a gender strategy to formalize and ensure implementation of gender empowerment measures. This would increase coherence between the gender-focused activities being implemented and would contribute to addressing remaining gaps. Formalizing a leaders’ view into a policy/strategy is crucial to ensure long-term sustainability and mainstreaming at organizational level. AfWA’s program management (coordination) with HR assistant and M&E officer. -AfWA has recruited a high number of women since 2015, but there is a significant gender imbalance in leadership positions at the Secretariat. -At STC and Board level, gender imbalance is quite significant and is mainly due to existing imbalance within water and sanitation utilities. -There is a good level of awareness of this situation at AfWA’s level and the Secretariat is currently preparing statutory modifications to address this issue. R5.1.3. Current efforts should be sustained to reach statutory representation of women in AfWA’s decision-making bodies as part of a global effort to increase representation within AfWA (gender, languages, geographical scope, and youth). Increasing coherence between what AfWA promotes (female empowerment in utilities) and AfWA’s internal structure. It would also empower women at utility level and contribute to engaging reflection on the Board composition. AfWA Secretariat, Board, STC and General Assembly -There is a high awareness of gender issues within AfWA’s Secretariat, but reinforcement is held in abeyance until the appointment of a Gender Specialist. R5.1.4. Definition and enforcement of a gender policy should be included as the responsibility of the Program Director and ensure capacity to lead this task. This would contribute to mainstream gender into program management and into AfWA’s organizational management. AfWA Secretariat | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 55 Source: AfriCAP/WA-WASH Evaluation (2019) R5.1.5. In case the organogram is modified, train the head of Administration/Finance in gender mainstreaming. R5.1.5. Should the current plan to recruit a gender specialist be pursued, one of his/her tasks should be capacity-building of existing staff on gender issues. -Women feel they need more interactive discussions in their networks for both topics and priorities. -They lack an AfWA focal point to ensure capitalization and coherence. 5.1.6. Designate a focal point at AfWA’s level to ensure coordination of women’s related activities, capitalization, and coherence. The focal point should not necessarily be a gender specialist as topics are related to capacity-building, sectoral coordination, or dissemination of effective service delivery models. AfWA has a coordination role and the networks need to entrust a staff to fulfill gender coordination activities. Gender issues can be dealt with at a programmatic level. SQ. 5.3: - Performance indicators are not sex disaggregated. R5.3.1. Initiate a process of reflection on the segregation of indicators to reflect gender but also youth and diversity of African members. This would support and highlight AfWA’s progress toward more gender-balanced representation. AfWA Secretariat and members SQ. 5.4: -AfWA is on the way to exceed AfriCAP’s targets to create 12 national networks of African Women professionals. Positive effects are observed on women in the WASH sector, on utilities, and on AfWA. R5.4.1. Conduct research on gender impacts in WASH utilities and promising gender- sensitive models in Africa (benchmarking). R5.4.2. Develop policy advocacy on gender management in utilities. Conducting research would support: • Visibility of outputs and outcomes to highlight benefits to utilities and on the sector from women’s network activities; and • Evidence to support policy advocacy on gender mainstreaming in utilities. The AfWA KM team SQ. 5.4: -Gender-sensitive budgeting, planning, and financial strategy is not integrated in the Strategic Business Plan and at program- management level. R5.4.3 Integrate gender-sensitive budgeting, planning, and financial strategy in the Strategic Business Plan and at program-management level. This would support the newly created professional women’s networks in implementing the activities from the road map and action plans (low-cost, high-impact activities). Board, program management, and financial management. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 56 2. SUMMARY CONCLUSIONS This section provides a high-level summary of conclusions derived from the Assessment Team’s response to the Evaluation Questions above. The efficacy of WA-WASH and AfriCAP was mixed with some meaningful results, as well as both minor and significant areas needing to be addressed as described below. 2.1. WAS USAID STRATEGY EFFECTIVE TO SUPPORT AFWA? • Yes, AfWA’s performance, stature and outputs were improved as a result of USAID’s strategy and support. • USAID’s strategy was and is led by its development hypothesis of “Building capacity of regional institutions in water supply, sanitation and hygiene leads to solutions for WASH-related challenges in the developing world. Persistently low coverage rates in WASH requires reaching a wider audience with the best practices and lessons learned from project implementation, policy harmonization and advocacy of WASH standards or approaches within the West African region. Improving institutional capacity will effectively address sustainability of WASH interventions and coordinate disparate approaches used for implementing WASH projects in the region. 29” • Where the USAID hypothesis/strategy should be adjusted is in the following areas: o Updated evaluation and focus on which institution(s) to support going forward. The evaluation will determine which institutions add the most value (e.g., AfWA and AMCOW) and which align most efficiently with others. The evaluation should also determine which institutions are best supported by USAID and which are best supported by others; o Updated evaluation and focus on high-value capacity improvement areas for the selected institutions; o The hypothesis/strategy should have a significantly greater focus on outcomes and measurable goals, encouraged by conditional funding – conditional upon results achieved; and o The strategy should increase focus on sustainability, sanitation, and gender promotion. 2.2. WAS THE SUPPORT PROVIDED BY AFRICAP AND WA-WASH SUFFICIENT TO BUILD THE CAPACITY OF AFWA? Both WA-WASH and AfriCAP made meaningful contributions to the capacity of AfWA. However, there were a few missed opportunities that would have enabled even greater successes: • Insufficient benchmarking of staff capacity and utility performance; and, without the benchmarking, output and outcome measurements were difficult. • A more radical paradigm shift from staff focus on activities/input to focus on outcomes/output. • Building a culture of AfWA staff/management accountability (to goals, objectives, deliverables, etc.) from the Board, to senior management, and throughout the organizational ladder. • In the planning of activities, more direct connections should have been made between the needs of AfWA members (particularly utilities), its clients (rate payers), and the required 29 See Annex I: Evaluation Scope of Work. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 57 improvements within AfWA to meet those needs; hence, the ability to better micro-target WA￾WASH and AfriCAP support. • Defining the capacity improvements necessary to address the situation of increased numbers of regular and affiliate members, payment of membership dues, and seeing to the successful capacity-building to meet these needs. • Defining the capacity improvements necessary to address the situation of few and marginalized Anglophone members and seeing to the successful capacity-building to address this need. • Senior management progress defining the policy and advocacy roles and responsibilities of AfWA vs. AMCOW. • Greater networking, coordination and use of the resources from USEPA, USAID, International Water Association (IWA), Water and Environment Foundation (WEF), International Development Association (IDA), etc. • Coming up with a strong and effective business plan - client and cost based, detailed and realistic. The Strategic Business Plan established with the support of WA-WASH and AfriCAP could have been significantly improved. The strategic and financial planning was mainly donor￾and project-based and did not include a financial sustainability plan. Donor confidence was and remains a strong asset for AfWA, hence, a great opportunity to expand AfWA’s client base and to develop income-generating services and products to meet the needs of its’ members. The current Strategic Business Plan (2018- 2022) did not take advantage of the financial cushion (due to donor support) over the next five years to develop activities to ensure sustainability and continental outreach. The Strategic Business Plan could have been more in line with AfWA’s strategic objectives, such as setting up the AfWA Academy and integrating support to women’s network and women’s equity and empowerment activities. 2.3. WHAT IS THE CURRENT CAPACITY OF AFWA TO DELIVER SERVICES TO ITS MEMBERS AND CARRY OUT ITS OWN MANDATE? • AfWA’s capacity has improved considerably and, in general, it at least has the foundation to cover the needs of its members, to varying degrees. • A thorough analysis of the needs of its members is required, as well as an updated AfWA capacity assessment before this question can be answered in the future. For example, should the members need greater support in policy formulation and advocacy or best practices areas of concern, then AfWA will likely need to improve its capacity in these areas. AfWA must be prepared to respond to its members once it learns what they want. • An area needing immediate attention is AfWA’s ability to self-fund its operations and the potential expansion of its operations and services (e.g., gaining members at universities, ministries, and private sector). It needs to look at KM as a source of revenue and a business (e.g. advertising in its quarterly magazine) – and it needs staff with the capability of “selling” these (KM) services. • Only 30 to 40 % of members regularly pay dues and there are almost no Anglophone members. • AfWA staff, together with their development partners, undertook successful activities that are relevant and effective to address sectoral needs such as peer-to-peer partnerships, youth networks and scholarships, and the WASH professional women’s network; but there is a lack of capacity to replicate and sustain these projects after they end. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 58 ANNEXES ANNEX I: EVALUATION SCOPE OF WORK USAID/WEST AFRICA EVALUATION STATEMENT OF WORK FOR A MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY BUILDING PORTION OF THE WA-WASH PROGRAM A1. Introduction and Background A.1.1. Description of the Problem, Development Hypothesis (es), and Theory of Change a. Development Challenge Globally, 780 million people lack access to safe drinking water supplies which negatively impacts both health and livelihoods. Low safe drinking water access rates in West Africa are exacerbated by rapid and unplanned urbanization, limited water resources, insufficient financing, and inadequate human and institutional capacity to address these challenges. In USAID West Africa’s 21 target countries 34% of a population of 346,182,000 people lack access to safe drinking water[1]. Moreover, while a great deal of attention has been paid to the issue of water supply, relatively little attention has been paid to the issue of water quality. Many development partners, NGOs, and organizing bodies in West Africa are actively partnering with national governments in the WASH sector to address access gaps and help the region meet its Sustainable Development Goals (MDGs) for improved drinking water and sanitation access. Most countries in the region are on track to meet their MDG goals for access to safe drinking water though the sustainability of those gains is jeopardized by inadequate management of drinking water infrastructure, including ineffective water quality assurance, and inadequate laboratory resources for water quality monitoring and analysis. In addition, the majority of countries in the region have fallen short of their MDG goals for access to improved sanitation due in large part to the inability of public sector approaches to adequately finance and manage sanitation services and ensure their sustainability. There are many active stakeholders responding to WASH sector challenges in West Africa, but strong and effective regional organizations are necessary to implement engagement strategies and regional initiatives across stakeholder communities. There is a clear need for (1) coordination among organizations in the region with a mandate to work in the WASH sector, (2) regional organizations with increased capability to formulate, coordinate, adapt, harmonize, and implement WASH policies across West Africa; and (3) regional organizations with enhanced ability to serve as effective regional leaders for knowledge sharing and public awareness in the WASH sector. Capacity building will enable regional partners to serve as catalysts to increase the coverage of quality WASH services and increase the number of West Africans who have access to them. Coordinating activities in the West Africa/WASH portfolio and monitoring past projects in the portfolio will strengthen sustainability, allow for sharing of lessons learned and best | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 59 practices in the sector regionally, and facilitate replication and scale-up through appropriate knowledge management platforms. b. Development Hypothesis Building capacity of regional institutions in water supply, sanitation and hygiene leads to solutions for WASH-related challenges in the developing world. Persistently low coverage rates in WASH requires reaching a wider audience with the best practices and lessons learned from project implementation, policy harmonization and advocacy of WASH standards or approaches within the West African region. Improving institutional capacity will effectively address sustainability of WASH interventions and coordinate disparate approaches used for implementing WASH projects in the region. A.1.2. Summary Strategy/Project/Activity/Intervention to be evaluated USAID/WA WASH Portfolio The USAID/West Africa (WA) Water, Sanitation and Hygiene (WASH) portfolio, approved under the WASH Project Appraisal Document (PAD) in January 2014, includes four mechanisms intended to achieve the goals of building the capacity of regional organizations working in the WASH sector, increasing access to improved water sources and sanitation services. First, a cost extension of the WA-WASH activity provided coordination of USAID WASH activities in the region, targeted capacity building to regional organizations and monitoring of past WA-WASH activities. The WA-WASH extension is the subject of this SOW and is discussed in detail below. Second, a cooperative agreement is in place with Population Services International (PSI) to provide improved sanitation service delivery in West Africa. Third, a Participating Agency Program Agreement (PAPA) is in place between USAID and the US Environmental Protection Agency (USEPA) to strengthen laboratory capacity in West Africa for drinking water monitoring and management. Finally, USAID/WA is providing direct assistance to the African Water Association (AfWA), a regional organization with a mandate to provide leadership in WASH in West Africa. The primary goal of the AfWA Regional Institutional Capacity Building Program (AfriCAP) is to strengthen the capacity of AfWA to coordinate, formulate, adapt, and harmonize WASH policies in order to improve the performance of the sector across Africa in general and West Africa in particular. The AfriCAP program needs to reach a wider WASH audience with best practices and lessons learned from well-founded knowledge and experience from previous programs and projects. Africa has made some progress towards meeting the Millennium Development Goal targets for water supply, however sustainability remains doubtful. Most African countries have not met the sanitation targets due to inappropriate approaches that do not attract the required funding. Nevertheless, there is consensus in Africa that solving the continent’s WASH–related challenges require reaching a wider audience with the lessons learned from best practices. AfriCAP’s vision is based on the premise that sustainable development in WASH requires regional coordination through the involvement of local institutions and organizations and also improving, if necessary, their functioning. This is in line with AfWA’s vision to become: a recognized and trusted leader in the WASH sector and a renowned platform for sharing the sector knowledge. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 60 In this context, AfriCAP’s specific objectives are to: 1. Build the capacity of AfWA to formulate, coordinate, adapt and harmonize WASH policies across West Africa 2. Develop and implement improved knowledge sharing and public awareness in the WASH sector through rapid broad-based dissemination of best practices and innovative knowledge products 3. Develop the capacity of drinking water quality laboratories in West Africa with the support of US EPA. Through this program, AfWA, USAID/WA and targeted partners will strategically engage to encourage transformational changes in regional coordination and communication for WASH, water quality management, and effective sanitation service delivery as an efficient mean of supporting the countries in the region towards reaching their WASH Sustainable Development Goals. Several regional WASH organizations operate in West Africa, including the African Water Association (AfWA), the Economic Community of West African States (ECOWAS) Water Resources Coordinating Center (WRCC), and Water and Sanitation for Africa (WSA). WA-WASH supported a capacity building effort that included identifying and measuring the existing capacity of these organizations, developing recommendations for strengthening them, and then conducting targeted capacity building efforts at each identified regional organization except for WSA. Activity Name Implementer Cooperative Agreement # TEC Life of Activity West Africa Water Sanitation and Hygiene Program (WA￾WASH) Florida International University AID-624-LA-11- 0001 $23,999,891 8/15/2011 - 12/29/2017 AfriCAP African Water Association (AfWA) AID-624-A-16- 00003 $2,000,000 11/27/2015 - 11/26/2019 This extension of the WA-WASH program supported three primary purposes: (1) coordinating and supporting activities being undertaken as part of the USAID/WA WASH portfolio, (2) providing monitoring of WA-WASH’s past projects to strengthen sustainability, knowledge management, sharing of best practices and scaling-up and (3) building institutional capacity of West African organizations with a mandate to work in the WASH sector on a regional scale. A2. Purpose of the Performance Evaluation | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 61 This evaluation will cover the final two years of WA-WASH implementation and the first two years of AfriCAP implementation. This evaluation is being conducted to assess and document any successes and/or failures of the capacity support to the African Water Association (AfWA), both provided under the extension of the West Africa -Water Sanitation and Hygiene Program (WA-WASH) cooperative agreement, and that support provided directly to AfWA through the AFRICAP Cooperative Agreement. The main audience for this evaluation is USAID/WA and the implementing partners in its WASH portfolio. The evaluation will be used to inform/adapt implementation of the AFRICAP activity as well as support learning by USAID/WA and its implementing partners. The evaluation report is expected to be available publicly on USAID’s Development Experience Clearinghouse, once final. A3. Evaluation Questions 1. Did the combined activities implemented by Florida International University and AfWA contribute to an improvement in the capacity of: (1) personnel working within AfWA in West Africa and (2) of AfWA as an association? 2. Did efforts to build regional organizational capacity result in improved coordination in the sector at a regional level and did these lead to a more effective and efficient delivery of sustainable WASH service delivery models by the targeted organizations? 3. How is AFRICAP contributing to AfWA’s regional coordination, advocacy and dissemination, and harmonization of policies that are informed by broad-based understanding of best practices and promising models for WASH service delivery? 4. Cross-cutting question: What have been the favorable or impeding factors that have affected the activities’ achievements? 5. How have investments focused on strengthening AfWA Secretariat’s strategic financial management capacity impacted on its ability to provide strategic financial and planning support to the organization’s programs and leadership decision-making? How do evaluators assess the current (and future) ability of AfWA Board and committees to incorporate strong financial planning into their strategic decision￾making? Gender Considerations In line with USAID’s Gender Equality and Female Empowerment Policy and Automated Directives System 203.3.1.5, the evaluation will consider gender-specific and differential effects of AfriCAP and WA-WASH activities. For example, the Assessment Team may investigate whether AfWA have considered gender differences in access in their WASH service delivery models. The Assessment Team might also assess the Network of WASH Women Professionals as a discrete set of activities to support women in the association. Any quantitative or qualitative data collected under this evaluation will be gender-disaggregated to identify gender differences with respect to benefits and outcomes. The Assessment Team will conduct further | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 62 inquiry on gender themes as they emerge during data analysis. The Assessment Team will be expected to apply gender-sensitive methods while conducting interviews to ensure that accurate data are collected. A4. Evaluation Design and Methodology The Assessment Team will be responsible for developing an evaluation design and methodology for this evaluation, which should include a mix of qualitative and quantitative data collection and analysis approaches, where appropriate. The Assessment Team’s inception report should detail, on a question￾by-question basis, the proposed design and data collection and analysis methods to be used for answering the evaluation questions. The following table is a suggested evaluation design matrix: Questions Suggested Data Sources (*) Suggested Data Collection Methods Data Analysis Methods 1.[Insert Evaluation question] [Documents (including. performance monitoring data, previous evaluations, etc.), national statistics, project staff, stakeholders, expert knowledge, beneficiaries…] [Key informant interviews, questionnaires or surveys, focus group discussions, direct observation, desk review…] [To be determined by Assessment Team] [Requested level of disaggregation—gender, ethnicity, location (district, province), etc.…] 2. [Insert Evaluation question] 3. [Insert Evaluation question] Data Collection and Analysis Methods: Data collection methods for this evaluation are likely to include, but will not necessarily be limited to: • In depth interviews: this is expected to include key informant interviews with program stakeholders, including national and municipal level government staff, USAID staff, AfriCAP and WA WASH and other implementation staff, water utilities, WASH organizations, etc. Where necessary and appropriate, a USAID/WA staff member may participate in meetings with regional economic community representatives; | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 63 • Focus group discussions: If schedule permits, focus groups may be conducted with AfWA members at one of the Scientific and Technical Council meetings. The Assessment Team should work to ensure diversity of participants in focus groups; USAID anticipates the use of mixed-methods analysis for interpreting and triangulating findings to answer the evaluation questions. The Assessment Team will prepare descriptive statistics for all quantitative data and calculate inferential statistics if appropriate. In addition, content and pattern analysis should be considered for interpreting and tracking trends to connect qualitative responses to answering the evaluation. Sampling Plan The Assessment Team should aim to obtain documentation and data from a representative sample of relevant countries in West Africa. The countries where field-based research will take place should be selected based on a sampling plan to be proposed by the Assessment Team in its inception report, and the selection should be done in collaboration with USAID/WA. A mix of Francophone and Anglophone countries may be selected for the majority of data collection. The extent of travel will be determined by the evaluation design and data needs as agreed upon between the Assessment Team and USAID/WA. A5. Deliverables A5.1. List of deliverables i. Kickoff Meeting Once ASSESS has proposed and received approval for the Assessment Team, the team will have a virtual (or in-person if team members are located in Accra) in-briefing with USAID/WA for introductions and to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, and/or to adjust the SOW, if necessary. ii. Inception report The Assessment Team will have 20 business days after the beginning of the evaluation period to produce the inception report and share it with USAID/WA. USAID/WA will be given no less than 10 business days to send their feedback to the Assessment Team. The Assessment Team will not be mobilized for field data collection until USAID approval is given. The Assessment Team shall prepare an inception report, which must contain the following information: • A work plan, which indicates the phases in the evaluation with key deliverables and milestones and key personnel responsibility. • Complete set of evaluation questions and elaborate on them as necessary. Any questions added during the contract negotiations must be clearly indicated and any deleted questions must be mentioned with a reason as to their exclusion. • Discussion of the overall approach of the evaluation, highlighting the conceptual model(s) adopted. This must incorporate an analysis of the intervention logic of the program. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 64 • Discussion of risks and limitations that may undermine the reliability and validity of the evaluation results. • Data Collection and Analysis Plan which includes a discussion of the data collection and data analysis methods that will be used for each question. It should also state the limitations for each method. The Plan should Include the level of precision required for quantitative and qualitative methods and value scales or coding used for qualitative methods. The standard data collection methods for USAID evaluations are: surveys, questionnaires, interviews, focus groups, document review, and observations. o Detail key data sources that will be selected to answer each evaluation questions. o Explanation of how existing data, such as indicators collected as activity monitoring and secondary sources will be incorporated and used to answer the evaluation questions. o Discussion of the sampling methods and details. Include area and population to be represented, rationale for selection, mechanics of selection, sample size (for each unit of analysis), sample precision and confidence and limitations. o Discussions on field design testing (testing on survey tools especially if translated) • Summarized evaluation methodology in an evaluation planning matrix that must contain the following column headings: evaluation question, measure (s) or indicator(s), data collection method(s), data source, design strategy / framework for each question, sampling methodology, data collection instrument(s) for each question and data analysis methodology on each evaluation question. • Timeline showing the evaluation phases (data collection, data analysis and reporting) with their key deliverables and milestones. This may be given in a number of days from inception but should be updated with actual dates once confirmed. • Specific responsibilities of each team member for each evaluation phase. Include any changes in the Assessment Team. • Discussion of logistics of carrying out the evaluation. Include specific assistance that will be required from USAID, such as providing arrangements for key contacts within the Mission or Government. • Appended draft instruments for data collection specific to questions and indicators in the evaluation. • Standard USAID work plan outlying timeframes, etc. for implementation. iii. In-Briefing/Presentation of the Inception Report Shortly after submission of the Inception report and at least 5 days prior to field mobilization, the Assessment Team will have an in-person in-briefing with USAID/WA to present the inception report, discuss evaluation questions, methodology, and work plan, and/or to adjust the SOW, if necessary. iv. Out-Briefing The Assessment Team is expected to hold a final out-briefing in person to discuss the summary of findings and recommendations to USAID/WA after field work has been completed. This presentation will be scheduled as agreed upon during the in-briefing. The PowerPoint presentation should be no more 30 minutes in length with 30 minutes left for Q&A and discussion. The Assessment Team may also at this | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 65 time share an early draft or detailed outline that includes main findings and bullets before finalizing the draft evaluation report. v. Draft Evaluation Report The draft evaluation report should be consistent with the guidance provided in Section A6:Final Report Format. The report will address each of the questions identified in the SOW and any other issues the team considers having a bearing on the objectives of the evaluation. Any such issues can be included in the report only after consultation with USAID. The submission date for the draft evaluation report will be determined in the evaluation work plan. Once the initial draft evaluation report is submitted, USAID/WA will have 10business days in which to review and comment on the initial draft, after which point the AOR/COR will submit the consolidated comments to the Assessment Team. The Assessment Team will then be asked to submit a revised final draft report 10business days hence, and again the USAID/WA will review and send comments on this final draft report within 10business days of its submission. vi. Final Evaluation Report The Assessment Team will be asked to take no more than 10business days to respond/incorporate the final comments from the USAID/WA. The Assessment Team leader will then submit the final report to the AOR/COR. All project data and records will be submitted in full and should be in electronic form in easily readable format, organized and documented for use by those not fully familiar with the intervention or evaluation, and owned by USAID. vii. Dissemination Products Once the final report has been approved, ASSESS will develop a one to two-page summary on evaluation findings and recommendations, which will include relevant data visualizations and infographics to help communicate evaluation findings/conclusions/results to stakeholders. viii. Learning Workshop and Report In coordination with USAID and the implementing partner, and in support of Agency collaboration learning and adaptation methodologies, ASSESS must organize a 1-day learning workshop in Abidjan or Accra and produce a learning report for stakeholders. If feasible, the learning event should be aligned with AfWA’s Scientific and Technical Council (STC) meeting and member(s) of the Assessment Team should present on their findings at the STC meeting. The annual STC Meeting schedule will be shared with ASSESS. The learning workshop will involve the use of West Africa Learning and Knowledge tool kit and processes. The specific objectives of the learning event are to: • Disseminate findings and recommendations from the evaluation; • Review in-depth the key lessons and their implication for programming; and • Discuss recommendations and generate a dialogue that captures stakeholder input, thoughts, and ideas on the technical approach used to achieve activity results as presented in the evaluation. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 66 • Capture specific, and actionable next steps used to inform how the learning captured in the evaluation will be applied, specifically, as it applies to the learning questions. • Build relationships with potential new partners and engage non- traditional actors within the space that might benefit from the knowledge though they might not have been involved in the activity (i.e. researchers, technologists, private sector/ CSR, other development partners, etc.) Prior to the planning of the learning event, ASSESS should facilitate an assessment with USAID that looks at objectives, outputs and outcomes of the learning event, as well as an after-action review plan that identifies how learnings from the report and event will be used in future activities, disseminated and tracked. Templates for these can be found on the Learning Exchange in the Learning Event Toolkit. ix. Submission to the DEC Per USAID policy (ADS 201 and the Evaluation Policy) ASSESS must submit evaluation final reports and their summaries to the Development Experience Clearinghouse (DEC) within three months of final approval by USAID. Per USAID ADS 579 and the Evaluation Policy, the Contractor must also submit to the COR and the Development Data Library (DDL), at www.usaid.gov/data, in a machine-readable, non￾proprietary format, a copy of any Dataset created or obtained in performance of this award. The Dataset should be organized and documented for use by those not fully familiar with the intervention or evaluation. A.6. Final Report Format The final version of the report must be submitted to USAID/WA electronically in Microsoft Word. The evaluation final report should include an abstract (of not more than 250 words); executive summary; background of the local context and the strategies/projects/activities being evaluated; the evaluation purpose and main evaluation questions; the methodology or methodologies; the limitations to the evaluation; findings, conclusions, and recommendations. The report should not be longer than be no longer than 40-50 pages (excluding table of contents, references, and annexes), written in English, single￾spaced in Gills Sans MT, size 11 type font. The final evaluation report must contain the following sections: • Executive Summary: This section shall be 3-5 pages in length and must summarize the purpose, project background, evaluation design and methodologies including main evaluation questions, key findings, conclusions, and recommendations and lessons learned from the evaluation. • Background: This section must provide a brief description of the project that highlights the scope of the project, project development hypothesis, activities undertaken in the project, key impact indicators of the project and impact areas of the project. Other activities that complemented the project activities directly or indirectly in the intervention districts must also be highlighted. • Methodology: This section must detail the methodology and related research protocols undertaken in conducting the evaluation, data collection, analysis, selection criteria/sampling, and related constraints or limitations encountered during the project implementation and evaluation. • Findings: Empirical facts collected during the evaluation: This section must present findings from the evaluation. The evaluation findings must be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay and/or the compilation of people’s opinions. The evaluation findings must assess key outcomes and impacts as structured around the organizational framework of the evaluation questions. The findings must be specific, concise and supported by strong | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 67 quantitative and qualitative evidence analyzed through scientifically plausible methodologies. Sources of information used in arriving at the findings must be properly acknowledged and listed in an annex. • Conclusions (Interpretations and judgments based on the findings): Evaluation conclusions must be presented for each key finding. The conclusions must logically follow from the gathered data and findings. These must be explicitly justified. If and when necessary, the evaluator must state his/her assumptions, judgments and value premises so that readers can better understand and assess them. • Recommendations (Proposed actions for management): This section must precisely and clearly present recommendations that must be drawn from specific findings. The recommendations must be stated in an action-oriented fashion, must be practical, specific, and with defined responsibility for the requisite action. The recommendations presented in this section must follow the evaluation questions as the organizational framework. • References: This section should include all documents reviewed, including background documentation and records of technical data application and decision-making. • Annexes: These may include, but not limited to, statement of work, tools used in conducting the evaluation such as questionnaires, checklists, discussion guides, sources of information, etc. • The annexes to the report shall include: o The evaluation SOW; o Any statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the Assessment Team; o All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; o All sources of information, properly identified and listed; and o Signed disclosure of conflict of interest forms for all Assessment Team members, either attesting to a lack of conflicts of interest or describing existing conflicts thereof. o Any “statements of difference” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the Assessment Team. o Summary information about Assessment Team members, including qualifications, experience, and role on the team. Once the report is finalized in English, it should be translated into French. Both English and French versions must be available in the DEC (see A.5 ix). A.7 Evaluation Team Composition The Assessment Team shall demonstrate familiarity with USAID’s evaluation policies and guidance included in the USAID Automated Directive System (ADS) in Chapter 200. ASSESS will propose a staffing plan for this evaluation including specific positions and CVs for proposed individuals to serve in those positions. USAID/WA encourages the use of local West African personnel on the team, as feasible and appropriate. All team members will be required to provide a signed statement attesting to a lack of conflict of interest or describing any existing conflict of interest. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 68 Core Evaluation Team Positions USAID expects Assessment Team members to have relevant prior experience in West Africa, familiarity with USAID’s objectives, approaches, and operations, and prior evaluation/assessment experience. In addition, individual team members should have the technical qualifications identified for their respective positions. The Assessment Team should consist of a range of experts and supporting staff, based on the specific tasks and sub-tasks required to carry out the activities described in this SOW. The Team Leader should be fully fluent in French with a minimum of 10 years’ experience in project M&E with at least three of them in West African countries. All other team members should have a minimum of working level fluency in French. An illustrative team composition is provided below, although ASSESS is free to propose a different configuration: • Team Leader (Evaluation Specialist, expertise in water supply and sanitation preferred but not required, capacity building expertise desired) • Member Association Specialist, with experience in Africa required, experience with Utility Associations preferred • Organizational Development Specialist, with experience in Associations The Assessment Team may also include a member from USAID/WA. In general, USAID staff participating in the evaluation should be anticipated to have less time available for the field visits than independent evaluators. The writing of the report will be the sole responsibility of the independent evaluators; however, if mutually agreed, the USAID staff members working on the evaluation may be asked to contribute to the report in their technical discipline. USAID team members will participate in interviews and meetings, but the independent evaluators may wish to have certain interviews without USAID staff present. ASSESS will provide home office support, as required, including quality assurance, research support, administrative oversight, and logistics. ASSESS may also hire facilitators for the Learning event. USAID Participation Technical oversight for the evaluation will be provided by USAID/WA. A designated contact person from the USAID/West Africa – Regional Economic Growth Office (REGO) Team will serve as the primary point of contact and Activity Manager for the Evaluation. The Assessment Team will keep this point of contact apprised of changes and developments that necessitate/require any significant decision￾making or modification of the approved Evaluation Design Proposal. A.8. Place of Performance The Assessment Team will focus on the capacity support provided to AfWA, which is headquartered in Abidjan, field work is expected to exclusively take place in Abidjan, though ASSESS may propose additional field sites based on the review of background documents. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 69 A.9. List of Background Documents USAID/WA will share the following documents with the Assessment Team in support of the desk review: ● AFRICAP Cooperative Agreement, including all modifications ● AFRICAP Activity Monitoring Evaluation Plan (AMEP), May 2016 ● WA-WASH Phase II Modifications ● WA--WASH Phase II AMEP, revised April 2016 ● All quarterly, semi-annual, annual activity reports for AfriCAP and WA-WASH Phase II activities ● Any Data Quality Assessments completed for AfriCAP and WA-WASH indicators ● USAID/WA’s Regional Development Cooperation Strategy, 2015-2019 ● AfWA NUPAS A.10 Other Requirements All quantitative data collected by the Assessment Team must be provided in machine-readable, non￾proprietary formats as required by USAID’s Open Data policy (see ADS 579). The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed. All modifications to the required elements of the SOW of the contract/agreement, whether in technical requirements, evaluation questions, Assessment Team composition, methodology, or timeline, need to be agreed upon in writing by the AOR/COR. Any revisions should be updated in the SOW that is included as an annex to the Evaluation Report. A.10.1. Criteria to Ensure the Quality of the Evaluation Report Draft and final evaluation reports will be evaluated against the criteria outlined in ADS 201, Criteria to Ensure the Quality of the Evaluation Report, to ensure the quality of the evaluation report. See ADS 201, USAID Evaluation Report Requirements and the Evaluation Report Review Checklist from the Evaluation Toolkit for additional guidance A.11 Evaluation and Learning Event Schedule To be completed and submitted by ASSESS. A.12 Evaluation and Learning Event Budgets To be completed and submitted by ASSESS. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 70 ANNEX II: EVALUATION METHODS AND LIMITATIONS EVALUATION METHODS The evaluation mainly relied on qualitative methods to ensure a rigorous and systematic analysis of data sourced from document review, KII, FGD, and surveys. The team also collected quantitative data from the existing M&E system of both WA-WASH and AfriCAP’s programs, and from AfWA’s financial and planning strategies. Qualitative Analyses. The qualitative data collection tools were structured to enable the team to progressively capitalize on early findings into a draft Evaluation Report. Qualitative data analyses included: ● Content analyses of relevant program documents and deliverables, ● Comparative analyses to assess effectiveness and efficiency and differentiated impact of the programs on AfWA, AfWA’s members, on different geographies and on women, ● A gender analysis for all components of the two programs, and ● Exploratory, inferential, predictive, and causal Quantitative Analyses. The Assessment Team collected quantitative data essentially from WA-WASH, AfriCAP and AfWA’s respective monitoring systems and related to AfWA’s financial status (current and projected) and to AfWA’s planning. Quantitative data and analyses triangulated and provided further evidence to probe the findings obtained through qualitative analysis. METHODS USED FOR DATA ANALYSES In the comprehensive Evaluation Matrix that was developed, column M, described the various methods available to analyze data and information collected. The methods are defined hereunder. Content Analysis. Broadly, content analyses refer to methods for studying and/or retrieving meaningful information from documents 30 in order to respond to the evaluation questions. By systematically evaluating texts, wherever possible, qualitative data was converted into quantitative data31. Consequently, content analyses lead to the collection and/or production of quantitative data (such as the achievement of quantitative program indicators) and qualitative data (such as processes and procedures employed in the programs). Descriptive Analysis. Descriptive analyses of quantitative data obtained from the content analyses helped to describe, show, or summarize the data in a meaningful way that made interpretation simpler. It 30https://en.wikipedia.org/wiki/Content_analysis accessed on March 7, 2017 31https://www.terry.uga.edu/management/contentanalysis/research/ accessed on March 7, 2017 | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 71 was also applied to program M&E performance indicator tables and other secondary data collected from the Assessment. Comparative Analysis. To explore criteria such as effectiveness and efficiency, comparisons were undertaken in order to grasp achievements and resources used commensurate with outputs and outcomes. The evaluation also explored the impact of USAID’s support on AfWA’s capacity and as a consequence on improved coordination, advocacy and harmonization of policies, which relied on comparing baseline and end line situations. In some cases, where baseline data was not available, proxy data (e.g., NUPAS reports) were used. Furthermore, the evaluation looked into the improvement of effectiveness and efficiency of WASH delivery models as an outcome of the programs, the impact of AfWA’s new capacities on members’ satisfaction and willingness to pay and the impact of gender mainstreaming on AfWA’s efficiency and profitability, all of which call for comparative analyses. Gender Analyses. Part of the comparative analyses was to conduct gender analyses. The analyses assessed the extent to which the programs contributed to raising the stakeholders’ awareness of the potential impact of gender diversity in the leadership of organizations. It also assessed the effects of this sensitization, in order to know if it translated into tangible actions. The analyses determined if the issue of gender was considered in the recruitment process of new staff, in training female staff of AfWA and in taking measures to promote their access to mid-management or leadership positions (for example in the HR procedure manual). It examined whether the topics of Gender in the WASH sector and in water utilities were given enough room in the topics discussed in WA-WASH and AfWA’s coordinating meetings. It also intended to look at gender considerations in the WASH service delivery models promoted by AfWA. Last but not least, the Assessment Team determined whether AfWA was successful in supporting 12 national networks of professional women in the WASH sector, as well as a regional network. In line with this, the team ensured gender balance in the sampling of stakeholders and included women in the WASH sector in the sampling of interviewees such as those at the AfricaSan5/FSM5 conference. The findings and conclusions of the analyses followed from the data and, where necessary, was caveated with care taken to avoid over-emphasizing conclusions based on limited information. Furthermore, we organized sequential meetings involving USAID AfWA, the Assessment Team to discuss recommendations collaboratively and seek incremental input. LIMITATIONS The limitations of the evaluation included: time limitations for the assessment, scope boundaries to exclude a robust set of interviews with members (e.g., utilities) of AfWA, confidence in the accuracy of published data, the departure of the WA-WASH team and their lack of availability, and inaccessibility to key stakeholders (e.g., the Chairman of the AfWA Board). The Assessment Team dealt with the above issues. For example, for the departure of WA-WASH, we conducted remote interviews with Lakdar Boukerrou and conducted a face-to-face meeting with a former member of the team Valentin K. YAO (African Young Water and Sanitation Professionals Program Coordinator). For accessibility of Key stakeholders, the Assessment Team worked in advance with AfWA (M. Akoava) to schedule interviews either face-to-face or remote. As gaining a broad sampling of utilities | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 72 was not possible, the Assessment Team also attended a regional conference (AfricaSan5/FSM5) where a number of utilities were represented and thus could be interviewed. The greatest limitation was that members of AfWA were scattered in many countries. We decided to employ a survey to capture their opinions however, the survey, despite encouragement from AfWA was only completed by a small fraction of the member utilities. It is important to note that, the evaluation questions focused on AfWA’s capacity building and not on the impact of new capacity on the utilities and ultimately on the populations. | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 73 ANNEX III: DATA COLLECTION INSTRUMENTS Document Review Guide The Assessment Team will employ the following template for each document reviewed that forms the body of the Document Review Evaluation Matrix Question Source of data (title, date, author): Method of analysis: Sub-questions Indicators and variables Information Found in Program’s Document Review FGD and KII guides These represent questions tailored to each type of stakeholder in coherence with the evaluation questions and the role of the stakeholder in the program. The questions aim at gathering mainly qualitative but to some extent quantitative data. The same questions are submitted to the same category of stakeholders, no matter which Assessment Team member is leading the interview or the group. Interview guides are for face-to-face interviews in Cote d’Ivoire and for remote interviews in other countries. FGDs are used when stakeholders can be physically gathered during the course of the field trip. Target groups for FGDs are participants of training and capacity building events, members of AfWA or groups of professional women in the WASH sector. The general format to be used is shown below and the specific questions to be asked are derived from the Evaluation Matrix by each Assessment Team member. Lead responsibility for each Evaluation Matrix question is divided as follows: Cecile – questions 4 and 6; Patrick – questions 2 and 3; Peter questions 1 and 5. FGD Evaluation Guide Category of Stakeholders Names of Stakeholders & Titles Sex Specific roles in AfriCAP or AfWA Interview Date & Time Name of Interviewer Evaluation Matrix Question Specific Question Answers - qualitative/ quantitative* Evaluation by Assessment team as it relates to Evaluation Matrix questions *indicate if the answer was translated from original French | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 74 KII Evaluation Guide Category of Stakeholder Name of Stakeholder & Title Sex Specific role in AfriCAP or AfWA Interview Date & Time Name of Interviewer Evaluation Matrix Question Specific Question Answer - qualitative/ quantitative* Evaluation by Assessment team as it relates to Evaluation Matrix questions *indicate if the answer was translated from original French Survey Surveys are used to reach indirect beneficiaries of the programs such as AfWA’s member organizations. In most cases the survey is intended to be completed face-to-face; but in some cases, it will have to be mailed to the person being interviewed – possibly requiring modifications. This is a short survey to collect mainly qualitative information and to some extent quantitative information. The survey questions aimed at providing a snapshot of AfWA’s member organizations feed-back. Survey Guide Category of Stakeholder Name of Stakeholder & Title Sex Specific role in AfriCAP or AfWA Interview Date & Time Name of Interviewer Evaluation Matrix Question Specific Question to AfWA member utilities Answer - qualitative/ quantitative* Evaluation by Assessment team as it relates to Evaluation Matrix questions AFWA’s progress in coordination in the sector and progress in the delivery of WASH delivery models at the national level (question 2) AfWA’s progress in the provision of regional coordination, advocacy, dissemination and promote harmonization of policies (question 3) The context of the programs’ delivery as well as existing favorable and impeding factors (question 4) Members’ satisfaction with AfWA’s products and services and as a trickling effect their willingness to pay | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 75 membership dues and events participation dues (question 5) Members level of awareness of gender equality and women’s empowerment as one of the strategies to reach SDG 6 *indicate if the answer was translated from original French | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 76 ANNEX IV: BIBLIOGRAPHICAL REFERENCES ADB and IWMI workshop, From the Shallows to the Deep: Who is Taking the Lead? Women, Water and Leadership: A Workshop for Asia and the Pacific. Manila. 13–14 February 2014.) ADB briefs, Women, Water and Leadership, No. 24, December 2014 ADS Chapter 205, Integrating Gender Equality and Female Empowerment in USAID’s Program Cycle, USAID, 2017. ADS Chapter 201 Program Cycle Operations 2017. Bordia Das M., & Hatzfeldt G., The Rising Tide. A New Look at Water and Gender, The World Bank Group 2017. Diversity and disruption in utilities: How four disconnects prevent greater gender diversity and innovation in power and utilities companies, Ernst & Young, 2016. DJIBUTI: Launching of the Network of Professional Women in Water and Sanitation Sector, Wednesday, 06 June 2018, in AfWA Flash News. Morgan et Al, Morgan P, Land T, Baser H (2205) Study on Capacity, Change and performance, Interim Report, Discussion Paper No 59a, European Centre for Development Policy Management The Bill and Melinda Gates Foundation wants to strengthen the capacity of AfWA, Thursday, 07 June 2018, in AfWA news The Malian Network of Professional Women in Water and Sanitation (REMAFPEA) received in audience by the New Malian Minister of Energy and Water, Saturday, 10 November 2018, in AfWA Flash News. The Network of Beninese Professional Women in Water and Sanitation Urges Young Bachelors to Orientate their Studies in the Water and Sanitation Sector, Saturday, 10 November 2018, in AfWA Flash News. The Untapped Resource: Gender and Diversity in the Water Workforce, IWA & USAID, 2016. Thompson K., O’Dell K., Syed S., & Kemp H., Thirsty for change. The untapped potential of women in urban water management, Deloitte Review, Issue 20, 2017 Uganda/ First Community Engagement as Professional Women for Water and Sanitation (Pw4watsan) at Kawempe Hospital, Monday, 22 October 2018, in AfWA flash news. UNEP (2002), Capacity building on environment, trade, and development trends, needs and future directions, Discussion paper prepared for UNEP workshop on Capacity building on environment, trade, and development 19 – 20 March 2002, Palais des Nations USAID SUWASA activity in Kenya developed a toolkit for gender integration in urban utilities | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 77 USAID, Water Current, Gender and WASH, December 21, 2017 Webinar: CS Week, Women in Utilities Breaking the Industry Glass Ceiling. A Perspective from a Woman in Water, 2015. Webinar: Integrating Gender Equality and Female Empowerment in USAID's Program Cycle, USAID Learning Lab, Feb. 2015 ANNEX V: LIST OF PERSONS INTERVIEWED No. NAME SEX POSITION ORGANISATION/BUSINESS 1 Monique S. Avoaka F M&E Manager African Water Association 2 Stephanie Ntickomon F Communication Manager African Water Association 3 Eveline Kahoue F Interim Communication Specialist African Water Association 4 Vanessa Tihi F Treasurer African Water Association 5 Assienin Courine S. F Assistant Human Resource Manager African Water Association 6 Sanon K.F. F Assistant African Water Association 7 Rogu-Dali Lea F Resp. Achts African Water Association 8 Claon J. S. M Head Office National de l'Eau Potable (ONEP)/LACQUE 9 Seka M. Arsene M Head Chemistry Department Office National de l'Eau Potable (ONEP)/LACQUE 10 Seguy Marcelle F Chemist Office National de l'Eau Potable (ONEP)/LACQUE 11 Kouassi K. Roger M Chemist Office National de l'Eau Potable (ONEP)/LACQUE 12 Aw Sadar M Director Société de Distribution d'Eau de la Côte d'Ivoire (SODECI) 13 Gnankpa Marius Olivier M Finance and Strategy Director African Water Association 14 Sylvain Usher M Executive Director African Water Association 15 Simeon Kenfack M Programs Director African Water Association 16 Djagoun Gilles M AfriCAP Coordinator African Water Association 17 Coulibaly Soumaila M Specialist GPC/KMSO African Water Association 18 Aime Kalou Digbeu M Manager Marketing Department African Water Association 19 Koukou Nicaise M IT Officer African Water Association 20 Koussi Alphonoe M Accounting and Membership African Water Association 21 Serge Seiba M SSD Country Coordinator PSI/Sanitation Services Delivery Program 22 Eddy Perez M Technical Director Global Communities 23 Peter Njaggah M Director Technical Services Kenya Water Services Regulatory Board (WSRB) | MIDTERM EVALUATION OF THE AFRICAP PROGRAM AND A FINAL EVALUATION OF THE CAPACITY - BUILDING PORTION OF THE WA-WASH PROGRAM USAID.GOV 78 No. NAME SEX POSITION ORGANISATION/BUSINESS 24 Barbara Kazimbaya F Africa Coordinator USAID WASH-FIN Program 25 Atatchi Gnancadja M Director Benin Sanitation Pit Emptying 26 Jonathan Kampata M Managing Director MD Of Lesotho Water and Sanitation Sector 27 Rich Rapier M Chief of Party USAID/Water for Africa through Leadership and Institutional Support (WALIS) Program 28 Neil MacLeod M X-CEO Durban Water & Honorary Research Fellow: UKZN 29 Margaret Maina F Managing Director Limuru Water, Kenya 30 Luke Wilson M Co-Founder and Deputy Director The Centre for Water Security and Cooperation (CFWSC) 31 Alberto Wilde M Country Director, Ghana Global Communities 32 Lusapho Tshangela M Senior Engineer EThekwini Water 33 Rose Kaggwa F Director Business National Water and Sewerage Corporation, Uganda 34 Truman Hardon M Senior Professional Engineer EThekwini Water and Sanitation, Durban 35 Justus Rwetabula M Managing Director Mwanza Urban Water and Sewerage Authority, Tanzania 36 Jackson Mutazamba M Assistant Director Operational Planning Tanzania Ministry of Water & Irrigation, Sanitation & Health 37 Amaka Godfrey F Sanitation, Water & Hygiene Consultant Consultant 38 Lucy Njambi F Technical Director Nairobi City Water & Sewerage Company (NCWSC) 39 Amani Nyekele M Engineer Tanzania Energy & Water Utilities Regulatory Authority (EWURA) 40 Charles Fall M Directeur Général Société Nationale des Eaux du Sénégal (SONES) 41 Lansana Gagny Saho M Directeur Général Office National de l’Assainissement du Sénégal (ONAS) 42 Mosbah Helali M Président Directeur Général Société Nationale d’Exploitation et de Distribution des Eaux (SONEDE) 43 Yemizanga Kone M Directeur Général Société Malienne de Patrimoine de l’Eau Potable (SOMAPEP-SA) 44 Silver Mugisha M Managing Director National Water & Sewerage Corporation (NWSC) 45 Thomas Odongo M Ag. Managing Director Kisumu Water and Sewerage Company Limited 46 Faustina Boachie F Women Professionals in WASH Ghana Water Company Limited 47 Lakhdar BOUKEROU M Regional Director, FIU/USAID WA-WASH 48 Papa Samba Diop M AfWA STC President and Technical Advisor to the Director General ONAS, Senegal 49 Francis Kere M AfWA Quality Auditor ONEA, Burkina Faso 50 Geraldine Mpouma F Regional coordinator Network of professional women 51 Mouminatou Adjibi F Head of the Network Beninese Professional Women in the Water and Sanitation Sector (RBFPEA) 79 ANNEX VI: FIELD DATA COLLECTION SCHEDULE In-Person Interviews in Cote d’Ivoire – January 14 – 19, 2019 Date Time Priority Level Preferred/Feasible Mode of Interview Name of Stakeholder Type of Stakeholder Sunday January 6, 2019 Cecile/Patrick/Peter Arrive in Accra, Ghana – staying at Alisa Swiss Spirits Hotel Monday January 7, 2019 All Day Preparation for In-briefing Tuesday January 8, 2019 08:30-17:00 - Meeting with ASSESS Staff and ASSESS COP; - Team Working Sessions Wednesday January 9, 2018 10:00 In-briefing at USAID/WA Thursday/Friday Jan 10- 11, 2018 08:30-17:00 Working session to update Inception Report and Survey Instruments Sat January 12, 2019 13:25-14:25 The team travels to Abidjan, Ivory Coast Monday, January 14, 2019 8:30-10:30 A - In a group following initial introduction - Later meetings with selected staff for further information AfWA: - Sylvain USHER, Executive Director - Simeon KENFACK, Programs Director - Gilles DJAGOUN, AfriCAP Program Coordinator - Monique SOBO AVOAKA, M&E Manager Beneficiary and Implementer 10:30-12:30 A Face to Face AfWA: - Ismael Coulibaly; Knowledge Management Coordinator Beneficiary and Implementer 14:30-16:30 C Face to Face AfWA: - Meeting with AfWA rep to speak to a wider WASH policy issues (Sylvain Usher, Executive Director & Simeon Kenfack, Director of Programs) Beneficiary and Implementer Tuesday, January 15, 2019 8:30-10:30 A Face to Face (as a Key Informant) SODECI: - Prof AW SADAT, AfWA Water Quality Auditor - (An AfWA certified water quality auditor) Beneficiary Mentee 14:30-16:30 A Face to Face (as a KI) Confirmed ONEP: Dr Stephane CLAON, LACQUE Director/ Mme Yobouet 48 23 40 70 II Plateaux 7ème tranche, après Save the Children Beneficiary Mentee Wednesday, January 16, 2019 8:30-10:30 A Face to Face (as a KI) PSI: Philippe Lemay, Chief of Party Partner 80 Date Time Priority Level Preferred/Feasible Mode of Interview Name of Stakeholder Type of Stakeholder Serges SEIBA, SSD Country Coordinator 11:00-13:00 C Face to Face Ivorian Committee of Professional Women: Ndioré Adele, President Partner 14:30-16:30 A Face to Face AYWSPP: Valentin K. YAO (African Young Water and Sanitation Professionals Program Coordinator) Thursday, January 17, 2019 8:30-10:30 B Face to Face AfWA Communication Team 11:00-13:00 14:30-16:30 Debriefing with AfWA Friday, January 18, 2019 8:30-10:30 11:00-13:00 Debriefing with AfWA Staff 14:30-16:30 Working Sessions at the Hotel Saturday, January 19 16:45-17:30 Travel to Accra, Ghana Sunday, January 20, 2019 All day Free 81 Remote Interviews in Cote d’Ivoire/Ghana – January 21-25, 2019 Date Time Priority Level Mode of Interview Name of Stakeholder Type of Stakeholder Monday, January 21, 2019 11:00-13:00 B Skype/Phone AfWA STC: Papa Samba Diop, STC President (Dakar, Senegal) Beneficiary Tuesday, January 22, 2019 8:30-12:30 A Face to Face GWCL: Mrs. Faustina Boachie (Ghana women professionals in WASH) Beneficiary B Skype/Phone FIU/USAID WA WASH: Lakhdar BOUKEROU, Regional Director, FIU/USAID WA-WASH (Miami, USA) Beneficiary Wednesday, January 23, 2019 8:30-10:30 B Skype/Phone AfWA Member utilities with promising service delivery models: - ONEA, Burkina Faso – Francis Kere, AfWA Quality Auditor Beneficiary 10:30-14:30 Skype/Phone Geraldine Mpouma, Regional coordinator for network of professional women from francophone countries Beneficiary Thursday, January 24, 2019 8:30-10:30 A Skype/Phone RBFPEA Mrs. Mouminatou Adjibi (Head of the Network of Beninese Professional Women in the Water and Sanitation Sector (RBFPEA) Beneficiary 11:00-13:00 B Skype/Phone AfWA Program Committee: Rose KAGGWA, Vice President (Kampala, Uganda) (NWSC) Beneficiary 14:30-16:30 USAID Debrief Friday, January 25 All day Cecile/Peter depart 82 ANNEX VII: DISCLOSURE OF ANY CONFLICTS OF INTEREST 83 84 85 ANNEX VIII: INFORMATION ABOUT TEAM MEMBERS The Assessment Team consists of three expert researchers with significant evaluation and technical expertise: Team Leader who is also a WASH Specialist, an Organizational Capacity Development Specialist, and an Evaluation Specialist. The qualifications of the proposed core team members are listed below, and their complete CVs are included in Annex 4. The ASSESS team provides technical and logistical support to the team along the entire phases of the evaluation. Peter Macy, WASH Organizational Development and Capacity Building Expert and Team Leader Mr. Macy has over 40 years of experience, worldwide, in all aspects of water supply, sanitation, and wastewater engineering and management in various sectors. He has been the regional and country manager in four countries, including S. Africa. He is an expert in drought management and water conservation including in the SADC region of Africa. He also has extensive experience in urban and rural water sanitation and hygiene (WASH), community water committees, program/project management, construction management/supervision, design, planning, maintenance, and operations. Mr. Macy has managed millions of dollars’ worth of projects, has started up and led the country and regional offices, marketed mega proposals, and served as a technical advisor for multiple sectors. Mr. Macy has served as a thought leader and guest speaker at numerous conferences and initiatives associated with international development and has authored and co-authored books and articles for professional organizations on the subject of water resources management. The main responsibility of the WASH Organizational Development and Capacity Building Expert and Team Leader will be to lead this evaluation and define the best interaction among team members towards the realization of the deliverables. Specifically, the team leader will: 1. Coordinate and provide direction to the implementation of this evaluation in relation to ASSESS and in compliance with the final SOW with the objective of addressing the questions leading to the production of the required deliverables; 2. Provide the technical direction to the implementation of the activity and together with the other team members define the appropriate and robust methodologies to address the objectives of the evaluation; 3. Lead the development of the inception report. The inception report should set out the methodology and approach to the evaluation, including how to evaluate the criteria and questions, their feasibility in the light of the situation, methods and data requirements, i.e., what kind of baseline data is needed to assess the criteria; 4. Participate in the in-briefing meeting/presentation of the inception report 5. Lead the development of data collection and survey instruments; 6. Lead the team in field data collection (interviews and document review); 7. Coordinate with team members in the selection of appropriate stakeholders to be interviewed; 86 8. Prepare and lead short PowerPoint presentation for the out-briefing outlining key preliminary findings during the field data collection. 9. Work with the other team members in the data analyses and reporting; 10. Organize and lead working sessions remotely and on the field; 11. Lead the development of draft and final reports. Collate inputs from team members and lead the development of the report. Prepare the final report based on comments on the first draft from ASSESS and USAID reviewers, and provide justification for comments not incorporated; 12. Contribute to the development of actionable recommendations from the evaluation; 13. Participate and present the evaluation findings and recommendations in a learning event; Mr. Patrick Apoya, WASH Specialist Patrick Apoya holds master’s degree in both Health Care Planning and Environmental Science. He has over 21 years of experience in the management and coordination of development work. He has developed and tested various training modules for the training of community-based workers, middle-level personnel and management staff in several institutions across different sectors. He has coordinated the development of business strategies and strategic plans for international clients such as WaterAid, CARE International, African Civil Society Network on Water and Sanitation (ANEW), World Vision International and the Ethiopian WASH Network. Mr. Apoya was the Program Manager for the design and construction services for the USAID/STEWARD WASH Program in West Africa from September 2015 to August 2016. He was Policy and WASH Advisor for the USAID/US Forest Service-International Programs from July 2014 to March 2016. He participated in the development of WASH Business Plan for the World Vision’s operations in West Africa from August to October 2013. From April 2013 to date, Mr. Apoya has been involved in the setup and direct implementation of the Africa Sanitation Think Tank, an initiative of Water and Sanitation for Africa (WSA) funded by Bill and Melinda Gates Foundation. In the field of research, M&E, he has participated in, or led scientific and socio-economic research in the WASH sector and coordinated various evaluation missions. Mr. Apoya is a member of several global boards and committees including the Sanitation and Health Guideline Development Group of the World Health Organization (WHO); as a scientific reviewer for the Water Resources Research, AGU Journal, University of North Carolina; Global Review Board, WASH Policy Research Digest, University of North Carolina; Global Advisory Board, Sanitation Pathways (SaniPath) Project, University of Emory; among others. Mr. Apoya has worked for several water companies and associations in West Africa. For instance, he was part of the team that conducted the baseline survey of the Ghana Water Company non-revenue water situation and also the market feasibility of self-billing systems for GWCL’s customs, which is scheduled to begin in July 2018. Mr. Apoya has several publications to his credit including Public-Community Partnership Model for Water Delivery in Ghana: A Case Study of the Savelugu Water System; Understanding Hand Pump Sustainability, Determinants of Rural Water Source Functionality in the Greater Afram Plains Region of Ghana; among others. He has experience in several African countries including Ethiopia, South Sudan, Uganda, Kenya, 87 Sierra Leone, Liberia, Nigeria, Zambia, Cote d’Ivoire, Cameroun, Senegal, among others. Mr. Apoya is fluent in English with a fair understanding of French. The WASH Specialist will: 1. Together with the other team members define the technical direction of the evaluation; 2. Contribute to the development of the inception report. 3. Participate in in-briefing meeting remotely; 4. Work with the other team members in defining appropriate and robust methodologies to address the objectives of the evaluation; 5. Coordinate with team members in the selection of appropriate stakeholders to be interviewed; 6. Participate in all field data collection (interviews and document review); 7. Contribute to the preparation of short PowerPoint presentation for the out-briefing; 8. Contribute to data analyses and the development of all evaluation deliverables; 9. Participate in working sessions both remotely and in the field; 10. Contribute to the development of actionable recommendations from the evaluation; 11. Participate and present the evaluation findings and recommendations in a learning event; Ms. Cecile Laborderie, WASH Evaluation, and Gender Specialist Ms. Cécile Laborderie has more than 10 years of experience in water supply, sanitation, hygiene, climate change, and solid waste management projects as researcher, socio-economist, evaluator, WASH expert, and project coordinator. Her academic background includes degrees in both economic and social development and political science with certificates and training in WASH and specialization in sustainable sanitation. Ms. Laborderie has conducted several baseline studies and research for WASH projects in rural, peri￾urban, and urban environments. She has worked in large array of contexts - in an emergency, post￾emergency, and development- particularly in peri-urban areas. She served as the Water and Sanitation Expert and Evaluator responsible for designing and carrying out a gender-sensitive mid-term evaluation of the Tajikistan Drinking Water Supply and Sanitation Sector Improving Social Accountability (TWISA) project, for OXFAM GB. She undertook several evaluations in the past and was the lead researcher for the baseline development and mid-term review of the Integrated Sanitation and Sewerage Infrastructure Project (ISSIP) in 3 governorates in Egypt and was also an evaluator for CARE USA’s programs in Tsunami affected areas in Indonesia within the Equal access to water and Sanitation (ACCESS) project. Ms. Laborderie has experience with Water utilities in Indonesia (2006), water association and utility in Egypt (2014 – 2015) and water utility in Tajikistan (2016). She also followed a UNITAR training on 88 Governance of Water in an Urban context which specifically addressed water governance, the role of utilities as well as economic and financial aspects of the project. In 2016 Ms. Laborderie served as the team leader in the midterm evaluation of the USAID/WA Sanitation Services Delivery (SSD) project and facilitated the USAID/WA Water Supply, Sanitation and Hygiene (WA-WASH) learning event in Ouagadougou. Since 2006, she has undertaken several assignments for various clients as sanitation or environmental expert, focusing on sustainable sanitation and development topics. Ms. Laborderie is an active member and forum moderator of the Sustainable Sanitation Alliance (SuSanA) and belongs to the French Ecological Sanitation Network. This provides her with excellent knowledge of current discussions on the design, implementation, and sustainability of sanitation projects such as sanitation planning, governance, value chains, private sector promotion, WASH and the New Urban Agenda, productive sanitation and WASH and nutrition, among others. Ms. Laborderie is fluent in both English and French. The WASH Evaluation and Gender Specialist will: 1. Work with the team members to develop the evaluation methodologies according to the USAID guidelines and regulations; 2. Contribute to the development of the inception report. 3. Participate in in-briefing meeting; 4. Together with team leader be directly responsible for the production of evaluation reports – inception, draft, and final reports, in accordance with the requirements of this evaluation and in line with the overarching USAID guidelines; 5. Together with the Team Leader ensure the appropriateness of and relevance of stakeholders identified to be contacted in the evaluation; 6. Participate in all field data collection (interviews and document review); 7. Compile and collate field data into machine-readable forms and generate data dictionary in a format consistent with USAID ADS 579 requirement; 8. Contribute to the preparation of short PowerPoint presentation for the out-briefing; 9. Contribute to data analyses and the development of all evaluation deliverables; 10. Participate in working sessions both remotely and in the field; 11. Contribute to the development of actionable recommendations from the evaluation; 12. Participate and present the evaluation findings and recommendations in a learning event; 13. Ensure that gender dimensions are considered and incorporated in all the stages of the evaluation. USAID Participation Technical oversight for the evaluation will be provided by USAID/WA. A designated contact person from the REGO Team will serve as the primary point of contact and Activity Manager for the Evaluation. The Assessment Team will keep this point of contact apprised of changes and developments that necessitate/require any significant decision-making or modification of the approved Evaluation Design Proposal. Evaluation Coordination and Technical Support 89 ASSESS with the support of its university partners will participate fully in all implementation phases of this evaluation. ASSESS will be responsible for both Technical and Operational Management of the evaluation. Under Technical Management, ASSESS will ensure the full application of USAID Evaluation policies and guidelines on the field. In terms of Operational Management, ASSESS will undertake both logistical and administrative duties such as arranging for in-country travels, lodging, meetings among others. The ASSESS technical and admin support staff comprises Fedelis Dadzie, Interim Chief of Party (COP); Ms. Barbara Arthur, M&E Specialist; and Nana Fredua-Agyeman, and Activity Manager-Evaluation Specialist. ● The Interim COP, Fedelis Dadzie, is responsible for the overall technical and logistical assistance needed by the Assessment Team for effective coordination and communication among the key implementing partners and stakeholders. He will provide technical support on evaluation approaches, methods, and data analysis and reporting. ● The Activity Manager-Evaluation Specialist, Nana Fredua-Agyeman, will serve as the focal point for this evaluation. He will provide technical support on evaluation approaches and data analysis and reporting. He will coordinate and facilitate field visits by the Assessment Team by scheduling appointments with the stakeholders to be reached for information, interviews, brainstorming sessions and workshops. ● The M&E Specialist, Barbara Arthur, will provide technical assistance on the field to the Assessment Team with respect to the development and administration of evaluation tools and technical backstopping of the analysis process. ● The Events Planning Specialist, Jennifer Tchorly-Boadi will liaise with USAID and the consultants to develop content for the learning event. She will also manage the logistics and make all necessary arrangements related to learning event. 90 ANNEX IX: CURRENT AND SUGGESTED ORGANIZATION CHARTS FOR AFWA AfWA organization chart adopted in the 2018-2022 Strategic Business Plan The Executive Director has eight people reporting to him and directly managed by him. This poses a problem of delegation of management and or prioritization. When an organization is growing, there should be a shift from direct to delegated management to ensure the Directors focus on the strategic goals and delegate operational management to technical and administration managers. An interesting example of reorganization is below. AfWA organization chart proposed in NUPAS (2015) The organization proposed by NUPAS is relevant and appropriate to support AfWA’s need to evolve with its ambition. The need to shift from a leadership model to a delegation with control model was also highlighted in the POE report. This proposed organization chart could be further improved by highlighting the relationships between programs and functional departments. 91 ANNEX X: EXCERPT FROM THE STRATEGIC BUSINESS PLAN The content lacks clarity as goals, challenges and lessons learned are all mixed up. Besides there are not details on how to achieve for instance improving of governance, increase of membership. There is a total lack of cost-based data and figure-based strategy to achieve sustainable financing. ANNEX XI: SERVICE DELIVERY MODELS Different Utility Service Delivery Models32 Municipality managed models, the norm, are found in countries such as Kenya, Tanzania and South Africa and face significant challenges and growing operational problems. These challenges such as high non￾revenue water and billings’ collections are well documented in recent reports such as the September 2015 USAID-funded SUWASA Final Report, the World Bank’s 2017 Performance of Water Utilities in Africa and the 2015 AfDB-funded Water Supply & Sanitation in Africa: Findings, Lessons and Good Practices to Improve Delivery. Private sector models (e.g., concession agreements) are considered well performing during the participation period of the private sector. A 2016 FIU and Oceane Consult International (OCI) report indicates there were 10 private sector operators and 15 public sector operators in Africa. The 2016 WA-WASH OCI report highlighted the public-private partnership (PPP) model used by the Ghana Water Company Ltd (GWCL) between 2006 and 201133.The risk to these utilities, however, 32A total of 11 WOPs, focusing on NRW and sanitation, were implemented with USAID support. 33Aqua Vitens Rand Limited (AVRL) was the “operator”. It managed urban water supply on behalf of the GWCL - though the latter remained the “asset holder”. The Public Utilities Regulating Commission (PURC) was (and still is) Cross Cutting Issue #6: Lack of understanding of the tenets of sustainability/durability, and absence of effort towards sustainable/durable interventions 92 is sustainability/durability of improvements when the private sector withdraws. For example, in Cameroun in 2018 the authorities did not renew the private sector concession agreement with Camerounaise des Eaux (CdE), resulting in challenges for CamWater afterwards such as the alleged embezzlement within the organization, discovered by the anti-corruption operation “Epervier”. Greater sharing of south-south and north-south lessons learned and best practices, by AfWA, would benefit utilities with this type of service delivery model where decades of experience and lessons learned could be shared. Fully corporatized member utilities such as the National Water and Sewerage Corporation (NWSC) in Uganda, having full autonomy to run their own budget with minimum interference, enjoys a positive relationship with Uganda’s Ministry of Finance (e.g., with regards to new investments). Investors can invest directly in NWSC without going through a government ministry. Ghana considered this model as well, but the initiative stalled with a change of government. Sanitation utilities exist in four out of 16 countries, namely Senegal, Cote d’Ivoire, Burkina Faso and Benin (new structure). For SOPs, in five mentee34 cities, three of them had existing sewer networks and two did not. The BMGF-funded RASOP, which supported “city-wide inclusive sanitation” to deliver on Sustainable Development Goal (SDG)6, explicitly considered more than just engineering solutions for sanitation. For cities that had some sewer infrastructure, understanding how to increase connections and addressing fecal sludge management (FSM) were challenging issues needing to be addressed. Lusaka, in Zambia, was a SOP mentee city where the Lusaka Water and Sewerage Corporation (LWSC) leveraged funds from the Lusaka Sanitation Program (LSP) ~$300 million in total with $280 million for expansion and $30 million serving the other 80 percent with FSM. AfWA along with the BMGF (e.g., via RASOP) maintains a sanitation focus. ANNEX XII: LOP SUMMARY West African utility members have improved their water quality management, but for primarily those few that participated in the peer-to-peer activities, and to levels that fall short of what is required. The Assessment Team observed that a systematic diagnosis was conducted by AfWA certified auditors and a member of AfWA’s Secretariat in each of the mentee laboratories. The improvement plans (Plan d’Amélioration des Performance (PAPs) or Performance Improvement Plans) drafted, following the diagnosis, reflect a high level of interaction between the auditors and the mentees, and a detailed action plan with identification of the person in charge and the budget needed35. Due to a late start (summer 2018), the first evaluation of the PAP’s achievements will only take place in summer 2019 so it was not possible to identify quantified improvements of laboratory processes. Furthermore, the labs contribute to the water quality process because they are a component of the whole quality process, but water quality can only improve if the other segments are integrated in the program (i.e., water treatment, pipe maintenance, households’ management of water at the point of use); and the program does not address these issues. Hence, there is an area of improvement for AfWA, which is to define clearly their goals (including their theory of change). If the goal is to improve water quality, then they should have a holistic approach of the the regulatory body for water services. AVRL was thought to have more power than GWCL but the former also had a monitoring responsibility. 34Part of the RASOP program 35PAPs do cost each identified action–but these neglect the cost of O&M, indirect support and other costs, which hinders effectiveness of investment. 93 water value chain. Otherwise, the expected results should be harmonized with the activities and in this case it is to improve laboratory water testing processes. Another lesson learned is that there are positive impacts and capacity building due to informal exchanges between stakeholders as a result of this program. However, there is no formalized support of mentor laboratories to mentees between the time of the diagnosis and PAP and the time of the assessment 12 months later. Designation of a focal person with monthly interaction seems to be lacking - a weakness in program management at AfWA’s secretariat level. Also, improvements for the larger body of West Africa utility members has been minimal– unless indirectly through knowledge management sharing from AfWA. Even for the peer-to-peer participants, while 10 labs have been equipped with PAP, the funding has not materialized to move beyond the planning stage. And, while capacity building can be provided through AfriCAP, required equipment cannot. This would normally be the responsibility of the utility, which, all too often, is an unmet responsibility for a variety of reasons including their real or perceived lack of sufficient funds. Utilities/labs appear to be reluctant to invest in lab equipment because they are not able to finance maintenance of this equipment. Hence there may be a gap in that there is no cost analysis including operational expenses (OpEx), indirect cost etc. See IRC WASH cost analysis36). Certainly, activity is taking place, but the ultimate efficacy of that activity is questionable. For example, AfWA developed water quality standards with support from USEPA. They conducted a national workshop in Burkina Faso and Ghana to promote the adoption of AfWA water quality standards. Both of these activities are a positive step. That said, as quoted in the 2015 IWA/WHO/USEPA report on Burkina Faso – a microcosm to the larger body of utilities - “Water quality information is not shared with the consumers therefore there has been a mistrust that has developed with regards to ONEA’s water quality” as well as “notable consumer complaints with regards to water quality issues (e.g. color and taste)”. In a follow up report, a year later, the situation had not improved.37The situation is not much better in Ghana. In a 2016 IWA/WHO/USEPA report it stated that “a study revealed that a number of drinking water quality parameters do not conform to the national water quality standards. Bacterial contamination is widespread in the country.” These cases point to the need for AfWA to be more output oriented and less input oriented. In another example, WA-WASH collected 600 water samples that proved serious quality problems in West Africa. In their opinion, improving the water quality of the labs was a step in the right direction, but would not ultimately improve the quality of the water in West Africa (and the training of the auditors was considered superficial). The auditors were sent to look at the quality of the labs38.Six exploratory audits missions were conducted in five countries: Cameroon (Camerounaise Des Eaux, CDE/CAMWATER), Mali (Société Malienne de Gestion de l'Eau Potable, SOMAGEP), Cote d'Ivoire (Office National de l'Eau Potable, ONEP; Société de Distribution d'Eau de la Côte d'Ivoire, SODECI), Guinea (Société des Eaux de Guinée, SEG) and Togo (Togolais des Eaux, TDE). But it remains unclear how these efforts are improving 36https://www.ircwash.org/washcost 37IWA/WHO/USEPA Water Safety Planning Baseline Assessments: West Africa Country Summary Report: Burkina Faso 6 May 2016 38Five exploratory audits missions were also conducted in the Federal Capital Territory (FCT) – the federal territory in central Nigeria containing Abuja (3 labs : Usuma DAM Lab1, Usama DAM Lab2, City Lab); in Ogun State in April 2018; and Niger (SEEN) in October, 2017. 94 the water quality being delivered to their customers. The real problem may actually be the water source, not the labs as proffered by one of the stakeholders interviewed. Note that the role of the mentor laboratory is limited to undertaking an audit, and there is no support from the mentor laboratory to the mentees apart from that. The labs are trained with USEPA standards which they prefer to the French standard. AfWA, in its role should indicate the preferred standard to use by member utilities and why. In any case, the training of auditors was superficial and AfWA lacks the authority to dictate to its member utilities about water quality. As such, auditors did assess the labs, but could not mandate anything be done for improvement. There is a need to focus more on member utilities to provide good quality water and for sharing their data to make them more accountable for water quality. All WASH programs should include a water quality component. However, newer and expanded programs, some with new members and countries, increase complexity and require new skillsets and capabilities AfWA may not yet have. AfWA is at least planning for evaluations of utility water quality improvements (and non-revenue water improvements).39 AfWA, including its members such as GWCL conduct audits of utilities including Cote d’Ivoire, and Nigeria40. AfWA developed a water quality audit certification program. From October 5 to 6, 2016, AfWA conducted the first training session for water quality auditors, which was attended by seven auditors from Burkina Faso, Côte d’Ivoire, Ghana, Guinea, Mali, and Senegal. A second training was organized from February 27 to March 2, 2017 in Accra, Ghana. This was attended by 13 auditors from the laboratories of water utilities in Benin, Burkina Faso, Côte d'Ivoire, Guinea, Ghana, Mali, Nigeria, Senegal, Sierra Leone and Togo. Training topics included: • The organization/ responsibilities of the laboratory, laboratory best practices and the collection and conservation of water samples; • Preparation of draft laboratory quality manuals; • Sharing knowledge of peer-to-peer laboratories and USEPA laboratory audit checklist; and • Discussions around the laboratory audit program. At the end of the two sessions, appropriate tools and materials for auditing laboratories in the region were developed by AfWA WQTF with support from USEPA. 3979th AfWA STC Report 40AfWA Audit Report “General Report on Exploratory Audit Mission in Ogun State”, 9-13 April, 2018 95 ANNEX XIII: DETAILED RECOMMENDATIONS AND TIMING This annex provides a summary of the top priority/key recommendations derived from answering the Questions above. • Sub-Question 1.1 In future donor-funded capacity building programs, the donors should require a comprehensive capacity gap analysis of staff at the Secretariat and a comprehensive organizational assessment of AfWA - to include concrete steps to address high-priority gaps. At a minimum the steps must include on AfWA’s capacity to deliver services to their members and to ensure AfWA’s financial sustainability • Sub-Question 1.1 Donors should formalize coordination regarding AfWA’s capacity building as well as coordination between programs of the same donor • Sub-Question 1.2 AfWA Secretariat, within the next 3 months, should review/reform HR management especially transparency of recruitment processes and harmonizing of salaries. In the next year, AfWA to review/reform organization chart including shifting from a hierarchical model to a delegation with distributed control model, creating links between programs and support functions, to facilitate staff autonomy and foster links between the Program and other departments (e.g. Communication, Membership, and Training). AfWA to diversify the gender, language and regional balance of their executive staff. • Sub-Question 1.2 Unless the situation of Anglophone vs. Francophone improves in the next six months, AfWA must secure a full-time translator for the Abidjan office for the KM team who will also assist with other AfWA external communication. • Sub-Question 1.3 The AfWA Board, particularly the Chairman, should strengthen its oversight role and accountability, to increase its effectiveness towards AfWA’s mandate and goals – particularly financial sustainability. Select a strong AfWA Board Chairperson willing and able to hold the Board accountable to measurable success • Sub-Question 1.3 AfWA, within three months, to complete a major analysis of its HR/staffing/appointments, starting with Board membership which, for example, should not exclusively be comprised of member utility CEO’s. STCs working groups should be comprised of experts • Sub-Question 1.3 AfWA, within a year, should move towards online member communication and virtual meetings. Also, reinforce the links between Programs and M&E, including indicators to track the impact of the programs and services on the utilities, affiliates, members and on the WASH sector writ large • Sub-Question 1.3 AfWA, with donor support and as a matter of urgency (e.g., within three months), conduct a members’ survey/needs assessment • Sub-Question 1.3 AfWA’s Secretariat and Board, within the next three months, identify and later implement activities that improve Secretariat rapport with members. AfWA should increase attention on members and allocate sufficient resources (human and financial) to members’ management. • Sub-Question 1.3 AfWA, within three months, should review and revise its policy with regards to membership • Sub-Question 1.3 AfWA is a member-based organization and these members are its “clients”. Clients come first and their needs must be met for AfWA to survive, let alone grow. Hence, AfWA should endeavor to determine what its members require (a.k.a. a detailed needs’ assessment) – a task that should be completed within three months. For example, using the knowledge of the Assessment Team, Figure 11 below maps the bottom rung of WASH sector needs – clients having quality water and sewerage services – and how to transfer these upward to where AfWA needs to focus. 96 • Sub-Question 2.1 AfWA should promote greater use of utility benchmarking (e.g., IB-NET, Agua Rating, GCR) among its member utilities. This can be done, internally by AfWA, over the period of a year if clear goals are set and followed. • Sub-Question 2.1 USAID should assist in professional-level benchmarking of AfWA staff capacity and member utility performance using SMART goals and surveys. This will be a six-month effort • Sub-Question 2.1 to prepare an action plan for the collection and dissemination of “lessons learned” specifically from member utilities, and to execute the plan– this is a long-term recommendation • Sub-Question 2.1 AfWA to establish within three months and then regularly update a database of specialists, who satisfy pre-determined criteria related to their areas of competency with select service delivery models, who would be willing and able to help member utilities with unique questions about their service delivery models • Sub-Question 2.1 AfWA, within a month, to develop member utility performance indicators (e.g., using AguaRating), and appoint a staff member to assemble/update (non-confidential) utility statistics • Sub-Question 2.1 AfWA to set immediate targets for itself to play a larger role in supporting member utilities to adopt best practices in the different service delivery models. AfWA, within six months, to improve/build on RASOP to have a package of sanitation models with best practices’ tools and institutional approaches; and later to replicate and scale these for city sanitation, and build capacity of the base level of sanitation • Sub-Question 2.1 At the higher levels of government where policy is discussed, AfWA must make itself better known. It must also help utility senior management to better understand what AfWA is about and it’s potential to help. AfWA to prepare proxy indicators to determine success of this effort. • Sub-Question 2.2 AfWA requires training on output based strategic planning and human resources development (e.g., with a focus on activities and input vs. outcomes/output). This should either be provided by a suitably experienced consultant to AfWA or with assistance from AfWA’s development partners and only requires one or two training workshops to accomplish this training suggestion. • Sub-Question 2.2 Regional offices of AfWA should be strongly considered while ensuring sufficient support to direct and hold a regional branch accountable. Make a decision within six months. • Sub-Question 2.2 AfWA must establish better SWOT's and accountability metrics in all planning documents. For existing documents, this would take no more than a week or two; but, as important, AfWA must incorporate this in all future planning documents. • Sub-Question 2.2 AfWA should conduct a comprehensive, three-month stakeholder mapping and lead role in coordination study - particularly of development partners such as AfriAlliance and the Public-Private Infrastructure Advisory Facility (PPIAF) and Global Water Operators’ Partnership Alliance (GOWPA), BMGF (RASOP II?), WOP-Africa, World Bank, and African Development Bank. This mapping should also include the many sources for online and in-situ training (e.g., IHE Delft Institute for Water Education). AfWA to host a donors’ conference within the next four months to help collate, efficiently, activities and financing. • Sub-Question 2.2 USAID should fund and execute a three-month master plan study for the headquarters and regional offices of AfWA, and assistance with securing funding for the same (e.g., headquarters in Cote D’Ivoire with regional offices in Morocco and S. Africa). This includes continued evaluation of an AfWA Academy to train utility senior management on leadership and change management • Sub-Question 2.2 USAID should use this Evaluation Report, and others, to conduct a one-month (independent41) SWOT analysis of AfWA and the sector 41AfWA has conducted its own SWOT analysis 97 • Sub-Question 2.5 Senior leadership of AfWA, with immediate effect, should insist that staff regularly use and display graphical information on progress of their objectives such as scheduled activities, showing baseline schedule vs. actual progress (on a monthly basis). AfWA needs to clearly define roles and responsibilities including for coordination. Focus on human resources strengthening in terms of definition of roles and responsibilities, restructuring the organogram and increasing the responsibilities at program management level. A focal point should be designated for cross sectoral and coordination activities. • Sub-Question 2.8 Improve the LOP program’s design in the following ways: o If at all possible, AfWA to increase the number of qualified mentors within a year and ensure a formalized support of them (to mentees) including creating communication channels for knowledge sharing (mailing list, discussion forum) o AfWA should ensure coordination of an integrated approach towards water quality testing in laboratories with the other components of the value chain to ensure that improved testing leads to acceptable water quality at the point of use o AfWA, within three months, prepare a road map of post-AfriCAP LOP activities with stakeholders to sustain the program and encourage replication and scaling up o Conduct a 3-month investigation for the purchase of lab consumables (e.g., reagents) and maintenance of lab equipment (labs/ utilities can fund) at a regional level to decrease costs and constraints. For example, establishing regional maintenance individuals/firms - funded by laboratories or water companies o AfWA, within three months, should determine required levels of water quality policy and advocacy to systematize water quality in all projects in Africa. For example, advocate for utilities to standardize analytical equipment and equipment maintenance • Sub-Question 3.1 Within two months, AfWA should develop and strictly follow an internal quality management protocol for all external documentation and communication • Sub-Question 3.1 AfWA, with USAID’s assistance, should consider extending SUWASA Pathways, or transfer this publication to AfWA News and ShareWater, to highlight current utility reform activities • Sub-Question 3.1 In order to address advocacy, future capacity building programs should support AfWA in addressing the following: o Decide if advocacy remains a priority for AfWA’s members and if yes, determine the priority topics for the members and the results required o Decide who should be responsible for advocacy within AfWA and clarify the respective roles of the STC and the Board. In parallel, identifying who has experience and skills in advocacy within AfWA • Sub-Question 3.2 In parallel to the recommendation of sub-Question 1.8, collect and act on a member survey – including regarding KM requests, especially related to policy. Based on member’s survey as well as an internal survey determine within the next four months what sort of database is needed, investigate what is available, and where data is missing, develop a plan for collecting that data and regularly updating the data. The plan should include the resources to perform this task, any costs, and any training required. • Sub-Question 3.3 With immediate effect, AfWA should identify who has the best leverage to conduct advocacy and if advocacy should also be assisted by utilities (with AfWA’s support), if it should be done at African Ministers Council on Water (AMCOW’s) level and even at donor’s level • Sub-Question 3.3 AfWA to propose, gain approval and then implement a structured policy formulation, harmonization and adoption strategy within the next three months. If current and recent 98 programs (e.g., AfriCAP and RASOP) result in policy recommendations, AfWA must ensure that these are taken up beyond their members, but also to the ministers and ministries that work in parallel and above the utilities’ organization • Sub-Question 3.3 AfWA and its development partners must better define the policy and advocacy roles and responsibilities of AfWA vs. AMCOW; and, growing the leverage of both organizations to effectuate positive change in the WASH sector. • Sub-Question 4.2 AfWA should consistently use information from its financial monitoring to implement measures that address negative trends in key financial metrics • Sub-Question 4.3 AfWA Secretariat should immediately identify up to four relevant financial metrics and include these in routine reporting, in addition to the budget execution rate and cash reserves rate that they currently monitor. • Sub-Question 4.3 In the next review of the Strategic Business Plan, AfWA should incorporate strong financial analysis as basis for strategic choices in the Plan. As per the findings of Question 5, integrate gender-sensitive budgeting, planning and financial strategy in the Strategic Business Plan and at program management level. • Sub-Question 4.5 AfWA, within the next six months should migrate to a more robust financial management software. AfWA should ensure that the selected vendor provides adequate training on the full functionality of the software. Initially AfWA should sign a one-year maintenance contract with the vendor to provide on the job support in the first year of the roll out of the software • Sub-Question 5.1 In future programs, women’s empowerment should be specifically targeted by integrating women’s needs assessment in the baseline capacity assessment. Draft a gender strategy to formalize and enforce gender empowerment. • Sub-Question 5.2 Immediately designate a focal point, at AfWA’s level, to ensure coordination of women’s related activities, capitalization and coherence • Sub-Question 5.3 Sustain current efforts to reach statutory representation of women in AfWA’s decision-making bodies as part of a global effort to increase representativeness within AfWA (gender, languages, geographical scope and youth) • Sub-Question 5.5 AfWA should include definition and enforcement of a gender policy as the responsibility of the Program Director and ensure capacity to lead this task. • Sub-Question 5.5 In case the organogram is modified, train the head of Administration / finance in gender mainstreaming. • Sub-Question 5.6 Within the next six months, conduct research on gender impacts in WASH utilities and promising gender sensitive models in Africa (benchmarking). Develop policy advocacy on gender management in utilities General: • AfWA should continue assistance towards setting up/improving regulatory authorities (e.g., in Nigeria) like what was done under SUWASA – this is a long-term recommendation. For example, share the documented successes of the National Water Supply and Sanitation Council (NWASCO) in Zambia and the use of a “governance indicator” (of utilities) by WSRB in Kenya. Bring regulators together to share lessons learned and challenges • AfWA should prepare an immediate action plan to expand private sector/vendors to re-engage or to up their engagement in the Africa utility market (e.g., by becoming AfWA members), and to execute the plan • AfWA should continue to be the advocate and key influencer to focus on the sanitation element of WASH – this is a long-term recommendation • Like the suggestion above, AfWA and development partners must establish and ensure obvious linkages between) all WASH activities, and b) the needs of AfWA members (particularly utilities) and 99 their “clients” (rate payers)– hence to better micro target support. This can be done, ad hoc and in a retrofit manner, with existing activities; but more importantly this needs to be institutionalized by AfWA and development partners for all future projects and activities. Two example needs of utilities, would be 1) guidance on how to achieve regulatory compliance and 2) strategies to close the “financing gap”. • AfWA, over the next 12 months, should prepare an action plan for peer-to-peer mentorships between AfWA and USAID/AWWA/EPA/WEF/IWA in a mix of the following areas: membership, conferences, website and publications as well as policy and advocacy, and to execute the plan • USAID should conduct a one-month R.O.C.K.S.42 analysis of sustainability/durability of AfWA to help ensure that future assistance addresses this critical issue Figure 11. Mapping Utility Customer Challenges to AfWA's Mandate 42R – resources, O – ownership, C – connection, K – knowledge, S – secondary backup http://www.wflrating.org/wp-content/uploads/2014/07/WASHSustainabilityForumReport.pdf PP10 100 ANNEX XIV: FINANCIAL METRICS Financial metrics have long been the standard for assessing a firm’s performance. They help in establishing and monitoring specific and measurable financial strategic goals on a coordinated, integrated basis, thus enabling the firm to operate efficiently and effectively. Financial goals and metrics are established based on benchmarking the “best-in-industry” and include: Cash Flow: This measures the net amount of cash and cash-equivalents being transferred into and out of a business. At the most fundamental level, an organization’s ability to create value for shareholders is determined by its ability to generate positive cash flows, or more specifically, maximize long-term free cash flow. Budget Execution Rate: Budget execution is the process by which the financial resources made available to an agency are directed and controlled toward achieving the purposes and objects for which budgets were approved. The process involves compliance with both legal and administrative requirements. Asset Management: This calls for the efficient management of current assets (cash, receivables, inventory) and current liabilities (payables, accruals) turnovers and the enhanced management of its working capital and cash conversion cycle. Profitability Ratios: This is a measure of the operational efficiency of an organization. Profitability ratios also indicate inefficient areas that require corrective actions by management; they measure profit relationships with sales, total assets, and net worth. Growth Indices: Growth indices evaluate sales and market share growth and determine the acceptable trade-off of growth with respect to reductions in cash flows and returns on investment. The above financial metrics together help firms implement and monitor their strategies with specific, industry-related, and measurable financial goals, strengthening the organization’s capabilities with hard-to￾imitate and non-substitutable competencies. They create sustainable competitive advantages that maximize a firm’s value, the main objective of all stakeholders.