3 MALI Midterm Performance Evaluation of USAID/Mali Funded Program Entitled USAID/Mali Sub-National Governance Project (SNGP) Final evaluation report January 3, 2019 Prepared by the Mitchell Group, Inc. (TMG) for the United States Agency for International Development. This document was prepared for the USAID/Mali as part of contract No. 720-688-19-F-00001 – Evaluation of the Sub-National Governance Project. Prepared by: Mr. Leslie Fox, Senior Evaluation Specialist (Team Leader) Mr. Jeremy Chevrier, Evaluation Technical Manager and Data Analyst, TMG, Inc. Mr. Brehima Doumbia, Public Financial Management Technical Specialist Mr. Boubacar Dramane Traore, Decentralization/Governance Technical Specialist Ms. Aisse Diarra, Gender Technical Specialist Main contacts: Mr. Leslie Fox, Team Leader. Email: devcondg@gmail.com Mr. Jeremy Chevrier, TMG, Inc., Washington, DC. Email: jeremyc@the-mitchellgroup.com The Mitchell Group, Inc. 1816 11th Street, NW Washington, DC 20001 Tel.: 202-745-1919 DISCLAIMER The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i | P a g e Table of Contents LIST OF ACRONYMS....................................................................................................................................iii Abstract ..............................................................................................................................................................v Executive Summary .........................................................................................................................................vi A. Overview ..........................................................................................................................................vi B. Key Findings .....................................................................................................................................vi C. Principal Conclusions....................................................................................................................vii D. Principal Recommendations.........................................................................................................vii I. EVALUATION/ANALYTIC PURPOSE AND EVALUATION/ANALYTIC QUESTIONS ...........1 A. Purpose of evaluation......................................................................................................................1 B. Audience ............................................................................................................................................1 C. Anticipated use .................................................................................................................................1 D. Evaluation lines of inquiry...............................................................................................................1 II. PROGRAM BACKGROUND...............................................................................................................2 A. Brief overview of context of decentralization and PFM in Mali ............................................2 History of Decentralization and PFM in Mali ............................................................................2 Public Financial Management Systems in Mali............................................................................3 B. SNGP Goal, Components and Theory of Change ...................................................................4 Project Overview ............................................................................................................................4 SNGP Theory of Change ...............................................................................................................5 C. Evaluation Report Structure and Flow........................................................................................6 III. EVALUATION/ANALYTICAL METHODS AND LIMITATIONS ...............................................7 A. Mixed methods research approach..............................................................................................7 B. Sampling methods............................................................................................................................7 C. Quantitative methods .....................................................................................................................7 Data collection - Mini-survey........................................................................................................7 Data collection - Public perceptions survey (i.e. radio survey).............................................7 Analysis ..............................................................................................................................................8 D. Qualitative methods........................................................................................................................8 Data collection – Key Informant Interviews (KIIs) & Focus Group Discussions (FGDs)8 Analysis ..............................................................................................................................................8 E. Limitations of the Evaluation .........................................................................................................9 IV. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS......................................................9 A. Frame 1: The Relevance and Soundness of the SNGP Theory of Change .........................9 ii | P a g e Principal Findings..............................................................................................................................9 SNGP Design Conclusions............................................................................................................9 Conclusions on TOC as the Foundation for the SNGP Design ........................................ 13 Recommendations: Relevance and Soundness of the SNGP TOC ................................... 14 B. Frame ii: SNGP Approach to Project Implementation......................................................... 14 Principal Findings: Tetra Tech’s Relationships with Partners and Beneficiaries ............. 14 Specific SNGP Approach Issues to Note ................................................................................ 16 Related Parameters and Contractual Requirements Affecting SNGP’s Approach ........ 18 Principal Conclusions: Tetra Tech’s Approach to SNGP Implementation...................... 21 Principal Recommendations: Tetra Tech’s Approach to SNGP Implementation .......... 21 C. Frame iii: SNGP Performance and Impact.............................................................................. 22 Principal Findings........................................................................................................................... 23 Gender Mainstreaming and Women’s Empowerment......................................................... 26 Principal Conclusions................................................................................................................... 27 Principal Recommendations: TT should revamp its performance monitoring plan ...... 28 V. SYNTHESIS AND OVERALL CONCLUSIONS............................................................................ 31 Annex I: TMG Responses to USAID Mali Comments on Draft Evaluation Report Annex II: Evaluation Statement of Work (SOW) Annex III: Evaluation/Analytic Methods and Limitations Annex IV: Data Collection Instruments Annex V: Sources of Information Annex VI: Summary Information about Evaluation Team Members iii | P a g e LIST OF ACRONYMS ACCM Association des Conseils de Cercles du Mali ADR Agence de Développement Régional (Regional Development Agencies) ANICT Agence Nationale d’Investissement des Collectivités Territoriales ARM Association des Régions du Mali CADD Cellule d’Appui à la Décentralisation et à la Déconcentration CAFO Coordination des Associations et ONG Féminines du Mali CFA West African CFA francs CFCT Centre de Formation des Collectivités Territoirales (Center of Training for Territorial Collectivitiés) CGSP Contrôle Général des Services Publics CREDD Cadre Stratégique de Relance Économique pour un Développement Durable CSO Civil Society Organizations CT Collectivités Territoriales DCPND Framework Document of the National Policy on Decentralization DGCT Direction Générale des Collectivités Territoriales DGI DRG Direction Générale des Impôts Democracy, Human Rights and Governance FNACT Fonds National d’Appui aux Collectivités Territoriales (National Fund to Support Local Collectivitiés) GOM Government of Mali GSB Groupe de Suivi Budgétaire IR Intermediate Result M&E Monitoring and Evaluation MEF Ministère de l’Economie et des Finances (Ministry of Economy and Finance) MoE Ministère de l’Education (Ministry of Education) MOU Memorandum of Understanding MSI Management Systems International NDI National Democratic Institute NGO Non-Governmental Organizations PDESC Programme de Développement Economique, Social et Culturel (Economic, Social, and Cultural Development Program) iv | P a g e PEDC Programme de Développement de l’Education au Niveau Cercle PFM Public Financial Management PMP Performance Monitoring Plan PNACT National Program of Support to Local Authorities PREM/GFP Programme pour la Reforme en Gestion des Finances Publiques Management in Mali SNGP Subnational Governance Program SOW Statement of Work TA Technical Advisor TT TDRL Tetra Tech / ARD Taxes de Development Local et Regional (Local Development Taxes) TOT Training of Trainers US United States USAID United States Agency for International Development WILDAF Women in Law and Development in Africa v | P a g e ABSTRACT A team of five The Mitchell Group (TMG) Inc. consultants conducted a midterm evaluation of the Sub-National Governance Project (SNGP), a five-year, $22 million USAID-funded project, awarded to Tetra Tech ARD (TT) and consortium of four sub-contractors. SNGP has an overall goal of advancing Mali’s decentralization process by creating and institutionalizing effective financial and asset management procedures. The evaluation utilized a mixed methods research approach, leveraging both qualitative and quantitative methods, around seven USAID-developed evaluation questions that formed the principal areas of inquiry and were in turn consolidated under three broader analytic frames: SNGP Theory of Change, SNGP Management Approach and SNGP Performance and Impact. The evaluation’s principal conclusion is that there were fundamental problems in many dimensions of the project, from its conception to its implementation and in the monitoring and evaluation of project activities. Some of these problems were due to TT’s choices in the way it managed the project; some were due to the faulty design of the project; and, yet others were due to external factors in SNGP’s operating environment. The evaluation team’s recommendations include: 1. SNGP should undertake a strategic moment of reflection retreat for its partners where identified problems can be discussed. 2. SNGP should transition to direct funding of partners showing that TT has confidence in their capacity to responsibly manage funds. 3. Revamp SNGP’s PMP so that outputs are tied to intermediate results that demonstrate change. 4. Reduce the number of activities that SNGP supports and in fewer regions, particularly in the North. vi | P a g e EXECUTIVE SUMMARY A. OVERVIEW A team of five TMG consultants conducted a midterm evaluation of USAID/Mali’s Sub-National Governance Project (SNGP). The methodology included a desk review of relevant documents; four weeks of field work including regional visits to three of eight regions in Mali, plus the capital Bamako, to collect both qualitative (KIIs and FGDs) and quantitative data (mini-surveys and perception surveys) from key stakeholder groups; and, a summary review of SNGP’s principal results and corresponding interviews. The evaluation used a set of three very broad areas of inquiry (Analytical Frames) which regrouped the seven evaluation questions and sub-questions into a logical analytical framework for data collection and analysis. The objective of this midterm evaluation was to review and assess the performance and impact of TT and its consortium over the first three years of the project with the ensuing findings to be used to determine recommendations for the final two out years of SNGP for USAID and to inform future initiatives of this type going forward. B. KEY FINDINGS The principal findings included that: 1. The SNGP design was lacking in analytical rigor, which led to the development of a theory of change that did not reflect the principal constraints to decentralization in Mali and in turn led to the development of program components that were ill-suited to the actual reality on the ground. The assumptions underlying the TOC, (a) that the GOM had the political will to support decentralization and regionalization; and, (b) that civil society had the capability to undertake a watchdog and oversight role vis-à-vis regional government were belied by the evidence collected by the TMG team and were in fact quite evident during the conception phase of the project. 2. Tetra Tech’s management and programmatic approach became significant constraints to the effective and efficient implementation of SNGP. The management was highly centralized and top-down and the impression among both local consortium partners and beneficiaries was one of disrespect for their interests and capabilities. The program approach of working in all eight regions of the country, including the insecure Northern Regions and, to a lesser extent, Mopti; a high-volume, transactional approach based on many activities spread out over the length and breadth of the country; and, the limitation of only being able to work at the regional level, all of which combined to dissipate the impact of project resources, which were imposed by the SNGP design. 3. Tetra Tech and its consortium partners’ performance and the activities that were undertaken were generally well received and appreciated by SNGP’s stakeholders, albeit with limited measurable impact. Their effectiveness and efficiency were mitigated by the aforesaid design-driven constraints imposed on them as well as the adverse effects of TT’s management approach. 4. Assessing impact after little more than three years is an unrealistic task, particularly in the challenging environment in which SNGP was implemented. It was made more difficult be the fact that all the PMP indicators formulated by TT measured very low-level outputs vii | P a g e that were untethered to any results which would have demonstrated meaningful change as a result of TT and consortium partner efforts. 5. Gender integration and women’s empowerment were a major focus of the project and considerable effort was made by SNGP to empower women to better engage with their local governments. Gender mainstreaming was effectively incorporated into the project’s design and a wide range of relevant activities were undertaken to increase women’s participation. Although, there was more focus on increasing the numbers of women attending training or other capacity building activities, rather than more focus on the quality of their engagement during these activities and thereafter. C. PRINCIPAL CONCLUSIONS The evaluation’s principal conclusion is that there were clearly fundamental problems in all dimensions of the project’s life to date, from its conception to its implementation and in the monitoring and evaluation of project activities. Some of these problems were due to TT’s choice in the way it managed the project; some were due to the faulty design of the project; and, yet others were due to external factors in SNGP’s operating environment. The challenge in the remaining two years of the project is how to make those activities that were highly-regarded more than one-off initiatives that dissipated as they moved down the Territorial Collectivity hierarchy. While the evaluation makes some rather pointed findings in the way the project was designed and implemented, the intent was not to cast aspersions but rather to provide a jolt of realism into the dynamics of this project as it has evolved over the past three years. Our conclusion is that SNGP can be repaired with some hard choices and a continued commitment to decentralization and subnational governance in Mali. We conclude that with modifications to project management’s approach, Tetra Tech and its partners are capable of making a constructive contribution to Malian decentralization, a commitment that USAID has demonstrated in 25 years of development assistance to Mali and its citizens. D. PRINCIPAL RECOMMENDATIONS The recommendations found in the body of this report should not be considered the basis for the design of a new project. USAID and the TT consortium have some difficult decisions to make but their result should not lead to a reconfiguration of the underlying purpose of the project— to support decentralization through regionalization and improved subnational governance. Even in the best conceived projects, there will be components or elements that do not work or fit an evolving context—failure is only a failure if the lessons it imparts are not used to improve a given situation. In fact, the great majority of recommendations found in this mid-term evaluation came from SNGP stakeholders. With regard to top-line recommendations, the Team recommends the following immediate and longer-term recommendations, as follows. Immediate Recommendations: 1. SNGP should undertake a strategic moment of reflection (SMR) retreat for all SNGP partners where all of the relevant issues can be discussed. 2. SNGP should transition to direct funding of partners and show that TT has confidence in their capacity to responsibly manage funds. 3. Revamp SNGP’s performance management plan (PMP) so that the outputs shown in the current PMP are more clearly tied to higher-level or intermediate results (IRs) that viii | P a g e actually demonstrate a change in condition, the fundamental principle of a results-based management approach. Longer-term Recommendations: 1. SNGP should develop joint work plans with its principle partners, which incorporate their priorities as well as those of SNGP. 2. Reduce the scope and coverage of the SNGP program - SNGP needs to reduce the number of activities it supports in fewer region. SNGP should move out of the three Northern Regions and consider whether Mopti itself should remain among the new focus of activities. 3. Consider ways of extending its training, technical assistance and awareness-raising initiatives, many of them worthwhile and appreciated, to the Circles and Communes. 4. Consider tweaking the program goal to focus on building the capacity of elective councils and CSOs to function effectively: “Deconcentrated state administrative and technical services, elected collectivity councils and civil society organizations are effective partners in democratic governance, including development planning and public policy making, at the regional level and below.” We believe the set of recommendations found in the body of this midterm evaluation report form a coherent and realistic response to the analysis and findings upon which they are built. They are intended to provide USAID and TT and its partners with a strategic context to give SNGP the best chance of succeeding and achieving results over the final two years of the project and that have a realistic chance for demonstrating development impact. 1 | P a g e I. EVALUATION/ANALYTIC PURPOSE AND EVALUATION/ANALYTIC QUESTIONS A. PURPOSE OF EVALUATION This purpose of this midterm performance evaluation of the USAID/Mali-funded program entitled USAID/Mali Sub-National Governance Project (SNGP), is to identify the extent to which the project is progressing toward achieving its goal and to develop practical recommendations based on lessons-learned from project implementation to inform USAID in the options years in order to achieve maximum impact and efficiency. B. AUDIENCE The primary intended audiences for this evaluation report are USAID and Tetra Tech. Secondary audiences for this report are Tetra Tech’s sub-partners under the SNGP. C. ANTICIPATED USE The evaluation will help to assess the effectiveness of the SNGP ahead of the decision to exercise the project’s option years. USAID/Mali seeks an external evaluation of the project thus far to identify the strengths and weaknesses of the activities and operations and if and how the approach needs to be refined in the out-years to achieve maximum results. D. EVALUATION LINES OF INQUIRY USAID/Mali has proposed seven lines of inquiry for this evaluation. The evaluation team reflected upon each of these lines of inquiry and their various sub-questions and noticed some overlap in the questions. The team leader proposed during his exit-brief to encompass the scope of the evaluation lines of inquiry within three broader analytic frames: 1. SNGP Theory of Change, 2. SNGP Approach and 3. SNGP Performance and Impact. The USAID team approved this approach. Each of the seven lines of inquiry are addressed in the narratives within the sections related to the three analytic frames in this report. Following are the seven lines of inquiry being explored in this evaluation: Q1 Efficiency, Efficacy and Appreciation of Activities: What activities have been most effective in line with the mandate of the project in terms of: 1.1) building the capacity and/or improving performance of sub-national entities (a concrete result in change or capacity), 1.2) appreciation by beneficiary (SNGP, GOM or civil society), 1.3) advancing a key issue such as transparency, gender, PFM, etc. and 1.4) In terms of cost-benefit (low cost, big impact)? Q2 Project results and design: 2.1) What results have been achieved for each component of the project? 2.2) Should the level of effort for each objective/sub-objective/activity remain the same in each of the three objectives? 2.3) Can any activities be graduated or ended? 2.4) What activities/interventions should be added in out years? Q3 Challenges and Bottlenecks to Implementation: 3.1) How has the security context impacted project implementation? 3.2) How have changes in the government and political will impacted the project implementation? 3.3) What have been the hardest things to overcome? 3.4) In what ways has the project failed? 3.5) How can the project better overcome these challenges and bottlenecks to implementation? 2 | P a g e Q4 Changes in approach: 4.1) What adjustments should be made in terms of activities, partners, operations, approach, etc.? 4.2) How can the project more acutely respond to needs, trends, the interests of GOM Ministries/officials and civil society/citizens? 4.3) How can the project better generate and channel existing political will for decentralization and regionalization? 4.4) How can the program better support the decentralization of transfers and service delivery in the education and health sectors? Q5 Technical Impact: 5.1) What is the impact of the institutional and organizational capacity building the project is undertaking with Malian government institutions and sub-grantees? 5.2) How has the GOM benefited from the program? 5.3) How has civil society been engaged? 5.4) How are the eleven (11) embedded Technical Advisors performing, and how can their role in the institutions be improved? Q6 Gender: How is the program’s gender integration strategy and plan progressing in terms of capacity building, awareness raising and lobbying, technical assistance to increase women and youth participation and inclusion and synergy with other stakeholders? Q7 Coordination and Synergy: 7.1) How is the program coordinating with other stakeholders (PADRE/GIZ, Lux-Development and other donors, other similar programs, etc.)? 7.2) What synergies or leveraging or layering of activities have occurred? 7.3) What opportunities exist to strengthen coordination and synergies? II. PROGRAM BACKGROUND A. BRIEF OVERVIEW OF CONTEXT OF DECENTRALIZATION AND PFM IN MALI HISTORY OF DECENTRALIZATION AND PFM IN MALI Sub-national governance in Mali is a hybrid system of two forms of decentralization; 1 devolution2 and deconcentration. Devolution of power, responsibilities and resources responds to the fundamental aspirations of Malians for self-rule, embodied in the constitution of the Third Republic (1992). The principal decentralization framework law (No. 93-008, February 1993) was adopted noting the conditions for the free administration of local authorities while respecting national unity, or the notion of the Unitary Malian State. The first iteration of decentralization was implemented through the country’s first general municipal elections (1999) and the establishment of elected bodies at the commune level (703). This marked the initial transfer of 17 functions and corresponding resources from the central state to local authorities, including civil status, census, archives and documentation, administrative police, hygiene and sanitation. These responsibilities were followed in 2002 by the transfer of three specific functions (competencies): education, health and hydraulics (Decrees 02-313, 314 and 315). 1 Decentralization is the process of distributing or dispersing functions, powers and resources (e.g., human, financial) downward from a central authority, in this case the Central State, to sub-national structures either autonomous from this central authority (devolution) or directly responsible to it (deconcentration). There are three dimensions to decentralization; the political, administrative and financial. 2 Devolution concerns the right of the democratically-elected collectivity councils to plan and manage their affairs with maximum political, administrative and fiscal autonomy. Devolution is essentially a bottom-up definition of power where citizens are given the freedom and resources to set their own collective agenda and to pursue it while being held accountable for the ensuing outcome. 3 | P a g e The 2012 coup, rebellion in the north and resultant political crisis effectively halted this nearly two-decade experiment in Malian decentralization and democratization. The signing of the Peace and Reconciliation Agreement (June 20, 2015) reaffirmed the historical importance of decentralization and the role of local authorities and reinforced it with a new policy of regionalization. 3 To make regionalization operational, several state organs were established, the most important of which were Regional Development Agencies (ADRs) (Order No. 2015-017/P￾RM of 02/04/2015). Their mission is to promote regional and local development in their areas of intervention by assisting local authorities in the exercise of project management, including training, as well as in the areas of planning, programming, carrying out development operations, service management and resource mobilization. To date, the so-called general powers relating to issues of functions and financial transfers are supplemented by specific powers covering the 22 functions for which decrees have been issued, setting out the details of the powers transferred from the state to the Collectivités Territoriales (CT – henceforth, Collectivitiés). To enable local authorities to manage these powers effectively and efficiently, the State has enacted The Act on the Free Administration of Territorial Collectivitiés stipulating that “any transfer of powers to a local authority must be accompanied by the concomitant transfer by the State to the latter of the resources and means necessary for the normal exercise of these powers” (Article 4 of Act No. 2017-052 of 10/2/2017), reinforcing Article 14 of the Peace and Reconciliation Agreement mandating the transfer of 30 percent of budget revenues to Collectivitiés by 2018. Decentralization as deconcentration established a parallel set of institutions, buttressed by a corresponding set of laws which reinforce Mali’s unitary system of governance with the extension of executive power to the local level through the appointment of Governors in the regions, Préfets in the Cercles and Sous-Préfets in the communes. Deconcentration, based on the system of tutelle (literally, ‘supervision’), has its antecedents in the colonial era. Today, the system of tutelle undertakes two well-established functions: 1) oversight of the decentralized (devolved) levels of government from the region to the commune to ensure compliance with the concerned laws; and, 2) to provide technical support to the Collectivitiés: i.e., Communes, Circles and Regions and their corresponding Councils in their discharge of mandated legal responsibilities. As will be discussed in the analysis in this report, it is the tension between these two forms of decentralization, devolution and deconcentration, and the underlying organizing principles that drive them that defines Mali’s difficulty in consolidating decentralization. PUBLIC FINANCIAL MANAGEMENT SYSTEMS IN MALI Public financial management (PFM) systems have been established at both the national and regional levels. At the center of the PFM system is the national budget which must first be approved at the national level, requiring a lengthy process of examination and inquiry by concerned executive-level ministries and agencies and a number of National Assembly (NA) committees. Once the NA passes the annual Finance Act, it is promulgated by the President, and 3 Regionalization concerns the process of giving more resources and authority directly to regional governments (Conseil Regionale) and, by extension, to other levels of the territorial collectivity; i.e., Circles and Communes. Regionalization was conceived largely around the need for promoting economic investments, both infrastructure and commercial, that transcend the more parochial or fragmented outlook of Circles and Communes. 4 | P a g e ultimately leads to the Minister of Finance’s issuance of an appropriation order making it possible to release budgetary allocations, including those to the Collectivitiés. At the subnational level, Collectivitiés, starting at the Commune level, collect and analyze demographic, economic and social (e.g., health and education) data which then allows the finance committees of the concerned Councils to develop a draft budget. The President of the CT Executive Body then organizes consultations with the councils of the grassroots communities (village, district, and fraction) before then, in principle, holding a public debate on the draft budget. The draft budget must be voted on by the concerned legislative body (e.g., communal council) and submitted to the supervisory authority for approval by October 31st of the year prior to that of the proposed budget. Budget implementation involves Authorizing Officers, Financial Controllers and Public Accountants (directly tied to the central state Treasury). Revenue issuance and recovery are done by the same departments. However, after recovery, all other departments make payments to the public treasury, which is the State's sole cashier (guichet unique). There are two types of budget control and oversight: internal control and external control. Internal control audits are carried out by concerned departments of the deconcentrated state administration; e.g., Inspection of the Finance Ministry and General Control of Public Services (Contrôle Général des Services Publics - CGSP). The CGSP is responsible for carrying out systematic control operations on the management and operation of all public services and establishments of the State, local authorities, authorizing departments, accounting services, State Companies and Enterprises and Mixed Economy Companies. Formerly under the authority of the Prime Minister, it has recently been moved to the Ministry of Administrative and Public Life Reform. External controls include audits that are carried out by the National Assembly and legislative bodies of the local authorities; e.g., the Supreme Court Accounts Section (SCAS) and the Office of the Auditor General (OAG). They are performed after the event and therefore complement or confirm internal control. B. SNGP GOAL, COMPONENTS AND THEORY OF CHANGE PROJECT OVERVIEW Since the country’s transition to democracy in 1992, USAID has been a steady and committed supporter of decentralized governance in Mali. The Mali Sub-National Governance Program (SNGP, Programme d’Appui à la Gouvernance Locale) is a third-generation project by the Democracy, Human Rights and Governance Office (DRG). The first generation or phase commenced with the Mission’s first strategic objective (1994), whose purpose was to strengthen local level civil society in order to be an effective partner alongside Communal government once elections were held. The second phase reflected a shared governance paradigm (CSPs 2002 – 2014) that supported communal government and local civil society to jointly plan, implement and achieve self-determined, sustainable local development objectives. This second generation DRG strategy was largely implemented through the Programme du Gouvernance Partagée (PGP) 1 and 2, which was cut short by the Crisis of 2012. SNGP, the third-generation program, was the result of two years of negotiations with the GOM following the 2013 Peace Accords, in which the USG agreed to support the GOM’s newly developed Regionalization Policy. The SNGP is a five-year, $22 million USAID-funded project, under contract number AID-688-C￾16-14-00003, awarded on February 9, 2016 to Tetra Tech ARD (TT). The overall goal of SNGP 5 | P a g e is “to advance Mali’s decentralization process by creating and institutionalizing effective financial and asset management procedures that contain corruption, and improves the overall management and operational efficiency of Mali’s financial resources between Bamako and the regions, thus ensuring that public funds intended for decentralized services are delivered effectively and transparently.” In addition to its core DRG funding, SNGP receives funding from USAID/Mali’s Education and Health Offices in order to support improved decentralization and delivery of services in the education and health sectors. Depending on the outcome of this mid-term evaluation showing the successful implementation of activities leading to the achievement of project objectives, USAID may exercise a two-year option period. SNGP has a national focus that covers all eight administrative regions of Mali plus Bamako; this does not include the two newly created regions in the North (Taouédeni and Ménaka). The project directly supports implementation of Mali’s peace agreement by bolstering the capacity of the ADRs and the responsiveness and vitality of services available to all Malians, regardless of region. The project focuses on Public Financial Management (PFM) as a catalyzing theme and consists of three components that: 1) strengthen national and regional PFM systems; 2) improve coordination and communication between national and sub-national government and civil society; and, 3) improve the capacity of the Mali’s Supreme Audit Institute, Controle General des Services Public (CGSP), to conduct performance audits that meet international standards, and to monitor public spending and service delivery at the national and subnational levels. The project was designed to take advantage of renewed efforts to decentralize government in Mali through regionalization; that is, the process of giving more resources and authority directly to the regional governments. Tetra Tech ARD (TT) engages four core sub-contractors that act as implementing partners: (a) Management Systems International (MSI), which focuses on public financial management; (b) Banyan Global, which addresses issues of gender integration and providing support to the third partner, Women in Law and Development in Africa (WILDAF); (c) WILDAF, which promotes gender equality and women’s empowerment with offices in the targeted regions; and, (d) Group de Suivi Budgétaire (GSB), which has expertise in budget monitoring and has offices in the targeted regions. In addition, TT has a Local Sub-Contractors Pool which it draws upon for technical assistance and support as needed. SNGP THEORY OF CHANGE The overarching Theory of Change (TOC) is based on the stated view in the RFTOP that the GOM does not have confidence in the Collectivitiés Territoriales (CTs) to effectively manage mandated state financial transfers, thus negatively affecting the ability of the CTs to deliver public services to their constituents, and, in turn, making the state reluctant to transfer these funds. SNGP thus addresses this overarching problematic through the implementation of each of its three project components and the TOCs that underlie them: Component 1: Strengthen National and Sub National Public Financial Management Systems. IF national and subnational public financial management systems and administrative systems are strengthened and subnational government institutions show improved understanding of and skills to manage the resources they are given, THEN the central government will have more confidence in subnational systems, and will therefore be more willing to transfer resources. IF decentralization 6 | P a g e occurs, THEN service delivery will be increasingly responsive and accountability to citizens and public trust in government will improve. Component 2: Improve Coordination and Communication between National and Subnational Government and Civil Society. IF coordination and communication processes between the national and subnational levels of government and civil society are improved, THEN budget planning/execution will be harmonized and civil society will have an increased capacity to undertake its watchdog and advocacy roles. IF both of these things happen, THEN the central government will have more confidence in subnational systems and will be more willing to transfer resources. IF [effective] decentralization occurs, THEN service delivery will be increasingly responsive and accountable to citizens and public trust in government will increase. Component 3: The “Contrôle Général pour Services Public” (CGSP) conducts performance audits that meet international standards and is able to monitor public spending and service delivery at the national and subnational levels. IF CGSP’s capacity to conduct performance audits and monitor public spending and service delivery at the national and subnational levels is improved, THEN the central government will have more confidence in subnational systems and will be more willing to transfer financial resources. IF decentralization occurs, THEN service delivery will be increasingly responsive and accountable to citizens, and public trust in government will improve. Finally, there are two critical assumptions which underlie the SNGP theory of change: 1. THERE IS POLITICAL WILL BY THE NATIONAL LEVEL GOVERNMENT TO FULLY SUPPORT DECENTRALIZATION AND REGIONALIZATION; AND, 2. CIVIL SOCIETY EFFECTIVELY DEMANDS ACCOUNTABILITY FROM GOVERNMENT AT BOTH THE NATIONAL AND LOCAL LEVEL. C. EVALUATION REPORT STRUCTURE AND FLOW As a general note, this evaluation report is based on data-driven findings and organized to enhance clarity and a logical flow from findings to conclusions to recommendations. Following this introductory chapter, Chapter II presents the overall evaluation approach and methodology, including a discussion of anticipated or unanticipated limitations encountered during the evaluation exercise. Chapter III presents findings, conclusions and recommendations as a coherent analytical framework. The findings, conclusions and recommendations will be divided topically into three key analytic frames that cross-cut across all seven of the evaluation’s lines of inquiry. These three analytic frames include: (a) the SNGP Theory of Change (TOC), (b) the SNGP Approach, (c) SNGP’s Performance and Impact. Within each analytical frame, findings related to challenges and successes, along with conclusions and recommendations, are discussed based on the evidence and analysis undertaken, while intentionally ensuring that the comprehensive scope of each of the seven evaluation lines of inquiry are fully incorporated and discussed. The introduction to each of the three frames will include the RFTOP questions / lines of inquiry that will be covered therein without the findings being classified by a given line of inquiry. Chapter IV of this report provides an overall Synthesis and Conclusions discussion that ties together and highlights the principal findings of this evaluation and provides a set of actionable recommendations. 7 | P a g e III. EVALUATION/ANALYTICAL METHODS AND LIMITATIONS A. MIXED METHODS RESEARCH APPROACH This evaluation utilized a mixed methods research approach, leveraging both qualitative and quantitative methods to enhance the validity of findings through triangulation. A desk review was done of relevant project documents, including indicators from the PMP, by the team in order to help better frame the qualitative and quantitative instruments to be utilized. Two different small quantitative surveys were done, a relatively larger, mini-survey of mostly project beneficiaries and a smaller public perceptions survey. For qualitative data collection, key informant interviews (KIIs) were conducted with key project stakeholders along with focus group discussions (FGDs). B. SAMPLING METHODS All data collection tools, with the exception of the small public perceptions survey, were focused on sampling across the following six stakeholder types: USAID, IP/Sub-IPs, Donors, GOM Technical Services, Collectives, schools and health centers. For the quantitative mini-survey purposive, non-probability sampling was used as the Team needed to select the appropriate mix from among the six stakeholder types for the survey. The public perception survey used random sampling of members of the public. The qualitative interviews all utilized purposive sampling, combining snow-ball sampling and sequential convenience sampling where appropriate when the team was able to build off contacts naturally from those interviewed as the field work progressed. Since the quantitative surveys had a relatively small sample size, the results may not be considered to be representative of the entire population. C. QUANTITATIVE METHODS DATA COLLECTION - MINI-SURVEY The approved mini-survey (Annex III.F) was comprised of 15 multiple choice questions querying project performance, the Theory of Change, gender, the impact of insecurity and a basic assessment of knowledge related to local budgets and local development plans. There were 766 people surveyed from the six stakeholder groups. Of all respondents, 39% were aware of the SNGP and 61% were not aware of SNGP. For all project performance-related questions which required some knowledge or engagement experience with SNGP, only those respondents who knew about SNGP (296 respondents) were considered in the data analysis of performance questions. The local enumerator team, managed by TMG’s local partner, CEDREF, conducted the survey in the following four regions: Bamako, Mopti, Sikasso and Koulikoro. DATA COLLECTION - PUBLIC PERCEPTIONS SURVEY (I.E. RADIO SURVEY) The evaluation team leveraged an innovative effort to capture local perceptions around decentralization and PFM through a short, six question survey broadcasted on radio, (see survey tool in Annex III.H), with responses called- or texted-in by the public. A one-hour radio show was created working closely with the Union des Radios et Televisions Libres du Mali (URTEL) and broadcast through URTEL partner radio stations in the following regions: Bamako, Kayes, Koulikoro, Segou, Mopti, Sikasso, Gao and Tombouctou. Unfortunately, only 12 citizens responded to these broadcasts by calling in or texting in their responses to the 6 questions on the public perceptions survey. The team had anticipated that 100-150 people might call in, but 8 | P a g e the low response rate may be evidence that Malians aren’t accustomed to call-in radio surveys. To mitigate for this lack of responses, TMG sent two enumerators into the field in Bamako and Koulikoro to ask randomly-selected men and women, ages 18 and up, the six survey questions. The team was able to garner 142 public surveys. ANALYSIS The mini-survey consisted of 15 questions geared towards one of four themes: knowledge of budget and plans (Q1, 2); project performance (Q3-4, 6-8, 12-14); insecurity impact (Q9); and theory of change (Q5, 10-11, 15). Based upon these four themes, the following disaggregated groups were compared: age, sex, region, stakeholder type and stakeholder group-level (See definitions in Annex II.B). Correlations were run between the four comparison groups and three main themes to tease out statistically-significant differences. Correlation tests were done via the Pearson correlation coefficient, which is used to examine relationships between two or more quantitative/numerical variables. Frequency distribution tables for each mini-survey question were created (Annex II.B for more details) where statistically-significant differences were identified between the disaggregation and the question. Disaggregated response rate tables were created for those questions and analysis and interpretation was provided. For the smaller public perceptions survey, frequency distribution tables for each question were created (Annex II.C), with analysis and interpretation provided. Triangulation between the two quantitative surveys along with qualitative findings were integrated into analysis and interpretation of data where appropriate. D. QUALITATIVE METHODS DATA COLLECTION – KEY INFORMANT INTERVIEWS (KIIS) & FOCUS GROUP DISCUSSIONS (FGDS) The team set out with the ambitious goal of conducting 120 KIIs and 12 FGDs across the four regions selected for field visits: Bamako, Mopti, Sikasso and Koulikoro. Logistical delays and difficulties getting contacts and meetings set up resulted in the evaluation team achieving 99 KIIs and 12 FGDs total. Stakeholders were interviewed from all six stakeholder groups. The interview instruments (Annex III.C and Annex III.E) centered around main lines of inquiry relating to the three analytic frames: 1) theory of change; 2) approach; 3) performance; and 4) impact, along with key issue-related questions. The team often diverged from central questions to explore unanticipated findings and perspectives encountered during the interview process. Counts of KIIs conducted per region are: 21 in Sikasso, 21 in Mopti, 23 in Koulikoro and 34 in Bamako. Counts of FGDs conducted per region are: 2 in Sikasso, 3 in Mopti, 3 in Koulikoro and 4 in Bamako. ANALYSIS The Team, with TMG home office support, utilized Nvivo qualitative analysis software to manually code all KII and FGD notes thematically. A hierarchy of dominant themes was created (Annex II.A), and frequency distributions were generated to understand which of the common themes were more dominant. These thematic frequencies were converted to response rates (percentages) to create pie chart visualizations of dominant themes that emerged across the totality of qualitative narratives analyzed. 9 | P a g e E. LIMITATIONS OF THE EVALUATION One of the main limitations that was encountered during the field work was that the team realized that not all contacts interviewed during KIIs and surveyed in the mini-survey actually knew about the SNGP project. Additionally, some had heard of SNGP, but only had a vague idea about the project. This made it difficult in some interviews and surveys to gather quality feedback on project performance. In the case of KIIs, the evaluation team adapted to this situation by reorienting the questions to be more general and focused on perspectives relating to the underlying logic of project components and garnered thoughts and reactions on the activities the project focused on. In the case of the mini-surveys, those who had no knowledge of SNGP were tracked in the database and excluded from the data analysis on all project performance-related questions. Another limitation is the relatively small sample size of the quantitative surveys; the responses, therefore, cannot be considered as statistically representative of the Malian population as whole. IV. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS As noted in Chapter I, principal evaluation findings, conclusions and recommendations are presented as a single, coherent investigative set for each of the three Analytical Frames that were developed to better address and organize each of the RFTOP areas of inquiry or seven questions. These three analytical frames are: (a) the SNGP Theory of Change (TOC), (b) the SNGP Approach, (c) SNGP’s Performance and Impact. A. FRAME 1: THE RELEVANCE AND SOUNDNESS OF THE SNGP THEORY OF CHANGE SNGP’s performance and achievement of results depends on the soundness and relevance of the TOC which was formulated during the design phase. This chapter examines the TOC, the fundamental conceptual foundation upon which the project rests. In this analysis, we address a number of the evaluation questions: (i) Q2: Project Results and Design; (ii) Q3.2: How have changes in the government and political will impacted the project implementation? (iii) Q4.3: How can the project better generate and channel existing political will for decentralization and regionalization? And (iv) Q3.3: What have been the hardest things to overcome? PRINCIPAL FINDINGS The Team reviewed the principal analytical documents that would have been used in formulating the SNGP design; i.e., the 2014 DRG Assessment of Mali, the 2017 SNGP RFTOP, and the 2016 Political Economy Assessment conducted by SNGP post-project start-up which reviews the justifications for SNGP’s TOC. In addition, the Team held a number of formal interviews with USAID staff, including one member of the SNGP design team, and an informal interview with a DRG consultant who had participated in the program’s design. The Team found no convincing evidence or justification for the choice of the overarching TOC or its three components. Other than the RFTOP, the Team’s review came up with no specific project design document that provided a more detailed explanation of the project and/or an evidence-based justification for its TOC. SNGP DESIGN CONCLUSIONS USAID’s commitment to supporting the GOM’s post-Peace Accord’s regionalization policy, and its strong desire to encourage a decrease in what analysts agree is endemic corruption at all levels of Mali’s government, is consistent with its historic support for a decentralized system of 10 | P a g e democratic governance. The 2012 crisis and its aftermath certainly changed the trajectory of decentralization in Mali and merited a corresponding change in USAID’s strategy to support it. However, the Team found no convincing analysis from any of the above referenced documents that would indicate why the SNGP’s overarching TOC and those of its three components were chosen: GOM’s lack of confidence in the financial management capacity of CTs, the resulting ineffective public service delivery (competencies), and thus, the GOM’s reluctance to release mandated transfers to the CTs. A close reading of the RFTOP indicates a number of reasons (see Section 2, following) as to why the GOM’s lack of confidence in the CTs was not the principal reason for its reluctance to make these constitutionally mandated transfers to the Collectivitiés, many of which were validated in our qualitative analysis (KIIs and FGDs). Similarly, while interviews with two of the principals who contributed to the formulation of SNGP provided insight into the eventual design, they did not, from the Team’s analysis, provide the justification for the ensuing TOC. While the Team found no strong evidence for the choice of the TOC chosen, there are elements of two SNGP components (see Section C, following) that have merit and have demonstrated their usefulness, despite being untethered to a justifiable TOC or the Mission’s broader DRG strategy. The above review of the SNGP design effort is now followed by the set of findings and corresponding conclusions and recommendations that were generated from the Team’s multi￾faceted application of both qualitative and quantitative methodologies. The Team’s overall finding is that while there were a number of Key Informants (KIs) and FGDs which seemed to agree with the basic logic of the TOC, the great majority did not. This analysis starts with the two principal assumptions underlying the TOC: Assumption 1: The existence of Political Will for decentralization and regionalization. According to the qualitative analysis and thematic coding conducted, as shown in Error! R eference source not found., the top problem cited by KIs and in FGDs undermining the theory of change was the existence of a culture of corruption in Mali that is the key driver behind why the State has been slow to effectuate transfers. The second most common problem noted with the TOC was that there is a lack of political will for decentralization at the national level. In fact, as a root cause, both of these problems are considered dimensions of a lack of political will. The analysis from our document review and interviews included the following: The constitution and relevant texts make it clear that the transfer of competencies to the Collectivitiés must be followed by the transfer of financial resources to ensure their implementation. On their face, the coherence and strength of the concerned decentralization texts do indicate an overt level of political will to make the mandated transfers. It is the lack of 11 | P a g e their application by the central state’s bureaucracy, what one respondent called the “deep state,” that divulge the true intentions of Mali’s leaders and their lack of political. While there have been and continue to be examples of misuse of funds by the Collectivitiés, this is less a matter of their financial management capacity, and more a matter of the State and its deconcentrated organs not undertaking their role in CT control or oversight. This will be discussed more in the sections on SNGP performance and impact. Respondents most often cited an underlying culture of corruption, normalized throughout Malian society, challenging the TOC’s logic. The overwhelming evidence in this evaluation points to corruption being the key “hidden hand” driving the true lack of political will for decentralization, and the claim that the Collectivitiés are not responsible enough to manage public finances is a State-perpetrated subterfuge. The Team’s qualitative and quantitative analyses strongly indicate the root problem is what is called a “Culture of Corruption,” one that is embedded internally across state and elected institutions, and which is not amenable to a solution by better or more PFM training and capacity-building alone. Question 5 of the Mini-Survey asks: “Given all the support that donors have provided to Malian decentralization over the past 20 years, why has so little progress been made to date?” It is clear from the frequency of four of the five possible responses to this question that they can all be attributed to corruption. Similarly, if we examine response rates to Question 1 of the Public Perception Survey – “What is necessary to ensure equitable decentralization of resources and reduce corruption in Mali?” – we find that 80% of respondents chose “An internal change – a cultural or mentality change needs to occur.” The other 20% chose “An external change – better systems of good governance and accountability need to be put in place.” Also, according to the mini-survey data findings, youth significantly felt that a culture of corruption is the underlying problem slowing progress in Mali and leading to poor behavior with regard to public financial management and good governance. It is clear from the evidence collected in this evaluation that a solution that also addresses the internal factors (i.e. internal moral sanctions) of corruption, and not just by increasing external sanctions and threats on bad behavior alone, needs to be implemented by any future project that wishes to robustly deal with the root factors underlying corruption. Assumption 2: Civil society has the capacity to hold CT governments accountable for their performance in the discharge of their mandated responsibilities, including service delivery. Mini-Survey Q5 – Why so little progress on decentralization over last 20 years in Mali? Frequency Percent Response 244 32% Lack of civil society participation and advocacy 222 29% Not enough transparency in political systems 163 21% Culture of corruption / Mentality is blocking progress in Mali 133 17% Weak accountability in governance systems 4 1% No answer 766 100% TOTAL 12 | P a g e As with Assumption 1, the Team found that Assumption 2 also does not stand up to close scrutiny. With the transition to democracy in 1992, civil society at all levels, but particularly the community level, saw a massive increase in the density and diversity of these voluntarily formed, newly autonomous and citizen-created organizations. USAID’s first DRG strategy sought to capitalize on this fact, through building their capacity in areas of advocacy and oversight, which was carried over, but to a lesser extent, into PGP 1 and 2. The assumption in the SNGP design is that this capacity for CS engagement with CT elected governments carried over into the post-Crisis era. In reviewing the relevant documents and in interviews with CSO leaders and members, the Team found that, in reality, the capacity of grassroots community organizations and their NGO partners to: 1) demand participation in the preparation of communal development plans (PDSEC) and budgets, or 2) analyze expenditure reports and conduct strategic planning tasks on their own was significantly overstated. SNGP’s regional focus and the targeting of ADR’s and Regional Councils, which left out civil society from project activities except for number of regional level trainings, did little to change this fact. In fact, participants in both KIIs and FGDs made it overwhelmingly clear that the SNGP regional level approach was incomplete, not just for civil society’s effective participation in regionalization, but also for Circle and, in particular, communal government strengthening. One final point related to the assumption of existing political will for decentralization and regionalization: Both the interviews and the analytical work undertook made clear that the hybrid or parallel system of decentralized governance – both the administrative deconcentration of the central state and its technical services on one side, and the devolved system of elected Collectivity Councils on the other – is considered a major indicator of the central state’s lack of commitment to supporting effective devolved power and providing adequate competencies and resources to these elected bodies. In interview after interview, respondents made clear that true power in the Collectivitiés rested with the Governor and his / her delegates at the Circle and Commune level; this is born out in earlier assessments in which the tutelle system was far more about maintaining central state control than in supporting the needs of the Collectivitiés. 4 4 See for instance, Fox, Leslie, February 2012, An Assessment of the Decentralization Sector in Mali: Is it really about CT capacity? Key Informant Mopti: “If the state is committed to decentralization, it must make the transfer of resources effective. The CTs have the capacity to manage the transferred resources well—as proof, the European Union, in a pilot, put money worth CFA 1 billion 49 million at the disposal of the Regional Councils of Ségou and Mopti. The result of the audits carried out on the proper use of these funds has been very conclusive. And since, this partnership between the EU and these two contiguous CTs has even been expanded to three other regions. So, how do we understand that we can manage partners' resources but not those of the state? Key Informant Bamako: “Despite the deadline for the transfer of 30% of state budget revenues to local authorities by 2018, the adopted Finance Act 2019 provides for only 21% of the budget transfer by the state to local authorities. This demonstrates the State’s lack of willingness to respect its own commitments for a greater responsibility of local authorities in relation to the management of local affairs.” 13 | P a g e CONCLUSIONS ON TOC AS THE FOUNDATION FOR THE SNGP DESIGN In conclusion, it is clear that, given the analysis generated from the Team’s field work, a significant majority of the key informants did not agree with or approve of the SNGP TOC. For these Key Informants (KIs), the weak capacity of Collectivitiés to effectively manage public finances cannot explain the state’s refusal to transfer more resources to them. In fact, many KIs even questioned how Collectivitiés who have weak PFM capacities are still getting transfers if PFM capacity was truly the key criteria for effectuating transfers by the State. In their view, the rationale that the Collectivitiés’ lack of management capacity is the reason for the slow and incomplete transfer of public resources to them ignores the facts - many of which were already well-known prior to the SNGP evaluation and which the Team’s analyses and findings have further confirmed. Continuing to accept the SNGP TOC going forward will not concretely solve the problem of the lack of effective and efficient management of public service delivery by the Collectivitiés. The concomitant conclusion is that the transfer of financial resources, consistent with existing laws for the management of the transferred competencies, is a legitimate and fundamental requirement for effective CT governance. The Team wants to be clear that this first set of findings and conclusions is neither a reproach to SNGP nor USAID for the program’s ultimate design. The country context in Mali in 2014 and 2015 was volatile and uncertain. The USG in general and USAID in particular had accompanied Malian democratization for the better part of 25 years before the 2012 crisis. The normal process of significant analytical work preceding project design was apparently set aside for SNGP; expediency in demonstrating continued USG support to post-crisis Mali was most likely an important policy consideration. This does not mean, however, that there are not lessons to be learned from this experience. One of these lessons, perhaps the most important one, is that political will cannot be strong-armed “or channeled” by donors in the short-term. It either exists or it does not, and if policy dialogue is to be effective, it must be concerted, firm and long-term. This would be considered the “hardest thing to overcome,” one of the principal evaluation questions. A Strategic Review with Recommendations for USAID/Mali’s Country Development and Cooperation Strategy; and, the aforesaid DRG Assessment. 14 | P a g e RECOMMENDATIONS: RELEVANCE AND SOUNDNESS OF THE SNGP TOC There are no ready or easy solutions to these findings that SNGP was designed around an inaccurate TOC. As noted in Section C (Analytic Frame 3), there are many good initiatives that have been undertaken by SNGP and appreciated by those individuals and organizations that have been supported by it. However, in order to become more effective in promoting “subnational governance,” these activities need a more coherent conceptual framework and a valid underlying rationale that addresses the real root causes of the problem as evidenced by the data – a lack of political will and weak civil society capacity. The remaining two Analytical Frames provide a number of critical recommendations that must be tied to the findings from this TOC analysis and will be addressed in Chapter V, Synthesis and Overall Conclusions. B. FRAME II: SNGP APPROACH TO PROJECT IMPLEMENTATION Normally, the Team would have incorporated Analytical Frame 2: SNGP’s Project Implementation Approach into Analytical Frame 3: SNGP Performance: Efficiency and Effectiveness in Project Implementation. However, early on in the evaluation, it became evident that SNGP’s interactions with its implementing partners (IPs) and beneficiaries was acting as an impediment to its effective implementation of project activities. This second Analytical Frame, thus, focuses on the nature of SNGP’s relationships with its core IPs and partner beneficiaries, and how it has affected and continues to affect project implementation. The principal evaluation question that is addressed in this section is: Q1.1: building the capacity and/or improving performance of sub-national entities (a concrete result in change or capacity), Q4.1: What adjustments should be made in terms of activities, partners, operations, approaches, etc.? Related issues are addressed in Analytic Frame 3: SNGP Performance in the next section, including: Q4.2: How can the project more acutely respond to needs, trends, the interests of GOM Ministries/officials and civil society/citizens? Q4.4: How can the program better support the decentralization of transfers and service delivery in the education and health sectors? And, Q5.4: How are the eleven (11) embedded Technical Advisors performing and how can their role in the institutions be improved? PRINCIPAL FINDINGS: TETRA TECH’S RELATIONSHIPS WITH PARTNERS AND BENEFICIARIES SUMMARY OVERVIEW OF PARTNERS AND BENEFICIARIES AND THEIR CONTRACTUAL RELATIONSHIP WITH TT A distinction is made between SNGP, a USAID-financed project, and Tetra Tech (TT), the for￾profit contract awardee and consortium management leader of four international and Malian implementing partners (IPs): MSI, Banyan Global, Group de Suivi Budgétaire (GSB) and Women in Law and Development in Africa (WILDAF) (See Chapter I for the specific roles). Contractual relationships with these four IPs had ostensibly been worked out during the development of Tetra Tech’s proposal submission to USAID. As leader of the management consortium, Tetra Tech was responsible for developing an approach, both programmatic and contractual, for working with its targeted regional beneficiaries, also referred to as partners: the eight regional ADRs plus Bamako, eight elected Regional Councils (Conseil Regional, CRs), the decentralization and deconcentrated support units (Cellule d’Appui à la Décentralisation et à la Déconcentrat, CADD) of three national Ministries (Health, Education and Finance), and Mali’s Central Government supreme audit institute, Controle General des Services Public (CGSP). 15 | P a g e Tetra Tech, representing SNGP, signed sub-contracts of various durations, terms and conditions with its four implementing partners (IPs). Tetra Tech provided the Team with each of the sub￾IPs’ Scopes of Work (SOWs), but, despite the Team’s requests, did not provide the actual sub￾contracts with the specific terms, conditions and budgets. With each of the ADRs and CADDs, Tetra Tech, representing SNGP, signed identical Memorandum of Understandings (Protocol de Partenariat) outlining the responsibilities of both parties / partners. In addition to a range of training activities to strengthen ADR and CADD capacities, the principal SNGP support to the ADRs was the engagement and transfer of an “embedded technical assistant” (TA) to provide a range of services to the ADR. SNGP also financed initiatives in the Collectivitiés managed by the ADRs. In turn, the ADR’s were to provide space for the TAs, introduce SNGP to and facilitate relations with the deconcentrated technical services of the central state and the Regional Councils (Tetra Tech signed no similar MOU with these Councils) and jointly analyze and develop capacity-building plans for the Councils and other regional level entities. As the chart to the left notes, there are a wide range of problems that were identified by both Tetra Tech IPs and SNGP beneficiary partners. Our interviews with both IPs and beneficiaries, including embedded TAs, demonstrated an across-the￾board frustration with TT’s / SNGP’s approach which was described as: highly- centralized and top-down; administratively cumbersome and time￾consuming; characterized by a lack of respect, if not arrogance, for both the local IPs and beneficiary partners; broken promises around protocols and pre-SNGP implementation understandings; highly-directive and unwilling to take partner needs into account; micro-managing the smallest details of an activity (e.g. coffee breaks); most, if not all, activities (whether with local IPs or ADRs / CADDs), were viewed as SNGP-activities and did not include direct funding of any beneficiaries; and, ultimately, a disempowering rather than an empowering approach to all of its partners. As one person put it, “it is paradoxical and ironic that a project that is supposed to be promoting decentralization is one of the most centralized organizations of its kind that we have seen.” To be certain, there were a number of interviewees that appreciated the SNGP approach, but as the following text box portrays, an overwhelming majority did not. 16 | P a g e SPECIFIC SNGP APPROACH ISSUES TO NOTE The sampling of interviewee responses provided above adds nuance and rich narrative to the analysis the Team undertook of the SNGP approach. Having said that, there remains a number of significantly more important issues that need to be discussed in more detail. The ADRs and SNGP Relationship TAs are recruited by TT / SNGP. They all have generic job descriptions that are not based on the ADR’s defined needs. When they initially arrive at an ADR, they are often not accompanied by an SNGP staff member, and sometimes do not even have a letter of introduction. They arrive, therefore, without an ADR job description (cahier de charge) and often have to determine with the ADR director what they will do for a given period of time. SNGP provides them with motorcycles and computers while office space is provided by the ADR. When a joint activity is undertaken, the TA accompanies the ADR team in their vehicle. Discussions with the concerned parties strongly indicated that TA accountability was as much if not more geared to SNGP (its employer) as it was to the concerned ADR. TAs, in addition to whatever they were working on with the ADRs, were responsible, inter-alia, for collecting data on SNGP’s PMP indicators from the deconcentrated technical services, and for organizing IP and Beneficiary Comments on SNGP Approach • SNGP manages coffee breaks and lunches itself during trainings. Its agents come to pay the per diems of participants who are often impatient. … This discredits the structure because it is us who distributes the invitations to participants. It makes us uncomfortable. Partnership must be based on trust! • Partners’ needs and perspectives are not properly taken into account … SNGP does not consider our aspirations. • SNGP does not fund our activities directly. We do not manage/receive the funds from SNGP. We want to feel accountable, capable of managing the situation even after the program ends. • People at SNGP say that SNGP is for-profit. • SNGP must respect the practices of partner’s manual of procedures… • Experienced NGOs in the field of decentralization like Helvetas and GIZ always consult with us and do not adopt such practices [such as not giving the funds to us directly] • SNGP is not as pragmatic as we are used to seeing with American projects. • SNGP is a project with a heavy administrative burden, a slow disbursement process and they are hesitant despite the partnership agreement. 17 | P a g e SNGP-financed activities, sometimes jointly undertaken with the ADR and sometimes simply as an SNGP activity. This dual-use of TAs was a point of tension with many of the ADRs and some of the TAs themselves. As KIs from Bamako, Mopti and Koulikouro noted, “the Technical Assistants are much more at the service of SNGP than the department where he/she is assigned.” In our interviews with several technical service personnel (e.g., health, education, Education Academies, regional planning officers) in the three study regions, there was little understanding why an SNGP-financed staff member working for the ADR was coming to their offices sometimes monthly, sometimes quarterly, to collect data from them. ADR Directors were also requested at times to write to the targeted technical services requesting their help on behalf of the SNGP-TA. Compounding these tensions was the manner in which ADR program needs were addressed by SNGP, as well as the way in which they were financed. Concerned parties gave a number of different accounts about how SNGP chose which ADR activities to support. While ADRs were asked to develop partnership plans (Plan d’Appui aux Partenarire Cibles), ultimately, the activities that were chosen were those that responded to SNGP’s program; specifically, those that were measured by SNGP’s PMP indicators. When asked about the criteria that SNGP used to select ADR activities for financing, the response was, “There is no set criteria per se.” As the above quotes demonstrate (e.g., lack of concern for ADR interests), this was not well-received by the ADRs – and similarly the CADDs – nor was the fact that SNGP would not provide the ADRs with their own funding to carry out their own identified activities. As the quotes also indicate, this showed a lack of trust and confidence by SNGP in the capacity of ADRs to manage funds that were intended to strengthen their ability to carry out their mandates—an eerily similar conception that underlies SNGP’s overall Theory of Change. The Local Implementing Partner Relationship The IPs were initially contacted by TT to gauge their interest in joining the consortium to bid on the SNGP RFTOP. Ostensibly, the two local IPs (GSP and WILDAF) were solicited because their organizations had near countrywide coverage and members in each of the targeted regions, which would allow TT to claim a readily-available infrastructure to extend its program quickly. According to the concerned partners, the proposal development process was an effective and cordial one with discussion on IP participation and eventual terms and conditions fairly-well fleshed out. The Team has reviewed both scopes of work for the local IPs and they seem fairly detailed with activities largely centered on training, document preparation and awareness-raising. The Team understands that sub-contracts were signed between TT and the IPs. However, it was not clear when they were signed or why both IPs were initially required to sign short-term contracts that were a month or months’ long, which have apparently since been extended to annual contracts. The Team did not receive the training plans for any of the IPs from TT but did receive a “Summary of Evaluation Findings and Proposed Action Plan,” dated February 2018, for one of the IPs and assumes that another was prepared for the other. The evaluation summarized the IP’s strengths and weaknesses by different categories related to IP / SNGP implementation capacity. It formed the basis for developing an IP capacity-building program for the future (remaining two years) and provided the IPs with the opportunity to comment on TT / SNGP’s evaluation findings. Staff in the IPs’ Bamako headquarters reported receiving training from SNGP, but it appears that most of it, like the evaluation itself, centered on ensuring that IPs understood TT financial, management, 18 | P a g e procurement and reporting systems so that they could be responsive to TT contractual and implementation requirements. This was once again compounded by the manner in which TT financed IP work plans, including both operational and program costs. One IP reported that it was forced, for a considerable period of time, to “pre-finance” its activities, which caused considerable stress and resentment and consideration by their board of directors of terminating their sub-contracts. Both IPs noted that, upon submission of their financial reports, including receipts, certain costs were rejected as unallowable. The receipts were never returned, and there was often no explanation as to why the expenses were deemed non-allowable. In one case, the IP has been waiting for over a year to get a response to its request for an explanation and the return of its receipts. The team met with IP staff in all three study regions. For the most part, the IP regional offices were either staffed by retired functionaries or volunteers. Several staff members reported receiving a CFA 75,000 monthly stipend and a computer, but were left to their own devices in terms of office space, transportation and communications. They reported receiving periodic training from SNGP, although the Team did not request a training plan for these regionally-based staff. These IP staff were expected to coordinate and organize trainings and awareness-raising activities (e.g., simplified budget training, women’s leadership training, public debates) in their respective regions. Like most joint, field-based activities, SNGP directly paid for all costs. This became problematic in a number of cases when TT / SNGP accounting or program staff were late in bringing per diems or related costs (e.g. training halls, training or transportation), leaving the IP in a difficult situation. Advocacy Committees (Comité de plaidoyer, Committees) were set up in each region by SNGP for the purpose of engaging Regional Councils over policy matters that were considered important to both civil society and the Collectivitiés. The Team’s interviews indicated that little if any funding for the Committees was made available by SNGP. The advocacy committees included members of both local IPs. From the Team’s discussions with members of the Committees who were also IP staff, it was clear that the Committees were largely dormant and that the IP staff were unmotivated to fully participate in them. SNGP Internal Management SNGP is primarily a high-volume activity-centered operation defined by many transactions and transaction costs, magnified by working in all eight regions and Bamako. Because SNGP not only manages and finances its own activities but those of its local IPs and beneficiary partners, SNGP staff reported being overworked and continually trying to manage the voluminous workload. This came out clearly in our interviews with SNGP staff. Added to this is the very hierarchical internal decision-making process and rigid financial, administrative and reporting requirements TT imposed on its many partners and beneficiaries. All of which has led to an unwieldly and often unresponsive management system which ultimately constrains SNGP efficiency and effectiveness. RELATED PARAMETERS AND CONTRACTUAL REQUIREMENTS AFFECTING SNGP’S APPROACH There are a number of contractual requirements that TT and its partners have undertaken which significantly impact SNGP’s approach to implementing activities and achieving project results. The following requirements are discussed in turn. SNGP Scope and Coverage: The Impact of a Countrywide Program 19 | P a g e Covering the whole of Mali would be a difficult task in the best of circumstances, i.e., a country at peace with easy access to all targeted program areas. However, this does not describe Mali. Mali is characterized by vast distances, low population density and difficult access to many regions, combined by a worrisome security situation, all of which make SNGP’s task of effectively implementing its program an arduous undertaking. Coupled with the significant number of activities SNGP has committed to implementing and its approach of directly implementing and/or paying the costs of project activities, whether its own or its partners, has led to what one KI called the “sprinkling” of SNGP resources over a great expanse of territory with little lasting impact to show for it. The consequence of this approach has been to spread activities throughout the country with little ability to follow-up or demonstrate sustainability. Question 9 of our Mini-survey asks: “As the security situation in Mali has gotten worse since 2012, overall, would you say decentralization and public financial management of resources improved, gotten worse or stayed the same?” Our findings show that more than half (54%) of all respondents said that decentralization and PFM in Mali has gotten worse since 2012 due to insecurity. This is one piece of evidence that the changing context of insecurity in Mali has rolled back prior progress on decentralization and PFM. Notably, the most insecure region the team sampled, Mopti, also had the largest percent of respondents who said that decentralization and PFM has “gotten worse” because of insecurity. The most secure region, Bamako, gave the least negative outlook for the impact of insecurity on decentralization and PFM. Furthermore, in our Mopti interviews, respondents noted that only two of eight Circles were secure enough to conduct SNGP (or any other donor-financed) activities. Embedded TAs in the North (Gao, Timbuctou and Kidal) noted that they were largely relegated to the regional capitals and, in the case of Kidal, the TA was currently stationed in Gao. The Impact of Regionalization Policy on SNGP The Mission’s decision to limit SNGP to the regional level with support largely aimed at strengthening ADRs and, to a lesser extent, Regional Councils, has had consequences, some predictable and some unforeseen. SNGP’s support to the ADRs has been relatively modest. Apart from the embedded TAs and some internal training for ADR staff, only a few programmatic activities have been financed by SNGP annually. In our KIIs and FGDs with ADR staff and embedded TAs, it was evident that the ADRs’ own funding was largely limited to covering staff costs and its day-to-day operations; ADR programs were, therefore, largely dependent on donor funding. Two activities that the ADRs were expected to become engaged in with the support of SNGP￾funding were resource mobilization, including studies and technical assistance, and support to Collectivitiés in the development of their respective Economic, Social, and Cultural Development Programs (PDSECs). Limited SNGP resources have, however, precluded much assistance in the area of development planning and budgeting, and SNGP has yet to make much progress on the issue of resource mobilization. To give an idea of SNGP’s limitations in supporting ADRs, CARE Mopti, which is managing the USAID-financed Harande project, told the team that SNGP had approached it to request financing for the Mopti ADR to conduct PDSEC technical assistance to three communes, which it did successfully. In fact, it might be a lesson that SNGP applies to addressing the limitations of its future support to the ADRs, i.e., working more closely with other donors, including other USAID projects, particularly the Community Engagement Project. Having said that, SNGP cannot be considered the sole champion for ADR effectiveness, nor the sole 20 | P a g e reason they succeed or fail; this is a GOM responsibility that has yet to receive its full commitment. Given the broad mandate of the ADRs as well as the significant number of Collectivitiés that they are expected to cover in their respective regions, their long-term solvency remains in question. An equally important consequence of regionalization on the SNGP program is the constraint placed on it to follow up on its regional initiatives (e.g., gender-related training, public debates and information dissemination) by moving down the Collectivity hierarchy to ensure that they have been internalized by those trained and, in turn, extended to other members of their communities and organizations. The lack of follow-up after training, particularly to ensure that any benefits and knowledge gained were shared and spread to sub-regional levels, was a widespread complaint heard from government (Regional Councils and technical services) and civil society organizations alike. The assumption that other initiatives, whether by government and/or civil society, would be able to ensure that SNGP training and TA at the regional level followed those trained to their Collectivitiés, does not appear to have held up. At this point, it is appropriate for the Team to provide its assessment of the performance of the 11 embedded TAs and how their role in the institutions can be improved. It is our conclusion that it would be unfair to evaluate TA performance in traditional terms, i.e., against detailed job descriptions, and their recruitment against a well-defined set of qualifications. It is true that the TAs all have extremely detailed job descriptions; the only task not included is solving the slow pace of decentralization in Mali. And, as far as we can determine from our discussions with the TAs, they were all recruited based on the same set of set of “generalist” criteria. None of the TAs was recruited on the basis of the needs identified by the ADRs themselves. And, as noted above, they were required, perhaps less so for the CADD Advisors, to negotiate their jobs with the ADRs. Most of them seem to have adapted reasonably well to what can only be described as a very poorly planned and executed assistance output to a key partner (the ADR). Perhaps the best way to summarize their performance is that after two plus years, they are still on the job and contributing to ADRs’ program—as effective as that has been. Their roles in the ADRs can only be improved through fundamental changes to both SNGP’s management and programmatic approaches. This is discussed in Frame C, following. The Appropriateness of SNGP Components While the Team has yet to discuss Component 3, Support to CGSP (see Part C, following), it is clear from our interviews as well as from the mini-survey that this component is neither well￾understood nor well-known, and has not had much impact to-date at the regional level. Of all the three components that comprise SNGP, this one seems the least relevant to making regionalization work, particularly given the relevance of the overall Theory of Change itself. This issue will be fully addressed in the following chapter. The Team believes that the issue of ADR sustainability is itself up for discussion in terms of its appropriateness to SNGP programming going forward. Not only are the ADRs not well known – a finding made clear by KIIs, FGDs and Mini-Surveys – but there is no indication that the political will exists to see them succeed. A significant number of respondents felt that the ADRs were just another “transaction” cost in an already financially unsustainable structure of territorial Collectivitiés that do not receive adequate financial transfers from the central state and have, for the most part, not been able demonstrate their ability to mobilize their own resources. Finally, the GOM’s policy of increasing the number of regions beyond the ten that now exist is not 21 | P a g e consistent with its financial capacity to sustain them, and would seem to fall into the category of a lack of political will. Whether or not this advances decentralization objectives will be discussed in greater detail below. PRINCIPAL CONCLUSIONS: TETRA TECH’S APPROACH TO SNGP IMPLEMENTATION There is clearly a fundamental problem with TT’s approach to managing SNGP, especially in its relations with partners. This includes both local IPs – note that there has been no discussion about the international IPs – and beneficiary partners. The Team would like to reiterate that the preceding findings were widespread and profound. Each of our methodological tools reinforced one another through the corresponding analytical optic used. We give sincere kudos to TT and its partners for their high energy and commitment to achieving project objectives and for navigating through the perils imposed by the external context. And, it should be noted that their task was not made easier by the less-than-optimal design which they were handed to implement. Having said that, the evidence has been overwhelming, indicating that a major course correction is in order concerning the way TT moves forward in the out years of the project, and the Team is confident that SNGP can change its approach. TMG was not privy to TT financial accounts, so our ability to fully assess SNGP’s “cost to benefit ratio (Q1.3: low cost, big impact)” is somewhat limited. Similarly, KI informants were, for the most part, unable to answer this question. However, from the above analysis of both TT’s management and programmatic approach, it would be safe to infer that SNGP was not a very cost-effective endeavor. Because there was no empirical way to measure impact – all PMP indicators were output-oriented – it not possible to discuss impact in any meaningful way. The issue of measuring SNGP performance and progress is discussed in the following Analytical Frame. PRINCIPAL RECOMMENDATIONS: TETRA TECH’S APPROACH TO SNGP IMPLEMENTATION Step 1: Clear the Air, Start Afresh The Team strongly recommends that this evaluation report serve as a tool to get SNGP moving in a new direction, both in terms of its management approach and in how to reorient its programmatic focus. The Team therefore recommends a strategic moment of reflection (SMR) retreat for all SNGP partners where all of the relevant issues can be discussed (several of which are to be discussed in the next section) in a neutral setting where participants can air their concerns frankly but civilly. The Team would recommend to TT that it acknowledges that mistakes have been made, not intentionally, but as the result of pursuing project results and a mandate beyond its manageable interest. Step 2: Demonstrate Change from the Start First, SNGP should immediately transition to direct funding of partners and show that TT has confidence in their capacity to responsibly manage the funds planned for their strengthening. There is no restriction imposed by USAID in this regard, as several other USAID-financed projects (e.g., FHI’s Community Engagement Program and CARE Mali’s Harande project in Mopti) are currently managing their own funds. Of course, there are risks in this type of approach, but the evaluation team thinks the rewards are greater. There are a range of financial management systems and procedures that reduce risk and give partners a sense of ownership. In fact, in the early 1990s, USAID / Mali promoted the notion that the flip-side of empowerment is 22 | P a g e accountability. No one is asking TT and partners to expose themselves to the impact of poor IP and beneficiary governance practices, but rather that it promote with its partners accountability, transparency and participation as principles of sound program management. Second, SNGP should develop joint work plans which identify ways to incorporate partner priorities—a key element in promoting true partnerships and, more importantly, achieving results. This will result in some significant programmatic changes, largely driven by TT and its partners, that will allow for a new, more relevant and feasible activities to address the problems facing Malian decentralization. This will allow for better planning and resource allocation and greater participation in and ownership of project results. And, at a minimum, it will reduce the cost-benefit ratio significantly. We strongly suggest that TT move away from an indicator-output￾activity-driven approach to one that better ties outputs and corresponding activities to higher￾level intermediate results that demonstrate actual change generated by the program. This is further discussed in section III. Third, there is a desperate need to reduce the scope and coverage of the SNGP program – to go deeper and less broad, and potentially piloting new ideas for scaling up. According to the mini-survey, when respondents were asked, “What is the biggest challenge faced by SNGP?”, the second largest percentage (31%) of respondents said the project was, "Trying to do too many things, and not focused enough on the key interventions needed." The team recommends that, at a minimum, SNGP should move out of the three Northern Regions and consider whether Mopti itself should remain among the new focus of activities. To achieve some semblance of impact, SNGP needs to reduce the number of activities it supports in fewer regions. Fourth, SNGP needs to consider ways of accompanying its training, technical assistance and awareness-raising initiatives, many of them worthwhile and appreciated, to the Circles and Communes. By working on a more focused set of activities in fewer regions, additional resources should be freed up to make this possible. In addition, there are other USAID-financed projects, and others supported by donor partners, that would complement SNGP’s regional efforts. Specifically, the Team refers to the Community Engagement Project managed by FHI, whose mission is to strengthen community-based organizations (e.g., ASACOs, CESCOMs, etc.) to better engage local government on policy issues through improved advocacy capacity, as well as holding them accountable for their performance through an enhanced oversight role. While FHI has indicated that it has been in talks with SNGP for a while now, it may be an appropriate time for both USAID IPs to sit down together and undertake a two-year joint planning exercise. Finally, the Team strongly recommends that the Mission consider creating an Ombudsman position, whether internally or externally, which can provide affected stakeholders with a means of raising their concerns about the impact of USG-funded projects on their organizations and the wider communities they often represent. We realize that USAID procurement instruments (grants, cooperative agreements and contracts) are designed to shield it from third party complaints, but situations such as those described herein should and can be avoided with early detection or later through mechanism that allows impacted partners to express their concerns. C. FRAME III: SNGP PERFORMANCE AND IMPACT Each of the three SNGP components is discussed in this section, including principal findings, conclusions and recommendations. RFTOP questions addressed in this section include: Q1: 23 | P a g e Efficiency, Efficacy and Appreciation of Activities: What activities have been most effective in line with the mandate of the project? Q1.1: building the capacity and/or improving performance of sub-national entities; Q1.2: appreciation by beneficiary (SNGP, GOM or civil society), Q1.3d: advancing a key issue such as transparency, gender, PFM, etc.; and, 1.4: In terms of cost-benefit (low cost, big impact)? Q2.1: What results have been achieved for each component of the project? Before beginning the analysis of these components, an overview and analysis of SNGP effectiveness and efficiency is presented. Q6 Gender: How is the program’s gender integration strategy and plan progressing in terms of capacity building, awareness raising and lobbying, technical assistance to increase women and youth participation and inclusion and synergy with other stakeholders? PRINCIPAL FINDINGS As a general finding, activities under Component 1 and Component 2 were the most appreciated, with awareness-raising activities having placed the highest, including: Public Debates and Training on Budget Process and planning (PDSEC), the Awareness Caravans undertaken with the ADRs, roundtables, radio broadcasts and culture of accountability initiatives. There was also considerable approval by both ADR and CADD personnel of the usefulness of IT and software application training. It is somewhat surprising, however, to see the relatively low scores for women’s capacity-building on rights and duties but perhaps not so much for the embedded TA for ADRs and CADDs, given the issues noted above. Not unexpectedly, there is little appreciation for Component 3 activities, as they are little evident to almost anyone interviewed (KIs and FGs) in the regions. SNGP is a project whose activities were designed to change the behavior of those responsible for the planning and management of financial resources, and those directly affected by these critical functions. SNGP wants government, both elected leaders in the Collectivitiés and central state actors and their deconcentrated services in the regions, to demonstrate increased transparency, accountability, responsiveness and inclusion in the way they plan and manage financial resources; and, to a more limited extent, citizens and local community organizations to increase their participation in these processes, including improved oversight. The awareness-raising activities detailed in components 1and 2, particularly the public debates, are major drivers of this kind of behavior change, which is evidenced by the high level of respondent appreciation for them. Capacity-building also has the aim of influencing behavior, although it seems to be somewhat less appreciated in this regard, particularly from the Team’s qualitative interviews. 24 | P a g e As the following table shows, those who know about the SNGP (and thus likely have benefitted from it), appreciated the capacity building they received: 76 percent of respondents thought the capacity building/training they received from SNGP was relevant and enabled them to make a positive difference. The principal government beneficiaries of SNGP capacity-building are the ADRs and CADDs and, to a lesser extent, the Regional Councils and the Comité Régional d’Orientation de Coordination et de Suivi des Actions de Développement (CROCSAD). On the non-governmental / CSO side, women’s groups and regional level networks were a principal target of SNGP capacity-building interventions (e.g., leadership training, planning and budget, roles and responsibilities); civil society more generally, particularly at the Commune level was rarely engaged at all. Outside of IT training and software provision, the team saw little actual strengthening of the ADRs or the CADDs based on the analysis derived from out KIIs and FGDs. While capacity￾building is certainly a need for any organization, ADRs have far more pressing problems than strengthening their skills and gaining greater knowledge. From our interviews with CADD TAs, we did not get the impression that these planning units were actually receiving much internal support from their host ministries. While there were a number of training and TA activities directed towards the Regional Councils (such as the development of regional planning documents), SNGP interventions were relatively modest and periodic, which is partly due to their (RC) infrequent meetings. The Advocacy Committees established by SNGP, which were intended to play a significant role in policy dialogue at the regional level, received little if any project support, including capacity￾building, which could explain why they are largely dormant. As for the CGSP, it is clear that a significant investment in capacity-building was undertaken. The question, discussed in more detail below in Component 3: CGSP Strengthening, is whether this was the right investment to improve PFM in the regions, and whether it can sustain its functions post-project. A summary of the status of each of the three project components and SNGP’s performance relative to the expected results is provided below. Component 1 – PFM Systems Strengthening & Component 3: CGSP Strengthening Mini-Survey Q8 - The CGSP capacity to monitor public spending and service delivery has been significantly improved at the national and sub-national levels on account of the SNGP Frequency Percent Question 8 responses Relative to other mini-survey questions, responses to this one is not as positive. Only half of the respondents agreed that the CGSP’s capacity to monitor public spending and service delivery has improved because of SNGP. It is likely that many respondents may feel this statement to be a bit of a stretch, particularly as they are well aware of the weak nature of PFM and the lack of quality service provision throughout their country. It is more likely that respondents had only a notional understanding of this central state agency and its functions. 49 50% Agree 76 26% Disagree 57 19% Not clear 8 3% Strongly Disagree 5 2% Strongly agree 1 0% No answer 296 100% TOTAL 25 | P a g e Component 1 was designed to provide technical assistance to strengthen overall budget development and PFM systems in Mali. According to the TOC, achieving this will ensure that financial transfers from the central state to the Collectivitiés will lead to improved management (and, implicitly, decreased corruption) of these transfers and thus greater effectiveness in delivering public services at the local level. When this is achieved, the central state will have more confidence in the Collectivitiés and it will ensure more timely and complete financial transfers. Since regionalization is a new part of the decentralization process, the capacity-building interventions are aimed at helping to establish regional structures focusing on institutionalizing systems and administrative processes and strengthening the concerned enabling environment. While Component 3 has its own distinct objectives, that is, strengthening CGSP’s capacity to conduct performance audits and monitor public spending and service delivery at the national and sub-national levels, it is also designed to enhance and complement the overall system of PFM in Mali and is, therefore reviewed and assessed together with component 1. The evaluation team’s assessment of both these components is that, given their intent to collectively enhance PFM at the regional level through existing institutions and structures, it appears that SNGP’s aim, based on the project design, was not on target. First of all, what regional structures were actually strengthened in PFM? The principal emphasis has been to strengthen the regional level ADRs whose mandate does not include improving overall budget development and oversight systems to ensure that financial management of Collectivitiés conforms to the country’s relevant policies and practices. In addition, embedded TAs in the CADDs were tasked with improving the reliability and consistency of financial information between the Ministry of Economy and Finance (MEF) and the line ministries (particularly in education and health), as well as between the national and subnational government. Whether or not the TAs have improved the effectiveness of the CADDs, our assessment is that the CADDs have not fulfilled this function as the units are largely marginalized within their ministries. Nor, as far as we have been able to determine, were the Ministries of Health and Education (MOH and MOE, respectively) provided with any assistance to diagnose their current sectoral PFM challenges and to develop a plan for their remediation. Apart from its support to the “supply-side,” SNGP’s emphasis has been on strengthening the demand-side of the equation, that is, through the training and awareness-raising of civil society (e.g., women’s associations), with some efforts made to improve the CROCSADs’ and Regional Councils’ understanding of PFM principles and practices. What structures were not strengthened in PFM? All of the Regional- and Circle-level deconcentrated services and their personnel discussed in Section I, whose mandate is precisely to ensure Collectivity finances are overseen and controlled. Rather, the emphasis was placed on 26 | P a g e strengthening the CGSP’s capacity to audit national and sub-national budgets and their PFM systems. While a few such audits have taken place in several regions, the overall demand for such services is high (ten regions) and the overall capacity, including resources available to the CGSP is very low both on the technical and human resources side. Component 2: Improve Coordination and Communication between National and Subnational Government and Civil Society The hypothesis underlying this second component is that with better communications and coordination processes between the national and sub-national governments and civil society, there will be an improved capacity for them to undertake harmonized budget planning and execution while civil society will have an increased capacity to effectively assume its watchdog and advocacy roles. There are a several problems with this hypothesis. The first one is that civil society below the regional level was not systematically targeted for strengthening under SNGP to undertake the aforesaid watchdog and oversight functions. While we applaud SNGP for a wide range of activities under component 2, such as training of CSOs in budget simplification, holding public forums on budgeting processes and supporting structures such as the CROCSADs to promote cross-sectoral communications and coordination, the Team did not see a viable pathway downward to the communes to extend and institutionalize these worthwhile activities. This was discussed in Section I: our review of the TOC assumptions and the fact that the statement of work for SNGP made it clear that was outside the project’s design parameters. One of our principal findings related to this component is the ongoing and subterranean conflict that exists between the deconcentrated central state administrative and technical services and the elected Collectivity Councils and their responsibility for the delivery of public services. As we noted in Section B above, our findings made it clear that the deconcentrated state at the regional level, politically and administratively led by the governor, was a far more powerful actor in regional governance than the elected president and her/his Regional Council. From a legal standpoint, the governor is not intended to be, a “development” actor. However, his/her administrative control over the technical services in the deconcentrated “governorate,” which includes deconcentrated ministries and the various inspectorates responsible for collectivity oversight in financial matters, gives him/her an outsized a say in regional development decision￾making and implementation. According to the relevant laws on decentralization, the deconcentrated technical services are to support to the Collectivitiés in their mandated role to deliver public services. Our interviews have clearly shown that these deconcentrated services are, in most cases, loathe to take orders from the Collectivitiés. We do want to acknowledge, however, the significant work that SNGP has done to improve coordination between national and regional level governments and with regional level civil society. The principal problem facing decentralization in the subnational context is the lack of coordination and communication between these institutional actors. This is the elephant in the room that no one wants to talk about, but the resolution of this impasse will be one of the principal indicators determining the success of regionalization. Someone needs to address it: why not SNGP? GENDER MAINSTREAMING AND WOMEN’S EMPOWERMENT The Team found that gender mainstreaming had been incorporated into the project’s design. This was witnessed by: 1) The existence of a gender equality analysis document and a Gender Integration Plan to guide SNGP staff and partners; 2) The contracting of international and 27 | P a g e domestic partners (Banyan Global and WILDAF Mali, respectively) for the capacity-building of local authorities, civil society and media organizations on gender issues; 3) The inclusion of skilled men and women, including a gender officer, on the SNGP team; and 4) The requirement for women to be included in all training activities, and specific activities exclusively for women. Respondents noted that the following activities were the most successful at including women: 1) Awareness-raising of Law 052 (mandating that 30% of elected positions be reserved for women); 2) Training women on planning and budgeting process at the Collective on the regional level; and 3) The use of radio debates to educate women about their rights and responsibilities. While respondents said that SNGP had helped increase women’s participation and capacity, many questioned the quality of their participation and noted a lack of follow-up or sustained involvement, one of the impacts of SNGP’s regional-only focus. “Women participate in quantity, but not in quality – they don’t participate in the meetings due to their weak grasp of the subjects and their low levels of education”. Many respondents also attributed the increase in the participation of women to “democratization” rather than SNGP. Respondents did not see a measurable increase in participation by youth, perhaps because youth participation was not deliberately written into the project design as women’s participation/gender integration was. “Young people do not participate as much as we expect them; only a few participate in public trainings and debates.” PRINCIPAL CONCLUSIONS At the start of our concluding remarks, the Team wants to acknowledge that SNGP has had significant success and has demonstrated its effectiveness in delivering a wide range of well￾regarded activities, primarily around Components 1 and 2. They have increased awareness among citizens and CSOs concerning their rights, including participation in the planning and budgeting process, and have the potential for lasting empowerment for all citizens, particularly women, youth and civil society. On the government side of the equation, the appreciation and effectiveness of SNGP interventions, while acknowledged, were assessed by the team as having less impact than would have been expected. The challenge in the remaining two years of the project is how to make those activities that were highly-regarded more than one-off initiatives that dissipated as they moved down the Territorial Collectivity hierarchy. For instance, what does it mean that 1,500 women were trained in budget preparation and analysis at the regional level, but were not supported in disseminating this information in their Communes and communities? The retention of information gained through capacity-building and awareness-raising initiatives could not be verified, but our interviews indicated that this was a significant problem. More importantly, the application of this new-found knowledge and awareness of the planning and budget process, for instance, was largely left to those trained; but again, there is no indication that it was either sustained or widespread. In terms of a results-based management approach and analysis, achieving outputs like the number of women who were trained or the number of days of technical assistance delivered leads to the question, so-what? 28 | P a g e As discussed below, part of the problem with assessing SNGP performance in terms of effectiveness and efficiency emanates from the project’s PMP and the modest measures of achievement that the indicators were formulated to report on. This is in addition to the significant problems already noted with the countrywide scope of the project; its transactional nature, including high transaction costs generated by so many activities spread out over such a large operating area; and the choice of activities driven by a Theory of Change that the Team deemed inconsistent with the actual problems constraining effective decentralization; or, whether regionalization itself is a justifiable policy meriting support. In summarizing our conclusion about TT and partners’ performance in implementing SNGP, it must be noted that many of the constraints facing the achievement of SNGP objectives were baked into the design and are issues over which TT and partners had no control. Thus, it is the Team’s considered view that it is more productive to look toward the future and provide some recommendations on the way forward. PRINCIPAL RECOMMENDATIONS: TT SHOULD REVAMP ITS PERFORMANCE MONITORING PLAN The team’s first performance-related recommendation is that TT should revamp its performance management plan (PMP), so that the outputs shown in the current PMP are tied to higher-level or intermediate results (IRs) that would actually demonstrate a change in condition (which is what a result is supposed to show). Attachment A of the SNGP RFTOP Statement of Work provides a set of higher-level indicators for each component that could be used in a reconfigured PMP. The current set of outputs could then be tied to one or more of the Intermediate Results (IRs) derived from the SOW. As we noted above, SNGP, its partners and USAID should conduct a mini strategic planning exercise that would examine how to reconfigure the current SNGP program while trying to retain those elements of the current program that do advance Malian decentralization. The process could look at a new two-year Theory of Change that is not inconsistent with the Mission’s commitment to support Malian decentralization. There are some hard questions to answer however, as to what should stay and what should go, which leads to a second set of recommendations. THE MISSION NEEDS TO DECIDE WHETHER TO CONTINUE WITH ITS SUPPORT OF THE GOM’S REGIONALIZATION POLICY In a perfect world, and looking over the horizon, the Team’s recommendation would be to disengage, but this is not a perfect world and politics and diplomacy are an integral part of it. So, our recommendation would be to keep supporting it, but with a new orientation. 29 | P a g e The new orientation requires answering the question that faced the designers during the conception stage of the project: “Is SNGP is a decentralization project or PFM project?” It is trying to be both and doing neither of them very well. As a PFM project, there is no justification for supporting the ADRs, and those resources could be better utilized elsewhere. As a decentralization project, there is no rationale for supporting CGSP while not supporting civil society in the communes. The Team’s view is that with some (major) tweaks, it may in fact be possible to do both, but with a new rationale and one harkening back to the pre-crisis days of USAID’s support to Malian decentralization, but with the advantage of the acquired wisdom that comes with time and the acceptance of mistakes. A new mini-strategic or development objective might read something like the following: Deconcentrated state administrative and technical services, elected collectivity councils and civil society organizations are effective partners in democratic governance, including development planning and public policy making, at the regional level and below. It would include a number of components that address the constraints to effective decentralization noted throughout this evaluation: Component 1: The Deconcentrated State The deconcentrated administrative and technical services would support devolution and the collectivity councils that are responsible for, inter-alia, the effective delivery of public services, a task that is consistent with the constitution and related decentralization texts. In terms of oversight functions, the concerned technical services (e.g., Inspectorates) at the regional and Circle levels that are responsible for the oversight of Collective budget expenditures would be supported with SNGP technical assistance and encouraged to do the jobs for which they were created. This would obviate the need for the current Component 3, CGSP strengthening, because the locus of the project would change from the national to the subnational level where it would have logically been placed to begin with. The line ministries (e.g., Health, the Education Academies, etc.) would reorient themselves to support the technical needs of the Collectivitiés as they develop and implement their development plans. The governors would be encouraged to stay out of the business of development and ensure that the Collectivitiés adhere to the country’s laws and policies, including those addressing effective public financial management, which are the governors’ actual mandate. Component 2: The Elected Collectivity Councils The primary responsibility of the Councils is to ensure the development of their Collectivitiés’ development plan (PDSEC) / budget, to see it through to approval, ensure that resources, including government transfers, are mobilized to implement it, deliver the services that are under their mandate and consistent with the PDSEC and then monitor the ongoing programs. SNGP would provide financial support and technical assistance indirectly to strengthen Collectivitiés’ 30 | P a g e capacity to plan, manage and monitor their development plans. A principal element of this component would focus on developing and implementing Collectivity resource mobilization plans and strategies, a task for which SNGP is already responsible. The ADRs, with additional direct support, including relevant activities are well-placed to do this. Component 3: Strengthening Civil Society as an Effective Partner in Sub-national Governance This component is consistent with the SNGP’s SOW but was retracted later on in the document. Civil society has both a demand and supply-side function to play in regional governance. As initially conceived in the SOW, the demand￾side role included a watchdog and oversight function as well as the capacity to engage Collectivity Councils and their executive bodies in policy making and decisions on Collectivitiés’ development priorities. The supply-side function dates back to USAID’s first generation DRG program that targeted community-based organization (e.g., ASACOs, CESCOMs) as the principal participants in the management of, inter-alia, local health and education facilities in the Collectivitiés. This describes the CEP project, which FHI 360 is now managing in many of the same regions where SNGP works. This third component would continue to give prominence to women’s associations and also support CSO regional and thematic networks that would act on behalf of local-level CSOs vis-à￾vis Collectivity councils and their executive bodies (e.g., the mayor) in the areas of policy advocacy and oversight. Each of these three components is consistent with the dual purposes of SNGP, 1) to promote improved sub-national PFM practices; and 2) to reinforce effective decentralization through the new regionalization policy. There are a number of specific recommendations that are consistent with the current program and with some minor modifications could enhance the likelihood of the next two years being a bridge to a new DRG program. They are: Reduce the number of regions in which SNGP works to no more than six and preferably five. This would permit a reallocation of resources to a targeted number of regions that could demonstrate an effective decentralization program; Graduate CGSP and reallocate funds to building regional level PFM (financial oversight) capacities; ADRs can maintain their role in supporting Collectivitiés to develop their PDSECs and budgets and also focus on resource mobilization, research, training and technical assistance. With additional resources available to SNGP, it should be in a better position to support the smaller number of targeted ADRs it will now be working with. GSB and WILDAF should take the lead in civil society-strengthening and bring in other CSOs with specialized expertise to support them. SNGP needs to support them accordingly. 31 | P a g e SNGP and the ADRs will need to reengage on the role of the embedded TAs, but one of the principal functions they could undertake, which most ADRs have expressed interest in, is developing realistic resource mobilization strategies. Reorient the communications and coordination component to promote a tripartite partnership for improved subnational governance among the above-mentioned partners. In this regard, linking mechanisms such as CROCSADs, rejuvenated Advocacy Committees (comites de plaidoyer) and other bridging mechanisms should be supported to smooth out regional level relationships that are currently impeding effective decentralization to take place. As discussed in the previous section, we strongly encourage the Health and Education teams and their implementing partners to engage with SNGP following this evaluation and its recommendation for a reconfigured SNGP program, to see how they can better work together to achieve the outcomes that both teams had anticipated. In this regard, we recommend a strategy that has both sectoral teams and SNGP working in the same Circles, if not Communes, where sectoral funding can serve as an incentive for concerned Collectivity Councils to develop and implement good development plans with the understanding that a modest amount of program or project funds will be set aside to fund some of their principal development priorities—a principal target would those community-based organizations that are already responsible for the management and oversight of health and education service delivery. In summary, this mid-term evaluation has laid out the principal findings, conclusions and recommendations that are intended to provide a strategic context to give SNGP the best chance of succeeding and achieving results that have realistic chance for demonstrating development impact. V.SYNTHESIS AND OVERALL CONCLUSIONS This evaluation was both a challenging and rewarding assignment for the TMG Evaluation Team. TMG deployed highly-experienced members with the right mix of skills and expertise to address the RFTOP SOW. We thank both the Mission and TT and its partners, who were extremely supportive of this evaluation, an indication of the high priority they have given to it. In return, we have provided one of the most evidence-driven exercises that TMG and each of the SNGP team members have participated in. The Team recognizes that this evaluation contains some rather pointed findings. The intent was not to cast aspersions on the lead organization, but rather to provide an objective view of the dynamics of this project as it has evolved over the past three years. If we did not believe that TT and its partners were committed to improving decentralization in Mali, our conclusions and recommendations would have been significantly different. We highlight, again, the fact that TT was handed a project design that was based on what we consider to be a faulty Theory of Change and, in many ways, TT and its partners did the best that they could with the cards they were dealt. We would like to point out that the recommendations made herein should not be considered the basis for the design of a new project. USAID and the TT consortium have some hard choices to make, but their resulting decisions should lead to a reconfiguration of the underlying purpose of the project, which is to support decentralization through regionalization and improved subnational governance. Even in the best-conceived projects, there will be components or elements that do not work or fit an evolving context—failure is only a failure if the lessons it 32 | P a g e imparts are not used to improve the situation. The majority of this mid-term evaluation’s recommendations came directly from SNGP stakeholders, and it was the Team’s job to put them in an analytical framework that packaged the massive amount of data collected from them into a coherent narrative of findings, conclusions and recommendations for use in moving forward to improve the project in its remaining time. Page | 33 Annex 1: TMG Responses to USAID Mali Comments on Draft Evaluation Report 1. Under point C, it is noted that technical services (heath and education etc.) had little understanding why the SNGP financed technical advisor working with the ADR was coming to their offices to collect data, so was SNGP's relationship with these technical services was just limited to this activity or did they receive any other support from them? Our summary response to this question is that the only contact between the technical services (e.g. health and education) and SNGP has been through the CADD at the ministry level. There has been no direct substantive contact at the regional level between either the regional Academies or the Directions Regionale de la Sante and SNGP. We note that the more detailed response below is based on information provided to the TMG evaluation team by, inter alia, SNGP-financed Technical Advisors themselves during our KIIs and FGDs. The following are examples of the contacts, or lack of contacts, between SNGP and regional level technical services: (a) According to an SNGP supported TA: “In fact, SNGP is not well known to the institutional landscape of the public administration of Mali, for lack of communication on its missions and objectives with the decentralized services of the State. So, this situation means that the deconcentrated services are reluctant to provide them with public data that do not know how [they are going] to use it. This is why the SNGP TAs are always accompanied by their ADR colleagues to facilitate the collection of data from these decentralized services.” Also, according to the key informants in the health sector (the Director of the Regional Director of Health / DRS of Sikasso and Mopti), and also the planning officer of the DRS of Koulikoro) “with SNGP we did not benefit from technical or financial support.” (b) One of the TMG Team members who interviewed several service sector key informants, asked if there was specific SNGP TA support for the deconcentrated education services. Regarding this question here is what the evaluation Team member stated: This problem was raised during one of my KII in Mopti. It turns out that, in search of information on behalf of SNGP, the TA goes into the services for the collection of this information, but the services are not always available to do this. The key informant said: “The [SNGP] Technical Assistant to ADR Mopti has been awarded by SNGP in 2018 for her performance in fulfilling the tasks entrusted to her. However, the Director of ADR is not at all satisfied with the role assigned to the SNGP technical assistant in the ADR and for good reason: she does not work in accordance with a specification, she does not participate in the mobilization of the resources of the communes, [she] has no decision-making power, her work focuses on collecting data from the deconcentrated technical services, the information which the SNGP needs and wants"; It’s possible to understand the relationship of the TA and the Technical Services through the words of this key informant from Bamako, who, in answering the question of whether the assistance of ADR by the TA is necessary, said, “If the TA [was] enterprising and [had a] sense of initiative, it would make more sense. The TA must understand that he has been secunded to the ADR to work [in support of the ADR].” Page | 34 Thus, is it critical that the TA’s workplan and purpose serves to support the needs of the ADR, best defined by the ADR. 2. Was TMG able to review the budget and contract conditions for all the sub￾contractors? If so, was there any analysis? We asked for the subcontracts / conventions for the IP’s but were told that TT would need to refer to their headquarters for approval. Given how long it took to get this response (two weeks), we only asked for the scope of work for the local IPs WILDAF and GSB. We did by accident, get a copy of MSI’s contract, but not budgets. Following are the communications exchanges we had with TT on this matter: (TMG) Does each of the member of Tetra Tech’s IPs have a convention and/or terms of reference, including the specific activities they are to undertake. Could we get copies please? If the specific activities are found in the annual work plans, can we get copies please. (TT) Yes, Tetra Tech singed a sub contract with all IPs. The role of each IP is identified in the contract. During annual WP preparation Tetra Tech works with all of them to identify activities. FO does not have copies of the subcontracts with international IPs. For local IP we may have to ask HO approval to share them with you. (TMG) we do not need to see the entire contract just the terms of reference (TT) We do have at FO level term of reference for our Local subs and could share with you. So, in direct response to the above Mission question, the answer is no, there was no review by the evaluation team of SNGP member budgets and therefore no corresponding analysis. This was due to not being able to get this information from SNGP; a recurrent problem that TMG encountered throughout the evaluation. As we noted in the body of the evaluation report, both local IPs were responsible (for varying periods of time) to “pre-finance” their own costs and then request reimbursements, a number of which were deemed unallowable by SNGP, often with no explanation. As discussed in the evaluation report, the IP job descriptions for both WILDAF and GSB, were largely identical, and required of them a wide range of responsibilities. Also, as we noted from our interviews with the sub-IPs, their goal posts and responsibilities often kept changing, yet any positive results were often appropriated by SNGP. 3. Was TMG able to interview representatives from MSI and Banyan Global? This would be helpful. Yes, we interviewed both MSI & Banyan Global and these interviews were included as part of our qualitative analysis. 4. On page 26, The awareness-raising activities detailed in components 1 and 2, particularly the public debates, are major drivers of this kind of behavior change, which is evidenced by the high level of respondent appreciation for them. Capacity-building also has the aim of influencing behavior, although it seems to be somewhat less appreciated in this regard, particularly in the qualitative interviews. • Who are the actors that have experienced behavior change? How have citizens benefited from the change? Unfortunately, as noted in greater detail below, it is not possible to determine with any certainty who, if anyone, experienced notable behavioral change on account of the project Page | 35 for two reasons noted in the report (please see, inter alia, pages 27 and 28 of the draft evaluation report): 1. SNGP indicators were only at the output-level, measuring only the number of days of SNGP TA delivered, numbers of persons trained, numbers of public debates held, etc. The SNGP PMP and the indicators that were measured and reported on simply gave no understanding of whether these actions / activities actually led to behavioral change. Which is why one of our recommendations was to revise the PMP in order to actually measure meaningful change (intermediate results), including behavioral change. 2. SNGP support to those trained or benefited from other interventions (e.g. public debates, women trained) ended at the regional level. Because the great majority of SNGP activities ended at the regional level and did not extend down to the commune level – where citizens reside – SNGP was unable to ensure that its interventions related to awareness raising and capacity building were shared and diffused with commune /community members. Apart from the lack of real measures of behavior change, we did come across qualitative evidence during interviews that many of the awareness raising activities – particularly the public debates and caravans – seemed to be highly appreciated. This level of appreciation may be assumed as a sign of a possible link to behavior change, but this cannot be certain without real indicators measuring such. • It will be good to elaborate more about the changes made by the intervention and support the positive appreciation by examples. Many of the beneficiaries of Components 1 and 2, that is, participants in training activities, public debates, etc. – those trained in the Simplified Budget by GSB, Women Leaders trained in the development of the communal development plans (PDSEC) and budgets – were all appreciative of these activities, but that is different than demonstrating behavioral change either among themselves (perhaps it did, but it was not measured and if it was not measured then, in RBM terms, it did not happen) or among citizens far removed from the regional level who may or may not have received a restitution from those benefiting from SNGP awareness raising and capacity building. Having said that and in fairness to SNGP, we do have several examples of Key Informants who have positive examples of their experience with the effects of SNGP interventions: According to a key informant, "this is the first time that all stages of the preparation of the original budget have been respected with the effective participation of all the actors". Another informant said, "As for the return of the administrative account of the mayor, it is done regularly for years in our commune, even if at the beginning there was not much enthusiasm, but now it became a reality and each village is represented by the first officials, in order to make a good restitution on return." Support of the DNCTP for capacity building of deconcentrated financial services and the own services of some CTs in the northern regions of the country on the COLLOC software (budgetary and accounting software for CTs). According to a key informant in Bamako: "thanks to this software, the Public Accountants' Accounts (CG) are produced in time and of better quality"; Page | 36 Finally, another key informant said "SNGP's support has greatly contributed to improving the quality of the activities of Koulikoro Regional Council (RC): the budget process, the realism of the budget where the budget forecasts of the Council budget have passed from 10 billion in 2018 to 17 billion in 2019 ". From these few examples it may be inferred that, in terms of behavioral change, the SNGP likely had some impact regarding raising awareness and understanding of the budget process which then may have led to improved budgets and budget processes in some cases. This being said, it must be kept in mind that these behavioral change examples were limited in scope and didn’t reflect the broad behavioral change that the project aspired to create. 5. On page 29, the report states that "We do want to acknowledge, however, the significant work that SNGP has done to improve coordination between national and regional level governments and with regional level civil society." Can TMG please elaborate more on the improvements? Per TMG, has the intervention improved the Coordination and communication between these actors? (Just for the record, the correct page is 27 of the submitted report) There are several examples of initiatives supported by SNGP in the area of coordination and communications among and between these actors that were noted in a number of KIIs: Among the most popular activities that were related to the team in KIIs were: • Support to the budget process of Regional Councils (RCs) and municipalities, one through ADR. This support had the merit of familiarizing the Collectivities with the steps of the process of drawing up their budget, the organization of a public debate on the draft budget in particular. According to a key informant from Koulikoro: "The support of SNGP has contributed a lot to the quality of the activities of the Koulikoro RC: the budget process, the realism of the budget, where the budget forecasts of the CR budget increase from ten billion in 2018 to seven billion in 2019, the public restitution of the CA"; • Support to the Public Restitution (PR) of the Administrative Accounts (CA) of the RC Presidents. PRs have created a constructive dialogue between RCs and civil society; • The support to an ADR in the drafting of a simplified guide for the elaboration of the PDSEC and the accompaniment of certain communes in the elaboration of their PDSEC; • Establishment of a consultation framework between the central control services in Mali (BVG, CGSP, SCCS). According to a key informant from Bamako: "the quarterly meetings instituted since the advent of the consultation framework, have not only helped to mitigate the rivalries between these services where everyone was jealous of his prerogatives, but also a better coordination between them in the conduct of control missions." • The organization of a meeting at the Hotel Kempeski regarding explaining to the public the missions of public finance management control structures in Mali. According to a key informant in Bamako: “the meeting that brought together SNGP, the CGSP, the OAG, the CSAS, allowed participants to understand that there are PS control structures, that monitoring missions are carried out, that reports are produced and forwarded to the competent authorities, that recommendations reports are applied by the Controlled Services. The audience was very interested and very much appreciated the discussions. These discussions and clarifications on pertinent issues concerning the public seem to have enhanced positive communications and enhanced (just a bit) public respect for and trust for government processes.” Page | 37 In general, as noted above and in the evaluation report, SNGP has promoted better coordination and communications through the public debates, its training with women’s CSOs to be able to better engage with regional governments and communes, and with its support to Regional Councils. While these examples provide a window into SNGP’s role in supporting Component 2, there were no PMP measures of communication or coordination improvements other than the number of TA days delivered, number of public debates, number of capacity building activities, etc. There were no SNGP PMP higher level indicators that would have noted an improvement in the quality of communications and coordination due to SNGP interventions, that is, a measurable change. It should be noted that during the KIIs many indicated that SNGP initiatives would have been more effective if they had been part of a concerted and coordinated strategy that was then measured for impact. All this being said, the larger issue that was articulated in the evaluation report about communication and coordination was not at the national level, but rather communication and coordination challenges at the regional level, and between the Regional Technical Services and their role and responsibilities to the collectivities at all levels. This problem was the principal one identified and one of the major reasons for the generally poor relationships that continue to persist between the deconcentrated administrative and technical services on the one hand, and the Collectivities on the other. 6. On page 30 under conclusions in the first paragraph: Has this increase of awareness among citizens has changed their behavior in claiming for more accountability from the elected authorities? If yes, please give some examples. What are the factors that have contributed to less impact according to Government? We clearly did not present this point as well as we should have. What we were trying to communicate was that SNGP was effective in undertaking several awareness raising and capacity building activities that were not targeted directly to broader level citizenry – but rather to more limited groups of beneficiaries, such as: women’s CSOs, or GSB (through the budget simplification training). We didn’t intend to suggest here that these activities had enhanced the capacity of citizens (broadly speaking) to engage Collectivity government at all three levels. This is consistent with the several earlier points made concerning TMG’s conclusion that SNGP did not work at the local commune / community level where the mass of citizens resides but tried to reach them through engagement through limited representatives that may have been trained (at regional level) in, inter alia, PDSEC and budget participation. Having said that, we do acknowledge that several SNGP activities, as previously noted, i.e., the organization of public debates, training workshops on the citizen's budget, leadership and gender can be seen as indirectly and potentially triggering factors for catalyzing citizens’ awareness of their roles and responsibilities in the management of local affairs, as some key informants testify: A key informant [not a community / commune member] said, "for us the budget was a question for economists and finance specialists, but with the accompaniment we received [from SNGP], we now have a good understanding of the budget." Another key informant added "since we understood that our contribution to budgeting is paramount, we mobilize and encourage our representatives to actively participate in all stages of budgeting.” Page | 38 7. Does TMG have any data/observations regarding citizens understanding of the complex decentralization process? Could this be an activity- general awareness raising about decentralization- that SNGP increases in the options years? There were no specific questions asked of the public regarding detailed knowledge of the decentralization processes in the public perception survey. The Team did ask the public if they had a rough idea of how much money was transferred in the past year from the central government to their local CT, and 95% responded that they did not know. Since this is a critical piece of information required for informed advocacy, it can be understood that citizens are not armed with even the most basic information they need in order to fulfill their civic responsibilities around PFM and decentralization. In the public perception survey, citizens were also asked if they knew what the money transferred to their local level CT was supposed to be used for (i.e. in terms of local development). In response to this question, only 15% of the public knew what the transferred money was supposed to be utilized for in their local community. This data point also reinforces the finding that the public doesn’t have the most basic information they need to effectively advocate for quality PFM and decentralization in their communities. A number of SNGP activities did address issues of decentralization (e.g. dissemination of relevant texts), but not in an organized way, with a set of specific and measurable outputs or outcomes about citizens’ knowledge of the decentralization process in Mali. Regarding whether general awareness raising about decentralization should be increased in option years – YES. This is consistent with the recommendations that we made in the body of the evaluation report. General awareness raising on decentralization, including simplifying and disseminating the relevant decentralization texts (particularly in local languages) – a recommendation made by KIs and FG participants in a majority of our interviews – would be important efforts going forward for SNGP. As one of TMG’s team members noted: The decentralization process remains unappreciated by the vast majority of the population in Mali. The process benefited from the support of a significant number of development partners in Mali, however, the results as far as the Team could determine were below expectations, due to several reasons: 1. The lack of information, understanding and awareness that has prevented people from taking ownership of the process of decentralization from its initial launching some 25 years ago; and, perhaps more importantly, its relaunching with the regionalization policy 2013; 2. The excessive involvement of politicians in the choice of the leaders of the CTs, thus contributing to the mistrust and negative perceptions the populations have of their leaders; 3. The very low educational level of the leaders of the CTs who have great difficulty in appropriating the texts that frame the process; 4. The overwhelming absence of the state to play its part in the process, leaving the CTs at the mercy of actors – such as administrative and technical services at the regional level who perceive that decentralization means a decrease in their power –who act against the process. For all these reasons, the team is strongly of the opinion that awareness-raising and capacity￾building activities should continue directly at the commune level, avoiding the mistakes of the past. To do this, SNGP can focus on activities such as: • Update the tax base of the CT by proceeding with the census of the taxable base; Page | 39 • Develop templates adapted to the level of understanding (including in local languages where relevant) of civil society for the organization of public debates on the budget and the planning of the communal assembly; • Periodically organize thematic exchange workshops led by a consultant, bringing together the CTs, the technical services concerned by sector and civil society. Set up a committee to follow up on the recommendations identified during the workshop and schedule follow-up on the implementation of recommendations in the field; • Organize proximity support for CTs aimed at improving the quality of public financial management tools and standardizing their behavior. • Advocate for a depoliticization of the choice of leaders and a revaluation of the elective functions of the CT. • Work on public dissemination (perhaps through radio) of the dollar amount value of money transferred to the communes AND what that money is planned to be utilized for in terms of community development. Armed with these two data points, i.e. 1) How much money is available in my commune for local development? and 2) What is that money supposed to be used for (based on local development plans)?, the public can better hold their local leaders accountable based on these available facts. Page | 40 Annex II: Evaluation Statement of Work (SOW) Page | 41 Page | 42 Page | 43 Page | 44 Page | 45 Page | 46 Page | 47 Page | 48 Page | 49 Page | 50 Page | 51 Page | 52 ANNEX III: Evaluation/Analytic Methods and Limitations Annex III.A – Qualitative Analysis: Frequency of Dominant Thematic Codes (arranged hierarchically) from KII and FGD notes The following is a list of the main dominant themes noted during qualitative interviews (KIIs and FGDs), ordered hierarchically. The frequency column reflects the number of times a key informant of focus group person mentioned/or discussed the relevant theme or topic. This qualitative data came as a result of a thorough and detail oriented manual coding of all interview notes using Nvivo qualitative analysis software. Response rate graphics based on this coded data were utilized and presented as supporting evidence for the findings presented in the evaluation report. Name Frequency Themes 0 TOC Positive 0 Basic logic is decent 6 TOC Negative 0 Culture of corruption is core problem 19 Lack of political will 12 State already transferred competencies but resources not transferred 6 Lack of Confidence in local PFM capacity is a State subterfuge to avoid effectuating transfers 4 State not implementing local PFM oversight and control 3 Decentralization not happening because of lack of funding 1 SNGP Challenges Successes Recommendations 0 WOMEN & YOUTH - Lack of meaningful engagement with marginalized groups 0 Youth 0 Youth not as intentionally included as women 6 Women's Empowerment 0 Women included but not well-engaged 10 SUCCESSES 0 Intentional effort to include women in activities 11 Page | 53 Awareness raising on 052 Law with respect to 30 percent gender in elected positions 1 RECOMMENDATIONS 0 Women-focused public radio debates 1 Start women's house for literacy and growth 1 Quota law for youth needed like Law 052 1 Learn from other approaches to meaningfully integrate women 2 Increase quotas for women's participation 1 Capacity-building for women's leadership 2 Impact difficult to measure 3 Democracy has increased women's participation more than SNGP 7 RECOMMENDATIONS 0 Women and Youth particularly relevant for improved service provision effort 3 Strengthen capacity to participate in community budget control and PDSEC 2 Meaningful engagement needed 6 Training and Capacity-Building Challenges 0 Trainings often not understood by key grassroots stakeholders 7 Trained in methods that participants don't have financial or HR capacity to actualize 2 RECOMMENDATIONS 0 Train in local languages 6 Limited or unclear impact of training 8 TOP-DOWN, heavy-handed approach lacking respect for and inclusion of partners 0 SNGP Workplans not well-aligned with Partner strategies and plans 13 RECOMMENDATIONS 0 Sensitize partners better on SGNP’s vision, approach and activities 2 Learn from approach of other USAID projects and other Donors 4 Follow signed partner agreements 1 Empower SNGP Strategic Partners 7 Develop more transparent and open comms with partners 3 Design strategy and activities jointly with partners 10 Poor comms - partners don't fully understand SNGP’s approach and strategy 15 Page | 54 Partners not empowered 8 Partners needs and perspectives not properly taken into account 26 Not respecting details of partner agreements 2 Lack of respect and proper protocol with partners 12 Too broad and ambitious, not deep enough to have impact 0 RECOMMENDATIONS 0 Strengthen capacity of ADR and identify unique comparative advantage 3 Pilot communes 2 Lack of resources required for a broad and impactful approach 0 Lack of human resources at CGSP 1 Regional Council’s capacity weak 5 ADRs don't have capacity for mandate 17 SUCCESSES and Appreciated Activities 0 Youth training for entrepreneurship 1 Women's capacity-building on rights and duties 1 Technical support from TA 3 Support for the CADD 2 Round tables 4 Radio broadcasts 4 Public restitution of Administrative Accounts 4 Public debates and training on budget process and planning (PDSEC) 35 Local Resource mobilization 3 Leadership training 1 IT training and software 7 Gender 4 Culture of Accountability initiated 5 CROCSAD meetings 1 Consultation spaces revitalized 2 Capacity-building of the GSB 3 Capacity-building of the CGSP (audit functions) 2 Page | 55 Awareness-raising caravan about ADR 7 STUDIES Needed 0 Support surveys and studies on decentralization needed by the State 0 Better understand constraints to local resource mobilization 0 Assessment of constraints and needs of CTs 0 A qualitative research study to understand the root reasons behind the lack of political will for transfers 0 SNGP UNKNOWN - Lack of general awareness and understanding of SNGP project 0 SNGP not widely known or understood 14 SNGP OVERALL IMPACT 9 SERVICE PROVISION - Lack of evidence of improved service provision 0 Weak integration perspectives and inputs of relevant stakeholders 3 RECOMMENDATIONS 0 Leverage existing laws and texts for decentralized management of services 2 Leverage existing consultation frameworks for local development 7 Leverage decrees 1 INNOVATION - Learn from positive deviants 1 Heavy focus on the inclusion and participation of women and youth 3 Focus efforts around one key community problem within projects to impact Health and Education sectors 0 fix teachers’ salary problem as a test case for success 2 Collaborate closely with relevant CSO Health and Education sector stakeholders 26 No indication of substantive progress 3 No inclusion on PDSEC final decision-making 2 REGIONAL LIMIT - Impact limited to Regional Level while unfulfilled need is greatest at local level 0 SUCCESSES 2 Regional level capacity-building 4 Regional Council's approach to development of local public businesses 1 Diverse actors included in regional planning and budget processes 3 RECOMMENDATIONS 0 Go local 10 Page | 56 CGSP should do more sub-regional audits and controls 1 Focus on regional projects of interest to local authorities 2 Catalyze capacity-building beyond regional level 1 Lack of reach outside of regional level 6 OLD SYSTEM still dominates 0 System before decentralization still alive and well 4 MISCELLANEOUS 5 M & E - Weak monitoring and evaluation of activities 0 Weak M&E from of SNGP activities 2 LOCAL RESOURCE - Weak Local Resource Mobilization 0 Weak Local Authority capacity to mobilize local resources 3 SUCCESSES 3 State taking local resources 1 RECOMMENDATIONS 3 Training on local resource mobilization needed 2 Studies and research needed 3 More communal authority to recover local resources needed 1 Develop local resource mobilization software 1 Coordinated effort needed 1 Poverty is one reason for lack of local resources 1 Lack of trust between population and CTs 1 Insecurity 0 RECOMMENDATIONS 0 More holistic approach needed in insecure zones 1 Consider not programming in insecure zones 2 Build on promising activities in insecure zones 1 Other Impacts: 2 Negative perception of regionalization as dividing Mali 2 Insecure regions have made programming difficult 5 Page | 57 GRASSROOTS ADVOCACY - Weak grassroots and umbrella level CSO advocacy and watchdog functions 0 Weak capacity and knowledge 2 RECOMMENDATIONS 0 Train and Equip grassroots and umbrella CSOs on advocacy and watchdog functions 15 Innovation - website on control of public services 1 Civic Education and Awareness-Raising on Rights and Duties 11 Awareness-Raising on controls texts and inspection protocols 5 Lack of robust civil society participation 4 DUPLICATION - Conflicting mandate between ADR and Regional Council 0 RECOMMENDATIONS 0 Clarify roles and responsibilities better 2 Duplication of efforts & overlapping mandate 8 DONOR COORDINATION - Lack of Development- Donor Project Synergy and Coordination 0 Weak coordination and synergies with other development actors 3 SUCCESSES 0 Examples of coordination with development partners 1 RECOMMENDATIONS 1 Leverage existing platforms for messaging 1 Learn from collaborative approaches of other development partners 1 Harmonize approach with other related development projects 2 CUMBERSOME, inflexible, red-tape-filled processes and admin 0 Per diem issues 1 Last minute invitations leading to substandard attendees 2 Inflexibility 1 Delays and slow implementation 5 CT LOW CAPACITY - Weakest capacity at CT - local level 0 RECOMMENDATIONS 0 Strengthen local level capacity and HR of CTs - particularly commune level 13 Page | 58 Leverage dynamism and creativity of youth at local level 1 Local-level corruption 1 Lack of human resources at lower levels 10 CONTROL ABSENT - Lack of State oversight and controls over PFM systems 0 Weak or non-existent monitoring and controls 13 RECOMMENDATIONS 0 Texts for monitoring and control MUST BE APPLIED 31 SNGP should collaborate on and support control missions 2 INNOVATION - Work with Malian Office for Combatting Illicit Enrichment 1 Framework of consultation of control services 1 Consider working with external auditors who have capacity 1 Build CGSP’s audit capacity 2 Lack of knowledge of texts 2 COMMS POOR - Weak comms between deconcentrated State and CTs 0 RECOMMENDATIONS 0 Specific Recommendations 7 Poor communications and coordination between State and CTs 4 ADR advocacy committee challenges 0 RECOMMENDATIONS 0 Budget for and support functionality of Advocacy Committee 0 Plan without a budget to implement 5 QUOTES 91 Page | 59 Annex III.B – Quantitative Analysis: Response Rates, Correlations and Analysis for Mini-Survey; and Response Rates and Analysis for Public Perceptions Survey Preliminary analysis of the quantitative data Executive Summary The mini-survey consists of 15 questions, geared towards one of four themes, with corresponding questions: 1. Insecurity impact - 9 2. Theory of Change - 5, 10, 11, 15 3. Knowledge of local budget and plans - 1, 2 4. Project Performance - 3, 4, 6-8, 12-14 The groups that were disaggregated to run analysis, filtered by only those who had knowledge of SNGP, included: 1. Sex 2. Youth vs non-youth 3. Stakeholder type 4. Stakeholder group-level 5. Region Four Categories of Respondents from the mini-survey • SEX: [A] MALE; [B] FEMALE • Youth: [a] Youth (age 18-25); [b] non-youth (age 26 and older); • Stakeholder Type: [a] Civil society (unclassified civil society, private sector, CGS / ASACO / APE, pupil and student, other school and health professionals, NGO, staff); [b] IPs / Subs (organization intervening in governance, SNGP’s partner); [c] GOM (education department, financial services, health department, school or health commission, local government administration, local government financial services, other GOM technical services); [d] donor; [e] USAID; and • Stakeholder Group-Level: [a] low-level stakeholders (civil society, school / health centers); [b] mid-level stakeholders (GOM technical services, Collective); [c] high-level stakeholders (donors, USAID, IP/SubIPs). Consolidated results from the analysis led to correlations of statistical significance between the following variables, with criteria being at least half of each theme being correlated: Page | 60 Table 1. Summary of correlations with # of questions correlated Insecurity impact (1 total question) Theory of Change - Situation (3 total questions) Theory of Change – Project (1 total question) Knowledge of local budget and plans (2 total questions) Project Performance (8 total questions) Region Correlated Not correlated (1 out of 3 questions) Not correlated Not correlated Not correlated Sex Not correlated Not correlated (1 out of 3 questions) Not correlated Not correlated Not correlated Stakeholder type Correlated Correlated (2 out of 3 questions) Not correlated Correlated (1 out of 2 questions) Not correlated (2 out of 8 questions) Youth vs non-youth Correlated Correlated (3 out of 3 questions) Not correlated Correlated (2 out of 2 questions) Not correlated Stakeholder group-level Correlated Correlated (3 out of 3 questions) Not correlated Correlated (2 out of 2 questions) Not correlated (2 out of 8 questions) Methodology The quantitative data analysis was conducted in SPSS using both descriptive and correlation analysis techniques. a) Descriptive Analysis In the report, the mini-survey data are used to conduct descriptive analysis of indicators describing participants perception on three key themes: knowledge of the SNGP program; components of the SNGP program; issues and challenges. Where applicable, indicator values are reported as percentages and means. • Percentages. For values provided in nominal scales (e.g., yes/no responses), percentages were computed using the weighted number of cases that provided a given response as the numerator, and the total weighted number of cases as the denominator. Single response variables add up to a maximum of 100 percent, while multiple response variables may total to more than 100 percent. • Means. For variables calculated in an interval scale format (e.g., Likert scale), means were computed using the weighted sum of values as the numerator and the total weighted number of cases as the denominator. Step-by-step breakdown of mini-survey data: 1. Reviewed the SNGP workplan 2. Used Excel PivotTables to take preliminary look of data Page | 61 3. Used preliminary analysis to clean up data 4. Disaggregated data by: a. Sex: female VS male b. Age: 18-25 VS 26+ (i.e. youth vs non-youth) c. Stakeholder type d. Stakeholder group-level: i. low-level stakeholders: civil society and school/health workers (the two stakeholder groups essentially) ii. mid-level stakeholders: Collectivité and GOM technical services (the two stakeholder groups) iii. higher-level stakeholders: IP/SubIPs, USAID and Donors e. Region f. Whether the respondent knows of SNGP 5. Recoded data from string to numeric variables in SPSS, as relevant 6. Used SPSS to run bivariate correlations between the IVs and DVs a. Pointed out the statistically significant results 7. Look more closely at the frequencies of questions that were statistically significant 8. Others, as emergent Step-by-step breakdown of radio text-in public survey questions (perception): 1. Similar process as mini-survey data analysis 2. Triangulation with mini-survey data Correlations: Insecurity impact question 9 Correlations on Insecurity Impact 9DecePubFinc Mgt region Pearson Correlation -.135** Sig. (2-tailed) .000 N 766 sex Pearson Correlation .037 Sig. (2-tailed) .308 N 766 stakeholder type Pearson Correlation .155** Sig. (2-tailed) .000 N 766 youth group Pearson Correlation .098** Sig. (2-tailed) .006 N 766 stakeholder group-level Pearson Correlation .124** Sig. (2-tailed) .001 N 766 **. Correlation at 0.01(2-tailed) *. Correlation at 0.05(2-tailed). This shows statistical significant on the following groups: region, stakeholder type, stakeholder group-level. Page | 62 Correlations: Theory of Change Context – questions 5, 11, 15 Correlations on Theory of Change 5WhyLittleProg 11BiggestObst 15WillingTransf region Pearson Correlation -.044 .108** -.054 Sig. (2-tailed) .224 .003 .133 N 766 766 766 sex Pearson Correlation -.081* -.038 .021 Sig. (2-tailed) .026 .287 .564 N 766 766 766 stakeholder type Pearson Correlation -.044 -.144** .075* Sig. (2-tailed) .226 .000 .039 N 766 766 766 youth group Pearson Correlation -.164** -.104** .180** Sig. (2-tailed) .000 .004 .000 N 766 766 766 stakeholder group-level Pearson Correlation -.103** -.143** .139** Sig. (2-tailed) .004 .000 .000 N 766 766 766 **. Correlation at 0.01(2-tailed) *. Correlation at 0.05(2-tailed). This shows statistical significant on the following groups (at least 2 out of the 3 theory of change-related questions): stakeholder type, youth, stakeholder-group level. Page | 63 Correlations: Theory of Change regarding Project – question 10 (only includes those with SNGP knowledge) Correlations on Theory of Change regarding Project 10BiggestChall region Pearson Correlation -.064 Sig. (2-tailed) .274 N 296 sex Pearson Correlation .014 Sig. (2-tailed) .811 N 296 stakeholder type Pearson Correlation -.105 Sig. (2-tailed) .070 N 296 youth group Pearson Correlation -.068 Sig. (2-tailed) .245 N 296 stakeholder group-level Pearson Correlation -.087 Sig. (2-tailed) .135 N 296 **. Correlation at 0.01(2-tailed) *. Correlation at 0.05(2-tailed). This shows statistical significant on the following groups: none. Correlations: Knowledge of local budget and plans – questions 1, 2 Correlations 1KnowAmtMon ey 2UseofMoney region Pearson Correlation -.001 -.001 Sig. (2-tailed) .976 .985 N 766 766 sex Pearson Correlation -.002 -.031 Sig. (2-tailed) .946 .398 N 766 766 stakeholder type Pearson Correlation .208** .040 Sig. (2-tailed) .000 .270 N 766 766 youth group Pearson Correlation .101** .164** Sig. (2-tailed) .005 .000 N 766 766 stakeholder group-level Pearson Correlation .251** .076* Sig. (2-tailed) .000 .036 N 766 766 **. Correlation at 0.01(2-tailed) *. Correlation at 0.05(2-tailed). This shows statistical significant on the following groups (at least 1 out of the 2 local budget and plans-related questions): stakeholder type, youth, stakeholder group-level. Page | 64 Correlations: Project Performance – questions 3-4, 6-8, 12-14 (only includes those with knowledge of SNGP) Correlations on Project Performance-related question 3CBtrai ning 4DecePu bRes 6StrFinc Mgmt 7CoorOpen Comms 8ImprCapM onitor 12Wome nEmp 13ProvHeal thEdu 14MobLo cRes region Pearso n Correl ation -.098 .084 .005 -.020 .100 .095 .112 -.080 Sig. (2- tailed) .092 .150 .936 .731 .087 .102 .054 .167 N 296 296 296 296 296 296 296 296 sex Pearso n Correl ation -.054 -.063 -.094 .013 .003 -.083 -.016 .007 Sig. (2- tailed) .353 .282 .106 .818 .960 .156 .788 .910 N 296 296 296 296 296 296 296 296 stakeh older type Pearso n Correl ation -.077 -.023 -.001 .078 .157** -.173** -.074 -.040 Sig. (2- tailed) .188 .693 .981 .183 .007 .003 .207 .488 N 296 296 296 296 296 296 296 296 youth group Pearso n Correl ation .076 -.041 -.060 -.041 -.048 .091 -.086 -.072 Sig. (2- tailed) .194 .481 .304 .487 .415 .118 .141 .218 N 296 296 296 296 296 296 296 296 stakeh older group￾level Pearso n Correl ation -.096 -.014 .033 .061 .121* -.144* -.063 -.021 Sig. (2- tailed) .101 .811 .574 .299 .037 .013 .279 .719 N 296 296 296 296 296 296 296 296 **. Correlation at 0.01(2-tailed) *. Correlation at 0.05(2-tailed). This shows statistical significant on the following groups (at least 4 out of the 8 project performance-related questions): none. Page | 65 Theory of Change Analysis MINI-SURVEY FINDINGS QUESTION 5: Donor funding has provided robust support for decentralization in Mali for 20 years now. Why do you think there has been so little progress to date? Table 1. Q5 - Donor funding has provided robust support for decentralization in Mali for 20 years now. Why do you think there has been so little progress to date? Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Response 244 32% Lack of civil society participation and advocacy 222 29% Not enough transparency in political systems 163 21% Culture of corruption / Mentality is blocking progress in Mali 133 17% Weak accountability in governance systems 4 1% No answer 766 100% TOTAL Across all respondents, the top three responses to why there has been so little progress on decentralization over the years are: 1) Lack of civil society participation and advocacy (32%); 2) Not enough transparency in political systems (29%); and 3) Culture of corruption/Mentality is blocking progress in Mali (21%). Since a lack of transparency and culture of corruption are related conceptually (i.e. the need for more transparency is based on the assumption that something unlawful might be going on that needs to be exposed), it is interesting to consider combining the frequency of responses to lack of transparency with the culture of corruption. If we consider the concepts of “transparency” and a “culture of corruption”, both allude to trying to uncover a more internal problem related to actors who lack of a moral compass or who may be motivated by personal financial gain. Civil society participation and accountability are both forms of external pressures to bring about change by sanctioning those who don't behave professionally as public servants. The deeper underlying question here is whether strengthening the internal moral sanctions (i.e. working on people) within the public sphere would be more effective than strengthening external sanctions (i.e. working on systems)? If we group the percentages by internal vs. external we get results seen in Table 5, below. *Data responses for Donors and USAID staff have been left out of this aggregated analysis due to the extremely small sample size for those stakeholders. Page | 66 Table 5. Q5 - Donor funding has provided robust support for decentralization in Mali for 20 years now. Why do you think there has been so little progress to date? Responses aggregated by Internal or External root problem Internal change needed External systems need to be strengthened No answer TOTAL 50.26% 49.21% 0.52% There appears to be a fairly even split between opinions across respondents when responses are grouped into the two categories of “internal change needed” and “external systems need to be strengthened”. Q5: Findings of statistically significant differences 1. Q5 - Youth Compared to Adults Table 2. Donor funding has provided robust support for decentralization in Mali for 20 years now. Why do you think there has been so little progress to date? Disaggregated by Age - YOUTH (18 - 25) Frequency Percent Response 25 46% Culture of corruption / Mentality is blocking progress in Mali 16 30% Not enough transparency in political systems 6 11% Weak accountability in governance systems 6 11% Lack of civil society participation and advocacy 1 2% No answer 54 100% TOTAL Table 3. Donor funding has provided robust support for decentralization in Mali for 20 years now. Why do you think there has been so little progress to date? Disaggregated by Age - ADULT (26 and higher) Frequency Percent Response 238 33% Lack of civil society participation and advocacy 206 29% Not enough transparency in political systems 138 19% Culture of corruption / Mentality is blocking progress in Mali 127 18% Weak accountability in governance systems 3 0% No answer 712 100% TOTAL Disaggregation by age group shows a clear distinction of opinion between the youth and respondents over 26. The youth have a very strong opinion that a culture of corruption/mentality issue is at the root of the lack of progress of decentralization in Mali, whereas those older (26 and up) feel that the lack of civil society participation and advocacy is the main reason (33% of respondents) progress has been so Page | 67 slow. For youth, their response rate for lack of civil society participation was much lower, at only 11%. For those older, the response rate for culture of corruption/mentality issue was 19%. These findings can be interpreted in a few different manners. The first interpretation is that youth have a much bleaker impression of the situation in Mali and feel that the root driver of many of the country’s problems are related to the culture of corruption and a mentality issue that is normalized and deeply embedded into the fabric of Malian society. It is very common to hear Malian youth and even adults proclaim very fervently that, “If the mentality doesn’t change in Mali, nothing will change!”. The difference of opinion with the older generation that lack of civil society participation is the core problem may be related to the education and experiential background those age groups have had, particularly since some of them have been instilled with democratic concepts, such as the “importance of civil society participation in a democracy”, for a longer period of time. It is also possible that the older generation embodies the culture of corruption the youth are pointing their fingers at, which could lead to a kind of defensiveness bias when older people respond to this question. This type of defensive bias could lead them to deny there is a deeply-embedded culture of corruption in Mali. 2. Q5 - Comparison across Stakeholder Group Levels Table 4. Q5 - Donor funding has provided robust support for decentralization in Mali for 20 years now. Why do you think there has been so little progress to date? Disaggregated by Stakeholder Group Levels Stakeholder Group Level Lack of civil society participation and advocacy Weak accountability in governance systems Not enough transparency in political systems Culture of corruption / Mentality is blocking progress in Mali No answer TOTAL Low-level (Civil society and Health and Education centers) 178 85 169 136 2 570 31% 15% 30% 24% 0% 74% Mid-level (Collective and GOM technical services) 54 37 49 21 1 162 33% 23% 30% 13% 1% 21% High-level (IP/SubIPs, USAID and Donors) 12 11 4 6 1 34 35% 32% 12% 18% 3% 4% TOTAL 244 133 222 163 4 766 32% 17% 29% 21% 1% 100% Across this unique group of stakeholder group levels, there was general agreement that the dominant reason behind the slowdown in progress of decentralization in Mali is the lack of civil society participation. Interestingly, the citizens closest to the “ground” appear to see a culture of corruption/mentality issue as more important than those operating at higher levels in their society (i.e. mid-level and high-level Page | 68 stakeholder groups). Those financing and implementing development work in Mali (i.e. high-level stakeholder groups) do not see transparency in political systems as important for decentralization in Mali as do the low and mid-level stakeholders. 3. Q5 - Comparison between the sexes Table 6. Q5 – Donor funding has provided robust support for decentralization in Mali for 20 years now. Why do you think there has been so little progress to date? Disaggregated by Sex – FEMALE only Frequency Percent Q5 61 30% Not enough transparency in political systems 54 27% Lack of civil society participation and advocacy 53 26% Culture of corruption / Mentality is blocking progress in Mali 31 15% Weak accountability in governance systems 2 1% No answer 201 100% TOTAL Table 7. Q5 - Donor funding has provided robust support for decentralization in Mali for 20 years now. Why do you think there has been so little progress to date? Disaggregated by Sex – MALE only Frequency Percent Q5 190 34% Lack of civil society participation and advocacy 161 28% Not enough transparency in political systems 110 19% Culture of corruption / Mentality is blocking progress in Mali 102 18% Weak accountability in governance systems 2 0% No answer 565 100% TOTAL We can see a small difference of opinion on the top two responses for both men and women. Men feel lack of civil society participation (34%) is the reason for so little progress on decentralization whereas women feel there is not enough transparency in politics (30%). The third most common response for both men and women was that a culture of corruption/mentality issue is blocking progress in Mali. Most notably, women responded at a much higher percentage (26%) regarding the culture of corruption being the root problem as compared to men who responded much less (18%). Since both a lack of transparency and a culture of corruption/mentality issue or related concepts and women prioritize both of these as root issues blocking progress in Mali more than man do, we can conclude that, overall, women see corruption/mentality and lack of transparency as the biggest challenges facing decentralization in Mali more than men do. Page | 69 QUESTION 10: What is the biggest challenge faced by the SNGP? Table 6. Q10 - What is the biggest challenge faced by the SNGP? Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Q10 105 35% Lack of political will 91 31% Trying to do too many things, not focused enough on key interventions needed 52 18% Lack of competence of the actors involved in the implementation of the program 38 13% There are fundamental problems with the logic, assumptions and approach of project 10 3% No answer 296 100% TOTAL The largest percentage (35%) responded that the biggest challenge faced by the SNGP is the “lack of political will”. This contradicts the underlying assumption in SNGP’s Theory of Change which states that there is political will for decentralization. After the lack of political will, 31% of respondents say that the project is, “trying to do too many things, not focused enough on key interventions needed”. This is a common cause of failure of many development programs globally – trying to take on more than the resources available will allow within the manageable interest. The clear recommendation for the future is to prioritize interventions and go deep rather than too broad in the programmatic approach of SNGP. Page | 70 Q10: Findings of statistically significant differences 1. Q10 - Comparison across Stakeholder Types Table 7. Q10 – What is the biggest challenge faced by the SNGP? Disaggregated by Stakeholder Type Stakeholder Type Lack of competence of the actors involved in the implementation of the program Lack of political will No answer There are fundamental problems with the logic, assumptions and approach of project Trying to do too many things, not focused enough on key interventions needed Tota l Civil society 9 22 0 14 14 59 15% 37% 0% 24% 24% 30% Collective (Local government) 3 11 0 1 7 22 14% 50% 0% 5% 32% 8% Donor (*anecdotal because of small sample size) 0 1 2 0 2 5 0% 20% 40% 0% 40% 1% GOM technical services 9 10 0 2 21 42 21% 24% 0% 5% 50% 13% IP / Sub IP 4 8 1 2 4 19 21% 42% 5% 11% 21% 3% School and health centres 27 50 5 18 43 143 19% 35% 3% 13% 30% 44% USAID (*anecdotal because of small sample size) 0 3 0 1 0 4 0% 75% 0% 25% 0% 1% Total 52 105 10 38 91 296 18% 35% 3% 13% 31% 100 % The most dramatic and interesting difference in the responses between the various stakeholder types is between the Collective (local authorities) and the GOM technical services. The Collective has the highest percent (50%) response rate that the biggest challenge is lack of political will, while the GOM technical services has one of the lowest response rates (24%) for lack of political will. The Collective believes that true political will doesn’t exist at the level of the GOM, as evidenced by the fact that transfers aren’t effectuated as they are supposed to. The GOM, however, denies there is a lack of political will. This is likely why only 24% of the GOM technical services respondents said the biggest challenge was political will. Page | 71 QUESTION 11: What is the biggest obstacle to decentralization of resources in Mali? Table 8. Q11 - Frequency Distribution across entire Mini-Survey Dataset What is the biggest obstacle to decentralization of resources in Mali? Frequency Percent Q11 216 28% Political will 191 25% Corruption 157 20% Lack of public financial management capacity at local level 111 14% Lack of financial and human resources 90 12% Lack of public pressure from civil society groups 1 0% No answer 766 100% TOTAL Interestingly, this data shows again that there is a lack of political will for decentralization in Mali – striking at the heart of the underlying assumption of SNGP’s Theory of Change. Corruption was a close second as the most oft-cited obstacle to decentralization in Mali. Since a lack of political will can be interpreted as being related to corruption, particularly as the lack of political will to decentralize resources by the State may be due to the State’s desire to hoard resources for purposes beyond the scope of the moral mandate of public servants. Interpreting the data in this light, we can say that about half (political will: 28% + corruption: 25% = 53%) of the survey respondents believe corruption to be at the heart of why transfers aren’t effectuated. Page | 72 Q11: Findings of statistically significant differences 1. Q11- Comparison across Stakeholder Types Table 9. Q11 What is the biggest obstacle to decentralization of resources in Mali? Disaggregation by Stakeholder Types Stakeholder Type Corruption Lack of financial and human resources Lack of public financial management capacity at local level Lack of public pressure from civil society groups No answer Political will Total Civil society 77 16 42 28 0 70 233 33% 7% 18% 12% 0% 30% 30% Collective (Local government) 2 28 10 5 0 18 63 3% 44% 16% 8% 0% 29% 8% Donor (*anecdotal because of small sample size) 1 1 0 1 0 4 7 14% 14% 0% 14% 0% 57% 1% GOM technical services 21 24 16 10 0 28 99 21% 24% 16% 10% 0% 28% 13% IP / Sub IP 3 3 3 5 1 8 23 13% 13% 13% 22% 4% 35% 3% School and health centers 87 39 86 40 0 85 337 26% 12% 26% 12% 0% 25% 44% USAID (*anecdotal because of small sample size) 0 0 0 1 0 3 4 0% 0% 0% 25% 0% 75% 1% Total 191 111 157 90 1 216 766 25% 14% 20% 12% 0% 28% 100% In this breakdown of the data comparing results by stakeholder group, it is interesting to note that the local authorities (Collectives) have a much lower response rate (3%) that the biggest obstacle to decentralization is corruption, versus civil society (33%) and the school and health center workers (26%), who had a much higher response rate that the biggest obstacle is corruption. It is also interesting to note that civil society and school and health workers were much less likely to believe that the lack of financial and human resources was the biggest obstacle (only 7% and 12% respectively). Whereas the Collective had a dramatically higher response rate (44%) pointing to lack of financial and human resources as the biggest obstacle. This is emblematic of the finger-pointing that the qualitative interviews has noted between the population/local communities and their local authorities. There is a large lack of trust and confidence in the Collective among the population. Page | 73 PUBLIC PERCEPTION SURVEY FINDINGS QUESTION 1: What is necessary to ensure equitable decentralization of resources and reduce corruption in Mali?? Table 10. Q1 - What is necessary to ensure equitable decentralization of resources and reduce corruption in Mali? Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Q1 114 80% An internal change - a cultural or mentality change needs to occur 28 20% An external change - better systems of good governance and accountability need to be put in place 142 100% TOTAL Here, it is very clear that the dominant perception (80% of population) of the public at large in Mali is that an internal change needs to occur in order for real decentralization and a reduction in corruption to occur. Essentially, technical training on PFM and external pressure from civil society, auditors and other external sanctions alone will not be enough to solve the internal challenge, which will require a more social science-based strategy and programmatic approach that can effectively address the mentality driving corrupt behaviors at the level of individual values, reasoning, beliefs and morals underlying a culture of corruption. This perceptions survey data points to an internal mentality issue (that should be researched and better understood) as the source of corrupt behaviors in Mali today. QUESTION 5: If money is fairly transferred to your Collectivité Territoriale (CT) from the central government, do you trust that your local government will use the resources appropriately to develop the community? Table 10. Q5 - If money is fairly transferred to your Collectivité Territoriale (CT) from the central government, do you trust that your local government will use the resources appropriately to develop the community? Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Q5 65 46% No 13 9% Yes 64 45% Maybe 142 100% TOTAL Since the respondents appear to be split between haven’t no confidence (46%) that transferred money would be managed well by their local authorities (CT) and being unsure (“maybe” at 45%), this clearly reveals a serious problem of trust among the population for their local authorities. As findings from the mini-survey confirm, much of the population believes that corruption is a core issue in the country and thus, having confidence in others is very difficult to come by in this type of climate. Page | 74 Project Performance Analysis MINI-SURVEY FINDINGS For all questions related to SNGP performance, data is only shown and analyzed for the 39% of all respondents who were aware of the SNGP project. All of the following data tables and correlations only include responses from the respondents who knew about the SNGP. Keep in mind that since these are respondents that know about SNGP, they are also likely beneficiaries. Thus, they may have a bias towards speaking well of the project that has supported them in some fashion. It is a Malian cultural trait to speak well of others who have supported you in some manner, even if imperfect. Knowing this, overly positive performance ratings should be taken with a grain of salt, particularly in light of the more negative reviews the evaluation team heard of the SNGP during qualitative KIIs and FGDs. QUESTION 3: The capacity building/training that I received from SNGP was relevant and has enabled me to make a positive difference in the work I do. Table 1. Q3 - The capacity building/training that I received from SNGP was relevant and has enabled me to make a positive difference in the work I do. Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 3 responses 224 76% Agree 27 9% Strongly agree 26 9% Not clear 17 6% Disagree 2 1% No answer 296 100% TOTAL *Q3: No statistically significant correlations found between Question 3 responses and any disaggregations (i.e. region, sex, stakeholder type, youth vs. non-youth or stakeholder group-level). Analysis and interpretation Since the SNGP is largely a project focused on capacity-building and training, it is not surprising that those who know about the project (and thus likely have benefitted from SNGP), would appreciate the capacity-building they received. Here we can see that 76% of respondents thought the capacity building/training they received from SNGP was relevant and enabled them to make a positive difference. It is important to note here that during the qualitative interviews, many respondents who may have started off the interviews saying generically positive things about the SNGP began to open up and speak frankly and much more critically about the trainings or other efforts of SNGP once they become more comfortable. Thus, since qualitative interviews allow for more probing and triangulation, sometimes the real feelings people have can come out better than in a more formal and rapid survey. Thus, both the more positive performance related findings from the quantitative mini-survey and the more critical findings around performance from the KIIs and FGDs, should be viewed together in balance – each moderating the other – as the truth is often near the middle. Page | 75 QUESTION 4: The SNGP has significantly helped to increase decentralization of public resources. Table 2. Q4 - The SNGP has significantly helped to increase decentralization of public resources. Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 4 responses 224 76% Agree 38 13% Disagree 21 7% Not clear 11 4% Strongly agree 2 1% Strongly Disagree 296 100% TOTAL *Q4: No statistically-significant correlations found between Question 3 responses and any disaggregations (i.e. region, sex, stakeholder type, youth vs. non-youth or stakeholder group-level). Analysis and interpretation Here, again, a generally positive response of 76% agree that SNGP has significantly helped increase the decentralization of public resources. Although in this case, different from the first question # 3 above, we see a larger number of respondents disagreeing (13%). It appears there may be some clashing opinions about the success of SNGP in helping decentralization, but most surveyed felt SNGP has helped the process. QUESTION 6: The SNGP has significantly strengthened National and Sub￾National Public Financial Management Systems. (comp1) Table 3. Q6 - The SNGP has significantly strengthened National and Sub-National Public Financial Management Systems. (comp1). Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 6 responses 196 66% Agree 55 19% Disagree 32 11% Not clear 7 2% Strongly Disagree 6 2% Strongly agree 296 100% TOTAL *Q6: No statistically significant correlations found between Question 3 responses and any disaggregations (i.e. region, sex, stakeholder type, youth vs. non-youth or stakeholder group-level). Analysis and interpretation Here, the positive responses are lower than the first two performance questions, as only 66% agree that SNGP has strengthened PFM systems, 19% disagree and 11% are not clear. This may reveal that many still feel PFM in Mali to be quite weak, even with the support of SNGP. This may also speak to the lack of trust and confidence Malians have in their authorities to manage public funds ethically. Page | 76 QUESTION 7: Coordination, openness and communications have improved between National and Subnational Government and Civil Society on account of the SNGP. (comp2) Table 4. Q7 - Coordination, openness and communications have improved between National and Subnational Government and Civil Society on account of the SNGP. (comp2) Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 7 responses 171 58% Agree 93 31% Disagree 18 6% Not clear 7 2% Strongly Disagree 7 2% Strongly agree 296 100% TOTAL *Q7: No statistically significant correlations found between Question 3 responses and any disaggregations (i.e. region, sex, stakeholder type, youth vs. non-youth or stakeholder group-level). Analysis and interpretation Here we see the first relatively lower positive responses on performance. Only 58% agree that SNGP has improved coordination, openness and communications between National and Subnational Government and Civil Society. In fact, 31% DISAGREE that SNGP has improved this situation. This is a clear sign that much more work needs to be done to not only address this core problem, which relates to the tension underlying the parallel systems of decentralization (i.e. deconcentration and devolution), but also to understand its dynamics and drivers better. QUESTION 8: The Controle General pour Services Public’s (CGSP) capacity to monitor public spending and service delivery has been significantly improved at the national and sub-national levels on account of the SNGP. (comp3) Table 3. Q8 - The Controle General pour Services Public (CGSP) capacity to monitor public spending and service delivery has been significantly improved at the national and sub-national levels on account of the SNGP. (comp3) Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 8 responses 149 50% Agree 76 26% Disagree 57 19% Not clear 8 3% Strongly Disagree 5 2% Strongly agree 1 0% No answer 296 100% TOTAL Page | 77 Analysis and interpretation Similar to the previous performance question, the responses to this question are not as positive for this one. Only half of the respondents agreed that the GGSP’s capacity to monitor public spending and service delivery has improved because of SNGP. It is likely that many respondents may feel this to be a bit of a stretch, particularly as they are well aware of the weak nature of PFM and the lack of quality service provision throughout their country. Q8: Findings of statistically significant differences 1. Q8- Disaggregation by Stakeholder Type Pearson Correlation = .157 / Sig. (2-tailed) = .007 / N = 296 Table 4. Q8 - The Controle General pour Services Public (CGSP) capacity to monitor public spending and service delivery has been significantly improved at the national and sub-national levels on account of the SNGP. (comp3) Disaggregation by Stakeholder Type showing frequency and response rate for each Stakeholder Type Agree Disagree No answer Not clear Strongly agree Strongly Disagree Total Civil society 20 19 0 18 0 4 61 33% 31% 0% 30% 0% 7% 100% Collective (Local government) 7 8 0 3 2 2 22 32% 36% 0% 14% 9% 9% 100% Donor (*anecdotal because of small sample size) 2 0 0 3 0 0 5 40% 0% 0% 60% 0% 0% 100% GOM technical services 30 7 1 5 1 0 44 68% 16% 2% 11% 2% 0% 100% IP / Sub IP 12 2 0 4 1 0 19 63% 11% 0% 21% 5% 0% 100% School and health centers 77 39 0 23 1 3 143 54% 27% 0% 16% 1% 2% 100% USAID (*anecdotal because of small sample size) 1 1 0 1 1 0 4 25% 25% 0% 25% 25% 0% 100% Total 149 76 1 57 5 8 296 50% 26% 0% 19% 2% 3% 100% Analysis and interpretation What appears to “pop” when comparing the response rates for this question across stakeholder groups is that the stakeholders closest to the reality on the ground (i.e. civil society, Collectivité, School and health centers) disagree the most with the statement that CGSP is doing a better job because of SNGP. Whereas, the higher-level stakeholders – GOM technical services and IP/Sub IPs – tend to agree more that CGSP is doing better and disagree less. Since those closest to the ground are the ones who would notice the benefits of good oversights of PFM and service provision in their communities, their responses should be given more validity than those who work further away from the context impacted. Page | 78 2. Q8- Disaggregation by Stakeholder Group Level Pearson Correlation = .121 / Sig. (2-tailed) = .037 / N = 296 Table 4. Q8 - The Controle General pour Services Public (CGSP) capacity to monitor public spending and service delivery has been significantly improved at the national and sub-national levels on account of the SNGP. (comp3) Disaggregation by Stakeholder Group Level showing frequency and response rate for each Stakeholder Group Level Agree Disagree No answer Not clear Strongly agree Strongly Disagree Total Low-level (Civil society and Health and Education centers) 97 58 0 41 1 5 202 48% 29% 0% 20% 0% 2% 68% Mid-level (Collectivité and GOM technical services) 37 15 1 8 2 3 66 56% 23% 2% 12% 3% 5% 22% High-level (IP/SubIPs, USAID and Donors) 15 3 0 8 2 0 28 54% 11% 0% 29% 7% 0% 9% TOTAL 149 76 1 57 5 8 296 50% 26% 0% 19% 2% 3% 100% Analysis and interpretation This breakdown by Stakeholder Group Type just reinforces the logic stated in the interpretation for the stakeholder types articulated above. We can see the same tendency in this chart – the closer stakeholder group types are “to the ground” (i.e. low-level), the less positive they are about progress with CGSP. QUESTION 12: Women have been more empowered to participate in decentralization and public financial management processes because of SNGP support. Table 3. Q12 - Women have been more empowered to participate in decentralization and public financial management processes because of SNGP support. Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 12 responses 223 75% Agree 38 13% Strongly agree 17 6% Not clear 16 5% Disagree 2 1% Strongly Disagree 296 100% TOTAL Page | 79 Analysis and interpretation These positive results can be difficult to interpret, particularly those relating to women’s empowerment. This also became apparent in the qualitative KIIs and FGDs. When asked about women’s engagement and opportunity because of SNGP, most respondents during the KIIs and FGDs would say that SNGP clearly made an intentional effort to always include and invite more women than normal for all their activities. The issue noted was that it seemed to be more of a numbers game, almost like meeting a quota of women participants, yet without the focus on the quality of the actual participation itself. Quantity over quality was emphasized in the qualitative interviews. Many also began to talk about how democracy has helped Malian women participate much more, although this was more to do with democracy than SNGP. Thus, we can see how this question’s positive response can sound decent on the surface, but there is more to explore on the qualitative side to understand the full picture of women’s engagement on account of SNGP. Q12: Findings of statistically significant differences 1. Q12- Disaggregation by Stakeholder Type Pearson Correlation = -.173 / Sig. (2-tailed) = .003 / N = 296 Table 4. Q12 - Women have been more empowered to participate in decentralization and public financial management processes because of SNGP support. Disaggregation by Stakeholder Type showing frequency and response rate for each Stakeholder Type Agree Disagree No answer Not clear Strongly agree Strongly Disagree Total Civil society 45 2 0 0 12 0 59 76% 3% 0% 0% 20% 0% 100% Collectivité (Local government) 18 1 0 2 1 0 22 82% 5% 0% 9% 5% 0% 100% Donor (*anecdotal because of small sample size) 1 0 0 3 1 0 5 20% 0% 0% 60% 20% 0% 100% GOM technical services 32 3 0 3 6 0 44 73% 7% 0% 7% 14% 0% 100% IP / Sub IP 11 1 0 3 3 1 19 58% 5% 0% 16% 16% 5% 100% School and health centres 114 8 0 5 15 1 143 80% 6% 0% 3% 10% 1% 100% USAID (*anecdotal because of small sample size) 2 1 0 1 0 0 4 50% 25% 0% 25% 0% 0% 100% Total 223 16 0 17 38 2 296 75% 5% 0% 6% 13% 1% 100% Analysis and interpretation When comparing across stakeholder groups, the same general positive response where most agree that women are more engaged on decentralization because of SNGP. The only nuance that appears in this breakdown is that Page | 80 the IP / Sub IPs seem to be less sure about the positive impact on women in the project. This could be on account of the two sub-partners who focus on gender issues for the project. They may understand more about the reality of “how the sausage was made” if you will. Some of that reality has in fact come to light in the qualitative interviews, where these partners often complain of a lack of substantive engagement with women and a particular lack of follow-up after trainings or other capacity building exercises are conducted. 2. Q12- Disaggregation by Stakeholder Group Level Pearson Correlation = -.144 / Sig. (2-tailed) = .013 / N = 296 Table 4. Q12 - Women have been more empowered to participate in decentralization and public financial management processes because of SNGP support. Disaggregation by Stakeholder Group Level showing frequency and response rate for each Stakeholder Group Level Agree Disagree No answer Not clear Strongly agree Strongly Disagree Total Low-level (Civil society and Health and Education centers) 159 10 0 5 27 1 202 79% 5% 0% 2% 13% 0% 68% Mid-level (Collectivité and GOM technical services) 50 4 0 5 7 0 66 76% 6% 0% 8% 11% 0% 22% High-level (IP/SubIPs, USAID and Donors) 12 1 0 6 4 1 24 50% 4% 0% 25% 17% 4% 8% TOTAL 223 16 0 17 38 2 296 75% 5% 0% 6% 13% 1% 100% Analysis and interpretation What is most interesting about this breakdown by stakeholder group type is the fact that the high-level stakeholders (i.e. IP/SubIPs, USAID, Donors) seem to have a much more negative view about the positive impact on women’s participation in decentralization and PFM processes because of SNGP. This may be due to the higher standards relating to women’ empowerment that many western donors and their core partners have. Page | 81 QUESTION 13: There have been notable improvements around health and education service provision because of the SNGP. Table 3. Q13 - There have been notable improvements around health and education service provision because of the SNGP. Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 13 responses 182 61% Agree 57 19% Disagree 30 10% Not clear 22 7% Strongly agree 5 2% Strongly Disagree 296 100% *Q13: No statistically significant correlations found between Question 3 responses and any disaggregations (i.e. region, sex, stakeholder type, youth vs. non-youth or stakeholder group-level). Here we see a little more than half of the respondents saying that service provision has improved because of SNGP, although 19% disagree. QUESTION 14: Mobilization of local resources (e.g. local tax collection) has been improved because of SNGP support. Table 3. Q14 - Mobilization of local resources (e.g. local tax collection) has been improved because of SNGP support. Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 14 responses 98 33% Agree 92 31% Not clear 89 30% Disagree 7 2% Strongly Disagree 6 2% Strongly agree 4 1% No answer 296 100% TOTAL *Q7: No statistically significant correlations found between Question 3 responses and any disaggregations (i.e. region, sex, stakeholder type, youth vs. non-youth or stakeholder group-level). Analysis and interpretation When comparing across stakeholder groups, the same general positive response where most agree is that women are more engaged on decentralization because of SNGP. Of all the performance measures in this survey, this one had the most negative responses. Only 33% of respondents agreed that mobilization of local resources has been improved because of SNGP. Another 30% disagreed and 31% were not clear. This lack of progress on local resource mobilization was also noted by many KIs during qualitative interviews. Local resource mobilization is seen by many as a real potential opportunity but is faced with many constraints. Page | 82 Analysis of Insecurity’s Impact on Decentralization and PFM MINI-SURVEY FINDINGS QUESTION 9: Since the security situation in Mali has gotten worse since 2012, overall, would you say decentralization and public financial management of resources improved, gotten worse or stayed the same? Table 1. Q9 - Since the security situation in Mali has gotten worse since 2012, overall, would you say decentralization and public financial management of resources improved, gotten worse or stayed the same? Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 9 responses 412 54% Gotten worse 123 16% Improved 2 0% No answer 16 2% Not clear 213 28% Stayed same 766 100% TOTAL Analysis and interpretation A little more than half (54%) of all respondents said the impact of insecurity has made the decentralization and PFM situation get worse. This is clear evidence that the changing context of insecurity in Mali has rolled back progress made on decentralization and PFM in the country. Page | 83 Q9: Findings of statistically significant differences 3. Q9- Disaggregation by Region Pearson Correlation = -.135 / Sig. (2-tailed) = .000 / N = 766 Table 2. Q9 - Since the security situation in Mali has gotten worse since 2012, overall, would you say decentralization and public financial management of resources improved, gotten worse or stayed the same? Disaggregation by Region showing frequency and response rate for each Region Improved Gotten Worse Stayed Same Not clear No answer Total Bamako 52 106 80 2 0 240 22% 44% 33% 1% 0% 31% Koulikoro 29 95 44 6 0 174 17% 55% 25% 3% 0% 23% Mopti 19 116 32 2 0 169 11% 69% 19% 1% 0% 22% Sikasso 23 95 57 6 2 183 13% 52% 31% 3% 1% 24% Total 123 412 213 16 2 766 16% 54% 28% 2% 0% 100% Analysis and interpretation The most insecure region sampled was Mopti and, not surprisingly, Mopti had the most negative impression of the impact of insecurity on decentralization and PFM. The highest response rate of “Gotten worse” was for Mopti – 69%. The most secure region, Bamako, gave the least negative outlook for the impact of insecurity on decentralization and PFM. It is clear from this contrast that the more insecure the region, the more decentralization and PFM systems have been negatively impacted. Page | 84 4. Q9- Disaggregation by Stakeholder Type Pearson Correlation = .155 / Sig. (2-tailed) = .000 / N = 766 Table 4. Q9 - Since the security situation in Mali has gotten worse since 2012, overall, would you say decentralization and public financial management of resources improved, gotten worse or stayed the same? Disaggregation by Stakeholder Type showing frequency and response rate for each Stakeholder Type Improved Gotten Worse Stayed Same Not clear No answer Total Civil society 27 134 63 8 0 163 10% 47% 26% 17% 0% 30% Collective (Local government) 7 48 7 1 0 63 11% 76% 11% 2% 0% 8% Donor (*anecdotal because of small sample size) 2 4 1 0 0 7 29% 57% 14% 0% 0% 1% GOM technical services 28 42 28 0 1 99 28% 42% 28% 0% 1% 13% IP / Sub IP 11 7 5 0 0 23 48% 30% 22% 0% 0% 3% School and health centres 47 177 107 6 0 337 14% 53% 32% 2% 0% 44% USAID (*anecdotal because of small sample size) 1 0 2 1 0 4 25% 0% 50% 25% 0% 1% Total 123 412 213 16 2 766 16% 54% 28% 2% 0% 100% Analysis and interpretation From the data analysis, it appears there is general agreement overall that the insecurity situation in Mali has had a detrimental impact on decentralization and PFM in the country. In fact, the highest response rate across stakeholders (except IP/SubIPs category) is that decentralization and PFM has “gotten worse” because of insecurity in the country. It is interesting that the IP/SubIPs have a more positive opinion, where 48% responded that the situation has improved because of insecurity. It is difficult to infer why IP/SubIPs have responded contrary to the other stakeholder types, but it is important to keep in mind the relatively smaller sample size for this stakeholder type (only 23 total), which may make this result less reliable. The last interesting finding of note here is that the local authorities (Collective) seem to have the most negative view of the impact of insecurity on decentralization and PFM – a whopping 76% of them said the security situation has made the decentralization and PFM situation worse. In fact, they would know best, since many fault lines of decentralization and PFM fall within their realm. Page | 85 5. Q9- Disaggregation by Youth vs. Non-Youth Pearson Correlation = .098 / Sig. (2-tailed) = .006 / N = 766 Table 4. Q9 - Since the security situation in Mali has gotten worse since 2012, overall, would you say decentralization and public financial management of resources improved, gotten worse or stayed the same? Disaggregation by Youth vs. Non-Youth showing frequency and response rate for each Youth or Non￾Youth Improved Gotten Worse Stayed Same Not clear No answer Total Youth (18 - 25) 1 37 15 1 0 54 2% 69% 28% 2% 0% 7% Non-Youth (over 25) 122 375 198 15 2 712 17% 53% 28% 2% 0% 93% Total 123 412 213 16 2 766 16% 54% 28% 2% 0% 100% Analysis and interpretation Although both youth and non-youth agree that insecurity has had a detrimental impact on decentralization and PFM, youth have a more negative outlook on the impact of insecurity on decentralization and PFM. This aligns with other studies done on youth in the Sahel having a strong sense of hopelessness and despair regarding their societal situation. Youth generally have a more pessimistic outlook regarding the situation in Mali. This is all the more reason that decentralization and proper PFM needs to happen, as the youth need hope in order to keep them away from more negative, delinquent and/or criminal/violent life decisions. Page | 86 6. Q9- Disaggregation by Stakeholder Group Level Pearson Correlation = .124 / Sig. (2-tailed) = .001 / N = 766 Table 4. Q9 - Since the security situation in Mali has gotten worse since 2012, overall, would you say decentralization and public financial management of resources improved, gotten worse or stayed the same? Disaggregation by Stakeholder Group Level showing frequency and response rate for each Stakeholder Group Level Improved Gotten Worse Stayed Same Not clear No answer Total Low-level (Civil society and Health and Education centers) 74 311 170 14 1 570 13% 55% 30% 2% 0% 74% Mid-level (Collectivité and GOM technical services) 35 90 35 1 1 162 22% 56% 22% 1% 1% 21% High-level (IP/SubIPs, USAID and Donors) 14 11 8 1 0 34 41% 32% 24% 3% 0% 4% TOTAL 123 412 213 16 2 766 16% 54% 28% 2% 0% 100% Analysis and interpretation Here, the data seems to show a trend that the closer stakeholders are to the ground-level context and reality, the more negative their opinion is regarding the impact of insecurity on decentralization and PFM. The high-level stakeholders, such as USAID, the donors and the IP/SubIPs, have a relatively more positive impression of the impact of insecurity on decentralization and PFM. Page | 87 Analysis of Citizen knowledge of local budgets and plans MINI-SURVEY FINDINGS QUESTION 1: Do you know approximately how much money was transferred to your Collectivité Territoriale (CT) by the central government for 2018? Table 1. Q1 - Do you know approximately how much money was transferred to your Collectivité Territoriale (CT) by the central government for 2018? Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 1 responses 630 82% No 136 18% Yes 766 100% TOTAL Analysis and interpretation Most respondents (82%) did not know how much money was in their local budget for development of their community. This clearly shows a knowledge gap that disempowers civil society in its watchdog and advocacy functions. In order for citizens to advocate for what is right, they need to know the amount available to local authorities AND the agreed upon development plans to which the budget is destined to be applied. With good knowledge of both of those, citizens and civil society can better hold their local authorities and government accountable. Page | 88 Q1: Findings of statistically significant differences 7. Q1- Disaggregation by Stakeholder Type Pearson Correlation = .208 / Sig. (2-tailed) = .000 / N = 766 Table 4. Q1 - Do you know approximately how much money was transferred to your Collectivité Territoriale (CT) by the central government for 2018? Disaggregation by Stakeholder Type showing frequency and response rate for each Stakeholder Type No Yes Total Civil society 211 22 233 91% 9% 30% Collective(Local government) 41 22 63 65% 35% 8% Donor (*anecdotal because of small sample size) 7 0 7 100% 0% 1% GOM technical services 65 34 99 66% 34% 13% IP / Sub IP 21 2 23 91% 9% 3% School and health centres 301 36 337 89% 11% 44% USAID (*anecdotal because of small sample size) 3 1 4 75% 25% 1% Total 630 136 766 82% 18% 100% Analysis and interpretation Here is made clear the unfortunate truth that the beneficiaries of the budget (civil society and school and health workers) know the least (only 9% and 11% respectively) about the budget. The local authorities responsible for properly managing the budget in line with citizen needs knows the most (35%). Again, this knowledge gap makes clear that more needs to be done by SNGP and other projects to help local populations understand how much money is available for local development plans in their communities. Page | 89 8. Q1- Disaggregation by Youth vs. Non-Youth Pearson Correlation = .101 / Sig. (2-tailed) = .005 / N = 766 Table 4. Q8 - Q1 - Do you know approximately how much money was transferred to your Collectivité Territoriale (CT) by the central government for 2018? Disaggregation by Youth vs. Non-Youth showing frequency and response rate for each Stakeholder Type No Yes Total Youth (18 - 25) 52 2 54 96% 4% 7% Non-Youth (over 25) 578 134 712 81% 19% 93% Total 630 136 766 82% 18% 100% Analysis and interpretation We can see a clear difference hear whereas youth are the most ignorant (only 4% know budget) regarding how much money is available in their local budget for development plans. Non-youth have significantly more knowledge (19% know) of the budget available for their community. Since the youth represent the most dynamic, and potentially volatile vector in society, it is critical they have the right knowledge and understanding necessary to put their abundant energy to good work as active and well informed citizens. 9. Q1- Disaggregation by Stakeholder Group Level Pearson Correlation = .251 / Sig. (2-tailed) = .000 / N = 766 Table 4. Q8 - Q1 - Do you know approximately how much money was transferred to your Collectivité Territoriale (CT) by the central government for 2018? Disaggregation by Stakeholder Group Level showing frequency and response rate for each Stakeholder Group Level No Yes Total Low-level (Civil society and Health and Education centers) 512 58 570 90% 10% 74% Mid-level (Collectivité and GOM technical services) 87 75 162 54% 46% 21% High-level (IP/SubIPs, USAID and Donors) 31 3 34 91% 9% 4% TOTAL 630 136 766 82% 18% 100% Analysis and interpretation This breakdown by Stakeholder Group Type makes it very clear that those closest to the local reality on the ground, know the least (only 10% know) about their community’s available budget. The fact that high-level Page | 90 stakeholders (i.e. USAID, donors, IPs) know very little about local budgets makes sense since they are more disconnected from the local population and their realities. The basic recommendation coming from this data is reinforced here – help the local populations better understand the total budget amount available in their communities for local development plans. QUESTION 2: Do you know what the money transferred to your Collectivité Territoriale (CT) in 2018 is supposed to be used for? Table 1. Q2 - Do you know what the money transferred to your Collectivité Territoriale (CT) in 2018 is supposed to be used for? Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 2 responses 629 82% Yes 137 18% No 766 100% TOTAL Analysis and interpretation Here we find contrasting results from the knowledge question on budget amount. 82% of respondents know what the budget is supposed to be used for in their local communities. This is a positive sign and likely means that many citizens pay attention to and/or are engaged in dialogue and decision-making processes, like the PDSEC, for their own local development plans. Q2: Findings of statistically significant differences 10. Q2- Disaggregation by Youth vs. Non-Youth Pearson Correlation = .164 / Sig. (2-tailed) = .000 / N = 766 Table 4. Q2 - Do you know what the money transferred to your Collectivité Territoriale (CT) in 2018 is supposed to be used for? Disaggregation by Youth vs. Non-Youth showing frequency and response rate for each Stakeholder Type No Yes Total Youth (18 - 25) 22 32 54 41% 59% 7% Non-Youth (over 25) 115 597 712 16% 84% 93% Total 137 629 766 18% 82% 100% Analysis and interpretation Once again, we can see that youth knowledge when it comes to local governance is much lower than non-youth. Only 59% of youth knew about what their local budget was supposed to be used for, whereas 84% of non-youth Page | 91 knew what the money was intended to be used for. The overarching recommend here is to better inform and engage youth in local development planning processes. Q2: Findings of statistically significant differences 11. Q2- Disaggregation by Stakeholder Group Level Pearson Correlation = .076 / Sig. (2-tailed) = .036 / N = 766 Table 4. Q2 - Do you know what the money transferred to your Collectivité Territoriale (CT) in 2018 is supposed to be used for? Disaggregation by Stakeholder Group Level showing frequency and response rate for each Stakeholder Group Level Yes No TOTAL Low-level (Civil society and Health and Education centers) 456 114 570 80% 20% 74% Mid-level (Collective and GOM technical services) 145 17 162 90% 10% 21% High-level (IP/SubIPs, USAID and Donors) 28 6 34 82% 18% 4% TOTAL 629 137 766 82% 18% 100% Analysis and interpretation Here we see that all stakeholder group levels have a fairly good idea of what their local budget is supposed to be used for. We can also see that the Mid-level stakeholder group (i.e. local authorities and GOM tech. services) seem to have a bit more knowledge (90%) of how the budget is supposed to be used. This makes good sense since they are the key actors behind the development of their local development plans and the budget itself. Page | 92 QUESTION 3: Do you know approximately how much money was transferred to your Collectivité Territoriale (CT) by the central government for 2018? Table 10. Q3 - Do you know approximately how much money was transferred to your Collectivité Territoriale (CT) by the central government for 2018? Frequency Distribution across Entire Mini-Survey Dataset Frequency Percent Question 3 responses 135 95% No, I don’t know. 4 3% Yes, I know. 3 2% I have a very rough idea 142 100% TOTAL Analysis and interpretation Here we can see that the same question asked in the mini-survey (to a mix SNGP beneficiaries and non￾beneficiaries) is being asked to random people in the public. We can see that a similar tendency appears, confirming the mini-survey findings. But, whereas in the mini-survey 82% didn’t know how much money was in their local budget, 95% of the general public has no idea about the budget amount. In this case, the general public has less knowledge and is more on a par with the mini-survey findings related to youth. QUESTION 4: Do you know what the money transferred to your Collectivité Territoriale (CT) in 2018 is supposed to be used for? Table 10. Q4 - Do you know what the money transferred to your Collectivité Territoriale (CT) in 2018 is supposed to be used for? Frequency Distribution across entire Mini-Survey Dataset Frequency Percent Question 4 responses 89 63% No, I don’t know. 31 22% Yes, I know. 22 15% I have a very rough idea 142 100% TOTAL Analysis and interpretation Here, again we can see that the same question asked in the mini-survey (to a mix SNGP beneficiaries and non￾beneficiaries) is being asked to random people in the public. Although the findings here are a bit contrasting. In the mini-survey 82% of respondents knew what budget was planned to be used for, whereas only 22% of the general public knows. This shows a dramatic need for more outreach to the general public, and particularly youth, on both the amount of money available for the development of their local community and also about the development plans for their community. This is the only way to ensure informed citizenship and civic responsibility. Annex III.C – Public Perception Survey Results Page | 93 PUBLIC PERCEPTIONS SURVEY FINDINGS Public Perception Survey Results by Response Rates This data is based on random interviews with Malians, aged 18 and older. Analysis: Clearly, the vast majority (80%) of Malians feel that a change of mentality or mindset is needed to address the deeply engrained culture of corruption that exists at all levels of society. Malians generally do not feel that overlaying systems, processes or technical training onto people will be enough to create the genuine change looked for. Malians feel that any real progress on decentralization and anti￾corruption efforts needs to emerge from a change from within the hearts and minds of the people themselves. A moral compass strengthening around public finance management is needed – a more social science-based approach to the problem set. Page | 94 Analysis: Clearly most Malians (82%) feel that the insecurity since 2012 has had an adverse impact on decentralization and public financial management. Thus the context of insecurity has likely rolled back years of progress made on decentralization and PFM. Page | 95 Analysis: Very few people in the public have any idea what their local level budget is for development activities in their communities. Without this knowledge, it is very difficult for civil society to have the information they need to advocate for the appropriate use of public funds at the community level. Page | 96 Analysis: Many more Malians know what their CT budget is supposed to be used for (63%) than know the actual budget amount available for those purposes. Knowing how much money is available in their community for development along with what the development plans are, can empower the local populations to play a more vital and informed advocacy and watchdog role. Page | 97 Analysis: This finding seems to reflect the lack of confidence and trust the local population has in the local authorities. This was also confirmed during many KIIs. This also speaks to the underlying culture of corruption that Malians are very aware of. Analysis: About half of the Malians surveyed feel women do have opportunities to participate and positively influence decentralization and PFM. The is fairly positive but clearly there is more work that needs to be done on this front. Page | 98 ANNEX IV – DATA COLLECTION INSTRUMENTS Annex IV.A - Draft Key Informant Interview Questionnaire Date: Region: Name: Stakeholder type: Organization: Position: Sex: Age: Notes on interview: KEY INFORMANT INTERVIEW QUESTIONS SUMMARY OVERVIEW All first order evaluation questions (i.e. the seven principal lines of inquiry) noted in the RFTOP are either presented as KII questions as formulated in the RFTOP or slightly modified but consistent with the intent of these questions. Sub-questions have been provided and are either taken directly from the RFTOP and TMG’s SNGP proposal or modified, but again consistent with the intent of RFTOP questions. All first order questions will be asked to each KI interviewed while sub-questions will be posed to and based on their relevance to concerned interviewee / stakeholder group. There is no intention of asking all sub-questions to any one respondent. One of the limitations in the use of this qualitative tool is that many questions will not have relevance to KIs because their knowledge of SNGP may be limited or sometimes even non-existent. Questions are broken down into two groups: 1) Core questions which correspond to the RFTOP main evaluation questions; and, 2) Supplementary questions which are not necessarily linked to SNGP’s mandate but which will provide critical data and analysis on contextual issues affecting the project and/or will better illuminate issues that the Core questions do not address. For instance, while Malian youth are not a principal target of SNGP, they are critical and defined societal group whose role and participation in decentralization and local governance has been largely marginalized. Similarly, while civil society and citizens are indirect beneficiaries of the SNGP, it will be important to get a better idea of their views on their role in decentralization and local governance as well as other stakeholder views. Page | 99 Before each KII interview, all KIs will receive an overall introduction to the evaluation including a description of SNGP, the purpose of the evaluation and the methodology that TMG will employ. A total of 120 are previewed for the KIIs and spread out over the five stakeholder groups noted in the RFTOP and TMG’s proposal A. CORE QUESTIONS 1. How would you characterize the overall performance of SNGP after 3 years of implementation in terms of the three components noted below, effectiveness in SNGP’s performance of its mandated responsibilities and the efficiency in which it undertakes its mandated functions? (Q1 – Efficiency, Efficacy and Appreciation of Activities: What activities have been most effective in line with the mandate of the project?) Can you provide some examples that exemplify the response you provided above? SNGP has three components: a) strengthening national and regional PFM systems (concerned national level ministries, and regional ADRs); b) improving coordination and communication between national and sub-national government and civil society; and c) Improving the capacity of CGSP to conduct performance audits and monitor public spending and service delivery at the national and sub-national levels. 1.1 Has SNGP been effective in building the capacity and/or improving performance of sub￾national entities (any of the three levels of the territorial collectivities)? ● Please provide an example of a concrete result in terms of a change in performance or the building of capacity 1.2 Can you cite an example of an SNGP-supported activity that has been appreciated by one or more project beneficiaries, including ● National level GOM Ministries, CGSP; ● sub-national entities: ADRs, circle and/or commune level organs; or, ● Civil society / Citizens: commune / community level associations, development NGOs, thematic or geographic CSO networks, cercle, regional or national level federated bodies (parent-teacher associations), think tanks, CSO representative / umbrella organizations? Page | 100 1.3 Can you cite an example of an SNGP initiative that…. ● Advances key project results / issue such as transparency, accountability, inclusivity and responsiveness by national and/or subnational government entities in public policy making; ● Promotes gender equality and women’s empowerment throughout the three levels of Malian governance; ● Demonstrates increased citizen participation national and sub-national decision making arenas; ● Treats Public Financial Management as an important tool in decreasing corruption; ● Promotes more effective performance of national and sub-national entities (ADRs, communal government, cercle and regional governments, ADRs; etc. 1.4 In your opinion have any SNGP interventions / activities had a major impact on one or more SNGP components: ● If yes, can you cite one or more examples? 1.5 In your opinion, do SNGP activities and results appear to be cost effective, particularly those generating high impact results? Can you provide an example? ● In your opinion, does SNGP appear to be an efficiently run project (resources such as number of staff, equipment/vehicles, etc., are appropriate for the achievement of results and activities defined by the project)? 1.6 In your opinion has SNGP been successful in improving coordination and communication between national and sub-national governments on the one hand and between them and civil society on the other; 2. How would you assess the impact and /or the achievement of results due to SNGP interventions in both the North and the South? (Q2 Project results and design – largely geared towards SNGP/USAID staff) 2.1 In your opinion, what is the most significant change that has taken place resulting from the project both in the North and/or the South? Why? 2.2 Does this change relate to a specific component of the project? And, what results have been achieved for each component of the project? Please explain. 2.3 Of which result are you most proud? (SNGP only) 2.4 If you were in charge of project implementation in the out-years, what would you recommend focusing on? Page | 101 2.5 Which activities do you feel are no longer needed because a) they have already been achieved; or, b) because they are no longer relevant to the current context? Please explain. 2.6 Can you suggest additional, perhaps innovative activities for the out-years of the project? 2.7 To what extent has national government confidence in sub-national government public financial management changed, and how has this impacted progress on transfer of resources to the local level? 2.8 Overall, how would you assess SNGP’s achievement of stated results and/or its three components? Please explain? 2.9 Looking only at SNGP’s efforts in the North, do you think conditions there are conducive to achieving any or all three project components, particularly those related to PFM? If yes, why and which ones? If no, why not and which components? 3. In your opinion: (a) what have been the principle challenges to and constraints impeding the implementation of the SNGP? (b) are there any changes in SNGP’s implementation, either its approach or the types of components or activities undertaken, that you would recommend? If so, what would they be? (Q3 Challenges and Bottlenecks to Implementation and Q4 Changes in approach) 3.1 From your point of view, how can these challenges be most effectively overcome? 3.2 From your experience with SNGP, has it made any mistakes in the implementation of project activities? If so, please name at least one example. 3.3 What significant adjustments have you observed in project management and implementation in the face of bottlenecks and problems? 3.4 From your perspective, do you think there is a need for the project to more proactively respond to the needs and the interests of GOM Ministries/officials? Civil society? If yes, what would you recommend? 4. From your knowledge of the Malian democracy and governance context, can you identify any opportunities or programmatic gaps that SNGP should consider pursuing in its final two years to increase the likely hood of achieving its results? If yes, what would they be? 4.1 What adjustments do you think should be made by SNGP in terms of the activities it funds, the partners with whom it works, the beneficiaries that it targets, etc.? 4.2 In your view can a project like SGNP influence the behavior of the Malian government and its political will to better support decentralization and regionalization reforms? If so, how? If no, why not? Page | 102 4.3 In your view, how can SNGP better support the decentralization of transfers and service delivery in the education and health sectors? 5. In your opinion, has the advent of Malian democracy increased the participation of Women in public policy making and decision making at the local level and beyond? (Q6 Gender: In your view, has SNGP’s gender integration strategy and plan been successful in terms of the technical assistance provided in the areas of capacity building, awareness raising and lobbying, for the purpose of increasing women participation and inclusion in formal decision making processes at all three levels decentralized governance?) 5.1 How have women and men been taken into account by SNGP in the design and implementation at all three levels of the Collectivite Territorial? 5.2 What gender strategy has the project taken to diffuse and put into practice gender principles and practices during the implementation of each of the project components? 5.3 Has the SNGP been able to measure progress of women’s and men’s participation in the process of Malian decentralization? 5.4 In your opinion, what should be done to address the challenges associated with the representation of women in determining public priorities, including how public resources are used in improving quality and access to decentralized services in education and health sectors? 5.5 What specific activities have been undertaken by SNGP to increase gender balance and to address gender disparities and inequalities in PFM and decentralization programs? 5.6 Have GOM actors developed new relationships or networks with CSOs to promote gender equity on account of the SNGP? 5.7 To what extent are regional governments formulating and executing budgets according to program areas and the needs of women and girls? 6. From your understanding of the project and its three components, what do you see as the impact of SNGP’s interventions in terms of increasing the capacity of concerned national and regional organs to improve the effectiveness and efficiency of decentralization including, decentralized service delivery? (Q5 Technical Impact): 6.1 What is the impact of the institutional and organizational capacity building the project is undertaking with Malian government institutions and sub-grantees? 6.2 How has the GOM, at both the national and regional levels benefited from the SNGP’s three component interventions? Page | 103 6.3 How are the twelve (12) embedded Technical Advisors performing and how can their role in the institutions be improved? 6.4 What is the most notable institutional change in the Malian government due to SNGP interventions? 6.5 To what extent is a verified increase in sub-national capacity in PFM contributing to the national government's willingness to decentralize? d) What support do the embedded TAs need in order to be more effective? B. SUPPLEMENTARY QUESTIONS S.1 In your opinion, how appropriate and relevant are SNGP’s three components to the current contextual reality of Mali, including the logic of the project design and its theory of change? If yes please explain. If no, please explain. (The overall project theory of change is that Central Government does not have confidence that local governments are capable of adequately managing the funds transferred to them supporting the discharge of their allocated competencies. From this overall theory of change, the three project components were developed.) ● In your view, does the validity of the SNGP’s design, including assumptions, make sense in light of today’s political realities, in terms of the role and functions of ADRs; the improved audit capacity (CGNP) of government; strengthening national and regional PFM systems; and, improving coordination and communication between national and sub-national government and civil society? If yes, why? If no, why not? ● One of the principal assumptions in the projects theory of change is that the political will exists to further decentralization reforms, including the transfer of local government allocations. In your opinion, did the political will exist at the time of the project’s design in 2013 and does it still exist today? ● Independent of SNGP’s performance in implementing the project, do you believe either the changing (from the time the project was designed in 2013 to the present) political and/or security context has had any impact on the project’s ability to achieve its results or demonstrate impact? S.2 In your opinion, has the advent of Malian democracy increased youth empowerment in areas of public policy making and decision making at the local level and beyond? ● Would you say that Malian youth have been marginalized in terms of their ability to participate in public life, at the local level and beyond, including public decision making? Page | 104 ● In your opinion, should SGNP add a new activity that target Malian youth? If so, what would you recommend? S.3 In your opinion has Malian Civil Society and citizenry been adequately and directly engaged by SNGP in its activities, including the three project components? If yes, how has it been engaged and in what ways? If no, how should it be engaged and in what ways? ● As a result of project activities, has decentralization increased CSO / Citizen’s: a) access to newly available government information? If so, to what extent has improved access to information resulted in increased public demand for accountability, transparency and responsiveness? b) Demand for improved access to and quality of services in health and education? c) effective participation in local governance matters, including decisions that determine the allocation and management of public resources; and, d) role in the oversight of local and regional governments (demand-side) and greater involvement in the delivery of public services (the supply-side)? e) To what extent has civil society engagement, resulted in more responsive and effective service delivery, especially for women, youth and vulnerable populations? Page | 105 Annex IV.B- Finalized Questions in French QUESTIONS D'ENTRETIEN AVEC L'INFORMATEUR CLÉ Nous sommes ici dans le cadre de l’évaluation à mi-parcours du programme SNGP au compte de l’USAID et vous avez été désigné comme l’un de nos informateurs. Nous souhaiterions avoir un entretien avec vous afin de connaitre votre avis sur certaines questions que nous nous posons dans ce cadre. Nous vous donnons l’assurance que tout ce qui sera dit ici ne sera utilisé que dans le cadre de cette évaluation. 1. Comment qualifieriez-vous la performance globale du SNGP après trois ans de mise en œuvre par rapport aux trois composantes indiquées ci-dessous, l’efficacité dans l’exécution des responsabilités qui lui ont été confiées et l’efficacité avec laquelle il s’acquitte de ses fonctions? (Q1 – Efficience, efficacité et appréciation des Activités: Quelles activités ont été les plus efficaces conformément au mandat du projet?) Pouvez-vous donner des exemples illustrant la réponse que vous avez donnée ci-dessus? ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… Date: Région: Name: Sexe: Organisation Fonction: Type d’organisation: Age: Notes sur l’interview: Page | 106 ………………………………………………………………………………………………………………………………………………………………… …………………………………………………………… 2. Comment évalueriez-vous l'impact des interventions du SNGP à la fois au Nord et au Sud? (Résultats et conception du projet Q2 – principalement destinés au personnel du SNGP / USAID) ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ……………………………………………………… 3. Selon vous: a) quels ont été les principaux défis et contraintes qui ont entravé la mise en œuvre du SNGP? (b) Y a-t-il des changements dans la mise en œuvre du SNGP, que ce soit son approche ou les types de composantes ou d’activités entreprises, que vous recommanderiez? Si oui, quels seraient-ils? (Q3: Défis et goulots d'étranglement dans la mise en œuvre et Q4 les changements dans l’approche) ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… Page | 107 ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ……………………………………………………………………………………… 4. A partir de votre connaissance du contexte Malien en matière de démocratie et de gestion des affaires publiques, pouvez-vous identifier les opportunités ou les lacunes programmatiques que le SNGP devrait envisager de poursuivre pour les deux années à venir afin d'accroître le potentiel de réalisation de ses résultats? Si oui, quels seraient￾ils? ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… …………… 5. Selon vous, l'avènement de la Démocratie Malienne a-t-il accru la participation des femmes à l'élaboration des politiques publiques et à la prise de décision aux niveaux National, régional et local (Q6 Genre: à votre avis, la stratégie et le plan d'intégration Page | 108 du genre de la SNGP ont-ils réussi? de l'assistance technique fournie dans les domaines du renforcement des capacités, de la sensibilisation et du lobbying, dans le but d'accroître la participation et l'inclusion des femmes dans les processus décisionnels formels aux trois niveaux de CT?) …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… ………………………………………………..………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… ……………………… 6. D'après votre compréhension du projet et de ses trois composantes, quel est l'impact des interventions du SNGP sur le renforcement des capacités des institutions nationales et régionales concernées afin d'améliorer l'efficacité et l'efficience de la décentralisation, y compris la prestation de services publics? (Q5 Impact Technique): …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… Page | 109 …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………….. QUESTIONS SUPPLÉMENTAIRES 1. À votre avis, dans quelle mesure les trois composantes du SNGP sont-elles appropriées et pertinentes pour la réalité contextuelle actuelle du Mali, y compris la logique de la conception du projet et sa théorie du changement? Si oui, veuillez expliquer. Si non, veuillez expliquer. (Selon la théorie globale du changement du projet dans son ensemble, le Gouvernement Central n’est pas convaincu que les acteurs locaux du gouvernement sont en mesure de gérer de manière adéquate les fonds qui leur sont transférés et d’appliquer les compétences qui leur ont été attribuées. Selon la théorie globale du changement trois composantes du projet ont été développées.) …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………………………………………………… ………………………………………….. Page | 110 2. À votre avis, l’avènement de la Démocratie Malienne a-t-il accru l’autonomisation des jeunes dans les domaines de l’élaboration des politiques publiques et de la prise de décisions au niveau local et au-delà? ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………… Selon vous, la Société Civile Malienne et la communauté ont-ils été correctement et directement impliqués par le SNGP dans ses activités, y compris les trois composantes du projet? Si oui, comment ont-elles été engagées et de quelle manière? Si non, comment devraient-elles être engagées et de quelle manière? ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… Page | 111 ………………………………………………………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………………………………………………… ……………………………………………………………. Page | 112 Annex IV.C - Key Focus Group Discussion Questions Date: Region: Stakeholder type: Number of participants: Names of participants and positions: Finalized Questions in French Finalized Questions in French FOCUS GROUP DISCUSSION LISTE DES QUESTIONS 1. Connaissez-vous le projet SNGP ou autre programme d’appui a la décentralisation au Mali ? 2. Comment qualifieriez-vous la performance globale du SNGP (après 3 ans de mise en œuvre) ou d’autre programme qui accompagne la décentralisation locale par rapport aux trois composantes indiquées ci-dessous, l’efficacité dans l’exécution des responsabilités qui lui ont été confiées et l’efficacité avec laquelle il s’acquitte de ses fonctions? (Q1 – Efficience, efficacité et appréciation des Activités: Quelles activités ont été les plus efficaces conformément au mandat du projet?) 3. Selon vous: a) quels sont les principaux défis et contraintes qui peuvent entraver la mise en œuvre du SNGP ou programme similaire? (b) Y a-t-il des changements dans la mise en œuvre du SNGP, que ce soit son approche ou les types de composantes ou d’activités entreprises, que vous recommanderiez? Si oui, quels seraient-ils? (Q3: Défis et goulets d'étranglement dans la mise en œuvre et Q4 les changements dans l’approche) 4. Dans quelle mesure l’état accepterait-il de transférer plus de ressources aux collectivités locales ? 5. Que pensez-vous des ADR dans le dispositif de régionalisation ? 6. À votre avis, l’appui du projet SNGP aux collectivités locales a-t-il accru la participation des femmes et des jeunes dans la gestion des affaires publiques. Comment ? Page | 113 Annex IV.D - Draft Mini-Survey Questionnaire – ENGLISH SNGP Mini-Survey Questionnaire Name of Region Respondents Age Group (tick) 18 - 25 26 - 35 36 - 45 46 - 55 56 and above Stakeholder Type USAID IPs/Subs Donor GOM Civil Society Sex Male Female Questions 1 Do you know approximately how much money was transferred to your Collectivité Territoriale (CT) by the central government for 2018? Yes No, I don’t know. I have a very rough idea. 2 Do you know what the money transferred to your Collectivité Territoriale (CT) in 2018 is supposed to be used for? Yes No, I don’t know. I know a little about that. 3 The capacity building/training that I received from SNGP was relevant and has enabled me to make a positive difference in the work I do. Strongly Disagree Disagree Not clear Agree Strongly Agree 4 The SNGP has significantly helped to increase decentralization of public resources. Strongly Disagree Disagree Not clear Agree Strongly Agree Page | 114 5 Donor funding has provided robust support for decentralization in Mali for 20 years now. Why do you think there has been so little progress to date? (choose best response even though more than one may be applicable) Lack of civil society participati on and advocacy Weak accountab ility in governanc e systems Not enough transpar ency in political systems Culture of corruption /mentality is blocking progress in Mali 6 The SNGP has significantly strengthened National and Sub-National Public Financial Management Systems. (comp1) Strongly Disagree Disagree Not clear Agree Strongly Agree 7 Coordination, openness and communications have improved between National and Sub-National Government and Civil Society on account of the SNGP. (comp2) Strongly Disagree Disagree Not clear Agree Strongly Agree 8 The Controle General pour Services Public (CGSP) capacity to monitor public spending and service delivery has been significantly improved at the national and sub-national levels on account of the SNGP. (comp3) Strongly Disagree Disagree Not clear Agree Strongly Agree 9 Since the security situation in Mali has gotten worse since 2012, overall, would you say decentralization and public financial management of resources improved, gotten worse or stayed the same? Improved Gotten worse Stayed same Page | 115 10 What is the biggest challenge faced by the SNGP? (choose best response) Trying to do too many things, not focused enough on key interventi ons needed There are fundamen tal problems with the logic, assumptio ns and approach of project Lack of political will 11 What is the biggest obstacle to decentralization of resources in Mali? (choose best response) Political will Corruptio n Lack of public financial manage ment capacity at local level Lack of public pressure from civil society groups 12 Women have been more empowered to participate in decentralization and public financial management processes because of SNGP support. Strongly Disagree Disagree Not clear Agree Strongly Agree 13 There have been notable improvements around health and education service provision because of the SNGP. Strongly Disagree Disagree Not clear Agree Strongly Agree 14 Mobilization of local resources (e.g. local tax collection) has been improved because of SNGP support. Strongly Disagree Disagree Not clear Agree Strongly Agree Page | 116 15 If sub-national public financial systems are strengthened and demonstrate an ability to manage resources better, the central government will be more willing to transfer resources to them. Strongly Disagree Disagree Not clear Agree Strongly Agree Page | 117 Annex IV.E - Draft Mini-Survey Questionnaire – French SNGP Mini-Survey Questionnaire Région Groupe d’âge Répondants 18 - 25 26 - 35 36 - 45 46 – 55 56 et plus Type d’Intervenant USAID SNGP/Part enaire Bailleur Gouverne mt Société Civile Sexe Homme Femme Questions 1 Connaissez-vous approximativement le montant des dotations financières reçus par votre collectivité territoriale en 2018 dans le cadre de la décentralisation ? Oui Non 2 Savez-vous à quoi doit servir l'argent transféré en 2018 à votre Collectivité Territoriale ? Oui Non 3 Est ce que le renforcement des capacités / formation donnés par le SNGP a été pertinent et a permis d’améliorer la façon de travailler chez les bénéficiaires. Je ne suis pas du tout d'accord Je ne suis pas tout à fait d'accord Je ne sais pas Je suis d'accord Je suis très d'accord Page | 118 4 Le programme SNGP a considérablement contribué à décentraliser davantage les ressources publiques. Je ne suis pas du tout d'accord Je ne suis pas tout à fait d'accord Je ne sais pas Je suis d'accord Je suis très d'accord 5 Les bailleurs de fonds ont fourni un appui solide à la décentralisation au Mali depuis 20 ans. Pourquoi pensez￾vous qu'il y a eu si peu de progrès à ce jour ? (choisissez la meilleure réponse, même si plusieurs peuvent être applicables) Faible participatio n de la société civile dans le plaidoyer, le suivi, et la planificatio n Faible redevabilit é dans la gestion des affaires publiques faible transpar ence dans la gestion des affaires publique s Une culture de corruption / mentalité bloque les progrès au Mali 6 Est ce que le SNGP a considérablement renforcé les systèmes de gestion des finances publiques au niveau du gouvernement central et des collectivités. (composante 1) Je ne suis pas du tout d'accord Je ne suis pas tout à fait d'accord Je ne sais pas Je suis d'accord Je suis très d'accord 7 Est ce que la coordination, la transparence, et les communications se sont améliorées entre le gouvernement central, les collectivités, et la société civile grâce au SNGP. (composante 2) Je ne suis pas du tout d'accord Je ne suis pas tout à fait d'accord Je ne sais pas Je suis d'accord Je suis très d'accord Page | 119 8 Est ce que la capacité du contrôle général des services publics (CGSP) à surveiller les dépenses publiques et la prestation des services a été considérablement améliorée aux niveaux de l’État et des Collectivités territoriales grâce au SNGP. (composante 3) Je ne suis pas du tout d'accord Je ne suis pas tout à fait d'accord Je ne sais pas Je suis d'accord Je suis très d'accord 9 Depuis que la situation sécuritaire au Mali s'est détériorée en 2012, diriez￾vous globalement que la décentralisation et la gestion des finances publiques des ressources se sont améliorées, se sont détériorées ou sont restées inchangées ? Améliorées Détériorée s Inchangé es Je ne sais pas 10 Quel est le plus grand défi du programme SNGP? (choisissez la meilleure réponse) Faire trop de choses a la fois, sans trop se concentrer sur les interventio ns clés nécessaires Il y a des problèmes fondament aux avec la logique, les hypothèses et l'approche du projet Manque de volonté politique Manque de compétence des acteurs de la mise en oeuvre du programme 11 Quel est le plus grand obstacle à la décentralisation des ressources financières et humaines au Mali ? (choisissez la meilleure réponse) VolontéPoli tique Corruption Faiblesse des capacités de gestion des finances publique s au niveau local Absence de pression publique de la part des organisations de la société civile Insuffisa nce des ressourc es financièr es et humaine s Page | 120 12 Les femmes ont été davantage habilitées à participer aux processus de décentralisation et de gestion des finances publiques grâce au soutien du SNGP. Je ne suis pas du tout d'accord Je ne suis pas tout à fait d'accord Je ne sais pas Je suis d'accord Je suis très d'accord 13 La fourniture de services de santé et d'éducation s'est considérablement améliorée grâce au SNGP. Je ne suis pas du tout d'accord Je ne suis pas tout à fait d'accord Je ne sais pas Je suis d'accord Je suis très d'accord 14 La mobilisation des ressources locales (par exemple, la collecte des impôts et taxes) a été améliorée grâce au soutien du SNGP Je ne suis pas du tout d'accord Je ne suis pas tout à fait d'accord Je ne sais pas Je suis d'accord Je suis très d'accord 15 Si la capacité de gestion des finances publique au niveau des collectivités est améliorée et que les CT démontrent plus de maitrise à gérer les ressources, l’Etat sera plus disposé à leur transférer plus de ressources ? Je ne suis pas du tout d'accord Je ne suis pas tout à fait d'accord Je ne sais pas Je suis d'accord Je suis très d'accord Page | 121 Annex IV.F - Radio text-in, public survey questions Question 1: What is necessary to ensure equitable decentralization of resources and reduce corruption in Mali? a) An internal change - a cultural or mentality change needs to occur b) An external change - better systems of good governance and accountability need to be put in place Question 2: Since the security situation in Mali has gotten worse since 2012, has decentralization, public financial management of resources improved, gotten worse or stayed the same? a) Improved b) Gotten worse c) Stayed the same Question 3: Do you know approximately how much money was transferred to your Collectivité Territoriale (CT) by the central government for 2018? a) No, I don’t know. b) Yes, I know. c) I have a very rough idea Question 4: Do you know what the money transferred to your Collectivité Territoriale (CT) in 2018 is supposed to be used for? a) No, I don’t know. b) Yes, I know. c) I have a very rough idea Question 5: If money is fairly transferred to your Collectivité Territoriale (CT) from the central government, do you trust that your local government will use the resources appropriately to develop the community? a) No b) Yes c) Maybe Question 6: Do women in your community have clear opportunities to positively influence decentralization and public financial management? a) No b) Yes c) I don’t know. Page | 122 Annex IV.G - Projet de questions de sondage public avec insertion de texte à la radio Question 1: Que faut-il pour assurer une décentralisation équitable des ressources et réduire la corruption au Mali? a) Un changement interne - un changement de culture ou de mentalité doit avoir lieu b) Un changement externe - de meilleurs systèmes de bonne gouvernance et de responsabilité doivent être mis en place Question 2: Puisque que la situation sécuritaire au Mali s'est détériorée depuis 2012, la décentralisation, la gestion des finances publiques des ressources s'est-elle améliorée, s'est-elle détériorée ou est-elle restée la même? a) s’est améliorée b) s’est empirée c) est restée le même Question 3: Savez-vous approximativement combien d'argent le gouvernement central a transféré à votre Collectivité Territoriale (CT) pour 2018? a) Non, je ne sais pas. b) Oui, je sais. c) J'ai une idée très approximative Question 4: Savez-vous à quoi doit servir l'argent transféré à votre Collectivité Territoriale (CT) en 2018? a) Non, je ne sais pas. b) Oui, je sais. c) J'ai une idée très approximative Question 5: Si le gouvernement transfère équitablement vos fonds à votre Collectivité Territoriale (CT), croyez-vous que vos leaders locaux utiliseront les ressources de manière appropriée pour développer la commune? a) non b) oui c) peut-être Question 6: Les femmes de votre communauté ont-elles clairement la possibilité d'influencer de manière positive la décentralisation et la gestion des finances publiques? a) non b) oui c) Je ne sais pas. Page | 123 Annexe IV.H – Radio text-in, public survey questions in Bamana Ni ye feere bolodalen yen ka jama gninika u hakilinaw la, n’u be se ka fili figne fe. Question 1: Mun de kan ka ke ka fangaseginniso sabati ani ka yurukuyuruku kele Mali la? a) Yelemali an yerebakun kan- Yelemali kan ka ke an ka taabolow la ani an ka hakilinaw la b) Yelemali an ka baaraketaw la- an ka sigidamara kecogo gnuman feere sigi sekan, ani ka bee joyoro yira i la walasa k’a minan fa. Question 2: Kabini Jamana ka basiki yoboyabara san 2012, fangaseginniso, forobanafolo barali oluw yiriwara wa walima u tignena de walima u tora u cogola? d) U yiriwara e) U tignena f) U tora u cogola. Question 3: Yala i b’a don nafolo hake min bilala ka na a’ ka Sigida la Govereneman fe wa san 2018? a) Ayi, n t’a don. b) Awo, n b’a don. c) N t’a don ten kosebe. Question 4: Yala i b’a don a’ ka sigidawari nanen ka k’an ka ke min ye wa? a) Ayi, n t’a don. b) Awo, n b’a don. c) N t’a don ten kosobe. Question 5: Ni govereneman ye a’ ka sigidanafolo bila ka n’a di a’ ka sigida gnemogow ma, yala I dalen b’u la k’u bena se k’a labaara a baaracogo gnuman na k’a’ ka sigida yiriwa wa? a) Awo b) Ayi c) Lala Question 6: Yala a se be a’ ka sigidamusow ye ka fangaseginniso ani forobanafolo baaraliw sinsin wa? a) Awo b) Ayi c) N t’a don. Page | 124 Annex IV.I - French Background Document for KIIs This document was shared with key informants before interviews, so they could have a background understanding of the purpose of the evaluation. Evaluation a mis parcours du Programme d’Appui à la Gouvernance de l'USAID au Mali Le but de cette évaluation est de fournir à l’USAID/Mali les constats, recommandations, et conclusions sur la performance des acteurs de mise en œuvre du projet et ses partenaires, et d’utiliser ces informations comme opportunité d’apprentissage dans la vie du programme afin de mieux éclairer les décideurs dans leurs prise de décisions par aux orientations futures pour les périodes à venir. L’objectif plus large est de mener cette évaluation de manière à contribuer à l’élaboration et à la mise en œuvre de politiques favorisant la décentralisation, la responsabilisation et la gestion saine des finances publiques au profit du peuple malien, et à renforcer ainsi son contrat social avec l’État. Cette évaluation évaluera l’efficacité des activités et des opérations du projet, y compris ses forces et ses faiblesses, et formulera des recommandations pour ajuster l’approche et les activités afin d’obtenir un impact et une efficacité maximum sur la base des enseignements récoltés. L’introduction de la prestation de services de santé et d’éducation comme priorité, ainsi que la contribution des ressources et des compétences du bureau de l’USAID pour la paix, la démocratie et la gouvernance, de la santé et de l’éducation, constitue également une étape importante dans l’intégration de la transversalité de la gouvernance dans les programmes de l’USAID. Notre équipe veillera à ce que cette évaluation soit pertinente et bénéfique pour tous ces bureaux, et contribue à une meilleure compréhension et coopération entre eux. Le projet met l'accent sur la gestion des finances publiques (GFP) en tant que thème catalyseur et comprend trois composantes: 1) renforcer les systèmes de GFP nationaux et régionaux; 2) améliorer la coordination et la communication entre le gouvernement national et sous-national et la société civile et 3) améliorer la capacité de l'Institut supérieur d'audit du Mali, le Contrôle général des services publics (CGSP), à mener des audits de performance conformes aux normes internationales et à surveiller dépenses et prestation de services aux niveaux national et sous￾national. Le projet fonctionne à l'échelle nationale et tirera parti des efforts renouvelés pour décentraliser le gouvernement malien par le biais de la régionalisation, c'est-à-dire le processus consistant à donner davantage de ressources et d'autorité directement aux gouvernements régionaux. Page | 125 TMG propose une approche avec des méthodes mixtes d’évaluation incorporant celles quantitatives et qualitatives qui ajoutent de l'ampleur et donne de la profondeur à l'évaluation tout en apportant des réponses approfondies aux questions posées. L’approche principale de TMG pour chaque question d’évaluation, comprend une description de (a) les données requises pour pouvoir y répondre, ainsi que des questions illustratives pouvant extraire les informations pertinentes, et (b) la source de données, y compris les documents, les individus ou les groupes les mieux placés pour les fournir. TMG envisage de mener des mini-enquêtes auprès de cinq grands types de parties prenantes, qui correspondent aux mêmes types de parties prenantes pour les Interviews des Informants clés (KII), et les discussions en focus groupe (FGD); à savoir: 1. USAID/Mali (y compris le personnel clé des bureaux pour la démocratie, la gouvernance, la santé et l'éducation) ; 2. Acteurs dans la mise en valeur (IP) et ses partenaires (y compris les 11 Assistants Techniques incorporés au sein de l’administration des collectivités) ; 3. Bailleurs de fonds et partenaires de développement (en particulier ceux travaillant sur la décentralisation, la gouvernance sous-nationale et la GFP) ; 4. Acteurs du gouvernement central et sous-national (y compris le personnel clé des ministères de l'économie et des finances, de la santé, de l'éducation, du développement local, des collectivités territoriales, du CGSP, ainsi que des entités régionales telles que les ADR, le CROCEP et le CROCSAD) ; 5. Société civile et citoyens (y compris les organisations de la société civile, les organisations de médias, les municipalités, les dirigeants et les membres de la communauté et les autres bénéficiaires officiellement reconnus). Vous avez été identifié comme élément de ces cinq groupes de parties prenantes qui contribuent ou bénéficient des interventions du SNGP, faisant de votre point de vue un élément important cette évaluation. Tout ce que vous direz est et restera strictement CONFIDENTIEL. Page | 126 Page | 127 Annex V: Sources of Information List of Persons Interviewed Region Organization/Title Name FGD KII Bamako SNGP Staff (4 KIIs) X SNGP/Directrice Administrative Mafoune Kone X SNGP Bamako /TA Mamadou Somboro X US Department of State (State) Phillip Gatins X State Housseini Intifaskwein X USAID Inna Bagayoko X USAID Sidi Yehia Cisse X USAID Mamoutou Coulibably X USAID Andrew Lucas X USAID Moussa Bambara X Support Cell for Decentralization and Deconcentration (CAAD) SISSOKO Fademba X Support Cell for Decentralization and Deconcentration / Education (CAAD) CAAD Educ. / Chargé communication & documentation X IP Partner / Representant WILDAF Bouare Bintou F. Samake X Chief of Section, Health / Support Cell for Decentralization and Deconcentration (CAAD) Dr. TRAORE Baboua X Health member, Support Cell for Decentralization and Deconcentration (CAAD) CAMARA Idrissa X FHI 360 CEP Program Quentin Kanyatsi X GSB (IP) staff Bassidy Diabate X District Mayor X District Mayor/SGAL KONE Mamadou Salah X Embedded TA / Tombouctou Yattara X CADD Health / SNGP Point Person DJIRE Bokary X Section Chief of Collectivités Territoriales / DNTC Publique TRAORE Pierre X CADD Health / Head of Public Health Dr. DEMBELE Marguérite X Officer of the General Inspector SOUMARE Samba X Representative of Banyan Global / (IP) Hadji Diakite X Representative of MSI / (IP) Diarra Zeinabou Ndiaye X President de la National Federation of Women’s Organizations (FENACOF) Ouleymatou Sow X Director of the National Association for Investment in the Territorial Communities (ANICT) Bamako X Finance for the Supreme Court SOW Abdoulaye X Training for the Territorial Communities Mr. Boubacar Maiga X Danish Cooperation Kola Sow X Ministry of Economy and Finance /Support Cell for Decentralization and Deconcentration (MEF/CADD) SIDIBE Seybou X Director General of Public Services KONATE Salimata DIAKITE X General Control of Public Services GOITA Bougouzanga X Director General/Regional Development Agency (ADR) of Bamako WADIDIE Mahamoudou X Assistant Director General/ Regional Development Agency (ADR) of Bamako Ag Maimoune X Page | 128 Head of Planning/ Regional Development Agency (ADR) of Bamako X Regional Development Agency (ADR) of Bamako/ Group 1 X Regional Development Agency (ADR) of Bamako/ Group 2 X Regional Development Agency (ADR) of Bamako/ Group 3 X Regional Development Agency (ADR) of Bamako/ Group 4 X Regional Development Agency (ADR) of Bamako/ Group 5 X Regional Development Agency (ADR) of Bamako/ Group 6 X Support Cell for Public Finance Reform CAMARA Boubacar X National Youth Council Souleymane S. Sidibe X Director General of Budget Sidiki Traoré X Budget Office Mr. Yanoga X Director General of Territorial Collectives (CT) Abdrahamane Cisse X Director of Public Function of the Territorial Collectives (CT) Brehima Bagayoko X Territorial Collectives (CT) Diakaridia Dembele X Director General of Taxation X President of the Association of Municipalities of Mali (AMM) Boubacar Bah X Decentralization Specialist Ousmane Sy X Coordinating President of the Chief of Neighborhoods, Bamako X President of the Parent/Teacher Association X Minister of Women’s Affairs and Parliamentarians X Swiss Cooperation Representative Mariam Farima Sissoko X Luxembourg Development Agency (LUXDEV) Representative Samba Biba X German Development Organization (GIZ) Representative Adama Sissouma X Mayor, 1st Commune KEITA Mamadou X Mayor, 2nd Commune NIARE Cheick Abba X Mayor, 3rd Commune DJIRE Mariam DIALLO X Mayor, 4th Commune BERETE Adama X Mayor, 5th Commune OUATTARA Amadou X Mayor, 6th Commune COULIBALY Alou X Mayor, 7th Commune BA Amadou X Secretary General, Town Hall, 1st Commune BA Amadou X Secretary General, Town Hall, 2nd Commune BOUARE Moussa X Secretary General, Town Hall, 3rd Commune SANOGO Kagnin X Secretary General, Town Hall, 4th Commune BOUGODOGOU Aboubacar X Secretary General, Town Hall, 5th Commune MAGASSOUBA Sidiki X Secretary General, Town Hall, 6th Commune Mme COULIBALY X SNGP Regional Embedded Technical Assistant X Mayors X Presidents of the Health and Education Committees X Women’s Organizations X Staff of the Regional Development Agency (ADR) X Regional Embedded SNGP Technical Assistant X Page | 129 Sikasso Governor Secretariat X Mayor Kalfa Sanogo X Mayor, SGAL X Second Deputy Mayor of the Urban Commune Adama Ballo X School Management Committee M. Ballo X Collectivities Treasurer Seybou Diakite X Director General Issa Kante X Deputy Director Moussa Kone X Head of Planning, Regional Development Agency (ADR) X Regional Development Agency Group (ADR) X Prefect of Sikasso Mahamadou T. Diarra X Sub-Prefect of Sikasso Mamoudou Diallo X Secretary of the Communal Council X Secretary General of the Regional Council Niara DIARRA X Head of Financial Services Abdrahamane Sidibe X Regional Director of Health Dramane Traore X Director of the Teachers’ Academy Baba Coulibaly X Deputy Department of Arts and Education X President of the Regional Council Yaya BAMBA X Vice President of the Regional Council X Regional Coordinator of Civil Society Organizations Mr. Dialakoro Keita X Regional Coordinator of Civil Society Organizations Issa Sissoko X Budget Monitoring Group (GSB) X Coordinator of the Association of Women’s NGOs (CAFO) Adjata Bamba X Rokia Coulibaly X Co-Coordinator of Women in Law and Development in Africa (WILDAF) Ma Berthe X Co-Coordinator of Women in Law and Development in Africa (WILDAF) DOUCOURE Awa Sidibe X Regional Director for the Promotion of Women and Children Keita Fatoumata S. X Regional Federation of Public Health Associations Daouda Sanogo X Regional Chamber of Commerce Brehima Coulibaly X UNICEF Partner Sekou Sidibe X President of CRJ Abdrahamane Saye X Women’s Civil Society Organization Adjata Bamba X Rural Development Agency Coulibaly OULIBALY X Mopti Governor Yacouba Maiga X Mayor Youssouf Nioumanta X Mayor, SGAL Joachim Traore X Second Deputy Mayor of the Urban Commune X School Management Committee X Collectivities Treasurer X Director General BOUNDY Hady X Deputy Director Magadougou Camara X Head of Planning, Regional Development Agency (ADR) SISSOKO Mady X Regional Development Agency Group (ADR) X Prefect of Sikasso Aliou Guindo X Sub-Prefect of Sikasso X Secretary of the Communal Council X Secretary General of the Regional Council Abdoulkarim TIMBINE X Page | 130 Head of Financial Services Moussa Samake X Regional Director of Health Aguissa Maiga X Director of the Teachers’ Academy Amadou DEGUENI X Deputy Department of Arts and Education Daouda Doumbia X President of the Regional Council Macki CISSE X Vice President of the Regional Council MAIGA Abdoulaye X Regional Coordinator of Civil Society Organizations Mme Diallo Tata X Regional Coordinator of Civil Society Organizations KANTA Amina X Budget Monitoring Group (GSB) MAIGA Ibrahim X Coordinator of the Association of Women’s NGOs (CAFO) Marie L. Haidara X Sire Cisse X Co-Coordinator of Women in Law and Development in Africa (WILDAF) DRABO Kadidia X Co-Coordinator of Women in Law and Development in Africa (WILDAF) X Regional Director for the Promotion of Women and Children X Regional Federation of Public Health Associations Fatoumata M. DEMBELE - X Regional Chamber of Commerce Ibrahima B. TOURE X UNICEF Partner Fatoumata Dicko X President of CRJ Djougal Goro X CARE Mali Hamouta Kornier X Administrative and Technical Services X Women’s Civil Society X Koulikoro Governor Mr. Samake X Mayor Mr. Eli Diarra X Mayor, SGAL BAGAYOGO Kaly X Second Deputy Mayor of the Urban Commune Mr. Kane X School Management Committee Mr Kontao X Collectivities Treasurer Lassana Doumbia X Director General KEITA N’Tji X Deputy Director TRAORE Nouhoum X Head of Planning, Regional Development Agency (ADR) KEITA Bakary X Regional Development Agency Group (ADR) SIDIBE Oumar X Prefect of Sikasso Abdoulaye Goita X Sub-Prefect of Sikasso Keita X Secretary of the Communal Council X Secretary General of the Regional Council Salif TAPILY X Head of Financial Services DIALLO Mahamoud X Regional Director of Health Dr. Youba Coulibaly X Director of the Teachers’ Academy Bakary Traore X Deputy Department of Arts and Education DOUMBIA Djigui X President of the Regional Council Abdoulaye Diakite X Vice President of the Regional Council X Regional Coordinator of Civil Society Organizations Mr. Lassine Mariko X Regional Coordinator of Civil Society Organizations FOMBA Youssouf X Budget Monitoring Group (GSB) X Coordinator of the Association of Women’s NGOs (CAFO) Ramata Diakite X Assitan Sidibe X Page | 131 Co-Coordinator of Women in Law and Development in Africa (WILDAF) KEITA Aminata X Co-Coordinator of Women in Law and Development in Africa (WILDAF) X Regional Director for the Promotion of Women and Children Awa Guindo X Regional Federation of Public Health Associations Fassery Doumbia X Regional Chamber of Commerce Baba Sanogo X UNICEF Partner X President of CRJ Sory Diarra X Director of ANICT Mohamed Ag. Faradji X Local Elected Officials X Civil Society X Regional Youth Council X Bibliography of Documents Reviewed ✓ GENDER ANALYSIS FOR THE SUB-NATIONAL GOVERNANCE PROGRAM – PRELIMINARY REPORT ✓ SECTION 3 RÉSUMÉ DES CONSTATIONS D’ÉVALUATION ET PLAN D’ACTION PROPOSÉ ✓ SUBCONTRACT NO. TETRA TECH-WILDAF-1085-004 SCOPE OF WORK ✓ SUBCONTRACT NUMBER TETRA TECH-GSB-1085-003 – GSB SCOPE OF WORK ✓ SUBCONTRACT NO. TETRA TECH-BG-1085-001 – BANYAN GLOBAL SCOPE OF WORK ✓ SUBCONTRACT - MSI SCOPE OF WORK ✓ YEAR 2 FY2017 ANNUAL REPORT WITH QUARTER 4 REPORT OCTOBER 2016- SEPTEMBER 2017 ✓ ANNUAL WORK PLAN PROJECT YEAR 3 ✓ QUARTERLY REPORT FY2018 - YEAR 3, QUARTER 1, OCTOBER-DECEMBER 2017 ✓ ECONOMY ASSESSMENT (PEA) DRAFT REPORT ✓ SNGP REVISED PMP FINAL APRIL 28, 2017 ✓ CONTRAT DE PARTENARIAT AVEC LES AGENCES DE DEVLOPPEMENT REGIONALES ✓ CONTRAT DE PARTENARIAT AVEC LES CADD MEF, EDUCATION, ET SANTE Page | 132 Annex VI: Summary information about evaluation team members TEAM COMPOSITION AND DYNAMICS The TMG evaluation had the right mix of team members with regards to analytical skills, technical skills and experience, but also critically, interpersonal skills and the ability to work positively together under stress, both with USAID staff and between the team members themselves, which is often key to the successful implementation of a robust and meaningful evaluation. To ensure the highest quality team for this evaluation, TMG invested heavily in the selection of the best mix of team members for this important mid-term evaluation. TMG even sent a technical person from our home office staff to the field in Mali to meet with local firms and local technical specialists to help vet the best candidates to work with for this evaluation. TMG also hired a local Malian expert to help with interviewing and seeking out the best local candidates, leveraging TMG’s vast network of Malians working in the governance sector from our experience working in Mali. It is because of this intentionally thorough vetting process that TMG was able to select the outstanding, holistic team presented here - where the whole is truly greater than the sum of its parts. Summary qualifications of our personnel are provided below (detailed resumes can be found in Annex B.) Mr. Leslie M. Fox is our Team Leader for this evaluation. Mr. Fox has over 30 years of experience working in international development and leading complex evaluations of development projects, particularly in the sectors of democratic governance and decentralization. Mr. Fox has a master’s degree from Johns Hopkins University’s School of Advanced International Studies in International Public Policy. He has produced 27 professional publications focused largely on democratic governance, decentralization, civil society and gender equity within the African context. Mr. Fox understands the Malian decentralization context from a historical perspective as he has been intimately involved with it starting with his facilitating the Mission’s first DG strategy and training the DG Results team in 1994; his 2002 “Assessment of Democratic Governance in Mali”. More recently, in 2012, he worked with USAID/Mali’s governance team and drafted the “Assessment of Decentralization and Local Governance in Mali” for the mission. Mr. Fox is fluent in both French and English and has a passion for working on decentralization related issues in the Malian context. We feel his insightful nature, wealth of knowledge and experience, clear writing skills, along with his excellent ability to lead evaluation teams has ensured a robust evaluation of the Mali SNGP that will exceed the expectations of the mission. Mr. Fox ensured that all team members understood their various roles and responsibilities from startup and provided oversight and management of the entire evaluation process. Mr. Fox has been the main communications point of contact with the USAID/Mali SNGP COR. Mr. Boubacar Dramane Traore is our Decentralization Specialist for this evaluation. Mr. Traore has a master’s degree in economics with a specialization in quantitative analysis and economics policies. He also has a bachelor’s degree in quantitative analysis and economic policies. He is currently working on his second master’s degree in project cycle management and development policies. Mr. Traore has 13 years of experience both managing and evaluating decentralization programs in Mali. Mr. Traore has also facilitated workshops and provided training and capacity building relating to decentralization processes in various capacities. He has also led dissemination workshops on decentralization and local resource mobilization, thus he can play a critical role in leading the dissemination workshop for this evaluation activity. Mr. Traore also conducted a study on the role of the territorial communities in the Page | 133 implementation of the Peace and Reconciliation Agreement resulting from the Algiers Process. Mr. Traore is fluent in French, with basic written and spoken English skills. He is also fluent in the local languages of Bamana and Malinke. Mr. Traore played a leading role on providing inputs for decentralization questions for both the qualitative and quantitative survey work. Mr. Traore also provided analytical support on decentralization findings for the final evaluation report. Ms. Kouyaté Aissé Diarra is our Gender Specialist for this evaluation. She is an experienced gender specialist with a career that spans over 20 years. She has a master's degree in rural sociology with academic and field research experience in health, local development and gender mainstreaming. In the health sector, she worked on many evaluation projects in the areas of maternal, neonatal and infantile health, fight against practices harmful to the health of women and girls, and in the fight against child marriage, obstetric fistula, etc. She also participated in the elaboration of the Ten-Year Program of Social Health Development of Mali. For 10 months, Ms. Kouyate worked closely with the CRM in the evaluation of gender aspects in Maternal, Neonatal and Infantile Health project in order to address the barriers related to gender in the regions of Sikasso and Koulikoro. Moreover, in the gender sector, after participating in a Training of Trainers at KIT in Amsterdam, she leveraged gender analysis tools for the training of project officers in gender equality and for studies conducted on the integration of gender equality for NGOs, African institutions. For instance, she conducted gender training for GEDEFOR, developed a gender profile of Burundi for AfDB, and facilitated gender integration in the Kandadji project and the OMVS Dam Project for BOAD. Her skills in English and her analytical and writing skills was a great asset to the team. Ms. Kouyaté ensure gender sensitivity during all survey work and interviews and she provided analytical support on gender issues for this evaluation. Ms. Kouyaté also ensured a health and education lens was brought to bear on the questions asked during field work, to ensure that USAID’s investments from the health and education teams were having an impact. Mr. Brehima Doumbia is our Public Financial Management Specialist for this evaluation. Mr. Doumbia has a master’s degree in Management with a focus on accounting. Mr. Doumbia has worked and led evaluations and various research studies relating to public financial management in Mali for over 15 years. His research and experience have provided him with expert knowledge of the local context in the areas of local resource mobilization and local systems of taxation, financial transparency, budget and accounting issues at the commune level and capacity building around public financial management in Mali more generally. Mr. Doumbia is fluent in French and Bamana with intermediate English speaking and writing skills. Mr. Doumbia provided leadership on providing inputs for public financial management related questions for both qualitative and quantitative survey work. Mr. Doumbia also provided analytical support on public financial management related findings for the final evaluation report. Mr. Cheick Coulibaly is our Field Work Coordinator for this evaluation. Mr. Coulibaly is a Senior Management Specialist with a multidisciplinary education. He has a bachelor’s degree in Computer Systems Engineering from the Capital Institute of Technology in Laurel, MD. Mr. Coulibaly worked for over 27 years with USADI/Mali in various capacities ranging from information systems management to working as the mission EXO. Mr. Coulibaly has a wealth of local contacts and expertise around project procurement, administration, logistics and reporting. He is fluent in written and oral English and is also fluent in French and Bamana. Since the timeframe for the field work in this evaluation was very tight, coupled with the strong need for a robust evaluation, Mr. Coulibaly’s skills in logistics and other gap￾filling functions for the evaluation team was critical to the team’s success. This position ensured that all Page | 134 logistics and administrative work for the evaluation team, along with coordination with the local firm working on the mini-survey were effective. The Field Work Coordinator allowed the evaluation team to focus 100% on their technical work without being burdened by logistics, admin functions and any other constraints that arose in the field while working. Mr. Coulibaly’s relationship with and knowledge of the USAID/Mali mission’s policies and procedures certainly enhanced the quality of the work and communications between USAID and the evaluation team. While the evaluation team focused on the qualitative work in the field, out local partner firm, the Center for Studies, Documentation, Research and Training (CEDREF), focused on the quantitative field work implemented by local enumerators for the mini-survey. CEDREF was selected amongst the various local firms TMG examined for this evaluation and it was determined they had the capacity, experience, time and passion necessary to effectively work with the team. CEDREF has been supporting evaluations around governance and decentralization in Mali since 2007. They are conveniently located five minutes away from the US Embassy and have experience working on evaluations with USAID (Mali Election Impact Evaluation), the World Bank and the French Development Agency, amongst others. CEDREF provided: office space for local meetings (e.g. KIIs, FGDs) for the evaluation team, tablet computers and database support for enumerators, training (working closely with evaluation team’s decentralization specialist who is a quantitative analyst) and oversight of data collectors, vehicles for field work, and other services. TMG’s Field Work Coordinator, Mr. Coulibaly, worked closely with the local firm on logistics as many of the same stakeholders that will be interviewed for the qualitative work in the field were also surveyed as part of the mini-survey. Since our Decentralization Specialist, Mr. Traore, also specializes in quantitative methods and analysis, he supported on oversight and quality assurance to ensure the proper training and understanding necessary for the enumerators. TMG home office provided extensive home office backstop support to the evaluation team. As TMG saw this evaluation as strategically important, the overall management and oversight of the contract was provided by Mr. Abou Kone, Program Manager for TMG. Mr. Kone, fluent in French and English, ensured that this contract had dedicated backstop support in handling logistics, contracts, administration and financial matters. Quality assurance for the final evaluation report, all data analysis and insights gleaned from data analysis were provided by TMG’s Senior Strategic and Technical Advisor, Mr. Jeremy Chevrier. Mr. Chevrier lived and worked in Mali from 2005 through 2012, including five years working directly for the USAID/Mali mission. Statistical correlations work for quantitative analysis was provided by Lam Huynh, a PhD graduate student of statistics and quantitative analysis. Mr. Huynh work as a fellow for USAID/Senegal for two years.