December 30, 2019 This report was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Denis Dunn, Team Leader, and Dr. Mehdi Ben Braham, Evaluation Specialist, for International Business & Technical Consultants, Inc. (IBTCI), under the Monitoring and Evaluation in Tunisia and Libya (METAL) activity, Contract No. AID-280-TO-17-00001. EVALUATION MIDTERM EVALUATION OF FISCAL REFORM FOR A STRONG TUNISIA (FIRST) ii MIDTERM EVALUATION OF FISCAL REFORM FOR A STRONG TUNISIA (FIRST) December 30, 2019 International Business & Technical Consultants, Inc. (IBTCI) In the US: 8618 Westwood Center Drive Suite 400 Vienna, VA 22182 USA Tel : +1.703.749.0100 In Tunisia: 1er étage Bloc Bleu Résidence Vital Cube 4 Rue de l’ile Tasmanie Les Berges du Lac 2 1053 Tunis AID-280-TO-17-00001 Prepared by Denis Dunn (Team Leader), Dr. Mehdi Ben Braham, Dr. Kamil Wanna, Ahmed Trabelssi, and Amal Lajmi. DISCLAIMER This report is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of IBTCI and do not necessarily reflect the views of USAID or the United States Government. iii ABSTRACT Fiscal Reform for a Strong Tunisia (FIRST) is a three-year US$17.3 million USAID activity that began in September 2017. FIRST aims to provide technical assistance (TA) and capacity-building to the Ministry of Finance (MOF) to strengthen the fiscal foundations for sustainable and inclusive growth and thereby improve the conditions for business investment and job creation. USAID/Tunisia requested a midterm evaluation of FIRST to identify the extent to which the activity is achieving intended results, and as well assess the relevance, effectiveness, and efficiency of the aid delivery. FIRST’s designed approach to facilitate fiscal reforms and build MOF capacity through demand-driven technical assistance, tools, training, and technology was relevant but an implementation strategy was not articulated. Year 1 delivery was not very effective and in Year 2, despite a more focused approach and accelerated activity progress, there were few concrete achievements. Efforts to reform tax administration are more likely than tax policy to yield results in the short term, and outreach efforts that make for a more open government can promote voluntary tax compliance. FIRST was able to mitigate many constraints on implementation, but needs to extend its partnering to donors, the private sector, and civil society. It should also continue to enhance the sustainability of its inputs, complete all main activities, and ensure interventions are adopted by the MOF. iv ACRONYMS AFD Agence Française de Développement AfDB African Development Bank AMEG Asia and Middle East Economic Growth Best Practices Project AMELP Activity Monitoring, Evaluation, and Learning Plan CD Capacity Development CDCS CGE Country Development Cooperation Strategy (USAID) Computable General Equilibrium CIME Centre d’Impôts des Moyennes Entreprises – Medium Enterprises Tax Office CIMF Centre Informatique du Ministère des Finances – Ministry of Finance Information Center CIT Corporate Income Tax COP COPIL Chief of Party Comité de Pilotage – Steering Committee COR Contracting Officer’s Representative CPF Country Partnership Framework CS Civil Society DGCPR Direction Générale de la Comptabilité Publique et du Recouvrement – National Tax Collection Department DGD Direction Générale des Douanes – National Customs Department DGE Direction des Grandes Entreprises – Large Taxpayers Office DGELF Direction Générale des Études et de la Législation Fiscales – General Directorate of Legislative and Fiscal Studies DGI DGRE DME Direction Générale des Impôts – National Tax Administration Department General Directorate for Resources and Equilibrium Directorate of Medium-Size Enterprise DMS Data Management System DO Development Objective DQA Data Quality Assessment EQ Evaluation Question ET Evaluation Team EU European Union FATCA Foreign Account Tax Compliance Act FAU Unité d’Analyses Fiscales – Fiscal Analysis Unit FG FGD Focus Group Focus Group Discussion FIRST Fiscal Reform for Strong Tunisia FIS Fiscal Information System FY Financial Year GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit GBO Gestion du Budget par Objectifs – Budget Management by Objectives GOT Government of Tunisia IACE Institut Arabes des Chef d’Entreprises IBTCI International Business & Technical Consultants, Inc. v ICT Information and Communications Technology IMF International Monetary Fund IP Implementing Partner IR Intermediate Result IT Information Technology ITCEQ Institut Tunisien de la Competitivite et des Études Quantitatives J2SR Journey to Self-Reliance KfW Kreditanstalt für Wiederaufbau KII Key Informant Interview KPI Key Performance Indicator M&E Monitoring and Evaluation MDiCI Ministry of Development, Investment, and International Cooperation MEGC Modèle d'équilibre général calculable (macro-economic model) MEL Monitoring, Evaluation, and Learning METAL Monitoring & Evaluation for Tunisia and Libya MOF Ministère des Finances – Ministry of Finance MOU Memorandum of Understanding MTFF Medium-Term Fiscal Framework NGO Nongovernmental Organization OEG Office of Economic Growth (USAID) PFM Public Financial Management PIT Personal Income Tax PMS Performance Management System PS Private Sector RNE Register National des Enterprises SCU SMEs Strategic Communications Unit Small and Medium-Sized Enterprises SOPs Standard Operating Procedures SOW STTA Scope of Work Short-term Technical Assistant TA Technical Assistance TADAT Tax Administration Diagnostic Assessment Tool TCP Tax and Customs Reform Pilot TOC Theory of Change TOT Training of Trainers TND Tunisian Dinar UGTT Union Générale Tunisienne du Travail/Tunisia General Labor Union USG United States Government USAID United States Agency for International Development UTICA UXP Union Tunisienne de l’Industrie, du Commerce, et de l’Artisanat/Union of Industry, Trade, and Handicrafts Unified Data Exchange Platform VAT Value-Added Tax vi GLOSSARY OF TERMS Country commitment: The degree to which laws, policies, actions, behaviors, and informal governance mechanisms—such as cultures and norms—enable a country to create and strengthen the institutions to solve its own development challenges. This includes commitment toward democracy (or open and accountable governance), inclusive development (inclusiveness across gender, social groups, and geographic sub-regions), and sound economic policy (microeconomic and macroeconomic). Country capacity:1 How far a country has come in its journey across the dimensions of political, social, and economic development, including the ability to work across these sectors. A country’s capacity to plan, resource, and manage its own development hinges on the capacity of the government (including the quality of government services and the competence of civil servants, as well as the government’s ability to mobilize domestic resources, and to maintain stability and security), the capacity of civil society including free media (as a means to hold government accountable and to provide mechanisms beyond elections by which citizens can be heard), the capacity of a country’s citizens (essentially, the extent to which citizens are engaged and informed, and leading productive and meaningful lives), and the productivity and functioning of the economy (including the extent to which the private sector can generate sustained, broad-based economic growth). Public financial management (PFM): 2 Effective administration of funds collected and spent by governments. It underlies all government activity and incorporates all components of a country’s budget cycle including the mobilization of revenue, the allocation of these funds to various activities, and the expenditure and accounting for spent funds. Capacity development (CD): Strengthening individual knowledge and skills, as well methodologies and procedures of the partner organization, and to a lesser extent, addressing administration structures, information technology (IT) infrastructure, and equipment. Change agent: Person within the target organization who facilitates strategic transformation. Champion: Person external to the target organization who voluntarily takes interest in and facilitates the adoption, implementation, and success of a cause, policy, program, project, or product. End user (end beneficiary): Person, group, or organization affected by the results that an intervention has on the performance of a target organization. Intervention: A donor-supported action, such as a program, project, or activity. Intervention logic: A description of a sequence of events that is expected to lead to a particular desired outcome. It includes the identification of the underlying assumptions and is often used interchangeably with theory of change (TOC). Legitimacy: The degree to which partner organizations accept and are satisfied with the donor’s support to development processes, in particular capacity development. Partner organization: Organization with which the project engages in the planning, implementation, and/or review of a given intervention. Responsiveness: The degree of alignment of the donor effort with the characteristics of a given intervention in terms of the legitimacy awarded by the cooperation partner(s), the intra- and extra-organizational context in which the intervention takes place, and the level of specification of and attention to capacity development results. Results-based management: A management strategy focusing on performance and the achievement of outputs, outcomes, and impacts. 1 USAID Self-Reliance Metrics FY2019 Methodology Guide. USAID. September 2018, p. 5. 2 Support to Capacity Development: Identifying Good Practice in Swedish Development Cooperation. SIDA, 2015. vii TABLE OF CONTENTS ABSTRACT iii ACRONYMS iv GLOSSARY OF TERMS vi EXECUTIVE SUMMARY viii INTRODUCTION 1 EVALUATION PURPOSE AND EVALUATION QUESTIONS 3 EVALUATION DESIGN, METHODS, AND LIMITATIONS 3 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 7 Findings and Conclusions 7 Recommendations 25 LESSONS AND OBSERVATIONS 26 ANNEXES 29 ANNEX I: FIRST LOGIC MODEL 30 ANNEX II: EVALUATION SCOPE OF WORK 31 ANNEX III: DATA COLLECTION INSTRUMENTS 38 ANNEX IV: DATA COLLECTION AND ANALYSIS PLAN 47 ANNEX V: SOURCES OF INFORMATION 50 ANNEX VI: FINDINGS-CONCLUSIONS-RECOMMENDATIONS (F-C-R) MATRIX 53 ANNEX VII: EVALUATION TEAM MEMBERS’ QUALIFICATIONS AND ROLES 61 ANNEX VIII: TECHNICAL NOTES 62 ANNEX IX: DISCLOSURE OF CONFLICT OF INTEREST 65 ANNEX X: FIRST COMMUNICATION MATERIALS 66 ANNEX XI: CAPACITY DEVELOPMENT: SIX KEY MESSAGES 68 ANNEX XII: TUNISIA FIRST KEY RESULTS – CUMULATIVE YEAR 1 & YEAR 2 69 ANNEX XIII: FIRST ORGANOGRAM 71 ANNEX XIV: FIRST CONSULTANTS AND ASSIGNMENTS 72 ANNEX XV: FIRST PROGRESS SUMMARY (SOURCE: YEAR 3 WORK PLAN TABLE 1) 73 ANNEX XVI: FIRST IMPLEMENTATION EFFICIENCY FACTORS 75 Table 1: Linkages Between Component 1, Workstreams 1-3, Key Results, and EQ 2a and 2b...........14 Table 2: Linkages Between Components 2 & 3, Workstreams 4 & 5, Key Results, and EQ 2c ...........15 Table 3: Linkages Between Component 4, Workstreams 6 and 7, Key Results, and EQ 2d.................16 viii EXECUTIVE SUMMARY Background The Government of Tunisia (GOT) is facing a considerable fiscal gap—the difference between public expenditures and government revenues. There is a pressing need to shrink the gap with policy reforms targeting sectors of the economy that underpay or are virtually untaxed, and with tax administration reform to address cumbersome processes for filing and payment. Lack of public awareness on fiscal issues must be addressed to counter the fierce resistance to changes in the tax code and/or subsidies. Fiscal Reform for a Strong Tunisia (FIRST) is a three-year, US$17.3 million USAID activity that began with an inception phase during September/October 2017. FIRST builds on USAID efforts to improve public financial management in Tunisia and follows on from the Public Financial Management (PFM)/Information and Communications Technology (ICT) Competitiveness and Tax and Customs Reform activities at the Ministry of Finance (MOF). FIRST’s goal is to provide technical assistance (TA) and capacity-building to strengthen the fiscal foundations for sustainable and inclusive growth with an aim to improve the conditions for business, investment, and job creation. FIRST has four objectives: (1) improve the efficiency, transparency, and cost of compliance of tax administration; (2) enhance MOF capacity to develop and manage tax policy; (3) enhance capacity to address other fiscal reform priorities as they emerge; and (4) improve communications, engagement, and consultation on priority reforms. Evaluation Purpose USAID/Tunisia requested USAID/Monitoring and Evaluation for Tunisia and Libya (METAL) to conduct a Midterm Performance Evaluation of FIRST. The evaluation purpose is to identify the extent to which FIRST is achieving intended results, and to assess the relevance, effectiveness, and efficiency of the approaches used to achieve USAID/Tunisia objectives. The evaluation also aims to identify successes, best practices, lessons learned, and challenges that affected the implementation of the activity, as well as identify potential course corrections before the project ends in August 2020. Evaluation Design The scope of work (SOW) for the FIRST evaluation specified the following five research questions: 1) To what extent was the designed activity approach relevant to the Tunisian government needs? Is the approach still relevant? 2) To what extent, and how did FIRST contribute to improving revenue collection, reducing taxpayers’ compliance costs, improving the budgeting and expenditure process, and enhancing public accountability? 3) What internal (organizational, resources, etc.) and external factors affected the efficiency and timeliness of the implementation and how? 4) To what extent has FIRST succeeded in identifying, partnering, and coordinating with various stakeholders? What other partnerships will strengthen FIRST, and similarly, USAID/Tunisia activities? 5) To what extent are FIRST activities sustainable and contributing to Tunisia government effectiveness (as per the Journey to Self-Reliance [J2SR] 4.GC.1)? The evaluation team (ET) gathered data and information from a desk review, 31 key informant interviews (KIIs), and five focus group discussions (FGDs) with MOF staff. The desk review comprised project documentation and other relevant literature. Interviews were conducted with FIRST activity ix staff, MOF counterparts, and representatives from the donor community, private sector, and civil society. Key Findings and Conclusions The evaluation’s key findings and conclusions are summarized below as per evaluation questions. For a fuller reading of the key findings below, refer to the main body of the report. The Evaluation Team also offers some lessons and observations at the end of the report. EQ 1: To what extent was the designed activity approach relevant to Tunisian government needs? Is the approach still relevant? The designed activity approach for the activity as outlined in the implementing partner (IP) contract was not elaborated by FIRST and a “reset” was necessary in Year 2 to put FIRST back on track. Key findings are as follows:  Although a technical approach was succinctly stated in the IP contract as “demand-driven technical assistance, capacity-building, facilitation, and commodities support,” an implementation strategy was not subsequently elaborated in FIRST work plans.  The proposed comprehensive assessment of Tunisia’s fiscal system in Year 1 was questionable when the MOF was initially apprehensive and skeptical – nevertheless the Tax Administration Diagnostic Assessment Tool (TADAT) proved to be highly effective in highlighting deficiencies in the fiscal system to better inform donor support, as well as establishing baselines for FIRST and benchmarking for the MOF.  In Year 1, FIRST also struggled to establish effective working relations with the MOF, resulting in few tangible accomplishments by year end.  Following a “reset” in Year 2, FIRST focused on digitization and e-services to facilitate core tax administration processes within the MOF.  FIRST was also urged in Year 2 to follow an approach that was led by the MOF and supported by experts to enhance partner “buy-in” or ownership of activities/results.  Although FIRST was certainly more responsive in Year 2,3 there were difficulties fully engaging the MOF with respect to obtaining input into work-planning, the use of TA, and feedback on reporting. The absence of an implementation strategy meant USAID and the MOF could not fully understand FIRST’s overall technical approach. Though TADAT was not a priority for the MOF, the comprehensive assessment nonetheless proved to be a worthwhile exercise for all concerned parties. However, the focus on a deep diagnosis and ensuing delays likely contributed to the tardiness of FIRST implementation in Year 1. The shift toward inherently complex IT solutions in Year 2 added to the technical challenges of support. In Year 2, FIRST proved to be more flexible and effective in its participatory approach toward the MOF. EQ 2: To what extent and how did FIRST contribute to (a) improving revenue collection, (b) reducing taxpayers’ compliance costs, (c) improving the budgeting and expenditure process, and (d) enhancing public accountability? It was not possible to determine the extent of FIRST’s contribution at the outcome and impact levels but FIRST productivity at the output level improved in Year 2. The key findings on to what extent FIRST contributed are as follows: 3 Doing Business, Foreign Account Tax Compliance Act (FATCA), customs post-clearance audit, and change management were added while fiscal decentralization, pensions, and public sector wage bill activities were dropped. x  Although the FIRST TOC/logic model is sound, many high-level indicators are not pragmatic or meaningful (FIRST effects on tax revenues or fiscal deficit are difficult to validate).  There were no process indicators to measure implementation progress, and no data management system to track and report on the indicators.  It is too early to look for outcomes within two years (confirmed by the European Union [EU], International Monetary Fund [IMF], and MOF), except in public accountability as a survey of taxpayers can provide feedback on the fiscal system.  Only three of 27 key results (main activities) were fully completed by the end of Year 2. The key findings on how FIRST contributed are as follows:  For improving revenue collection (workstreams 1 and 3), FIRST introduced the Unified Data Exchange Platform (UXP) that is a valuable source of data for analysis of tax gaps, replaced lengthy and traumatic audits with a more effective audit strategy supported by rapidly collected payments for back taxes, set up the Medium Taxpayer Office, and improved the system for value-added tax (VAT) refunds.  For reducing compliance costs/easing burden on taxpayers (workstream 2),4 FIRST is broadening the tax base by registering taxpayers, upgrading the taxpayer registry, revising the penalty system, digitalizing MOF (e-services) to facilitate filing and payment, training tax collection staff in customer services, and training private sector accountants on e-filing.  For improving the budgeting and expenditure process (workstreams 4 and 5), FIRST is delivering models and methodologies to better understand tax gaps, tax incidence, tax expenditures, and specific subsidies, and improving Fiscal Analysis Unit (FAU) capacity to quantify policy impacts on revenues and to better communicate with stakeholders on proposed tax reforms.  For public accountability5 (workstreams 6 and 7), FIRST is delivering webcasting (intranet) to link Tunis to eight regions, strengthening MOF’s strategic communications unit and Call Center, supporting media campaigns, redesigning the MOF website, and producing a Taxpayer’s Practical Guide.  For gender equity, there was a gender balance in FIRST staffing, in MOF beneficiaries, and in training participants, but not in short-term technical assistants (STTAs) deployed by FIRST. FIRST delivery was lacking in Year 1 (except for TADAT) and, although activity accelerated in Year 2, there were few concrete accomplishments. Understanding real progress on key results without process indicators and milestones is difficult.6 In the short term, efforts to reform tax administration are more likely than tax policy to yield tangible results, while outreach efforts can rapidly make government more transparent and promote voluntary tax compliance. Little was done by FIRST to advance opportunities for women in the fiscal domain, despite FIRST’s gender strategy and indicator.7 EQ 3: What internal and external factors affected the efficiency and timeliness of the implementation and how? Many factors facilitated and/or hampered the efficiency and timeliness of FIRST delivery— some were “success factors” but most were constraints on implementation. Key findings are as follows:  Activity duration of only three years is insufficient for meaningful fiscal reform so there is real concern that FIRST’s ambitious program will not be fully completed in Year 3. 4 TADAT yielded a compliance strategy for the MOF together with a detailed action plan. 5 Meaning raising taxpayer awareness, promoting voluntary tax compliance, and fostering public trust in the tax system. 6 Tables on pp.15-17 show links between objectives, sub-objectives (or workstreams), key results, and evaluation questions. 7 There is no reporting on gender in the FIRST 2nd Annual Report or QPR J/A/S, and no gender planning in Year 3 work plan. xi  For sustainability, the MOF needs to “institutionalize”8 reform initiatives but the final step (“last mile”) is out of FIRST control.  A participatory approach is good practice but slows the process of implementation, whereas bypassing the partner on input into TA use speeds up delivery but is not good practice.  Not being embedded in partner premises to be close to counterparts and beneficiaries diminishes the efficiency and effectiveness of FIRST experts in terms of communication, collaboration, and capacity development (CD).  FIRST experts are undoubtedly highly qualified technically but working relations with MOF in Year 1 were lacking. In Year 2 there was improved engagement with MOF.  FIRST staff turnover, especially in management, is disruptive to activity implementation.  FIRST proved to be flexible and responsive as per the activity design.  FIRST’s Monitoring, Evaluation, and Learning (MEL) system was not functioning so it could not serve as a management tool.  Context offers constraints (e.g., 2019 was an election year so there was no political will for reform) and opportunities (e.g., persuading the donor community to impose budget support conditions).  FIRST needs to navigate and/or overcome obstacles in MOF’s difficult work environment.  The presence of high-level “champions” (agents of change) within MOF fast-tracks reforms.  The shift to IT solutions and applications complicates fiscal reform interventions.  The absence of a Memorandum of Understanding (MOU) between the MOF and USAID meant there was a lack of clarity on institutional governance, and working arrangements for FIRST.  The Tunisian “brain drain” undermines MOF plans for digitalization because it needs local IT talent. FIRST operates in a challenging environment so it needs to play to its strengths while applying tactics to mitigate various obstacles. Due to constraints, completion of all FIRST activities is unlikely by mid￾2020, especially the “last mile” by MOF for sustainability. The low quality of work-planning and reporting documents is hampering the MOF’s and USAID’s oversight role. EQ 4: To what extent has FIRST succeeded in identifying, partnering, and coordinating with various stakeholders? What other partnerships will strengthen FIRST and similar USAID/Tunisia activities? FIRST partnering and coordinating was effective but limited in scope. The key findings are as follows:  FIRST established solid working relations with MOF counterparts in Year 2 through “joint working” and through the Steering Committee forum.  FIRST interacted effectively with the Ministry of Major Reforms and the Ministry of Development, Investment, and International Cooperation (MDiCI).  FIRST has good communication with multilateral institutions (EU, IMF, and World Bank) but weaker relations with bilateral donors.  FIRST’s collaboration with subcontractor SOLIDAR Tunisie is lacking while its collaboration with subcontractor PricewaterhouseCoopers (PwC) is strong.  FIRST was required by its contract to cooperate with the Tunisian private sector, civil society, and United States Government (USG) agencies but engagement was limited. 8 “Institutionalization” refers to embedding a tool (model, methodology), TA (systems upgrade, standard operating procedures [SOPs]), technology (IT hardware and software), or training introduced by FIRST within the ministry so it becomes fully adopted and utilized. Also see Year 1 Annual Report, September 2018, p. 13. xii FIRST successfully partnered with MOF, multilaterals, selected GOT Ministries and its sub-contractor PwC in Year 2, but partnering with bilateral donors, the Tunisian private sector and civil society was lacking. To build a broader reform constituency and foster outside pressure, FIRST needs to work closer with MDiCI to harmonize the role for TA, with bilateral donors for sharing information and experiences, and with organizations such as Institut Arabes des Chef d’Entreprises (IACE) and Union Tunisienne de l’Industrie, du Commerce, et de l’Artisanat (UTICA) for input and feedback on reforms. EQ 5: To what extent are FIRST activities sustainable and contributing to Tunisia government effectiveness (J2SR 4.GC.1)? It is premature to determine the extent to which FIRST activities are sustainable and contributing to government effectiveness as few key results were completed and adopted. Key findings are as follows:  FIRST is aiming to deliver 27 main activities or key results, of which 24 are ongoing.  FIRST relies on the MOF to “go the last mile” (i.e., fully adopt reform initiatives).  FIRST’s TA sustainability is enhanced by being enshrined in legislation and regulations (e.g., fiscal code for targeted auditing, and Budget Law for budget support conditionalities); Tools and Training sustainability is enhanced through FIRST’s “virtual resource library” and Training of Trainers (TOT), and IT sustainability is enhanced via security measures, MOF’s IT center budget for upkeep, perpetual licenses for software, and keeping software codes in-house. In terms of the Journey to Self-Reliance (J2SR), FIRST is contributing to GOT effectiveness by enhancing taxpayer services, building independence, strengthening the policy framework, and directly contributing to mobilizing (tax) revenues and budget management, but it is too early to say to what extent because there has been no measurement of effects and few completed main activities thus far. For long-term impact, FIRST needs to strengthen the sustainability of its inputs, complete all main activities, and ensure interventions are “institutionalized” by the MOF. FIRST is advancing improvements in GOT effectiveness, but the credibility of government commitment to policies will depend to a large extent on how reform-minded the new government will be in 2020. Recommendations Based on its findings and conclusions, the ET developed the following recommendations. FIRST should do the following:  Include MOF activities necessary to adopt reforms (the ‘final mile”) in FIRST’s Sustainability Plan, and agree to this in the Steering Committee forum to ensure long-term impact.  Articulate a detailed strategy for implementation that reflects its technical approach as well as capacity development, and incorporates tactics for overcoming constraints, including MOF challenges (e.g., encouraging competition among Director Generals for FIRST resources).  Ensure MOF has input into FIRST’s Year 3 work plan, approves FIRST quarterly and annual reports, and is involved in all aspects of delivery (e.g., STTA SOW, selection, and deployment).  Ensure its planning and reporting documents are consistent, comprehensive, and coherent.  Consider preparing action plans for the 24 ongoing main activities listed in FIRST’s Year 3 work plan, with milestones as process indicators, to aid monitoring of progress on key results.  Ensure all activities in its Year 3 work plan Gantt chart are carried out by August 2020 to enhance prospects for sustainability of results.  Establish a data management system for tracking and reporting on indicators, and revise indicators and MEL Plan as necessary.  Reach out to and partner with donors, private sector, and civil society to build a broader constituency for advancing PFM reforms. xiii  Work with the MOF on operationalizing its Performance Management System (PMS).  Include in the Year 3 work plan a gender analysis that identifies ways to mitigate gender issues affecting this activity, and systematically consider the gender aspects and implications of specific interventions.  With USAID, open a dialogue with IMF, Kreditanstalt für Wiederaufbau (KfW), and other donors on fiscal reform budget support conditionalities in 2020.  Conduct a public survey via MOF/ Strategic Communications Unit (SCU) for taxpayer opinion on tax matters and the MOF. The MOF should do the following:  Inform FIRST what MOF needs to do to fully adopt reform initiatives (e.g., UXP legislative measures for sharing information and development of IT applications for data exchange).  Make suitable office space available for FIRST experts inside relevant departments of the MOF.  Sign an MOU with USAID to provide formal institutional governance structures and protocols for the oversight of FIRST activity.  Schedule Steering Committee meetings to discuss FIRST Year 3 work plan and quarterly reports as well as FIRST achievements and challenges.  Expand the Steering Committee membership to include key external stakeholders (e.g., representatives from multilateral institutions, the Tunisia private sector, and civil society. USAID should do the following:  Request a detailed implementation strategy in the overall work plan of all future USAID activities at the outset of the activity to properly understand the implementing partner (IP) technical approach to delivery of aid.  Lengthen the duration of PFM-CD activities in future programming for more meaningful reforms (e.g., structural changes) to take hold.  Ensure that an MOU is signed with the partner at the outset for project governance and protocols (e.g., roles, responsibilities, expectations, working arrangements, etc.).  Require high-quality planning and reporting documents from the IP to facilitate USAID’s monitoring function.  Request that FIRST revise its MEL Plan and Sustainability Plan. 1 INTRODUCTION This document is a Midterm Performance Evaluation report of USAID’s Activity “Fiscal Reform for a Strong Tunisia” or FIRST, as described in the SOW prepared by USAID/Tunisia’s Economic Growth Office. The evaluation was carried out under the terms of USAID’s agreement with International Business & Technical Consultants, Inc., under the Monitoring and Evaluation in Tunisia and Libya (METAL) Activity (AID-280-TO-17-00001). PROJECT BACKGROUND After the Arab Spring in 2011, the opportunity arose in Tunisia to address socioeconomic issues, but unfortunately structural impediments have endured to this day. The economy is dominated by large, inefficient state monopolies (state-owned enterprises), and heavy government regulation hampers growth. Barriers to entry and competition, corruption, and cronyism persist. As a result, the economy has languished with only 3 percent annual growth, unemployment remains high (35 percent youth and 24 percent women), and acute regional disparities continue. Tunisia faces a myriad of challenges in its fiscal space: tax leakages, low revenue collection, tax avoiders, a narrow tax base, 43 percent of potential VAT liabilities are not paid, liberal professionals (doctors, lawyers, and architects) pay a fraction of potential liabilities, regime forfaitaire used by 60 percent of active taxpayers contributes less than 1 percent of total tax receipts, and the 369 tax expenditures (exemptions, holidays, and other tax breaks) represent substantial foregone revenue for the government. The tax burden is borne by a diminishing small group of taxpayers, fueling a growing sense of unfairness. As a result, the budget deficit and public debt ratio have all increased in Tunisia since 2011.9 There is urgent need for the government to reduce wasteful spending, such as unsustainable subsidies on energy and food, and increase tax revenues by increasing tax system efficiencies, improving compliance, reducing tax expenditures, and closing tax gaps. But there is resistance from powerful trade unions and interest groups that dilute fiscal reform efforts, and MOF directorates, especially the Direction Générale des Impôts – National Tax Administration Department (DGI), cling to ineffective and reactive enforcement strategies that do little to raise taxpayer awareness, promote voluntary compliance, or foster public trust in the tax system. FIRST is a three-year US$17.3 million USAID activity that began operations in September 2017. The goal of this activity is to provide technical assistance and capacity-building to key GOT institutions, particularly the MOF, to strengthen the fiscal foundations for sustainable and inclusive growth with an aim to improve the conditions for business, investment, and job creation. FIRST builds on US Government (USG) efforts since 2012 to improve public financial management (PFM) in Tunisia. The information and communication technology (ICT) competitiveness project worked with the MOF on micro-simulation models for personal income tax (PIT), value-added tax (VAT), and corporate income tax (CIT). USAID expanded and deepened these initiatives in 2014 with the Tax and Customs Reform Pilot (TCP) under the Asia and Middle East Economic Growth (AMEG) best practices activity that helped to establish the Fiscal Analysis Unit (FAU) within Direction Générale des Études et de la Législation Fiscales (DGELF), introduced risk management principles in tax administration, developed policy reforms to broaden the tax base, simplified the tax structure, and reduced the tax burden on small business and low-income persons. 9 Interview with IMF in Tunis, September 16, 2019. 2 The development hypothesis or TOC for FIRST10 is that if the government can maintain macroeconomic stability while achieving greater simplicity, equity, and efficiency in the tax system, the private sector will respond by increasing investment, expanding business activity, and hiring more workers. Business, investment, and job growth, in turn, will broaden the tax base, bolster the public’s confidence in government, and over time create the fiscal space for both lower taxes and increases in public investment and services. A longer term “governance dividend” can also be expected from this activity, insofar as reforms and improvements in fiscal policy and administration help to legitimize the state, increase political stability, and enhance social cohesion. The intervention logic for PFM reform is widely accepted. The FIRST activity contributes to the following higher-level objectives:11  Objective 1: Improve the efficiency, transparency, and cost of compliance of tax administration.12  Objective 2: Enhance MOF capacity to develop and manage tax policy.13  Objective 3: Enhance capacity to address other fiscal reform priorities as they emerge.14  Objective 4: Improve communications, engagement, and consultation on priority reforms.15 These activity objectives are mutually reinforcing. Modernizing tax administration will close the compliance gap through incentive-based systems that encourage taxpayers to pay their fair share of taxes for public services, and will lead to greater revenue collection within existing laws and regulations. By enhancing the MOF’s capacity to undertake fiscal policy analysis, decision makers will be able to assess the potential impacts of policy proposals that can help close the fiscal gap. By improving public outreach and stakeholder consultation, the MOF will increase the willingness of taxpayers to pay. Thus, all FIRST components converge toward the same goal: to improve tax revenues in a more equitable manner. Achievement of the above objectives is also expected to help Tunisia reduce the annual fiscal deficit to a level that is sustainable and consistent with a safe level of public and external debt. The Logic Model for FIRST can be found in Annex I. FIRST represents one of USAID’s core contributions to helping Tunisia surmount its economic and social challenges, as identified in the GOT’s 2016–2020 National Development Plan. In the context of USAID Tunisia’s Country Development Cooperation Strategy, FIRST activity contributes directly to Development Objective (DO) 1 “Inclusive private sector employment increased,” especially Intermediate Result (IR) 1.3 “Accountable public financial management systems enhanced,” and it reinforces Sub-IR 1.1.2 efforts to improve the business enabling environment. By supporting reforms that improve equity, efficiency, and transparency in the mobilization and use of public resources, this activity simultaneously contributes to IR 2.1 “Accountable Governance Strengthened.” 10 Development hypothesis for FIRST in contract section C1 p. 7 paragraph 2. 11 See FIRST’s Logic Model in its MEL Plan April 2018. 12 To modernize and streamline the tax administration system. 13 Tax policies that are sound, predictable, and fair. 14 Better targeting of social spending and increase the fiscal space for public investments and service delivery. 15 FIRST has four components aligned to its four objectives but in practice components 2 and 3 are combined. SUMMARY TABLE Activity name Fiscal Reform for a Strong Tunisia (FIRST) activity USAID office USAID/Tunisia Implementer Chemonics 3 EVALUATION PURPOSE AND EVALUATION QUESTIONS Evaluation Purpose USAID/Tunisia requested METAL to conduct a midterm performance evaluation of the FIRST activity. The evaluation purpose is to identify the extent to which FIRST is achieving intended results, as well as to assess the relevance, effectiveness, and efficiency of the approaches used to achieve USAID/Tunisia objectives. The evaluation aimed to identify successes, best practices, lessons learned, and challenges that affected the implementation of the activity, as well as identify potential course corrections before the project ends in August 2020. The final report will also propose recommendations based on the findings that would help inform USAID/Tunisia programming strategy and decision-making. The primary audience for this evaluation is USAID/Tunisia, particularly the Office of Economic Growth (OEG). Evaluation Questions The SOW for the FIRST midterm evaluation specifies the following five core research questions: EQ 1: To what extent was the designed activity approach relevant to the Tunisian government needs? Is the approach still relevant? EQ 2: To what extent, and how did FIRST contribute to improving revenue collection, reducing taxpayers’ compliance costs, improving the budgeting and expenditure process, and enhancing public accountability? EQ 3: What internal (organizational, resources, etc.) and external factors affected the efficiency and timeliness of the implementation and how? EQ 4: To what extent has FIRST succeeded in identifying, partnering, and coordinating with various stakeholders? What other partnerships will strengthen FIRST and similar USAID/ Tunisia activities? EQ 5: To what extent are FIRST activities sustainable and contributing to Tunisia Government effectiveness (as per the J2SR 4.GC.1)? Findings are disaggregated by gender, age, geographical location, and institution where feasible. EVALUATION DESIGN, METHODS, AND LIMITATIONS The aim of this study is to produce coherent answers to key questions posed in the midterm performance evaluation SOW based on sound evidence and analysis. This section describes how the evaluation was designed and specifically the methodology that was followed, i.e., what will be done to develop evidence-based findings (facts or evidence), deduce conclusions (interpretation of facts), and make recommendations (judgment). Cooperative agreement/Contract # Contract under AID-OAA-I-12-00035/AID-664-TO-17-00001 Total estimated ceiling of the activity (TEC) $17,325,000 Life of activity (period of performance) August 25, 2017–August 24, 2020 Active geographic regions Tunisia Development objective(s) DO 1: Inclusive private sector employment increased External or internal evaluation? External 4 Data Collection Methods Data and information were collected from three main sources: one secondary source (desk review of FIRST documentation) and two primary sources (focus group discussions [FGDs] and Key Informant Interviews [KIIs]). Following the analysis of data and information gathered, the evaluation team (ET) then discerned its findings, conclusions, and recommendations, and prepared the evaluation report. The ET engaged with USAID at each step of the process: during an in-brief, through regular updates during fieldwork, an exit-brief meeting, and a remote debrief presentation on the draft evaluation report. During the in-brief meeting with USAID on September 5, 2019, the draft work plan was discussed in depth and subsequently approved by USAID. All evaluation activities were consistent with USAID’s Evaluation Policy (January 2011) and ADS 201. Secondary Data Sources Desk review data and information were drawn primarily from FIRST activity files. Specific FIRST documents provided by USAID and the IP for this desk review included FIRST Contract, Annual Work Plans, Monitoring and Evaluation (M&E) Plan, Quarterly Reports, and the Annual Report. The ET also reviewed technical papers, articles, documents, and reports provided by other agencies, and derived from internet research, as deemed appropriate, such as country reports and strategies (see bibliography in Annex V).16 Primary Data Sources Evaluation data and information were drawn from two primary sources: • Key Informant Interviews with internal and external stakeholders—those having a strong interest in the outcomes of the activity such as senior government officials, especially MOF “counterparts” (internal stakeholders); FIRST Chief of Party (COP), Team Leaders, Deputy Team Leaders, MEL Manager, Human Resources, Procurement; USAID representatives such as FIRST Contracting Officer’s Representative (COR); representatives from relevant international agencies/organizations such as the EU, World Bank, Agence Française de Développement (AFD), African Development Bank (AfDB), Kreditanstalt für Wiederaufbau (KfW); and private sector and civil society, including UTICA (larger business community), IACE (think tank), and SOLIDAR (social justice advocacy). • Focus Group Discussions with selected direct beneficiaries, primarily MOF technical and professional staff as well as some staff from other relevant GOT units. FIRST provided a contact list of all activity counterparts to enable individual face-to-face interviews, plus MOF staff (beneficiaries to date) so that they could be invited to participate in focus groups. USAID/OEG and FIRST suggested international agencies/organizations and other external stakeholders to be invited to the KIIs. A list of interviewees and focus group participants can be found in Annex V. Research Methods (Data Collection) Key Informant Interviews (KIIs) The ET carried out KIIs to obtain opinions and views on FIRST from activity stakeholders, in particular its challenges and achievements in Years 1 and 2, as well as suggestions for Year 3 implementation 16 FIRST 2nd Annual Report and QPR J/A/S 2019 were produced after the evaluation period and were not part of the desk review. 5 improvements. The ET arranged for 31 KIIs with FIRST activity counterparts and other MOF Units, external stakeholders, the IP/FIRST, international agencies, and USAID. A KII Guide (qualitative instrument) was drafted with open-ended questions to steer the conversation. Detailed context or background for each question is included in the tool to aid the ET probe or “drill down” on specific matters (see data collection instrument in Annex III). During each KII, the interviewee comments were captured with careful note-taking. The conversation was not recorded because this limits responses. After typing, responses were manually coded to facilitate analysis. Focus Group Discussions (FGDs) The ET arranged FGDs to interview direct beneficiaries of the activity (MOF technical and professional staff) for the purpose of capturing their perceptions of FIRST activities, achievements, challenges, etc. The 61 people invited to participate in FGDs were drawn from the six working groups established by FIRST on e-filing, UXP, information exchange, communication, fiscal gap, and fiscal civism. Five focus groups were hosted with approximately seven persons in each group. A Focus Group Guide (qualitative instrument) was drafted containing a slightly different introduction to the KIIs but the questions remain the same to stimulate and focus discussion (see tool in Annex III). The ET facilitated the discussion and ensure it remained on track. During each focus group, comments were captured with careful note-taking. The event was not recorded because this would limit interviewees’ responses. After typing, responses were manually coded to facilitate analysis. Data Analysis and Synthesis As mentioned, this evaluation relied on two primary sources of data (FGDs and KIIs) and one secondary source (desk review). The methodology behind the desk review (analysis of documentation) was to gather all pertinent literature and then review, discuss, and synthesize understanding to develop a comprehensive context for the activity evaluation. Project and other relevant documents were analyzed up to the start of fieldwork and may continue if additional documents are obtained while in the field. Data analysis for KIIs and FGDs commenced at the beginning of the fieldwork. The fieldwork commenced September 4, 2019, and was scheduled for four weeks to allow for in-brief, exit-brief, 31 KIIs, and five FGDs. Data from interviews (FGDs and KIIs) were abstracted and shared among team members. A thematic analysis was run against evaluation questions throughout the fieldwork. Analysts looked for emergent themes, contextual factors, and trends or patterns in the responses. Results of the thematic analysis assisted evaluators in developing findings and conclusions. This analysis produced specific comments/concerns of key informants that were presented in the final report. Analysis is reflected in the Findings-Conclusions-Recommendations Matrix (Annex VI). The Data Collection Instrument, Analysis Plan, and Sources of Information are found in Annexes III, IV, and V. Data Quality and Limitations The ET was concerned that evaluation reliability and validity might be constrained by security and logistical factors beyond the control of IBTCI/METAL, USAID, or the ET, but this was not the case. The ET operated in the capital city and remained vigilant in case of risks and threats and was flexible regarding rescheduling of KIIs and FGDs. 6 Gender Considerations The evaluation also addressed the robustness of the FIRST’s attention to gender equality issues in the documentation review, KIIs, FGDs, and various analyses. Gender was not a separate component of review, but rather an integrated cross-cutting theme addressed throughout the evaluation. The subcontractor SOLIDAR was supposed to have assisted in ‘gender sensitizing.’ Efforts were made to disaggregate data by sex, ensure full participation by women in FGDs, and document key metrics that can be used to parse a full range of considerations that may affect women’s participation in FIRST assistance. 7 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Following data collection, the ET analyzed the data and information to determine fact-based qualitative evidence, and this is written up as specific and concise findings. Conclusions are a synthesis or interpretation of the findings. Recommendations are proposed actions for concerned parties drawn from specific findings. FINDINGS AND CONCLUSIONS EQ 1: To what extent was the designed activity approach relevant to the Tunisian Government’s needs? Is the approach still relevant? The designed activity approach for the activity was relevant but not elaborated by FIRST and a “reset” was necessary in Year 2 to put the activity back on track. EQ 1 Findings: To what extent was the designed activity approach relevant? FIRST has a dual purpose—to strengthen Tunisian fiscal foundations through PFM reforms while also building sustainable capacity of MOF: “to assist the GOT strengthen the fiscal foundations for sustainable and inclusive growth, with an aim to improve the conditions for business, investment, and job creation” and “above all the emphasis of this activity must be on building sustainable local capacity.”17 The contract between USAID and the IP outlines a design for FIRST in terms of rationale/needs, indicative interventions, and partnering, but the only mention of how to implement the activity is as follows: “Results will be achieved through a combination of TA, capacity-building, facilitation, and commodities support” and “while one would expect a detailed technical approach and interventions aimed at achieving the results, . . . because TA should be demand-driven, the approach would be expected to be flexible in order to be responsive to emerging needs and opportunities.”18 From the project documents reviewed, there was no articulation of a technical approach by FIRST to delivery, only detail on planned interventions. Work plan documents mentioned “assisting, supporting, or helping MOF,” but did not outline a strategy for implementation. One would normally see this in an overall work plan at the outset of an activity, and there was an expectation in the contract that a technical approach be developed by FIRST. It is important to distinguish between FIRST Year 1 and Year 2. The Year 1 work plan contained an activity schedule (Gantt chart), a listing of indicators, and a brief (technical) explanatory note on main activities including a call for setting up a Project Steering Committee and establishing formal project work teams. The Steering Committee (COPIL) meetings were coordinated by FIRST’s principal point of contact at the MOF and in Year 2 the meetings became much less frequent.19 Preexisting working groups under predecessor activity TCP included fiscal gap, fiscal expenditure, communications, and fiscal policy. Following TADAT and its fiscal civism strategy, FIRST established additional working groups according to the four pillars: legislation, fiscal control, tax recovery, and ICT.20 17 Award/Contract USAID Middle East Regional Platform and Chemonics August 8, 2017, p. 10 and 15. 18 Award/Contract USAID Middle East Regional Platform and Chemonics August 8, 2017, p. 10. 19 ET was told that seven COPIL/SC meetings took place in Years 1 and 2 but some of them were “thematic meetings.” 20 There is confusion over working groups because the first formal Steering Committee in May 2018 created four groups (e￾services, external data exchange, internal data exchange, and communications). Quarterly Progress Report April–June 2018. 8 The remaining 17 main initiatives in the Year 1 work plan (total 88 activities) demonstrated that the initial FIRST team had a reasonable grasp of the needs of the Tunisian fiscal system and were continuing the work started by TCP. In the Year 1 Annual Report, FIRST described how it met with key partners and counterparts during the start-up phase to better understand the economic and tax environment in Tunisia. FIRST was criticized by the tax administration focus group as being too Tunis-centric, saying there was no regional dimension to its program and the regions were not consulted. The Year 1 work plan called for a comprehensive diagnostic of the Tunisian tax system to identify areas for improvement. It indicated that TADAT21 would be carried out in November 2017 and afterwards a three-year work plan developed. However, there are other less- demanding consultative tools for assessment and setting baselines. According to one focus group, “We spent a lot of time on the diagnosis while everyone knows it (needs).” Inside the MOF, some were questioning the utility of a comprehensive diagnosis and in fact “they did not want it.” Senior management were worried about outsiders probing their operations and the possibility for this to lead to some sort of blacklisting. FIRST had to convince the MOF of TADAT merits, address their concerns, and secure their “buy-in.” A letter was eventually sent by the MOF to TADAT Secretariat mid-December 2017 and the TADAT team mobilized in January/February 2018. The mission occurred in March, but obtaining the necessary data proved to be challenging.22 The report presented to the MOF in May 2018 was not well received because Tunisia was ranked poorly by the tool, a negative unintended consequence. Nevertheless, TADAT is now considered a success—despite the difficulties encountered. The deep “holistic” diagnostic showed deficiencies in Tunisia’s fiscal system but was good for benchmarking and for establishing baselines. Other value-added provided by TADAT was that it led to development of a voluntary tax compliance strategy, provided a learning opportunity for senior MOF managers (workshop training), was good for informing future programming support, and helped to inform which direction to take, i.e., it provided a starting point for selecting FIRST interventions. The 28 TADAT metrics also helped to inform MOF performance indicators. Finally, TADAT did ultimately serve to “shake up the ministry” and as such contributed to a change in mindset, i.e., modern thinking about the fiscal system. Apart from TADAT, FIRST’s tangible accomplishments in Year 1 were minimal. FIRST struggled in Year 1 to establish effective working relationships with the MOF. In Year 2, shortly after the arrival of the new COP, FIRST was “reset” to focus more on IT solutions, applications, and web services (e￾governance), while continuing to upgrade systems and build capacity.23 This “shift in implementation” was suggested by USAID to put FIRST back on track. There was also a focus on capacity development that was to be led by the ministry and supported by experts to get better “buy-in” or ownership for sustainability. COPIL was supportive of this fresh start and the redesign gave the activity new momentum. FIRST used a variety of tactics to move forward on activities, including taking advantage of windows of opportunity such as creating “competition” between counterparts (DGs). FIRST demonstrated its responsiveness in Year 2 by dropping unwanted activities (fiscal decentralization, review of pensions, subsidies and public sector wage bill) and adding priority items (FACTA,24 webcasting, and customs 21 TADAT is an IMF best practice diagnostic tool used worldwide for assessing strengths and weaknesses of a fiscal system. 22 According to the FIRST Year 1 Annual Report September 2018, TADAT comprised initiation, preparation, in-country assessment, and post-assessment phases to report on 28 indicators in nine key performance areas. 23 FIRST introduced into the MOF 20 different software programs and spent USD 1.2 million on hardware. The IP contract states “contractor will pay special attention to using technology and e-services to facilitate core tax administration processes.” 24 Foreign Account Tax Compliance Act compels US citizens to file on foreign bank accounts. 9 post-clearance audit). However, in terms of engagement, FIRST failed to properly seek input from MOF on work plans and on use of TA, and did not to obtain MOF feedback on reports. EQ 1 Findings: Is the approach still relevant? As for second part of EQ 1, the answer is a qualified yes. The current FIRST team verbally reported its delivery approach as follows: engage with stakeholders; assess their needs (constraints, prevailing capacity); set priorities (within project scope); prepare action plans and strategies; deploy STTA; introduce and test models (simulations); adapt tools; procure “commodities;”25 prepare standard operating procedures (SOPs), guides/manuals, and modules; and deliver training and coaching. This “theory and practice” approach was confirmed by beneficiaries in focus groups, so it would appear that an understood approach exists in practice but not in writing. FIRST’s delivery in Year 2 conforms to international best practice in capacity development.26 This stipulates an approach that puts the client/MOF in the driver’s seat (i.e., the process should be demand￾driven and client-led with donor support).27 In practice this is nuanced—FIRST experts often propose interventions that may be rejected initially by MOF but then they use arguments to convince counterparts that proposals have merit based on best practice, experience, etc., which leads to a “dynamic dialogue.” The tax administration focus group said they wanted to see more practice from developed countries used, but FIRST tended to push tools from developing countries, whereas the tax policy focus group stated that consultants spent too much time talking about foreign experiences instead of developing tailor-made solutions for Tunisia. There is an important distinction between need and demand; there may be multiple areas for improvement in Tunisia’s fiscal system (generating multiple proposals for intervention from international experts), but successful intervention requires committed insiders to lead the change process with strong leadership and widespread support (see CD paradigm parameters Annex XI). It is prudent for foreign experts to drop their proposals when they are unwelcome and/or unlikely to succeed—FIRST examples include the unified tax code arising from increased reliance on local government as a result of fiscal decentralization, the Tax Policy Board to facilitate communications between the MOF and the private sector, the Tunisia pension system, and the public sector wage bill.28 There were limitations regarding FIRST’s engagement with the MOF on planning. Summary work plans were presented to COPIL/SC in the form of PowerPoint slides, the first being Year 2/3 life-of￾project slides in October 2018 and again in April 2019.29 However, the ET was informed by FIRST that there was negligible partner feedback on draft work plan documents despite being widely distributed.30 In addition, the ET was informed by a civil society representative that “there is no correspondence between what the Minister wants to do and what FIRST wants to do.” FIRST communication and coordination with the partner MOF improved in Year 2,31 but with regard to TA, MOF beneficiaries said that their input into SOW and selection of CVs was somewhat ad hoc. According to the tax policy focus group, “Sometimes FIRST proposes things that are incompatible 25 FIRST has already disbursed approximately US$1.12 million on computer equipment (servers, laptops, tablets, printers, webcasting system) and 20 software items. 26 See OECD, UNDP, EU, SIDA documents in Annex V sources of information. 27 Supporting Capacity Development in PFM: A Practitioner’s Guide. Vol. 1. 4th High-Level Forum on Aid Effectiveness. Busan, Korea. 2011. 28 IMF informs that government payroll and energy subsidies are a significant burden on the state budget. 29 COPIL met January 25, 2019, but focused on upcoming STTA missions and resolving pressing activity issues. 30 FIRST Year 1 Annual Report states that the Year 1 work plan was approved, yet the MOF Coordinator informs the work plan was not validated, in part due to lack of a formal agreement (MOU) between MOF and FIRST. 31 Tax policy focus group confirms FIRST improved communications: “They have been listening.” 10 with the Tunisian taxpayer mindset.” In Year 2 the FIRST quarterly reports were no longer shared with counterparts. A civil society source observed that FIRST “wants to work alone,” and that is detrimental to reform success because outsiders are often necessary to facilitate consultation with top-level government officials, ministers, and parliamentarians (see later under EQ 4). EQ 1 Conclusion: To what extent was/is FIRST’s approach relevant? According to the contract, the original design detailed a rationale for FIRST, together with illustrative interventions and expected results that are relevant to reform in the Tunisian fiscal system. However, the contract did not spell out the approach for implementation in detail (apart from TA and capacity￾building), leaving it up to the contractor to elaborate. The absence of an implementation strategy made it difficult for USAID and the MOF to understand FIRST’s overall technical approach. Although TADAT is a rigorous tool for identifying areas for improvement and benchmarking the fiscal system,32 the focus on a deep diagnosis, when the partner was reluctant, likely delayed or subsumed many reform-related activities planned in Year 1. Initially, the MOF was not convinced about TADAT’s value and was concerned that there may be repercussions (e.g., blacklisting). Interestingly, after it was concluded in May 2018, MOF found TADAT to be “well done and the accompanied training useful.” FIRST delivery was more effective in Year 2 as it heavily focused on digitalization and e-services (despite the added complexity of IT solutions support) and on joint working at the ministry or “reform from within.”33 The tax administration focus group stated that “the context is not conducive to structural (radical) reform,” so this limits FIRST’s ability to have a significant fiscal impact. That FIRST went for the “low-hanging fruit” is understandable given the difficulties encountered in Year 1, but it also limited the likelihood of FIRST acting as an agent of real change (i.e., disruptor to the status quo). FIRST’s approach in Year 2 aimed to balance being proactive (suggesting proposals based on international best practice and know-how—its “value proposition”) while at the same time being reactive (responding to the partner request for assistance). To operate on a demand-driven basis alone can be unproductive if the partner is not particularly active, not ambitious, or is pursuing a flawed direction. However, there were shortcomings in the Year 2 approach because FIRST was unable to secure feedback from MOF on draft work plans despite repeated request to the Ministry. In addition, terms of reference for international experts were shared with DGI and DGCPR prior to missions presumably to give them an opportunity to comment but it is not clear if feedback was received. EQ 2: To what extent and how did FIRST contribute in (a) improving revenue collection, (b) reducing taxpayers’ compliance costs, (c) improving the budgeting and expenditure process, and (d) enhancing public accountability? It is too early to say to what extent FIRST contributed at outcome and impact levels; however, FIRST productivity in terms of outputs improved considerably in Year 2. EQ 2 Findings: To what extent has FIRST contributed? The main findings in the four areas of EQ 2 require the measurement of results (contributions) using indicators (metrics). EQ 2 (a) and (b) are impact indicators while (c) and (d) are outcome indicators. The problem with impact measurement is attribution because it is very difficult to assign cause and effect,34 and the problem with outcome measurement is the time it takes to realize a concrete effect 32 The EU has a high opinion of the tool and is using some TADAT metrics. 33 Paper on fiscal policy components provided to ET by FIRST tax team dated July 30, 2019. 34 Even if tax revenues go up, FIRST cannot take credit due to the “disconnect.” 11 that can be measured,35 especially because FIRST has only operated for two years. For example, the EU states that it took Tunisia 15 years to fully accept program budgeting, and IMF support to Liberia tax modernization saw positive impacts (increased revenues) after 10 years.36 The exception is in public accountability because the MOF’s Strategic Communications Unit (SCU) is able to carry out a taxpayer survey to obtain feedback on taxes and fiscal reforms. Nevertheless, the DGI-DG is expecting a positive effect on tax revenues in the near future, at least through tax administration reforms. The Centre Informatique du Ministère des Finances (CIMF) DG said FIRST accelerated aid in Year 2, and that FIRST assistance with specifications and procurement is fast compared to the public sector. CIMF is the only directorate to provide office space to FIRST, and this has helped with experts being more on-site to help support and coach various IT teams. Interestingly, several sources reported that the focus of reforms should be on fairness and tax equity37 and not on increasing domestic tax revenues—Tunisia’s tax revenue is already 22 percent of GDP (rising to 31 percent with social security taxes), which is high for a middle-income country.38 The FIRST MEL system was not functioning. METAL’s June 2019 Data Quality Assessment (DQA) stated that FIRST had not tracked or reported on indicators because there was no data management system in place. Furthermore, it said some of the high-level indicator are not meaningful—they need to be more relevant and useful—and there are no process indicators to measure implementation progress. With the refocus in Year 2 on IT solutions, FIRST’s MEL Plan should have been revised to reflect the shift in technical approach to digitization and e-services. Indicators, results framework (logic model), and the Activity Monitoring, Evaluation, and Learning Plan (AMELP) are all being revised, and FIRST is developing a data management system to capture information on activities and results.39 FIRST’s dashboard using DevResults software will also be amended to reflect MEL modifications and future reporting. FIRST’s risks and assumptions still apply going into Year 3. The FIRST results framework shows IR 3 indicators for revenues, compliance, and fiscal deficit at impact level and indicators (objectives) at outcome level. What is missing are indicators for each of the seven workstreams (sub-objectives) as FIRST delivery is through this modality.40 These intermediate objectives are the building blocks for the fiscal foundation as per FIRST’s purpose. Despite spending considerable time and effort in Year 1 on assessment, strategizing and action planning, FIRST managed to carry out a few other activities.41 FIRST’s Year 1 annual report lists achievements in tax administration as follows: TADAT was completed yielding a compliance strategy and action plan, a scoping mission for UXP was carried out, a plan was devised for modernizing returns through e-filing and e-payment, and private sector accountants were trained. In tax policy, a tax gap analysis was launched and computable general equilibrium (CGE) and medium-term fiscal framework (MTFF) models introduced. In communications, taxpayers were surveyed, a Taxpayer Guide prepared, and a digital strategy formulated for the MOF. Year 2 has been more productive in terms of progressing important reform initiatives, but concrete achievements are few. Only TADAT diagnostics and accompanying compliance strategy and action 35 See DQA June 2019 that was reinforced by the DGs, CIMF, and DGI. 36 Interview with EU representatives in Tunis, and IMF website. 37 IMF informs that the informal sector in Tunisia is about half of the economy yet escapes many tax obligations. 38 Tunisia Country Development Cooperation Strategy. USAID/Tunisia. September 2016, reinforced by IMF. 39 New MEL Manager arrived in May 2019. 40 FIRST Life-of-Project 20-month work plan PowerPoint presentation to COPIL October 2018, and FIRST Year 3 work plan. 41 Current COP stated that FIRST activities were “sluggish” in Year 1, so there was not much to report on. 12 plan, tax expenditure report, and post-clearance audit procedures for customs were completed. The vast majority of FIRST main activities are ongoing at the end of Year 2.42 The table in Annex XV shows 24 ongoing main activities with progress made on them by activities. The percentage complete column indicates that a major proportion of work remains for Year 3. There is only one action plan—for the compliance strategy. FIRST insists that progress can be deduced from the table in Annex XV and remaining activities from its Gantt chart in its work plan. A discrete action plan for each main activity would help in understanding precisely what has been done to advance the key result and what remains to be done with a clear timeline. The milestones found within an action plan would act as process indicators for tracking progress on achieving key results. FIRST is supporting MOF to operationalize its performance management system.43 For units that partner with FIRST, reporting on work plans should indicate improved performance, that is, whether the beneficiaries are adopting/using the tools and training provided (management by objectives or change management). EQ 2 Conclusions: To what extent has FIRST contributed? It cannot be determined that FIRST interventions had any effect at outcome or impact levels because there was no MEL system for tracking and reporting on indicators. Furthermore, it may not be feasible to measure outcomes after only two years of operations because of the time it takes to institutionalize fiscal reform efforts and see high-level results. Detailed action plans would help understand the process of implementation and help track progress of main activities. Milestones constitute process indicators to measure progress of delivery. FIRST can measure MOF performance improvements at the individual or unit level when the performance management system (PMS) is operationalized. EQ 2 Findings: How is FIRST contributing? Although FIRST is structured as four components, it delivers in seven workstreams (see Annex XII). The table in Annex XV lists the ongoing 24 key results as well as activities carried out.44 To better understand FIRST interventions, especially the main activities or key results, there follows a brief description of activities within the framework of the seven FIRST workstreams. Workstream 1: Create a streamlined, transparent, and predictable tax administration. • The TADAT diagnostic tool is excellent for international benchmarking, understanding shortcomings (inefficiencies) in a country fiscal system, prioritizing needs, and setting a reform agenda. As such, TADAT can be considered an intervention by itself. Although TADAT found many areas for improvement in the Tunisian fiscal system, it was not accepted initially by the MOF because of a lack of understanding of the value of a detailed assessment. • UXP is a standardized system to improve data exchange between government agencies with strict data security45 through strong encryption. Installation of the unified data exchange platform46 is connecting eight institutions [Centre Informatique du Ministère des Finances (CIMF), Direction Générale des Impôts (DGI), Direction Générale des Douanes (DGD), ATTT, CNAM, CNSS, Register National des Enterprises (RNE), and Country Partnership Framework (CPF)] to streamline data sharing. Although UXP is now “concrete and functioning” according to CIMF 42 FIRST also refers to these main activities as key results—see Annex XII. 43 Radical reform of management of public policies and administration started in the 2014 Budget Law and is being linked with TADAT performance indicators. FIRST is helping MOF to operationalize the new performance framework. 44 FIRST has also delivered 15 different training courses to over 1,400 MOF staff thus far. 45 CIMF suggests future donor support through a project on data security using artificial intelligence (AI). 46 UXP, the so-called “superhighway of data exchange,” is the basis for all e-government functions in Estonia. 13 (MOF’s IT center), it is not yet operational because applications need to be developed by MOF to permit communication among the entities involved. The ET was told by CIMF that regarding the UXP initiative, “the delay is with us.” • Other technical assistance (TA) was provided by FIRST as follows: a strategy for improving Tunisia’s score on the “Paying Taxes” indicator of the Doing Business survey (note the very bad score on VAT payments); support to MOF in setting up cybersecurity protocols and assistance to Tunisian Bankers Association on FATCA compliance requirements to avoid being blacklisted for money laundering and terrorism financing; support for setting up a post-clearance audit unit at Tunisian customs enabling pre-cleared companies to benefit from streamlined customs clearance processes/procedures; and change management whereby FIRST assisted with an assessment of the current performance management system at MOF. Workstream 2: Enable easier citizen and business tax compliance. • TADAT informed the development of a fiscal civism47 compliance strategy and action plan to “make it easier for taxpayers to file and pay taxes in a timely manner.” The strategy attempts a shift in MOF mindset from being repressive or “muscular” toward a new vision of fiscal civism.48 The ET was informed that Tunisian “fiscal citizenship” has deteriorated since 2011. The new strategy emphasizes voluntary tax compliance (spontaneous declaration) through four pillars: a redesigned fiscal information system (FIS) required a substantial IT architecture upgrade at CIMF necessary to make the taxpayer registry (database) more reliable49 by removing inactive, duplicate, and “ghost” files, and introducing a unique identifier/number for each person and company so it can access all their data (taxpayer account); reorienting the tax audit procedure (see Workstream 3); reviewing legislation on penalties for proportionality (ensure that the punishment fits the crime) and introducing a hierarchical penalty system to penalize the intentional non-filers in addition to charging interest on late payments; and simplifying (reengineering) business processes/procedures to improve taxpayer services in 300 tax offices nationwide. • FIRST support for DGI deployment of IT solutions to streamline the tax filing and payment system means development of portals and web services for online e-filing, e-registration, e-declaration,50 and e-payment. Electronic taxpayer registration and an improved taxpayer account/database will expand the taxpayer base,51 especially for those in the informal sector. Obtaining legal authority for electronic notification52 or messaging (automate via text messages and emails) for reminding non-filers of VAT should also increase compliance and improve Tunisia’s ranking on WB Doing Business Survey. The Year 2 work plan committed FIRST to help upgrade the IT system at CIMF prior to starting to use e-services. Interestingly, Morocco is seen as a role model for modernization of African governments through digitalization.53 • FIRST also delivered customer service training for tax collection agency staff to increase compliance (1,000 Direction Générale de la Comptabilité Publique et du Recouvrement – National Tax Collection Department [DGCPR] agents in five regions), and shifted responsibility for taxpayer education to private sector (e.g., e-filing training to 58 tax accountants from private 47 Sense of civic responsibility and duty to voluntarily file and pay taxes without government involvement. 48 Tax Administration Focus Group, September 18, 2019. 49 Paradox software proposed by FIRST to “clean up” the taxpayer registry needs to be adapted to MOF needs. 50 E-declaration is aimed at 20,000 taxpayers representing 3 percent of “natural persons” who contribute 90 percent of tax revenues. Source: CIMF. 51 More people paying taxes means more equity and fairness in the tax system (tax justice). 52 This is termed “dematerialization” by CIMF. 53 https://www.moroccoworldnews.com/2019/01/263828/morocco-digital-economic-social-development/ 14 companies). FIRST supports DGI with traditional and social media campaigns to reach out to the public by outlining the benefits of paying taxes and the penalties of not paying, and thereby encourage voluntary tax compliance. Workstream 3: Implement an effective and efficient tax audit system. • Rolling out a risk-analysis system for delinquent taxpayers means shorter, more frequent, and better targeted audits that will mean greater revenue from audits (e.g., recover VAT taxes from VAT-exempt exporting companies that sell domestically and collect VAT). A new targeted audit procedure (issue-oriented audits) manual was prepared and used to train DGI auditors in the regions on the risk-based approach to audit selection and collections, and this will enhance revenue-raising opportunities. Computer tablet devices are being purchased for field verification. • Strengthen collection policy to repeal the idea of optional payment of taxes (filing without payment). Review forced collection (recovery) policy because the previous punitive system was not successful in recovering old debts from bad taxpayers. It is better to focus on fiscal civism (voluntary compliance) and also on collecting large arrears. FIRST also help to set up a Directorate of Medium-Size Enterprise [DME] to offer focused e-services. • Support DGI in implementing a risk-based approach, with relevant controls and security, to accelerate VAT refund payments for low-risk, trusted taxpayers, and provide training on new procedures for VAT refunds. FIRST also encouraged the MOF to reverse the mechanism allowing export companies in free zones to benefit from VAT-free purchases and to repeal a VAT-free scheme. Changing the basis for calculating tax officers’ performance awards from acquired debts to received debts will help avoid perverse effects (the current method encourages the exaggeration of uncollected payment of back taxes). Table 1: Linkages Between Component 1, Workstreams 1-3, Key Results, and EQ 2a and 2b Components (objectives) Workstreams (sub-objectives) Achievements (key results or main activities) Sub-questions C1: Improve the efficiency, transparency, and cost of compliance of tax administration W1 to create more streamlined, transparent, and predictable tax administration W2 to enable easier citizen and business tax compliance W3 to implement an effective and efficient tax audit system  TADAT comprehensive diagnostic  UXP platform installation  Other TA is Doing Business “paying taxes,” FATCA supports customs post-clearance audit, change management (management by objectives) Improving revenue collection  Fiscal civism compliance strategy and action plan  Fiscal info system (FIS) – TA  IT solutions  Training in customer services for tax collection staff  Training on e-filing for private sector accountants Reducing taxpayers’ compliance costs  Risk analysis system (targeted audit procedures)  Collection policy and SOPs, and training  DME – Medium Taxpayer Office TA  VAT refunds – SOPs and training Improving revenue collection Workstream 4: Strengthen the capacity of MOF (FAU) to independently produce relevant and good tax policy analysis. 15 • Update the micro-simulation models54 for high-quality analysis of personal income tax (PIT), corporate income tax (CIT), and VAT to calculate tax gaps.55 Transfer the model on tax redistributive effects to better understand distortions in tax system (i.e., tax incidence).56 Support the analysis and publication of an annual report on tax expenditures for estimating significant foregone revenues through exemptions, exclusions, and deductions from the base of a tax, credits against a tax, tax deferrals, and preferential tax rates, as well as for assessing the effectiveness of tax expenditures (through cost-benefit studies) in terms of promoting economic growth and job creation. Although a finalized product, the report will be in the new budget law as an annex and as such will be debated by the new government next year so action by decision makers, with shifts in tax policy and effect on tax revenues, is likely a long way off. • Assist with studies on tobacco taxes,57 energy subsidies58 (oil price hedging), and Treasury Special Funds—note that several items were dropped (public sector wage bill,59 pension system, fiscal decentralization) because they were deemed politically contentious but FIRST could have reached out to civil society and others to help process such sensitive files. Build FAU capacity to use and update models, and to produce meaningful analysis of potential revenue impacts of changes to tax policies. Workstream 5: Ensure the use of tax policy analysis and proposals by decision makers. • Develop a medium-term fiscal framework (MTFF) model for developing the government budget, specifically estimating the impacts of budgetary policy changes over a three- to five-year window. Develop a computable general equilibrium (CGE) model at General Directorate for Resources and Equilibrium (DGRE) for quantifying macroeconomic and sectoral impacts of fiscal policies on the economy and a macroeconomic forecasting model capable of predicting fundamental economic indicators such as GDP and then apply a baseline tax law to determine how much revenue could be available for tax expenditures and for spending (ministry budgets ceilings). Reinforce DGRE’s ability to use the CGE model with General Algebraic Modeling System (GAMS) software. • Support the implementation of changes to organic budget law through a transparent budget process. Build FAU capacity through training and coaching to present, explain, and defend policy options and proposals to decision makers and the public. The MOF should reach out through the SCU with communication products, and through public forums to educate, inform, and obtain feedback on reform initiatives. Table 2: Linkages Between Components 2 & 3, Workstreams 4 & 5, Key Results, and EQ 2c Components (objectives) Workstreams (sub-objectives) Achievements (key results or main activities) Sub￾questions C2: Enhance MOF capacity to develop and manage tax policy W4 to strengthen the capacity of MOF (FAU) to independently produce  Micro-simulation models for VAT, CIT, PIT  Tax incidence analysis and report  Tax expenditure analysis and report Improving the budgeting and 54 The tax team in Year 2 followed its own participatory process comprising research, identification, selection, database, model code, testing, calibration, flowcharts, user guide, training, and coaching – Tax Team Paper, July 30, 2019. 55 Tax compliance gap is the difference between potential revenues that could be collected under the current tax regime and actual revenues collected. 56 Tax incidence or tax burden is the effect of a particular tax on the distribution of economic welfare—to distinguish between the entities who ultimately bear the tax burden and those on whom tax is initially imposed. 57 IACE informs that the VAT losses on tobacco alone are 900 million TND. 58 The energy sector is heavily subsidized in Tunisia. In 2012, it amounted to 5,600 million TND, which is 20 percent of the state budget or 9 percent of the GDP. This level is not sustainable and decreases public investments. Source: Energypedia. 59 IMF reports that the Tunisian public sector has expanded 40 percent since 2011. 16 C3: Enhance capacity to address other fiscal reform priorities as they emerge60 relevant and good tax policy analysis W5 to ensure the use of tax policy analysis by decision makers  MTFF – medium-term fiscal framework model  CGE – computable general equilibrium model expenditure process Workstream 6: Strengthen internal communication capacities. • Efforts to strengthen internal communications include setting up a webcasting (intranet) system at CIMF to virtually connect four MOF offices in Tunis and eight regional offices to facilitate the exchange of information and distance learning; strengthening the Strategic Communications Unit (SCU) at the MOF with strategic plan and other supports; and supporting improvement of internal communications and coordination between DGI and DGCPR—one is tax administration and the other collections—and best if they are merged but unlikely due to “protecting turfs.” Workstream 7: Strengthen external communications for greater transparency toward taxpayers.  There are many activities under external communications: improving efficiency of the Call Center via training in customer service and information on the tax system so that it is better able to serve the public; renewing annual national survey of taxpayer perceptions to gather feedback on issues related to the fiscal system, monitor compliance improvements, and assess changes in public perception of taxpayer services; preparing communications products (e.g., videos) for traditional and digital/social media campaigns to raise public awareness on features of the budget law such as de-cashing and tax amnesty; creating channels of consultation and dialogue with professional organizations, civil society, and the private sector (e.g., Small and Medium-Sized Enterprises [SMEs] Roundtable or Enterprise Days); implementing a communications strategy targeting young people (e.g., “Youth Know Taxes” campaign) and arranging events for young accountants to discuss tax issues; redesigning the MOF website and migrating the new user-friendly website to the MOF’s IT Department (before it was hosted off-site) for posting valuable information to taxpayers online; developing a mobile application (TUNIMPOT) concerning tax registration; and assisting with the Taxpayer’s Practical Guide (an informational booklet that consolidates all the tax obligations for Tunisian citizens and businesses). Table 3: Linkages Between Component 4, Workstreams 6 and 7, Key Results, and EQ 2d Components (objectives) Workstreams (sub-objectives) Achievements (key results or main activities) Sub-questions C4: Improve communications, engagement, and consultation on priority reforms W6 to improve internal communication W7 to strengthen external communications for greater transparency toward taxpayers  Webcasting (intranet) link Tunis to eight regions  Strategic Communications Unit strategy  Strengthen coordination DGI and DGCPR Enhancing public accountability  Call Center efficiency  Media campaigns support (de-cashing and tax amnesty)  MOF website redesign  Taxpayer’s Practical Guide Regarding cross-cutting gender, according to the contract: 60 In practice, FIRST has combined C2 (tax policy reform) and C3 (other fiscal reforms)—see Year 2 Work Plan, p. 7. 17 “The contractor shall systematically consider the gender aspects and implications of specific interventions, including opportunities to reduce gender-related biases or risks embedded in tax and revenue policies, systems, and procedures. As part of the annual work plan, the Contractor shall include a gender analysis that identifies ways to mitigate gender issues affecting this activity. Further, the contractor shall integrate gender considerations into all planned deliverables.” In terms of gender equity, the ET found a gender balance in FIRST staffing, MOF beneficiaries, and training participants, but not in the STTAs deployed by FIRST.61 EQ 2a: Conclusion for increasing tax revenues (WS1, 3) Though the TADAT analysis of Tunisia’s tax system deficiencies eventually fed into work planning for Years 2 and 3 (compliance strategy and action plan), the tool was not embraced by the MOF because there were concerns that an assessment by outsiders could lead to future “blacklisting,” and when the report was issued it painted a bleak picture of the fiscal system. The UXP platform, on the other hand, was strongly supported at the highest levels of the MOF. It is seen as a valuable tool that will enhance data compilation for auditing and facilitate the analysis of sources of tax gaps thereby yielding greater tax revenues. UXP was installed in record time and is regarded as a success story. In addition, FIRST’s ad hoc TA support (FATCA, etc.) demonstrated its flexibility and responsiveness to MOF counterparts, and lengthy and traumatic audits without any real budgetary effect are being replaced with a more effective issue-oriented audit strategy supported by rapidly collected payments for back taxes. FIRST efforts to build DGI capacity through revised policy, upgraded systems, SOPs, computer tablets, and practical training will improve the deterrent effects of tax audits, and therefore also lead to raising tax revenues. EQ 2b Conclusion with respect to compliance (WS2) FIRST efforts aimed at broadening the tax base by registering taxpayers, upgrading the taxpayer registry, revising penalty system, and digitalization (e-services) to facilitate filing and payment, and training, are all effective for reducing compliance costs/easing burden on taxpayers. EQ 2c Conclusion of FIRST efforts in improving budgeting and expenditures (WS4, 5) FIRST’s development of FAU capabilities for analyzing tax issues and formulating policy options and proposals for targeted sectors that underpay or are virtually untaxed is a long-term undertaking. Improved understanding of tax gaps, tax incidence, tax expenditures, and specific subsidies, though worthwhile, is unlikely to produce concrete results in the short term. Likewise, efforts to improve FAU capacity to quantify policy impacts on revenues and on the broader economy, better communicate with stakeholders on proposed tax reforms during the budgetary process, and garner support for reforms is worthwhile but unlikely to improve the budgeting and expenditure process in the short term. EQ 2d: Conclusion for public accountability (WS 6, 7) A stronger SCU and setting up the intranet will improve internal communications and understanding, and in turn positively support MOF external communications efforts to engage and consult with the private sector and civil society. The diverse external communications initiatives by MOF supported by FIRST (see Annex X for full listing of communication products) reflects the desire for a more open and transparent government, and aim to get in front of the public discourse and shape public opinion. Communication efforts that provide information and assistance to citizens and businesses to better 61 In response to the ET question on PFM and gender equity in FGs, some responded, “There is no issue as women are everywhere” and “We are all feminists.” 18 understand their rights and obligations, and to get their feedback on fiscal policy and tax reform initiatives, not only enhances public accountability but also promotes fiscal civism (culture of voluntary compliance). Despite FIRST having a gender strategy and MEL Plan indicator 18, which states that “policies/regulations should have a gender review,” very little was done by FIRST to advance opportunities for women in the fiscal domain. EQ 2: Conclusion Overall: How is FIRST contributing? Without a clear process for implementing main activities (i.e., detailed action plans with clear milestones for each initiative), it is difficult to understand FIRST’s real progress, bearing in mind the distinction between main activities (key results) and activities leading to achieving the main activity. FIRST provided a listing of 27 key results (Annex XII) but only three have been achieved to date. According to the Gantt chart in the Year 3 Work Plan, there are approximately 70 activities to complete in Year 3, which is a considerable challenge.62 In addition, the MOF has to undertake activities to institutionalize these initiatives (i.e., make them operational and sustainable; see EQ 5). Some external stakeholders expressed a concern that by the end of FIRST there would be no discernable effect on tax justice/equity, expansion of tax contributor base, and better fiscal controls. Although there is urgent need to accelerate the pace of implementation in Year 3, in fairness FIRST is “a short-term project with long-term objectives,” capacity-building is intensive and time-consuming, and there are many factors impeding success (see EQ 4). FIRST is executing a very ambitious and extensive PFM reform program in a challenging environment. The prospects for outcomes and impact to be realized (and measurable) by the end of Year 3 are in doubt because initiatives first need to be completed by FIRST and then adopted by the MOF. FIRST’s Sustainability Plan needs to consider what to do about uncompleted main activities (within FIRST control), and what to do about initiatives not yet institutionalized (MOF responsibility). EQ 3: What were the internal (organizational, resources, etc.) and external factors that affected the efficiency and timeliness of the implementation and how? There were many factors facilitating and/or hampering the efficiency and timeliness of FIRST delivery— although there were some success factors, most of the factors were constraints on implementation. EQ 3 Findings: What factors affected implementation, and how did they affect it? Internal factors are found within FIRST activity itself and therefore are fully within its control, while external are outside the activity. Internal factors include the design of FIRST, the comprehensive assessment, work plans, office space, activity staffing, STTAs, PFM tools, technology, flexibility, MEL, and reporting. External factors include the election, international organizations, MOF champions, MOF staffing, MOF adoption, MOF “appetite,” MOF project framework, MOF traditions, PFM complexity, activity governance, external stakeholders, and IT talent. Below is an explanation of how the factors affected FIRST operational efficiency and timeliness. Internal factors FIRST duration was insufficient: In terms of FIRST design, the contract is succinct on “why” (justification for interventions), “what” (expected outputs) and “who” (internal and external stakeholders), but on “when,” FIRST activity duration is too short for structural reform. Not only is there insufficient time 62 The tax policy focus group stated, “It will be impossible to finish everything by the end of the third year.” 19 for meaningful reform to take hold but there is strong likelihood that some of FIRST’s main activities will not be completed (institutionalized). Some of FIRST objectives and targets are not feasible within the allotted time frame, and in light of the organizational situation and wider context. Assessment should be appropriate: TADAT is an excellent diagnostic tool, especially to feed into work planning Years 2 and 3, but the start of comprehensive assessment in Year 1 was delayed and it was time-consuming. TADAT was supposed to run in November 2017 and then prepare a three-year overall work plan, but the report was not issued until May 2018. The focus of effort, time, and resources on the TADAT tool contributed toward the tardiness of FIRST activities implementation in Year 1. Planning should be participatory: Though work plans built on prior reform interventions such as TCP and there was home office support for preparing the AMELP and the gender strategy, the MOF felt mostly excluded from activity work planning. In addition, work plan documents lacked an explanation of challenges and strategy. Being embedded is essential for CD: USAID and the IP tried to secure office space for FIRST experts to facilitate communication, coordination, and skills transfer but the MOF counterparts did not make any available. The exception was CIMF that offered office space to enable experts to help supervise various IT teams. Staffing changes are disruptive: Currently, FIRST has five expatriates and 23 local staff. The MOF respondents considered the in-house experts in Year 1 to be technically proficient, but said that communication and working relations were lacking. At the end of Year 1, there was turnover of core staff (the COP and team leaders), followed by a significant number of staff departures in late 2018/early 2019. Under new leadership in Year 2, FIRST needed to build relations with MOF counterparts as well as integrate staff replacements. STTA deployment is critical to success: FIRST brought in many experienced international consultants, mostly international, and focus group participants stated, “We learned a lot” (see Annex XIV for list of consultants and assignments). However, beneficiaries also noted that many consultants were not familiar with the Middle East North Africa region or Tunisia, or have a command of French/Arabic, and that some training was too theoretical. In terms of planning for consultancies, MOF respondents said they were not involved in preparing SOWs, reviewing CVs, and approving deliverables. Some said FIRST consultants pushed “best practice” tools when what was needed was “best fit.” In other words, state-of-the-art models introduced by FIRST have to be adapted to the MOF environment. Technology solutions add to technical complexity: IT solutions and associated applications are inherently complex because they tend to involve data flows (shift to digitization), process flows (systems upgrade and/or methodological modifications), and work flows (procedural changes and SOPs), as well as staff re-skilling (training). In addition, FIRST encountered delays with the MOF who ran into legislative barriers and technical issues (web services). There was also concern at the MOF over the control and security of sensitive data. Flexibility is paramount: FIRST demonstrated responsiveness by adding several initiatives (FATCA, webcasting, UXP) and dropping planned activities likely to cause political difficulties (review of pensions, public sector wage bill, fiscal decentralization). MEL is an essential management tool: According to the recent DQA, some indicators are not relevant/useful, there was no data management system so no reporting on indicators, and given the short time no effects could be realized at outcome levels. This meant the MEL system was inadequate as a management tool. It is being revised. 20 Reporting has to be timely, accurate, and succinct: There was also no substantive discussion of obstacles or blocking points in reports. However, the COP stated he had access to the Secretary-General to take his concern for a resolution, or via the FIRST Coordinator at MOF who would set up a COPIL. External factors Elections are unsettling: 2019 is an election year in Tunisia and as such there is no political will for major reform. The ET was informed that “everyone is waiting for the next government” (January 2020). The hope is that a new government will shake up the political establishment and push for real reforms such as eliminating inefficient tax expenditures and unsustainable subsidies. Budget conditions are helpful to reform efforts: FIRST has good relations with external stakeholders EU and IMF, and together with USAID, met them to discuss budget support conditionalities (budget law) to force MOF changes (e.g., targeted audits, tax expenditure report, and post-audit control). FIRST and USAID should continue this tactic with KfW as they are also considering budget support in 2020. MOF is a difficult environment: Many constraints arise from the ministry side—MOF leadership is not particularly reform-minded. However, when a need emerges that they support, things move quickly. For example, FATCA was driven by the MOF Minister, UXP was driven by the MOF Secretary￾General, and webcasting by the FIRST Main Focal Point at the MOF. Working groups are beneficial: Initially, FIRST asked for formal project teams to be directed by senior MOF managers and staffed by motivated employees assigned to work on FIRST initiatives on a full￾time basis for the duration of the activity. MOF did not allow secondment to occur, and instead FIRST used existing working groups carried over from TCP. In Year 2, FIRST identified focal points within MOF corresponding to the TADAT pillars to facilitate new activities on an unofficial basis. MOF is a traditional institution: MOF is characterized by antiquated processes, archaic systems, and a “silos” structure that hamper communications and coordination.63 It features an old-school top-down management style, and there is an “old guard” that forms pockets of resistance to change.64 In addition, MOF staff have a busy workload, so at times they could not dedicate effort to FIRST, especially during budget time (view expressed by several FGs). There was also MOF staff turnover affecting FIRST (e.g., vacant positions of focal points of tax gap and scoring working groups). Adoption is the final step in the reform process: As a result of many of the factors cited above and evidenced by the EU and IMF, PFM reform can often take a long time from initiation to institutionalization. A completed activity on FIRST’s part does not mean it is fully operational—the final step is for all changes and improvements to be adopted (embedded) by the MOF. Beneficiaries at the MOF need to use the tools that were introduced and apply the know-how acquired through TA and training. This last step is out of FIRST’s control, but if there is commitment by MOF to the intervention, then this final step will likely be taken by MOF—the “final mile.” Computer tablets as well as study tours to Estonia for UXP and to Morocco for tax administration and e-government are being touted by the partner as a “reward” to its staff for performance and cooperation with FIRST. The Steering Committee is good for resolving issues. FIRST envisaged the Steering Committee comprising its MOF counterparts and meeting quarterly to set priorities, review key results, help clear roadblocks, and facilitate access to data for studies and analysis (see Year 1 Work Plan in Annex 2). However, the 63 Observations of MOF from various representatives of the donor community and civil society shared with ET. 64 See FIRST Annual Progress Report, Year 1, September 2018, pages 11 and 12. 21 first formal SC (COPIL) did not occur until May 2018 and has then met intermittently. In Year 1 the SC met on “thematic issues” only, i.e., to resolve problems.65 A Memorandum of Understanding (MOU) provides clarity. The lack of a formal signed agreement (MOU) between the activity and partner was to the detriment of good governance and good operations. There was no clarity on expectation, roles, responsibilities, protocols, etc., and working groups were not formalized.66 Not all working groups set up at end of Year 1 are active. Currently, they are headed by “focal points” and membership depends on their mandate. PFM reform is a highly technical process. PFM reforms are challenging (see technical note in Annex VIII). Examples include UXP data management system and taxpayer information system (registry). It is easy to underestimate the time and effort it will take to implement for example, upgrading the MOF website was targeted by FIRST as a “quick win” because it appeared to be relatively straightforward but instead of the planned six months it took two years to complete. In addition, vested interests such as liberal professions (doctors, dentists, lawyers, etc.) lobby to undermine and block reform efforts.67 “Brain drain” is undesirable: Young educated Tunisian talent is leaving for Europe, particularly those skilled in ICT.68 IT solutions and digitalization of MOF requires dynamic energetic and highly qualified people to succeed so this is a concern. FIRST has focused its coaching and mentoring support on young persons in the MOF. EQ 3 Conclusions: What were the factors affecting implementation, and how? In conclusion, there are many factors promoting and impeding the success of FIRST implementation, both internal and external. FIRST needs to be aware of all factors that affects the efficiency and timeliness of its implementation. It is important to focus on those aspects that facilitate implementation (success factors) and to come up with mitigation measures to minimize impacts of negative factors. The table in Annex XVI summarizes the factors and highlights their positive and negative aspects. EQ 4: To what extent has FIRST succeeded in identifying, partnering, and coordinating with various stakeholders? What other partnerships will strengthen FIRST and similar USAID/Tunisia activities? FIRST partnering was effective but limited in scope and insufficient for building a broad constituency for supporting meaningful reform efforts. EQ 4 Findings: To what extent has FIRST succeeded in partnering? FIRST contract: As stipulated on page 10 and reiterated in the Year 2 Work Plan, FIRST must work in close collaboration with the MOF, and especially units within MOF that focus on tax policy, revenue administration, and wider macro-fiscal concerns; work selectively with other GOT units, key stakeholders in Tunisian Parliament, the private sector, civil society, and donors and multilaterals, including the EU, IMF, and World Bank; and coordinate with other USG agencies such as the US State Department, US Department of Treasury, and US Customs and Border Protection. FIRST partnering inside: There was strong coordination with MOF internal stakeholders (counterparts) in Year 2. Working relations with the MOF Secretary-General, and with relevant DGs heading workstreams are effective. FIRST also works closely (joint working) with beneficiaries at lower levels on specific activities. Any emerging issues have been taken to the MOF main focal point for resolution 65 The Evaluation Team requested Minutes of COPIL meetings but none were forthcoming. 66 See FIRST Annual Progress Report, Year 1, September 2018, page 11. 67 See IP-USAID contract award for FIRST, August 2017, top of page 9. 68 https://7dnews.com/news/why-tunisia-expels-its-talent 22 that sometimes requires a COPIL meeting. But this strategy seems to have limits. Many projects are underway, and the focal point at the MOF no longer has a clear idea of the progress of the various projects. Interestingly, the Tax Administration FG informs that MOF sees FIRST as being part of the MOF, so it is not necessary for FIRST to interact with people from outside the ministry. FIRST partnering with other GOT entities: The COP stated there are no GOT partners outside MOF except for customs who are “independent,” the Ministry of Major Reforms that reports to the PM on oil price hedging, and the MDiCI on the World Bank Doing Business/paying taxes element. The tax policy focus group said, “There was no relationship with ITCEQ” (for multisectoral general equilibrium model (modèle d'équilibre général calculable or MEGC)).69 ITCEQ has been developing MEGC for a long time, and a collaboration with this institution would have allowed the MOF to advance more quickly by capitalizing on what already exists. FIRST partnering outside: As for other external stakeholders, FIRST has good contacts with multilaterals IMF,70 EU,71 and World Bank,72 but less so with donors such as AfDB, AFD, KfW, and Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), some of whom were somewhat aware of FIRST but had very little information about achievements to date. FIRST had no real dealings with parliamentarians, the private sector (except Enterprise Days) and labor unions. FIRST collaborated with the Academie des Banques et Finances (Tunisian Bankers Association training institute) on organizing FATCA training for local financial institutions as well as with the Millennium Challenge Corporation (MCC) on technical missions related to the compact with Tunisia. Regarding civil society, SOLIDAR Tunisie is a social justice nongovernmental organization on the winning bid for FIRST, but according to the General Director, during start-up it was sidelined because there was a “difficult relationship with FIRST.” Although working relations are much better with the current COP and SOLIDAR Tunisie wrote the revised political-economy analysis (PEA), they have been informed that further involvement with FIRST is being blocked by the MOF. IACE (a think tank) observed that FIRST has not collaborated with civil society on tax policy reform, and this was a missed opportunity. UTICA represents the private sector (mostly SMEs) and is only vaguely familiar with FIRST. UTICA states that there are 400,000 businesses in Tunisia, so it is impossible for the MOF to follow them all, and that the MOF database is woefully outdated. No respondents reported any coordination with other USG agencies. EQ 4 Findings: What other partnerships will strengthen FIRST? Strengthening FIRST partnering with GOT: It would be useful to harmonize all TA and other programs for Tunisia by strengthening the role of the MDiCI to strengthen cooperation between the various international institutions. This should lead to scale efficiencies and increased efficiency. The MDiCI should centralize all programs and transparently inform each other’s activities. Programs that develop certain skills within a department should also consider the existence of these competencies in other departments. It would be useful to strengthen the skills of all concerned for greater sustainability and ownership of projects at the national level. Strengthening FIRST partnering with donors: To some extent the effectiveness of FIRST partnering is dependent upon USAID’s relations with other donors and multilaterals. It appears that with the latter there is good communication, but ET has learned from some donors that while there may some coordination at the project level, collaboration on initiatives is negligible. Donor cooperation is not 69 Institut Tunisien de la Competitivite et des Études Quantitatives; multisectoral general equilibrium model. 70 IMF has met with FIRST on several occasions to be apprised of their work. 71 EU representatives expressed a very high appreciation for the work done by FIRST. 72 World Bank focus in the fiscal domain is on customs. 23 always easy and sometimes donors compete over partners (GIZ). The IMF program ends in April 2020, so there will be an opportunity then to work with the new government on tax revenues, provided it seeks IMF assistance. Strengthening FIRST partnering with others: External stakeholders include private sector, civil society, unions, and think tanks. All reform projects need cooperation with civil society. For example, SOLIDAR has expertise in several areas related to taxation, can carry out advocacy, and has links to Parliament. Outside entities such as IACE are valuable to facilitate consultation with top-level government officials, Ministers, and Parliamentarians. UTICA has worked on many position papers pertinent to taxation. As such, they could help FIRST by providing news on taxation from the private sector perspective. EQ 4 Conclusions: To what extent did FIRST succeed in partnering, and what other partnering will strengthen FIRST? Best practice suggests that reforms require the building of a broad constituency both to facilitate implementation and for future sustainability, and FIRST collaboration with external stakeholders is good with multilaterals (facilitated by USAID) but it has not reached out to donors such as GIZ and AFD, private sector, labor unions, and civil society. Collaboration with civil society can help to clearly identify needs in terms of structural reforms, and also to define priorities. It is important to work with key stakeholders for a pragmatic vision and maximum impact. In addition, civil society can help unlock certain situations and advocate at the political level. In the current context, projects without political support will find it difficult to succeed in Tunisia. EQ 5: To what extent are FIRST activities sustainable and contributing to Tunisia Government effectiveness (as per the J2SR 4.GC.1)? It is premature to determine the extent to which FIRST activities are sustainable and contributing to government effectiveness because few main activities have been completed and adopted. EQ 5 Findings: To what extent are FIRST activities sustainable? Long-term impact: Sustainability is defined as “all the changes made in partner organization that were supported and facilitated through FIRST continue long after the activity is ended.” There are two issues in sustainability: (a) will FIRST complete the remaining 24 main activities within its control by end of Year 3, and (b) will the MOF fully adopt the various tools, systems upgrade, SOPs know-how, and technology? In other words, will FIRST interventions become completed and institutionalized? FIRST has to complete all activities to ensure sustainability— UXP for example is installed and partners are “all onside,”—but some legislative modifications are needed before the activity can be fully adopted. Without clear action plans for each main activity, it is not clear what FIRST needs to do to complete these, and more importantly what the MOF needs to do (the last mile) as a final step for institutionalization and sustainability. FIRST inputs: FIRST provides technical assistance, tools, training, and technology (4 Ts): TA is provided by both FIRST in-house experts and outside consultants via STTAs. Tools comprised strategies, models, methodologies, SOPs/manuals, user guides, modules, communication products, etc. FIRST arranged training in 15 different courses73 for 1,428 MOF staff (736 female and 692 male) totaling 2,348 training days, but there has been no follow-up to determine effectiveness of training. Post- 73 FIRST offered training in CGE modeling, compliance strategy, customer service, fiscal control policy, MTFF model, post￾clearance audit, collections, contributable services, TADAT, tax expenditures, tax gap, UXP service administrator, UXP service developer, UXP TOT, and punctual verification. 24 training surveys on competency will reveal whether the training was useful and is being applied in the workplace. Beneficiaries stated they wanted training to be more practical, but FIRST argues that transfer of knowledge is considered as “theoretical” by trainees. Two study tours may be arranged in Year 3—one to look at Morocco tax administration and the other to go to Estonia for UXP. Study tours are usually designed to reinforce training but MOF wishes them to be deployed as a “reward” for beneficiaries’ cooperation and successful completion of interventions. As for technology, FIRST has committed US$1.12 million to date (nothing planned for Year 3) for computer hardware and software. Twenty software programs have been introduced by FIRST so far,74 and this creates the impression that FIRST has become an IT project. Sustainability of inputs: TA is enshrined in legislation and regulations (e.g., fiscal code for targeted auditing and budget support conditionalities in Budget Law); tools are being put into FIRST’s virtual resource library to be handed over to partners in mid-2020; training materials are also put into the library; and Training of Trainers have been arranged to train new recruits as well as refresher courses. Regarding technology, the MOF has security measures to prevent theft, particularly of mobile equipment (laptops, printers, tablets); the CIMF budget will cover maintenance and repair to keep equipment functioning, and software license renewal and perpetual licenses have been secured for software to minimize recurrent costs (FIRST paid for three years) for UXP and GAM software; in some instances the MOF has kept software codes in-house. FIRST has a Sustainability Plan within its draft Year 3 work plan. EQ 5 Findings: To what extent is FIRST contributing to GOT effectiveness? Tunisia J2SR FY2019 Country Roadmap:75 USAID is reorienting its policies, strategies, and practices to improve its support to a partner country, specifically its capacity to plan, finance, and implement solutions to local development, and its commitment to see these through effectively, inclusively, and with accountability. In the J2SR, there are seven commitment metrics and 10 capacity metrics.76 One government capacity metric sub-dimension relates to government effectiveness (e.g., quality of public services, quality of civil service, and its independence from political pressure), quality of policy formulation and implementation (including efficiency of revenue mobilization and budget management), and credibility of the government’s commitment to stated policies. GOT currently scores 0.51 out of 1.0 on government effectiveness. EQ 5 Conclusion: To what extent are FIRST activities sustainable? FIRST COP is confident that it will complete all activities in Year 3 in its control (transfer of technology, SOPs/manuals, know-how through TA, training, procurement, and tools) but some may not be fully institutionalized by project end, meaning the MOF may not have fully adopted and applied these inputs. This is outside the control of FIRST. All FIRST can do is to ensure that it completes the main activities. The FIRST Year 3 draft work plan Gantt chart shows all activities necessary under each main activity. EQ 5 Conclusion: To what extent do FIRST activities contribute to GOT effectiveness? As for J2SR, FIRST is advancing improvements in public (taxpayer) services, building capacity for civil servants, building capability for independent tax policy analysis and policy formulation, enhancing efficiencies in revenue mobilization and in budget management. But credibility of government commitment to policies will depend on new government in 2020, the effective use of FIRST tools and the sustainability of the project. But we cannot say to what extent FIRST is contributing to GOT 74 See listing in Year 3 Work Plan in Annex I. 75 https://selfreliance.usaid.gov/country/tunisia 76 Self-Reliance Metrics. FY2019 Methodology Guide. USAID September 2018. 25 effectiveness because there has been no measurement of the effects of FIRST’s contributions on Tunisian government effectiveness thus far. RECOMMENDATIONS Based on the findings, the ET developed the following recommendations: FIRST should do the following:  Include all MOF activities necessary to adopt reforms (the final mile) in FIRST’s Sustainability Plan, and discuss and agree this with USAID and MOF in the Steering Committee forum to ensure long-term impact.  Articulate a clear strategy for implementation that reflects its technical approach including capacity development (CD) best practice and incorporates FIRST tactics for overcoming external constraints, including MOF challenges (e.g., encouraging competition among DGs for FIRST resources).  Ensure MOF has input into FIRST’s Year 3 work plan, approves FIRST quarterly and annual reports, and is involved in all aspects of delivery (e.g., STTA SOW, selection, and deployment).  Ensure its planning and reporting documents are consistent, coherent and comprehensive.  Consider preparing action plans for the 24 ongoing main activities (key results) listed in FIRST’s Year 3 work plan, with milestones as process indicators, to aid monitoring of progress.  Ensure all activities in its Year 3 work plan Gantt chart are carried out by August 2020 to enhance prospects for sustainability of results.  Establish a data management system for tracking and reporting on indicators, and revise indicators and MEL Plan as necessary.  Reach out to, and partner with, donors, private sector, civil society, and unions to build a broader constituency for advancing PFM reforms.  Work with the MOF on operationalizing its Performance Management System.  Include in the Year 3 work plan a gender analysis that identifies ways to mitigate gender issues affecting this activity, and systematically consider the gender aspects and implications of specific interventions.  With USAID, open a dialogue with IMF, KfW, and other donors on fiscal reform budget support conditionalities in 2020.  Conduct a public survey via MOF/SCU for taxpayer opinion on tax matters and the MOF. The MOF should do the following:  Inform FIRST what the ministry needs to do to adopt reform initiatives (e.g., to institutionalize UXP legislative measures for sharing information and IT applications for data exchange).  Make suitable office space available for FIRST experts inside the relevant departments of the MOF.  Sign an MOU with USAID to provide formal institutional governance structures and protocols for FIRST activity.  Schedule Steering Committee meetings to discuss FIRST Year 3 work plan and quarterly reports, as well as FIRST achievements and challenges.  Expand the Steering Committee membership to include key external stakeholders, i.e., multilateral institutions, Tunisia private sector, and civil society. USAID should do the following: 26  Request a detailed implementation strategy in the overall work plan of all future USAID activities to properly understand the IP approach to delivery of aid.  Lengthen the duration of PFM-CD activities in future programming for more meaningful reforms (e.g., structural changes).  Ensure that an MOU is signed with the partner for establishing governance and protocols (roles, responsibilities, expectations, working arrangements, etc.).  Require high-quality planning and reporting documents from the IP to facilitate USAID’s monitoring function.  Request that FIRST revise its MEL Plan and Sustainability Plan. LESSONS AND OBSERVATIONS FIRST is a fairly standard PFM intervention (progressing fiscal reforms at Tunisia’s Ministry of Finance) encountering typical developing country challenges. As such, lessons and observations derived from the experience are relevant to other projects in similar circumstances. The first lesson is that all projects should have a detailed implementation strategy at the outset, i.e., a clear and coherent approach to how assistance will be delivered and not just what and when activities will be delivered as reflected in activity scheduling (Gantt chart). In the case of PFM reforms, this will involve interventions such as introducing models, upgrading systems and technology, enhancing know-how, etc., and these changes need to be managed using a capacity development methodology. Although project staff may comprise high-level experts with vast technical knowledge of fiscal reforms, they may not fully understand the current thinking on capacity-building or consistently follow the process in practice. CD has evolved over the years and the current thinking is that meaningful institutional reform is not only participatory but takes a long time for resistance to change to be overcome, and for capacities to be developed and take root. It has been stated that PFM is not for the faint-hearted or the time constrained. To undertake effective public financial management reforms and capacity-building, the following steps are considered best practice: Understand Stakeholders In-house project experts should not only engage with the partner (internal stakeholder) but also with external stakeholders (i.e., persons outside the organization having an interest in or influence over such as private sector, unions, civil society, think tanks, donors, politicians, and media). Broader constituency support is vital for change management, especially structural (major) reforms such as conditionalities in donor budget support. A political–economy analysis (PEA) can help with understanding the wider context—the “enabling environment” can exert powerful influence on reforms. Project managers should communicate regularly with other relevant projects and avoid competitive stance (because of bilateral relations, donors usually communicate with other donors). Support Partner Ownership and Leadership Aid recipient countries should lead reforms, with donors playing a supporting role (i.e., aid should be beneficiary-led and facilitated and supported by experts but not defined by them). Putting the partner in the driver’s seat greatly enhances efficiency (implementation), effectiveness (results), and sustainability (long-term impacts). PFM-CD projects should take a participatory approach because that will build trust and commitment; this will also enhance ownership, which is critical to project success vital for sustainability. Partners should be held responsible for reform initiatives. Strong leadership with 27 clear vision is crucial for project success and it is important to have champions who are drivers of change. Without genuine commitment to reforms at the highest level within an organization, it will be an uphill battle and for aid deliverers will feel like “spinning wheels.” Project governance is needed for oversight and accountability. Projects should start with direct involvement of those affected, meaningful engagement that goes beyond box-ticking consultation and participation. Projects need a clear written agreement to reflect understanding of agreed working arrangements. A signed MOU and a Project Steering Committee will ensure there is a clear understanding of expectations, roles, and responsibilities, and foster partner ownership of the process while at the same time providing safeguards for the project. The MOU can also help initiate working group structure formed on the basis of key results; it is best if ministry staff are dedicated full time (i.e., on secondment). Assess Partner Needs and Priorities In project design, experts should not crowd out the voices of those who understand the local challenges best, the end users of aid. Most aid projects fail because they do not genuinely understand the wider context and the organizational situation. It is essential that experts develop a deep understanding of institutional constraints. During the project start-up, it is necessary to properly assess institutional arrangements (e.g., policies, procedures, structure, leadership, communication, coordination, etc.). Engagement should consist of a dynamic dialogue between the expert and the end user. CD works best when it combines expert supply and demand side push. Experts should not impose comprehensive time-consuming diagnostics if that is unwanted or deemed unnecessary; use appropriate assessment tools to assess preexisting capacities. It is vital to distinguish needs from demands—the former are issues/constraints while the latter reflect urgent priority supported at the highest levels. Plan Interventions with Partners CD occurs at two levels within an organization—enhancing knowledge and skills of individuals as well as strengthening the institutional reform. The planning of interventions draws on local conditions, best practice, innovation, and alternatives/options. Project design and sequencing should align with the specific circumstances of the country, and not reflect imported solutions; avoid placing square pegs in round holes. An approach should be used that fits a country’s circumstances and needs, not necessarily best practice. Support provided should be coherent, coordinated, and programmatic. So-called “joint working” yields true tailor-made solutions and interventions that will be regarded as legitimate. The vital role of project experts is as change agents to overcome resistance to change. A more realistic approach may be step-by-step or incremental rather than major, profound structural changes, and it is best if CD interventions are integrated into existing processes. Proposed IT solutions to improve capacity should be appropriate technology. Project and partner must discuss and agree on the logic of interventions, targets and indicators, implementation strategy, and intended results as reflected in the AMELP and annual work plans. Experts should not insist on specific TA if it is not requested/supported by the partner, and recognize that partners are entitled to reject proposed interventions. Deliver Aid Jointly Enhancing the performance of the public sector is not easy due to technical complexity, political sensitivities, resistance to change, etc. Project implementation should be responsive to emerging needs and opportunities; so, be flexible. Getting direct feedback on an ongoing basis from the end 28 user/beneficiaries is much improved when the project office is inside a ministry. Being embedded enables experts to get close to the end user and deeply immerse in their day-to-day work challenges. Project experts need to focus more on the process of aid delivery, but this will take time, and donors are driven by results, so divergences of opinion will occur. Lack of political support limits success of CD efforts—you cannot push reform too hard, especially during an election year. Ensure the partner is actively involved in delivery of TA, from SOW preparation to CV selection to approving deliverables, or in the case of procurement from drafting specifications to selection of vendor and facilitation installations. Provide high-quality inputs (4 Ts) technical assistance, tools, technology, and training, both for strategic and operational support for real development and not merely to manage (“burn”) donor funds. Monitor Progress and Assess Results The results framework is important for day-to-day management. It is important that goals, objectives, and targets be modest and realistic rather than ambitious and aspirational. Measure changes in institutional performance for accountability. The MEL system will capture information and data on deliverables—tangible, concrete outputs that monitor and measure effects of inputs on both individuals and on the organization/institution for outcome results. Report on Activities and Results Project reporting must be regular, frequent, consistent, clear, coherent, and comprehensive; it must cover activities, achievements, and challenges. Reports should be shared with partners on a regular basis for feedback and approval. Ensure Sustainability The project exit strategy should ensure results are sustainable. MEL needs to follow up on activities to determine whether deliverables (models, methodologies, SOPs, manuals, and training) have indeed been adopted (i.e., used and cared for by the partner) via spot checks, post-training surveys, observations, etc. If interventions are not properly institutionalized, then there will be no long-term impact. 29 ANNEXES 30 ANNEX I: FIRST LOGIC MODEL Below is FIRST’s high-level Results Chain or Logic Model linking project activities to their stated objectives. 31 ANNEX II: EVALUATION SCOPE OF WORK OFFICE OF ECONOMIC GROWTH STATEMENT OF WORK MIDTERM EVALUATION OF FISCAL REFORM FOR A STRONG TUNISIA [FIRST] PURPOSE OF THE EVALUATION USAID/Tunisia requested Monitoring and Evaluation for Tunisia and Libya (METAL) to conduct a Midterm performance evaluation of the Fiscal Reform for a Strong Tunisia (FIRST) activity. The purpose is to identify the extent to which FIRST is achieving intended results, and assess the relevance, effectiveness, and efficiency of the approaches used to achieve USAID/Tunisia objectives. The evaluation will identify successes, best practices, lessons learned, and challenges (internal and external factors) that affected the implementation of the activity. The evaluation will also help identify potential course corrections before the project ends in August 2020. The final report will propose recommendations based on the findings that would help inform USAID/Tunisia programming strategy and decision-making. The primary audience for this evaluation is USAID/Tunisia, particularly the Economic Growth Office. SUMMARY INFORMATION Activity name Fiscal Reform for a Strong Tunisia (FIRST) activity USAID office USAID/Tunisia Implementer Chemonics Cooperative agreement/Contract # Contract under AID-OAA-I-12-00035/AID-664-TO-17-00001 Total estimated ceiling of the activity (TEC) $17,325,000 Life of activity August 2017–August 2020 Active geographic regions Tunisia Development objective(s) DO 1: Inclusive private sector employment increased Required evaluation? Yes External or internal evaluation? External 32 BACKGROUND Tunisia faces two interrelated challenges: sustaining the country’s ongoing effort toward improved governance and inclusive development. USAID Tunisia supports these efforts through economic and governance assistance programs to promote both economic participation and democratic consolidation. Fiscal Reform for a Strong Tunisia (FIRST) assists the Ministry of Finance to rationalize tax policy, modernize tax administration, and undertake other fiscal reforms to secure a sound fiscal foundation for economic stability and long-term growth. The project aims to help the Tunisian government improve revenue collection, reduce taxpayer compliance costs, improve the budgeting and expenditure process, and enhance public accountability. Description of the Problem and Context The weak political and economic institutions inherited from previous regimes have not been able to martial effective pressure on the government to reform the anachronistic tax system also inherited from the prerevolution government. As a result of the economic liberalization unleashed by the revolution of 2011, entrepreneurship has created a growing cadre of small and medium-sized, largely family-owned companies, as well as a growing class of increasingly wealthy independent professionals, most of whom are virtually untaxed. While the tax base has remained static, growth of the public-sector employment, public demands for improved services, and public employee demands for increased wages have pushed government spending to increasingly higher levels. Description of the Intervention to Be Evaluated and Theory of Change The development hypothesis for this activity is that if the government can maintain macroeconomic stability while achieving greater simplicity, equity, and efficiency in the tax system, the private sector will respond by increasing investment, expanding business activity, and hiring more workers. Business, investment, and job growth, in turn, will broaden the tax base, bolster the public’s confidence in government, and over time create the fiscal space for both lower taxes and concomitant increases in public investment and services. A longer-term “governance dividend” can also be expected from this activity, insofar as reforms and improvements in fiscal policy and administration help to legitimize the state, increase political stability, and enhance social cohesion. The project aims to improve the efficiency, transparency, and cost of compliance of tax administration as well as enhancing capacity to develop, manage tax policy, and address other fiscal reform priorities as they emerge. Besides the project intends to improved communications, engagement and consultation on priority reforms.77 EVALUATION QUESTIONS The Evaluation Team Leader (TL) and USAID will review the evaluation questions (EQs) below and adjust them if needed prior to finalizing the evaluation design. EQ 1: To what extent was the designed activity approach relevant to the Tunisian Government needs? Is the approach still relevant? EQ 2: To what extent, and how did FIRST contribute to (a) improving revenue collection, (b) reducing taxpayers’ compliance costs, (c) improving the budgeting and expenditure process, and (d) enhancing public accountability? EQ 3: What were the internal (organizational, resources, etc.) and external factors that affected the efficiency and timeliness of the implementation and how? EQ 4: To what extent has FIRST succeeded in identifying, partnering, and coordinating with various stakeholders? What other partnerships will strengthen FIRST and similar USAID/Tunisia activities? EQ 5: To what extent FIRST are activities sustainable and contributing to Tunisia Government effectiveness (as per the J2SR 4.GC.1)? EVALUATION DESIGN AND METHODOLOGY This performance evaluation will utilize a mixed-methods research design, employing both quantitative and qualitative methods to strengthen the validity of the findings and provide room for data triangulation. METAL will describe and document the methodological approach that will be used, which follows USAID Evaluation best practices. The model will include an evaluation framework and assessment tools for each evaluation question, and highlight the conceptual model(s), specifying the measurement criteria to be used to respond to each question. It will discuss any risks and limitations that may undermine the reliability and validity of the evaluation results. 77 https://www.usaid.gov/tunisia/fact-sheets/first 33 In order to ensure the maximum value for learning and use, a description of the proposed evaluation methodology will include the following: • Review of the existing baseline relevant in data analysis • Methods of data collection • An evaluation design that shows how sampling will be done and appropriate sample sizes required to ensure scientific rigor (if applicable) • Use of primary and secondary data in data analysis and a plan for analysis Evaluation Design Matrix Template (To be finalized by the TL) Questions Suggested Data Sources Suggested Data Collection Methods Suggested Data Analysis Methods 1. [Evaluation Question] 2. [Evaluation Question] 3. [Evaluation Question] DELIVERABLES AND REPORTING REQUIREMENTS Evaluation Work Plan Within one week of the evaluation kickoff call with USAID, METAL will present a draft work plan to include the following: • Draft schedule and logistical arrangements • Members of the evaluation team, delineated by roles and responsibilities • Evaluation milestones • Anticipated schedule of evaluation team data collection efforts • Locations and dates for piloting data collection efforts, if applicable • Proposed evaluation methodology including selection criteria for comparison groups, if applicable • Evaluation Report outline METAL will update the Contracting Officer’s Representative (COR) on any updates (interviewees, survey participants, schedule) as needed. 1. Evaluation Design Within one week of the work plan approval, METAL will submit an evaluation design, to include the following: • Detailed evaluation design matrix that links the Evaluation Questions from the SOW to data sources, methods, and the data analysis plan • Draft questionnaires and other data collection instruments or their main features • List of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan (will include sampling methodology and methods, including a justification of sample size and any applicable calculations) • Limitations to the evaluation design • Dissemination plan (designed in collaboration with USAID) The data analysis plan will clearly describe the TL’s approach for analyzing quantitative and qualitative data, including proposed sample sizes, specific data analysis tools, and any software proposed to be used, with an explanation of how/why these selections will be useful in answering the evaluation questions for this task. Qualitative data will be coded as part of the analysis approach, and the coding used should be included in the appendix of the final report. Gender, geographic, and role (beneficiary, implementer, government official, nongovernmental organization, etc.) disaggregation must be included in the data analysis where applicable. All dissemination plans will be developed with USAID and include information on audiences, activities, and deliverables, including any data visualizations, multimedia products, or events to help communicate evaluation findings, conclusions, and recommendations. 34 2. Inception Report: The inception report will describe the conceptual framework the evaluator will use to undertake the evaluation and the justification for selecting this approach. It will detail the evaluation methodology (i.e., how each question will be answered by way of data collection methods, data sources, and sampling). The report will also contain a work plan, which indicates the phases in the evaluation with key deliverables and milestones. USAID/Tunisia will review this report and the evaluation team will submit it to the COR for approval before it begins implementing the evaluation plan. The inception report will clearly document and discuss how gender analysis will be integrated into the design of the evaluation. 3. Preliminary Presentation for Recommendations Development The TL must debrief USAID on the preliminary findings of the evaluation. This meeting will provide a summary of any analytical results and will discuss challenges, successes, and the way forward. The TL will deliver an oral presentation of the evaluation findings, conclusions, and recommendations for each question to USAID prior to finalizing the draft evaluation report. The team leader will be required to routinely update the evaluation point of contact on the progress of the evaluation. 4. Final Presentation In coordination with USAID, and in support of Agency collaboration learning and adaptation methodologies, METAL will organize a learning event and produce a learning report. The focus of the event is to generate varied and diverse learning points including useful and actionable suggestions or proposals with the end goal being to enhance achievement of USAID objectives. The specific objectives of the learning event are as follows:  To disseminate findings and recommendations from the evaluation  To review in-depth the key lessons and their implication for future programs  Most importantly, to engage stakeholders on the evaluation topic, to share lessons learned, barriers, successes, discuss recommendations and to generate a dialogue that captures stakeholder input, thoughts, and ideas on the technical approach used to achieve activity results as presented in the evaluation 5. Final Evaluation Report The TL will take five working days to respond to and incorporate final draft evaluation report presentation comments from USAID/Tunisia. METAL will then submit the final report to the COR. The TL will submit a final evaluation report that is based on analyzed facts and evidence and fully addresses all the evaluation questions. The report will be 15–20 pages in length (excluding annexes) and comply with the Checklist for Assessing USAID Evaluation Reports (see annexes). After taking into account all the new information and feedback provided on the final oral briefings and draft evaluation report, METAL will submit hardbound copies and an electronic version of the report to the Mission. METAL will also submit an electronic version in an appropriate media including all tools and products of the evaluation, including instruments and data in data formats suitable for reanalysis. METAL must ensure that Appendix I of the USAID Evaluation Policy – Criteria to Ensure the Quality of the Evaluation Report is followed. This includes the following requirements:  The evaluation report will represent a thoughtful, well-researched, and well-organized effort to objectively evaluate what worked in the project, what did not, and why.  Evaluation reports will address all evaluation questions included in the scope of work.  The evaluation report will include the scope of work as an annex. All modifications to the scope of work—whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline—need to be agreed upon in writing by the technical officer.  Evaluation methodology will be explained in detail, and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an annex in the final report.  Evaluation findings will assess outcomes and impact on males and females.  Limitations to the evaluation will be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings will be presented as analyzed facts, evidence, and data, and not based on anecdotes, hearsay, or the compilation of people’s opinions. Findings must be specific, concise, and supported by strong quantitative or qualitative evidence. 35  Sources of information will be properly identified and listed in an annex.  Recommendations will be supported by a specific set of findings and will be action-oriented, practical, and specific, with defined responsibility for the action.  All quantitative data collected by the evaluation team will be provided in machine-readable, non￾proprietary formats as required by USAID’s Open Data policy (see ADS 579). The data will be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed.  All modifications to the required elements of the SOW of the contract/agreement—whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline—need to be agreed upon in writing by the COR. Any revisions should be updated in the SOW that is included as an annex to the Evaluation Report. The final evaluation report must contain the following sections:  Abstract: This section will be no more than 250 words and will describe what was evaluated, evaluation questions, methods, and key findings or conclusions.  Executive Summary: This section will be 2–5 pages in length and will summarize the purpose, project background, evaluation design, and methodologies including main evaluation questions, key findings, conclusions, recommendations, and lessons learned from the evaluation.  Background: This section will provide a brief description of the project that highlights the scope, project development hypothesis, activities undertaken, key impact indicators, and impact areas. Other activities that complemented the project activities directly or indirectly in the intervention districts must also be highlighted.  Methodology: This section will detail the methodology and related research protocols undertaken in conducting the evaluation, data collection, analysis, selection criteria/sampling, and related constraints or limitations encountered during the project implementation and evaluation.  Findings: This section will present findings from the evaluation (empirical facts collected during the evaluation). The evaluation findings will be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions. The evaluation findings must assess key outcomes and impacts as structured around the organizational framework of the evaluation questions. The findings will be specific, concise, and supported by strong quantitative and qualitative evidence analyzed through scientifically plausible methodologies. Sources of information used in arriving at the findings must be properly acknowledged and listed in an annex.  Conclusions (interpretations and judgments based on the findings): Evaluation conclusions will be presented for each key finding. The conclusions will logically follow from the gathered data and findings. These will be explicitly justified. If and when necessary, the evaluator will state his/her assumptions, judgments, and value premises so that readers can better understand and assess them.  Recommendations (proposed actions for management): This section will precisely and clearly present recommendations that must be drawn from specific findings. The recommendations will be stated in an action-oriented fashion and must be practical, specific, and with defined responsibility for the requisite action. The recommendations presented in this section will follow the evaluation questions as the organizational framework.  References: This section will include all documents reviewed, including background documentation and records of technical data application and decision-making.  Annexes: These will include the following: o Evaluation SOW (updated, not the original, if there were any modifications) o Evaluation methods o All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides o All sources of information or data identified and listed o Statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team, if applicable o Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of or describing existing conflicts of interest o Summary information about evaluation team members, including qualifications, experience, and role on the team 6. Submission of Dataset(s) to the Development Data Library Per USAID’s Open Data policy (see ADS 579, USAID Development Data) METAL will submit to the COR and the Development Data Library (DDL), at www.usaid.gov/data, in a machine-readable, non-proprietary 36 format, a copy of any dataset created or obtained in performance of this award, if applicable. The dataset should be organized and documented for use by those not fully familiar with the intervention or evaluation. 7. Submission of Final Evaluation Report to the Development Experience Clearinghouse Per USAID policy (ADS 201.3.5.18) METAL will submit the evaluation final report and its summary or summaries to the Development Experience Clearinghouse (DEC) within three months of final approval by USAID. EVALUATION TEAM COMPOSITION The team should consist of a team leader, a technical specialist, and local evaluation support staff. The team will possess the skills and experiences below: Team Leader Preferred Knowledge, Skills, and Abilities:  Minimum 10 years’ experience in designing and/or conducting evaluations of international development projects and leading evaluation teams  Minimum five years’ experience in capacity development of government and/or public sector agencies as a civil servant/employee or and external advisor/consultant  Prior experience as an evaluation team leader is desirable  Demonstrated skills in qualitative data analysis  Experience managing performance evaluations of similar size for USAID activities in Middle East/North Africa region is desirable  Knowledge of the public financial management reform sector  Knowledge of USAID Evaluation Policy and evaluation reporting requirements  Excellent communication skills, both verbal and written, are required  Demonstrated organizational skills and attention to detail; ability to work independently to meet deadlines and adhere to high-quality standards  Strong professional written and verbal proficiency in the English language; French or Arabic language knowledge is desirable  Ability and willingness to travel to Tunis for fieldwork Minimum Requirements:  Minimum 10 years’ experience in designing and/or conducting evaluations of international development projects and leading evaluation teams  Minimum five years’ experience in capacity development of government and/or public sector agencies as a civil servant/employee or and external advisor/consultant Education:  Master’s degree in finance, political science, international relations, social sciences, economics, or any other relevant field Evaluation Specialist Preferred Knowledge, Skills, and Abilities  Experience in performance evaluations for USAID activities in Middle East/North Africa region  Knowledge of the public financial management reform sector  Demonstrated organizational skills and attention to detail; ability to work independently to meet deadlines and adhere to high quality standards  Strong professional written and verbal proficiency in the English language; French and/or Arabic language knowledge is required Minimum Requirements:  Minimum 8 years of experience in designing and/or conducting evaluations of international development projects.  Demonstrated skills in qualitative data analysis  Knowledge of USAID Evaluation Policy and evaluation reporting requirements  Excellent communication skills, both verbal and written, are required Education: 37  Master’s degree in degree in finance, political science, international relations, social sciences, economics or any other relevant field. EVALUATION SCHEDULE The evaluation will start on August 5, 2019. Estimated level of effort (LOE) in days by activity for a team of two Tasks Team Lead Specialist 1) Preparation – In-briefings, understanding assignment and documentation, Desk Review, Tool Instruments development, and Inception Report 10 5 2) Field Work – Data Collection in DC 18 18 3) Data Analysis including oral debriefing, presentation, and preparations, and submission to USAID for review and comments 10 10 4) Draft Report to USAID for review and comments 10 2 5) Final Draft Report to USAID 5 Total LOE 53 35 CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure quality:  Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity.  Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly.  The Executive Summary should present a concise and accurate statement of the most critical elements of the report.  Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID.  Evaluation methodology should be explained in detail and sources of information or data properly identified.  Limitations to the evaluation should be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings should be presented as analyzed facts, evidence, and data, and not based on anecdotes, hearsay, or simply the compilation of people’s opinions.  Conclusions should be specific, concise, and include an assessment of quality and strength of evidence to support them supported by strong quantitative and/or qualitative evidence.  If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females.  If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. See ADS 201mah, USAID Evaluation Report Requirements and the Evaluation Report Checklist and Review Template from the Evaluation Toolkit. 38 ANNEX III: DATA COLLECTION INSTRUMENTS Key Informant Interviews (KIIs) – GUIDE For Internal and External Stakeholders Interviewee’s name and designation: Tel.: E-mail: Interviewer: Date of interview: Location: Time interview started: Means of recording: a. Note taking b. Voice recording Time interview ended: ___________________________________________________________________ USAID/Tunisia has requested Monitoring and Evaluation for Tunisia and Libya (METAL) to conduct a midterm performance evaluation of the Fiscal Reform for a Strong Tunisia (FIRST) activity. The evaluation purpose is to identify the extent to which FIRST is achieving intended results, as well as to assess the relevance, effectiveness, and efficiency of the approaches used to achieve USAID/Tunisia objectives. The evaluation will identify successes, best practices, lessons learned, and challenges (internal and external factors) that affected the implementation of the activity. The evaluation will also help identify potential course corrections before FIRST ends in August 2020. The final report will also propose recommendations based on the findings to help inform USAID/Tunisia programming strategy and decision-making. The primary audience for this evaluation is USAID/Tunisia, particularly the Economic Growth Office. The evaluation team is Denis Dunn and Dr. Mehdi Ben Braham, and we are carrying out data collection in Tunis through Key Informant Interviews and Focus Groups. We have questions (and follow-up questions as needed) to pose to you and very much appreciate your frank and candid responses. Note our findings will be disaggregated by gender, age, geographical location, and institution where feasible. Ask the following questions (and follow-up questions as needed) to probe: 1. Please tell us briefly about your role and the type of involvement that you had with FIRST? 2. To what extent was the designed activity (FIRST) approach relevant to the Tunisian Government needs? EQ 1: [EQ focus is relevance – How relevant is the activity to problems and priorities of stakeholders and beneficiaries? And references intervention logic, activity plan/design, developmental hypothesis, TOC.] FIRST TOC and goal/objectives. The development hypothesis or TOC for FIRST activity is that if the government can maintain macroeconomic stability while achieving greater simplicity, equity, and efficiency in the tax system, the private sector will respond by increasing investment, expanding business activity, and hiring more workers. Business, investment, and job growth, in turn, will broaden the tax base, bolster the public’s confidence in government, and over time create the fiscal space for both lower taxes and increases in public investment and services. A longer-term “governance dividend” can also be expected from this activity, insofar as reforms and improvements in fiscal policy and administration help to legitimize the state, increase political stability, and enhance social cohesion. The FIRST activity goal is stronger fiscal foundation for sustainable and inclusive growth with the aim to improve the conditions for business, investment, and job creation. FIRST activity contributes to four high-level objectives. Objective 1 is more efficient and transparent tax administration (modern), O2 is enhanced MOF capacity to develop and manage tax policy (that are sound, predictable and fair), O3 is enhanced capacity to address other fiscal reform priorities as they emerge (improve targeting of social spending and increase the fiscal space for public investments and service delivery), O4 is improved communications, engagement, and consultation on priority reforms, and cross-cutting is gender review to maximize opportunities for women in tax administration/policy reform. 39 The activity objectives are mutually reinforcing. Modernizing tax administration will help close the compliance gap through incentive-based systems that encourage taxpayers to pay their fair share of taxes for public services, and lead to greater revenue collection within existing laws and regulations. By enhancing the capacity of the MOF to undertake fiscal policy analysis, decision makers will be provided with tools to assess the potential impacts of revenue raising policies that can also help close the fiscal policy gap. By improving internal and external communications and fostering stakeholder engagement, the MOF will be able to increase taxpayer willingness to pay. Thus, all activity components converge toward the same goal: improve tax revenues in a sustainable and equitable way by closing the tax gap. Achieving the above objectives will also help Tunisia reduce the fiscal deficit to a level that is sustainable, and consistent with safe levels of public and external debt. FIRST Workstreams. See Year 3 Work Plan. Gantt chart (activity schedule) and Year 3 Work Plan Sustainability Plan for sub-objectives below: A.1 Institutional reform/organizational development, change management (PMS, RBM, BPI) 1.1 Create more streamlined, transparent, and predictable tax administration through UXP data exchange (modify legal framework, train in fiscal information, and apply to tax gaps, tax expenditure, risk analysis, FATCA?) 1.2 Structure and services enabling easier citizen and business compliance (TADAT diagnostic, fiscal civism compliance strategy, policy on forced recovery, taxpayer registry, risk analysis system, DME piloting, e-filing) 1.3 Effective audit system implemented (more frequent and targeted audits, change fiscal procedures manual, train DGI, review collection policy) 2.1 Policy analysis produced independently (microsimulation models – e.g., VAT, PIT and CIT, model training, tax expenditure report, macro MTFF, CGE model, CD FAU) 2.2 Policy analysis used by decision makers (CD MOF to present and defend policy options) 3.1 Improve internal communications (webcasting intranet link Tunis to regions, videoconferencing, Strategic Communications Unit) 3.2 Improve external communications for enhanced transparency of MOF toward taxpayers (taxpayer surveys, media monitoring, traditional media and digital/social media campaigns, hotline/call center, train customer services, website, SME Roundtable) 3. Is the approach (of FIRST) still relevant? Should the approach remain the same, or should there be a new approach or modified approach, for Year 3? EQ 2a: [EQ focus is effectiveness – how well did the results (outputs) contribute toward achieving objectives? Results]. WS2 and WS3 4. To what extent did FIRST contribute to improving revenue collection? Indicator(s) KPI-2, as well as 4, 5, and 8 Expected Results • There are more streamlined, transparent, and predictable tax administration processes. • Risk-based approaches to audit, collections, and other aspects of compliance management have been institutionalized (e.g., fewer but more productive tax audits). • The public has increased access to information, assistance, and services and better understands their rights and obligations that facilitate voluntary compliance. 5. How did FIRST contribute to improving revenue collection? Objective 1: Improve the efficiency, transparency, and cost of compliance of tax administration. Transforming the DGI into an efficient and service-oriented organization will require fundamental changes in the way it operates and interacts with its customers. In addition to ineffective laws and low levels of taxpayer awareness, there are a number of barriers to reform, including an organization structure built around 27 regional silos, the separation of tax functions between DGI and DGCPR, and disjointed systems 40 and procedures that prevent the authorities from using the vast information at their disposal to support compliance management. Moreover, DGI leadership has articulated a vision of change that has not yet been effectively communicated or broadly accepted by staff. The continued reliance on audit (“verification”) as the sole means of improving compliance causes costly disruptions for businesses while producing little additional revenue for the government. The contractor will support the DGI to enhance its ability to effectively, efficiently, and transparently enforce revenue and tax policies and laws and carry out its revenue collection mandate. Improving DGI performance will require extensive capacity-building, change management, and close coordination with other MOF units, including DGCPR, DGELF, and the MOF’s data processing center (CIMF). The contractor will pay special attention to using technology and electronic services to facilitate core tax administration processes, with a view to reducing the costs of administration and the costs and burdens of compliance for taxpayers of all sizes. The contractor will address needed reforms to the structure of the tax administration, including efforts to more closely align audit (“verification”) and collection functions, to introduce the full suite of compliance functions within the Large Taxpayer Office, and to stand up a dedicated Medium Taxpayer Office. The contractor will also support efforts to strengthen coordination among different units within the DGI, between the tax administration and tax policy units (e.g., DGELF), and between DGI and the General Directorate of Customs (DGD). Related areas, such as customs modernization and trade facilitation, may also be supported, provided there is a sustained demonstration of commitment and engagement among government and private sector stakeholders. Illustrative Interventions – may include but are not limited to the following: ● Support MOF to develop and operationalize a modern organization structure for tax administration. ● Support MOF review of the existing legislative regime to identify, draft, and promulgate laws that support effective tax administration. ● Support efforts to reengineer business processes and systems to better manage risk, foster a customer service orientation, and encourage voluntary compliance. ● Support the development and implementation of information technology78 that effectively supports workflows, facilitates taxpayer compliance, supports risk identification, generates management information, and enhances internal control. ● Support planning and rollout of “e-services” to be managed and maintained by the GOT with an aim to reduce compliance burdens, improve revenue collection, and ensure implementation and sustainability of these services. ● Assist efforts to increase registration rates, particularly within the informal and small/medium size enterprise sectors. ● Deepen DGI capacity to implement a robust, risk-based tax audit program, including institutionalization of the DGI’s audit criteria selection system (ACSS). ● Support the DGI to implement and institutionalize annual updates of its Taxpayer Compliance Improvement Plan (TCIP). ● Support the development and implementation of external communications strategies. ● Assist the MOF to develop and roll out competency-based tax administration training, including in taxpayer services, audit, collections, IT, and management. ● If directed by USAID, assist DGD and other GOT institutions to streamline and standardize border clearance procedures, strengthen private sector consultation, and address barriers in the import/export supply chain. 78 According to IP contract, $US 1 million set aside for ‘commodities support’ i.e., computer hardware and software. 41  1.1 Create more streamlined, transparent, and predictable tax administration through UXP data exchange (modify legal framework, train in fiscal information, and apply to tax gaps, inefficient tax expenditure, risk analysis, FATCA?)  GOT only collects half of potential tax revenues.  Use TADAT diagnostic tool to identify constraints and priorities for TA and CD,  Broaden tax base [increase registered taxpayers].  UXP platform is standardized, and there is a predictable communications channel for data exchange between government agencies (strict data security and confidentiality).  UXP will enable tax administrators to better evaluate sources of tax gaps and raise revenues.  Shifting to VAT will speed up tax refunds.  Add DGI CD and substantial IT architecture upgrade (CIMF).  Implement e-services (test pilot DME structure in Tunis).  Aim for budget self-sufficiency.  Provide in-service technical training for CS to reduce “leakage” in PFM (anti-corruption).  Conduct post-training competencies surveys EQ 2b: [EQ focus is effectiveness, results}. WS4 and WS5. 6. To what extent did FIRST reduce taxpayers’ compliance costs? Indicators KP1-1 as well as 6 and 7 Expected Results ● Better compliance by taxpayers (e.g., net increase in registered taxpayers, higher on-time filing rates) ● Reduced wait times for VAT and other tax refunds ● Reduced costs and burdens of taxpayer compliance 7. How did FIRST reduce taxpayers’ compliance costs?  1.2 Structure and services enable easier citizen and business compliance (TADAT diagnostic, fiscal civism compliance strategy, policy on forced recovery, improved taxpayer services, taxpayer registry, risk analysis system, DME piloting, e-filing).  DGI’s tax compliance strategy is to “make it easier for taxpayers to file and pay taxes in timely manner,” and includes a compliance improvement plan to address gaps.  Update taxpayer registry to increase number of registered taxpayers, and offer online registration.  De-link tax declaration and payment.  Reduce tax system complexity and ease tax burden on low-income groups (revise PIT regime).  Reengineer or streamline systems (BPI) – modify tax administration process (workflows, rules/regulations, procedures, guides).  Inform the public (successful outreach) to facilitate voluntary compliance.  1.3 Effective audit system implemented – DGI continues to rely on verification as sole means to improve compliance, but this causes disruptions for businesses while little additional review emerges.  As targeting improves, hold fewer (but more productive) audits (based on risks, change fiscal procedures manual, train DGI, review collection policy).  Avoid protracted audits (new risk-based audit selection system), and have better targeted tax audits.  Review enforced collections legislations, especially penalties on non-filers.  Focus on fiscal civism (voluntary compliance) rather than punitive (recovery of bad debts from bad taxpayers. EQ 2c: [effectiveness, results] WS6 8. To what extent did FIRST improve the budgeting and expenditure process? Indicator[s] 8, 9, 10, 11, and for Objectives 3–13, 14 Expected Results (tax policy) ● The MOF has more sustainable, competency-based capacity to develop, manage, and analyze revenue policy, legislation, and regulations. ● Increased MOF capacity to plan, coordinate, and implement policy reforms. ● Increased MOF capacity to communicate reforms and conduct political and public consultation on changes to policies, laws, and regulations. 42 Expected Results (other reforms) ● Improved skills and capacity to produce analysis and options that help the MOF make informed policy decisions. ● Development and delivery of economic policy options and proposals that the MOF can use to promote policy changes to achieve sustained savings in budget resources and more and better targeting of public investment and social spending. 9. How did FIRST improve the budgeting and expenditure process? Objective 2: Enhance the capacity to develop and manage tax policy. The purpose of this objective is to improve the capacity of the GOT to create predictable, transparent, and fair revenue policies that are broad-based, business-friendly, and conducive to improved job creation. The contractor shall focus activities toward building and sustaining MOF capacity, particularly in the DGELF’s Fiscal Analysis Unit (FAU), to generate high-quality, substantive tax analysis and studies to steer the adoption of reforms and help the GOT gain support for new tax measures from key Tunisian stakeholders, including the parliament, the private sector, and civil society. Emphasis shall be placed on cultivating sustainable local capacity in revenue forecasting, tax impact analysis, and strategic communications, including communications and outreach to garner support for priority policy reforms. The contractor shall ensure that its advisors work side by side with the FAU to respond to MOF requests, providing technical assistance, guidance, and instruction in advanced modeling and forecasting techniques to ensure high-quality FAU deliverables. The tools and products resulting from these shall be used to strengthen the GOT’s campaign for tax reform. As appropriate the contractor may also work with the General Directorate of Fiscal and Financial Incentives (DGAFF) and other MOF units influencing tax system design. Illustrative Interventions – may include but are not limited to the following: ● Develop an effective plan for further recruitment and training of FAU professional staff. ● Support analysis, consultation, and advocacy toward advancement of policy reform priorities. ● Assist with drafting policy, legal, and regulatory reforms. ● Support the development of policy briefs, informational guides, and other user-friendly materials to guide political and public consultation and education on tax issues. ● Support the quantification and reporting of tax expenditures (i.e., revenues foregone due to exemptions, exonerations, and other tax relief). ● Assist the decentralization transition, including planning and analysis related to revenue sharing and local government own-source revenue. ● Strengthen the capacity of the MOF’s communications team to implement ongoing strategic communications and public outreach. Objective 3: Enhance the capacity to address other fiscal policy and management priorities as they emerge. The purpose of this objective is to strategically assist the MOF to use targeted analysis to advance key policy reforms that enhance efficiencies in government administration, improve targeting of social spending, and increase fiscal space for priority investment and service delivery. The contractor will support the MOF’s cadre of analysts to undertake research and provide robust and data￾driven analysis of public policy issues that the Minister of Finance will need to present to the Council of Ministers, Parliament, key stakeholders, or in other policy forums. Assistance under this objective will strengthen MOF capacity to conduct economic policy analysis, market ideas effectively, and plan and oversee the implementation of reforms. Assistance under this objective will be limited to support for research and analysis and development of policy options; it will not involve implementation activities. 43 The contractor will selectively support the MOF to study and assess options for addressing issues that pose threats to fiscal sustainability as well as emergent priorities of strategic importance to Tunisia’s ongoing transition. Illustrative areas of focus may include but are not limited to the following: ● Assessment and options for containing the public sector wage bill ● Proposals for subsidy reform, and reform of public enterprise and the pension system ● Review of the “Special Treasury Funds” and the taxes that fund them ● Identification of other resource-saving reforms in selected government programs. ● Assessment and options for implementation of intergovernmental fiscal arrangements to support Tunisia’s decentralization vision ● Assessment and modernization of the present labor law regime ● Trade and customs reforms and modernization.  2.1 Policy analysis is produced independently – enhanced MOF capacity to plan, manage analyze, coordinate, and consult on changes to tax policies, laws, and regulations (simple/fair/transparent regulations and laws)  Develop and institutionalize micro and macro simulation models.  Conduct tax gap analysis on VAT, PIT (professionals’ low tax), and CIT; regime forfeiture (flat tax) 8 percent.  Create microsimulation models, e.g., VAT, PIT and CIT, model training, CGE (general equilibrium) model at DGRE, CD FAU.  Develop macro model MTFF for high-quality analysis on tax options and potential impacts on revenues.  Expand tax gap modeling to excise taxes and customs duties (post clearance audit system).  Provide formal and informal training (and track change in competencies) FAU and DGI in methodologies/modeling for revenue forecasting and for tax impacts.  Provide TA to assist analysis of benefit of tax expenditures and phase out inefficient tax expenditures.  Conduct a study tour in Morocco and France – why not?  Enhance the ability to forecast and formulate tax policy and annual budget law to improve budget credibility.  Use FAU to better identify targets for future policy reforms – ‘knowledge products’ (publications and policy advice).  Conduct CD for studies on treasury funds, energy subsidies, tobacco taxes, public sector wage bill, pension system, SOE subsidies, and fiscal decentralization to rationalize payroll taxes and close the fiscal deficit.  2.2 Policy analysis used by decision makers – CD MOF to present (via communications) and defend policy options (via consultations). EQ 2d: [Effectiveness, result here is strategic communications, outreach/engagement, collaboration] 10. To what extent did FIRST enhance public accountability? Indictor[s] 15, 16, 17 Expected Results ● Expanded and improved opportunities for the MOF, private sector, and civil society to engage and consult on tax and broader fiscal issues. ● Improved MOF capacity to engage with the public on tax and broader fiscal issues. ● Increased MOF capacity to utilize cost-effective tools to facilitate two-way communication with the public on tax and broader fiscal issues. ● Increased MOF capacity to communicate internally. ● Increased public awareness of tax and broader fiscal reforms. ● Improved public trust in the government. 11. How did FIRST enhance public accountability Objective 4: Improve communications, engagement, and consultation on priority reforms. 44 The purpose of this objective is to support the MOF to strengthen communications with policymakers and the general public, particularly on strategies and measures that the MOF shall be proposing and implementing to reform the tax regime and address broader fiscal concerns. To supplement (and lead) the respective MOF directorates in this communication effort, MOF leadership will need to adapt and retool for more effective consultation and outreach than in the past in order to forge a more successful consensus among stakeholders on important reforms. Key stakeholders include not only parliamentarians and civil society organizations, but also groups representing vested institutional interests, such as the Union of Industry, Trade, and Handicrafts (UTICA) and the Tunisia General Labor Union (UGTT). The contractor will collaborate with the MOF to improve engagement, expand the dialogue, and broaden the reach and effectiveness of public outreach and consultation. Strategic communications and outreach to diverse segments of the Tunisian population will help to inform the public debate and address perceptions about sensitive issues such as the tax system, the government’s role in the economy, and the efficacy of GOT policies and programs. Illustrative Interventions – may include but are not limited to the following: ● Support the development and institutionalization of forums and platforms for the MOF to engage and consult on tax and broader public policy issues. ● Improve the institutional capacity of the MOF to effectively communicate and consult the public on tax and broader public policy issues. ● Improve MOF capacity to conduct internal communications to facilitate and manage organizational change efforts. ● Support the development of cost-effective content and tools (e.g., social media, television, radio) to enhance and extend the reach of public communications EQ 3: [Efficiency in implementation – How were inputs and activities converted into results (outputs or deliverables)?] – should come before EQ 2 12. What were the internal (organizational, resources, etc.) and external factors that affected the efficiency and timeliness of FIRST implementation? 13. How did these factors affect the efficiency and timeliness of implementation? EQ 4 [stakeholders’ analysis – linked to EQ 3 implementation] 14. To what extent has FIRST succeeded in identifying, partnering, and coordinating with various stakeholders? The IP expected under contract to collaborate not only with specific units within Tunisian MOF that focus on tax policy and administration, but also members of Tunisian Parliament, private sector, civil society, donors, and multilaterals (EU, IMF, World Bank), as well as USG agencies Department State, Department Treasury, and US Customs and Border Protection. 15. What other partnerships will strengthen FIRST and similar USAID/Tunisia activities? EQ 5: [long-term impact] 16. To what extent are FIRST activities sustainable and contributing to Tunisia Government effectiveness (as per the J2SR 4.GC.1)? Sustainability is integral to the design of this activity. Tunisia FIRST is fundamentally about strengthening the fiscal foundations for sustainable and inclusive growth. With this in mind, reforms supported by FIRST must be sufficiently robust, and command a sufficiently broad constituency, to hold up through changes in administration and future political transitions. The contractor’s approach to implementation must also be sufficiently flexible to adapt to changes in priorities or leadership within key institutions. For example, contractor efforts to facilitate the GOT rollout of web-based and other “e-services” should be supported 45 only if there is a strong likelihood that the Tunisians can effectively implement and sustain these services independent of donors. Above all, the emphasis of this activity must be on building sustainable local capacity—to develop and manage tax policy, administer taxes efficiently, and generate and deliver evidence-based analysis and recommendations to support home-grown reforms that support and facilitate business, investment, and employment growth in Tunisia. At all times, the contractor should exercise flexibility, embrace facilitation, foster local accountability, and cultivate capacity and demand for monitoring and evaluation (M&E) as a tool for enhancing performance, monitoring progress, and ensuring sustainability. Learning from the lessons of previous fiscal reform activities, FIRST will also ensure that systematic and sustained engagement of all stakeholders spanning the public sector, private sector, and civil society to ensure that transparent, fair, and predictable policies, systems, and practices are institutionalized and sustained.  A.1 Institutional reform/organizational development, change management, change management (PMS, KPIs, RBM,  Refer to Sustainability Plan – ownership, buy-in  CD tools, methodologies, manuals, systems/IT upgrade, checklist, etc. [WS1] 17. How was FIRST concerned with gender equality (i.e., opportunities for advancing women in the context of tax/policy reform)? Indicator 18 Women account for less than one-third of Tunisia’s workforce, but social and cultural norms still restrict women’s access to economic opportunity. Nevertheless, women can be found in all areas of trade and commerce, and they represent an important, under-utilized segment of the working age population. The contractor will systematically consider the gender aspects and implications of specific interventions, including opportunities to reduce gender-related biases or risks embedded in tax and revenue policies, systems, and procedures. Policy scenarios for reform of the VAT, for instance, must take into account the different effects of exemptions and differential rates on men and women as well as rich and poor. Moreover, proposal options to reform subsidies, or for that matter the pension system, must be complemented by proposals and options to better target transfers to meet the needs of women and other underserved groups. As part of the annual work plan, the contractor should include a gender analysis that identifies ways to mitigate gender issues affecting this activity. Further, the contractor shall integrate gender considerations into all planned deliverables. The contractor will also seek opportunities to promote the equitable professional development of women in assisted GOT institutions. To that end, all data collected for monitoring and evaluation purposes shall be sex-disaggregated and surveys and other monitoring and evaluation tools shall include questions to elicit information that allows differentiation of impacts based on gender. 18. In light of what worked well and not so well in the FIRST activity, please tell us about the lessons learned from this project. 19. How can FIRST implementation be improved in Year 3? 20. Do you have any suggestions to USAID for future similar programming? 21. Do you have any other comments on FIRST? 46 FOCUS GROUP DISCUSSIONS [FGDs] - GUIDE - Names of participants and contact info See Sign-In Sheet Location of FGD FGD # Date Time FGD started: Time FGD ended: Facilitator: Other ET member: Note-taker: Recording: Y/N Ask them to sign Sign-In Sheet USAID/Tunisia has requested Monitoring and Evaluation for Tunisia and Libya (METAL) to conduct a Midterm performance evaluation of the Fiscal Reform for a Strong Tunisia (FIRST) activity. The evaluation purpose is to identify the extent to which FIRST is achieving intended results, as well as to assess the relevance, effectiveness, and efficiency of the approaches used to achieve USAID/Tunisia objectives. The evaluation will identify successes, best practices, lessons learned, and challenges (internal and external factors) that affected the implementation of the activity. The evaluation will also help identify potential course corrections before FIRST ends in August 2020. The final report will also propose recommendations based on the findings to help inform USAID/Tunisia programming strategy and decision-making. The primary audience for this evaluation is USAID/Tunisia, particularly the Economic Growth Office. The evaluation team is Denis Dunn and Dr. Mehdi Ben Braham, and we are carrying out data collection in Tunis through Key Informant Interviews and Focus Groups. We have questions to pose to you and very much appreciate your frank and candid responses. Our findings will be disaggregated by gender, age, geographical location, and institution where feasible. Ensure no one dominates the discussions and all participants have an opportunity to speak. ET QUESTIONS – Focus Groups QUESTION TIME 1 Introduction and involvement with FIRST? 10 2 What is FIRST approach for capacity development of MOF? Does it address all/most pressing fiscal reform issues? 15 3 What are the main achievements of FIRST so far (i.e., completed deliverables)? Which outputs are ongoing (i.e., expected) in Year 3? 55 4 Are there/were there any challenges with project implementation? 15 5 To carry out activities may require collaboration with people outside MOF – who were/are the project partners? 15 6 Regarding the long-term effect of FIRST interventions, what needs to be done/completed in Year 3 to ensure sustainability of results? 20 7 Are there any efforts by FIRST to consider the gender aspects at MOF or within the tax system? 5 8 Can you deduce any lessons learned from FIRST project? 10 9 Do you have any suggestion for improving FIRST in Year 3? 10 10 Do you have any suggestions for future USAID programming in PFM? 5 47 ANNEX IV: DATA COLLECTION AND ANALYSIS PLAN The table below provides data sources, data collection methods, and analysis approaches for each evaluation question. EQ 1: To what extent was the designed activity approach relevant to the Tunisian Government needs? Is the approach still relevant? Sub-question Data Collection Sampling Approach Data Analysis Method Source Tool Method 1.1 To what extent was the designed activity approach relevant to the Tunisian Government needs? Stakeholders79 Activity Beneficiaries80 Project and Other Relevant Documents KII Guide FG Guide In-person Interviews (KIIs) Focus Groups Desk research Non￾probability Sample Descriptive and content analysis and triangulation of findings 1.2 Is the approach still relevant? Stakeholders Activity Beneficiaries Project and Other Relevant Documents KII Guide FG Guide In-person Interviews (KIIs) Focus Groups Desk research Non￾probability Sample Descriptive and content analysis and triangulation of findings 79 Internal stakeholders are top officials/counterparts of FIRST at MOF, FIRST management; external stakeholders are international donors and multilaterals, private sector representatives, labor union, etc. 80 Beneficiaries are government staff receiving direct benefits from FIRST activities through training, mentoring, tools, systems upgrade, manuals, methodologies, checklists, etc. 48 EQ 2: To what extent, and how did FIRST contribute to improving revenue collection, reducing taxpayers’ compliance costs, improving the budgeting and expenditure process, and enhancing public accountability? Sub-question Data Collection Sampling Approach Data Analysis Method Source Tool Method 2.1 To what extent and how did FIRST contribute to improving revenue collection? Document review Stakeholders Activity Beneficiaries M&E Plans, DQAs Baseline and progress tracking data KII Guide FG Guide Desk research In-Person Interviews Focus Groups Descriptive and content analysis 2.2. To what extent and how did FIRST contribute to reducing taxpayers’ compliance costs? Document review Stakeholders Activity Beneficiaries M&E Plans, DQAs Baseline and progress tracking data KII Guide FG Guide Desk research In-Person Interviews Focus Groups Descriptive and content analysis 2.3 To what extent and how did FIRST contribute to improving the budgeting and expenditure process? Document review Stakeholders Activity Beneficiaries M&E Plans, DQAs Baseline and progress tracking data KII Guide FG Guide Desk research In-Person Interviews Focus Groups Descriptive and content analysis 2.4 To what extent and how FIRST contribute to enhancing public accountability? Document review Stakeholders Activity Beneficiaries M&E Plans, DQAs Baseline and progress tracking data KII Guide FG Guide Desk research In-Person Interviews Focus Groups Descriptive and content analysis 49 EQ 3: What were the internal (organizational, resources, etc.) and external factors that affected the efficiency and timeliness of the implementation, and how? Sub-question Data Collection Sampling Approach Data Analysis Method Source Tool Method No sub-questions. Stakeholders Activity Beneficiaries Project and Other Relevant Documents KII Guide FG Guide In-person Interviews (KIIs) Focus Groups Desk research Descriptive and content analysis and triangulation of findings EQ 4: To what extent has FIRST succeeded in identifying, partnering and coordinating with various stakeholders? What other partnerships will strengthen FIRST and similar USAID/Tunisia activities? Sub-question Data Collection Sampling Approach Data Analysis Method Source Tool Method 4.1 To what extent has FIRST succeeded in identifying, partnering, and coordinating with various stakeholders? Stakeholders Guide KII Descriptive and content analysis 4.2 What other partnerships will strengthen FIRST and similar USAID/Tunisia activities? Stakeholders Guide KII Descriptive and content analysis EQ 5: To what extent are FIRST activities sustainable and contributing to Tunisia Government effectiveness (as per the J2SR 4.GC.1)? Sub-question Data Collection Sampling Approach Data Analysis Method Source Tool Method No sub-questions. Stakeholders Activity Beneficiaries Project and Other Relevant Documents KII Guide FG Guide In-person Interviews (KIIs) Focus Groups Desk research Descriptive and content analysis and triangulation of findings 50 ANNEX V: SOURCES OF INFORMATION Key Informants Interviews (KIIs) Designation Organization Meeting Chief of Party FIRST Sept. 6 Chief of Party FIRST Sept. 9 Tax Administrator TL FIRST Deputy TL, Tax Admin FIRST Sept 9 Deputy TL, Tax Admin FIRST Strategic Communications TL FIRST Sept 9 Strategic Comms. Deputy TL FIRST Tax Policy Team Leader FIRST Deputy Tax Policy TL FIRST Sept. 10 Deputy Tax Policy TL FIRST Operations & Finance Director FIRST Sept. 10 Sr. Procurement Manager FIRST Sr. HR and Logistics Manager FIRST Sept. 10 MEL and Gender Manager FIRST Sept. 10 Chief of Party FIRST Chief of Party FIRST Sept. 11 MEL and Gender Manager FIRST Sept. 11 Strategic Communications TL FIRST Strategic Comms. Deputy TL FIRST Sept. 11 Outreach Specialist FIRST Director General MOF – DGI Sept. 13 Director General MOF – CIMF Sept. 13 Secretary-General MOF Sept. 13 Resident Representative IMF Sept. 16 Chief, Economic Development EU Delegation, Tunis Attaché de Cooperation, Fiscal EU Delegation, Tunis Sept. 16 Attaché de Cooperation BEE EU Delegation, Tunis Sr. Governance Specialist AfDB Sept. 17 Charge de Mission Governance AFD Sept. 18 Coordinator for FIRST MOF Sept. 20 Communications Director MOF Sept. 20 Sr. Specialist, OEG USAID Sept. 23 OEG COP FIRST Sept. 24 Chef de cluster GIZ Sept. 28 51 COP [Skype call] FIRST Sept. 30 Resident Director KfW Oct. 2 Charge de mission General Director SOLIDAR Oct. 3 Coordinator for FIRST MOF Oct. 4 Executive Coordinator IACE Oct. 5 In charge of fiscal issues UTICA [SMEs] Oct. 9 Partner PwC Oct. 10 Sr. Advisor, Bureau Econ. Growth, Washington, DC USAID Dec. 6 Focus Group Discussions [FGDs]: Name and Titles of individuals are withheld under ADS 540 Provision BIBLIOGRAPHY Project Documents 1. Annual Work Plans. Year 1 (26.10.2017), Year 2 (Sept. 2018), and Year 3 (undated). FIRST. 2. Selected monthly reports and quarterly reports for Year 1 and Year 2. FIRST. 3. Annual Report Year 1. FIRST. Sept. 2018. 4. Contract Award for FIRST. USAID. 08.08.2017. 5. Data Quality Assessment Report (DQA) on FIRST. METAL, Tunis. 20.06.2019. 6. Activity Monitoring, Evaluation and Learning Plan (AMELP). FIRST. 13.04.2018. 7. Activity Dashboard. DevResults. FIRST. Sept. 2019. 8. Stakeholder Analysis. FIRST. Sept. 2018. 9. PPT Presentation to COPIL (Steering Committee). FIRST. 25.01.2019. 10. PPT Presentation to COPIL on 23m (Life-Of-Project) Action Plan. FIRST. 31.10.2018. 11. Gender Strategy. FIRST. 15.11.2017. 12. Creating a Communications Unit at the Tunisian MOF. FIRST. June 2019. 13. Select SOWs for FIRST staff. 14. Political Economy Analysis. FIRST 20.11.2017. 15. Political Economy Analysis (updated). FIRST. Nov. 2018. 16. Training Report “Customer Service”. Sample. 27.06.2019. 17. Success Stories, MOF website (access to tax information, mobile app. compliance strategy). 18. MOU sample for UXP data exchange. DGI-REN. FIRST. 19. Tax Compliance Strategy and Action Plan. FIRST. June 2018. Other Documents 52 1. A Transatlantic Strategy for a Democratic Tunisia. F.G. Burwell, A. Hawthorne, K. Mazran, and E. Miller. Atlantic Council. June 2016. 2. Tunisia: Country Fact Sheet on Food and Agriculture Policy Trends. Food & Agriculture Organization of United Nations. FAO. Aug. 2017. 3. Tunisia Country Report. Bertelsman Stiftung’s Transformation Index. BTI 2018. 4. Strengthening EU Support for Tunisia: Joint Communication to the European Parliament and the Council. European Commission, Sept. 2016. 5. Economy of Tunisia, Constitution of Tunisia and History of Tunisia, Wikipedia website. Accessed 05-06.08.2019. 6. Tunisia. 3rd Review Under Extended Fund Facility. International Monetary Fund. IMF Country Report No 18/218. July 2018. 7. Capitalizing on Tunisia’s Transition: The Role of Broad-Based Reform. M. Muasher, M. Pierini, and F. Aliriza. Carnegie Endowment for International Peace. Nov. 2016. 8. Tunisia in 2019: A Pivotal Year? A Commentary. International Crisis Group. Feb. 2019. 9. Tunisia in Struggles with Scourge of Tax Evasion. R. Bouzza. Arab Weekly. 21.04.2019. 10. Tunisia Country Development Cooperation Strategy. USAID/Tunisia. Sept. 2016. 11. Tunisia: Economic Survey. Organization for Economic Cooperation and Development. OECD March 2018. 12. The Challenge of Capacity Development: Working Towards Good Practice. OECD. 2006. 13. Supporting Capacity Development in PFM – A Practitioner’s Guide. 4th High Level Forum on Aid Effectiveness. 29Nov-1\|Dec 2011. Busan, Korea. 14. Support to Capacity Development – Identifying Good Practice in Swedish Development Cooperation. Sida Evaluation Report for the Joint Scandinavian Evaluation of Support to Capacity Development. Sida 2015. 15. Tunisia Country Strategy 2018-23. European Bank for Reconstruction and Development. EBRD. Dec. 2018. 16. Regional Development in Tunisia: the Consequences of Multiple Marginalization. L. Sadiki. Brookings Doha Centre. Policy Briefing. Jan. 2019. 17. World Bank in Tunisia – Tunisia Overview and Strategy. WB website accessed 06.08.2019. 18. CIA Factbook. Tunisia. https://www.cia.gov/library/publications/the-world-factbook/ geos/ts.html 19. Democracy in Tunisia: Façade or Reality? A. Ameur and E. Neale. Atlantic Council. March 2018. 20. Le Projet de loi de finances 2019: Sous les projecteurs de l’equité fiscal support. ALBawsala 2018. 21. Institutional Assessment and Capacity Development: Why, what and how? European Commission. Sept. 2005. 22. Tunisia Country Roadmap. USAID. FY 2019. 23. Adrian Flint & Christian Meyer zu Natrup, “Aid and development by design: local solutions to local problems” Development In Practice, 29:2, 2018. 24. USAID Self-Reliance Metrics. FY2019 Methodology Guide. USAID Sept. 2018. 25. Tunisian Tax and Customs Reform Pilot Activity. Phase 1 Final Report. USAID. May 2016. 53 ANNEX VI: FINDINGS-CONCLUSIONS-RECOMMENDATIONS (F-C-R) MATRIX Key Questions Findings-Conclusions-Recommendations Matrix EQ 1 Key Question Findings Conclusions Recommendations To what extent was the designed activity approach relevant to the Tunisian Govern-ment needs? Is the approach still relevant?  The activity approach was not elaborated in the contract – just demand-driven TA, CD, facilitation, and commodities support.  Neither the IP nor USAID could obtain official written agreement (MOF) to steer Steering Committee and Working Groups.  The FIRST team in Year 1 prepared a work plan that comprised 88 activities, some carryover from TCP.  The FIRST team insisted on a comprehensive diagnostic of the Tunisian fiscal system (TADAT) that was supposed to run in Nov. 2017 leading to three-year overall work plan but instead ran Jan￾May 2018.  TADAT was delayed by MOF apprehension and skepticism, and FIRST struggled to convince the partner of its merits.  There was improved engagement (“dynamic dialogue”) in Year 2 with MOF, and FIRST was “reset” with a focus on IT, applications, and web services.  FIRST encountered difficulties with respect to fully involving the partner in work planning, TA use, and reporting.  FIRST could not push profound reforms due to it being an election year, MOF resistance, etc., but multilaterals can promote reform through budget support conditionalities.  The presence of “agents of change” inside MOF can rapidly move initiatives forward.  As requested, FIRST dropped planned activities (study of pensions, decentralization, etc.) and added new activities (FATCA, Doing Business, etc.).  Although an implementation approach was followed in practice, it was never articulated.  In Year 2, absence of a signed MOU meant no formal project governance structure for oversight and delivery.  TADAT was successful in highlighting the Tunisia tax system deficiencies and informing future interventions.  FIRST focus on TADAT and ensuing delays contributed to negligible results in Year 1.  FIRST demonstrated flexibility and responsiveness in Year 2.  FIRST’s approach was reset in Year 2 in terms of content (IT) and process (CD).  FIRST’s participatory approach in Year 2 had shortcomings.  External stakeholders can be persuaded (by FIRST and USAID) to exert pressure for reform.  MOF champions can fast-track reform initiatives when there is strong commitment.  FIRST should articulate a clear strategy for implementation that reflects CD best practice.  USAID should insist on an implementation strategy in work plan of future activities.  FIRST should be client-led, meaning MOF input into and approval of plans, approval of reports, and involvement in all aspects of delivery.  USAID should sign an MOU with the MOF to provide formal institutional governance structures and protocols for FIRST.  FIRST should remain responsive to MOF bearing in mind that flexibility is limited in final year. 54 EQ 2 EQ 2: To what extent did FIRST contribute to improving revenue collection, reducing taxpayers’ compliance costs, improving the budgeting and expenditure process, and enhancing public account￾ability?  The sub-questions pertain to outcome and impact. Contributions at this level have not been reported because no data management system exists and some of the indicators are not useful.  There is an argument that instead of aiming to increase tax revenues, FIRST should focus on fairness and equity (e.g., tax justice, as Tunisia taxes 31 percent of GDP, which is high for a middle-income country).  FIRST is missing intermediate indicators for its 7 workstreams.  Year 1 results were negligible while Year 2 was much more productive in terms of activities.  However, Year 2 had no concrete achievements – only 3 key results were completed (incl. TADAT) by the end of Year 2.  24 main activities are still ongoing and need to be finalized in Year 3 (a total of 70 activities to be carried out).  Only TADAT has a detailed action plan – Gantt chart is an activity schedule.  FIRST is helping the MOF operationalize its performance management system.  It is impossible to say whether FIRST had any effect on outcomes or impact because (a) there is no data management system in place so results were not reported, (b) it is too early to measure effects, and (c) it is difficult to measure impact due to attribution (causal linkages).  The C4 communications survey of public opinion on fiscal issues will provide feedback on public accountability.  PMS can measure performance of individual or unit, i.e., effect of FIRST interventions.  If all main activities had action plans, then there would be a better understanding of the implementation process, especially what needs to done as the “final mile” by the MOF for adoption.  Milestones can be converted into process indicators for tracking and reporting.  FIRST should establish data management system for tracking and reporting on indicators.  FIRST should revise indicators for relevance and pragmatism as well as MEL Plan.  FIRST should conduct a public survey via SCU for opinion on tax matters.  FIRST should work with the MOF on operationalizing PMS.  FIRST should ensure all activities are completed in Year 3, otherwise a contingency plan is needed.  FIRST should support and persuade the MOF to adopt and utilize (institutionalize) reform initiatives. EQ 2: How did FIRST contribute to improving revenue collection? Workstream 1  TADAT comprehensive diagnostic (complete)  UXP platform installation  Other TA (Doing Business ‘paying taxes’, FATCA support customs post clearance audit, change management/mgt by objectives) Workstream 3  Risk analysis system (targeted audit procedures)  Collection policy and SOPs, and training  DME – Medium Taxpayer Office TA  VAT refunds – SOPs and training  Only 3 main activities (or key results) were fully completed—TADAT, compliance strategy, and tax expenditure report.  There are 24 main activities to be completed in Year 3.  Although a comprehensive best practice tool for assessing Tunisia’s fiscal system, TADAT was not initially embraced by MOF.  UXP, on the other hand, received solid commitment throughout the ministry and was strongly supported at the highest levels. Furthermore, it is widely seen as a valuable source of data for analysis of tax gaps.  Lengthy and traumatic audits without any real budgetary effect are being replaced with a  FIRST should prepare action plans for all ongoing main activities with clear milestones.  FIRST should ensure all activities in its Year 3 work plan Gantt chart are carried out by August 2020. 55 more effective issue-oriented audit strategy supported by rapidly collected payments for back taxes. FIRST efforts will improve the deterrent effects of tax audits, and therefore lead to raising tax revenues. EQ 2: How did FIRST contribute to reducing taxpayers’ compliance costs? Workstream 2  Fiscal civism compliance strategy and action plan (complete)  Fiscal info system (FIS) upgrade - TA  IT solutions – computer hardware and software  Training in customer services for tax collection staff  Training on e-filing for private sector accountants  FIRST efforts under W2 aimed to broaden the tax base by registering taxpayers, upgrading the taxpayer registry, revising the penalty system, and digitalization (e-services) to facilitate filing and payment are all effective for reducing compliance costs and easing burden on taxpayers.  FIRST should prepare action plans for all ongoing main activities.  FIRST should ensure all activities in its Year 3 Work Plan Gantt chart are carried out by August 2020. EQ 2: How did FIRST contribute to improving the budgeting and expenditure process? Workstream 4  Micro-simulation models for VAT, CIP, PIT  Tax incidence analysis and report  Tax expenditure analysis and report (complete) Workstream 5  MTFF – medium-term fiscal framework model  CGE – computable general equilibrium model  Improving understanding of tax gaps, tax incidence, tax expenditures, and specific subsidies, though worthwhile, is unlikely to produce concrete tangible results (increased revenues or improved expenditure process) in the short term.  Improving FAU capacity to quantify policy impacts on revenues and on the broader economy, to better communicate with stakeholders on proposed tax reforms during the budgetary process, and to garner support for reforms is worthwhile but unlikely to improve budgeting and expenditure process in the short term.  FIRST should prepare action plans for all ongoing main activities,  FIRST should ensure all activities in its Year 3 Work Plan Gantt chart are carried out by August 2020. EQ 2: How did FIRST contribute to enhancing public accountability ? Workstream 6  Webcasting (intranet) link Tunis to 8 regions  Strategic Communications Unit strategy  Strengthen coordination DGI and DGCPR Workstream 7  Call Center efficiency  Media campaigns support (de-cashing and tax amnesty)  MOF website redesign  Taxpayer’s Practical Guide Cross-Cutting  Stronger SCU and setting up the intranet will improve internal communications and understanding, and in turn positively support MOF external communications efforts.  The diverse external communications initiatives by MOF supported by FIRST reflect the desire for a more open and transparent government, and aim to get in front of the public discourse and shape public opinion.  FIRST should prepare action plans for all ongoing main activities.  FIRST should ensure all activities in its Year 3 work plan Gantt are carried out by Aug. 2020.  FIRST should assist MOF￾SCU with a public survey to 56  In terms of gender equity (a cross-cutting item) the ET found a gender balance in FIRST staffing, in MOF beneficiaries and in training participants, but not in the STTAs deployed by FIRST Efforts that provide information and assistance to citizens and businesses to better understand their rights and obligations, and to get their feedback on fiscal policy and tax reform initiatives, not only enhance public accountability but also promote fiscal civism.  Very little was done by FIRST to advance opportunities for women in the fiscal domain, despite FIRST having a gender strategy, and AMELP indicator 18 stating that “Policies/regulations should have a gender review.” assess taxpayer opinion of tax system.  FIRST should systematically consider the gender aspects and implications of specific interventions, and in Year 3 work plan include a gender analysis that identifies ways to mitigate gender issues affecting this activity. EQ 3 EQ 3: What were the internal (organiza￾tional, resources, etc.) and external factors that affected the efficiency and timeliness of the implemen￾tation and how?  Internal factors include the design of FIRST, TADAT comprehensive assessment, work plans, office space, activity staffing, inputs (TA, tools, technology, training), flexibility, MEL and reporting.  External factors include the election, international organizations, MOF champions, MOF staffing, MOF “final mile,” MOF “appetite,” MOF project framework, MOF traditions, PFM complexity, activity governance, vested interests, and young talent.  FIRST intervention logic/TOC/development hypothesis is sound, but having only three years meant FIRST had to go for shorter￾term measures (“low-hanging fruit”) that is less impactful.  Smarter engagement in Year 2 meant movement on progressing main activities but at the end of Year 2 most are ongoing.  Activity runs for three year (minus startup and close out, as well slow progress in Year 1), so there is not much time for real delivery. There is concern that in Year 3 some main activities might not be completed by FIRST and/or adopted by MOF (institutionalized).  Pressure for results means FIRST was inclined to bypass the partner in planning, use of TA, and reporting to speed up delivery but this dilutes commitment, ownership, and sustainability.  There were many factors that facilitated and impeded the efficiency and timeliness of activity implementation.  Internal factors are found within the FIRST activity itself, and therefore are fully within its control, while external factors are outside the activity and therefore outside its control.  The FIRST activity duration is not long enough for meaningful reform.  Year 2 was more productive in terms of progressing activities but still with little concrete accomplishments (except TADAT).  There is concern that some activities will not be complete by the end of Year 3, and that the MOF will not fully adopt them (sustainability).  Following a participatory approach is good practice but slows the process of implementation – likewise bypassing the partner (STTAs) speeds up delivery but is not good practice.  FIRST should include its successful tactics (for overcoming MOF constraints) in its revised implementation strategy.  The MOF should make available suitable office space for FIRST inside MOF.  FIRST should incentivize its staff to stay until end of project.  USAID should lengthen the duration of PFM-CD activities in future programming.  FIRST should revise its MEL system.  FIRST should determine precisely what is required in initiatives for MOF to adopt/institutionalize.  FIRST together with USAID should open a dialogue with 57  FIRST offices are excellent for administrative functioning but not for technical—FIRST in-house experts operate remotely and this means less effective communication and coordination, and less opportunity for joint working, to “supervise” MOF staff and push reform, and awareness of issues and concerns.  The departure of the COP and two TLs after Year 1 disrupted smooth operations and the new leadership had to play “catch-up” in Year 2.  FIRST can rapidly mobilize STTAs and the temptation is to by￾pass the partner in SOW, CV selection, quality check, and approval of deliverables, which again diminishes ownership.  FIRST procurement of computer hardware and software is relatively fast but installation/testing is slow at MOF due to concerns for legislation, technical issues (web services), data control, security, etc.  FIRST is flexible (dropped items unwanted by MOF) and responsive (added new emerging priorities) as per contract design.  There was no MEL data system so there was no reporting on indicators; management could not use this tool to track achievements and take necessary corrective actions.  During an election year there is no political will for major reform so FIRST had to go for less impactful “low-hanging fruit”—but the new government in 2020 may have an appetite for structural reforms.  Reforms may hit obstacles within MOF (or even pressure from outside groups and vested interests to block and undermine reform) but FIRST/USAID can convince multilaterals (EU, IMF, and KfW) to include conditionalities in budget support to ensure reforms go forward.  The MOF is a difficult environment for change—leadership is viewed as not being so reform-minded, there are pockets of resistance, antiquated systems, archaic structure silos, and old￾school top-down management. Staff are not dedicated to FIRST and are very busy at budget time.  FIRST was able to overcome constraints through various means such as working with champions of priority items to fast-track  Not being embedded in partner premises diminishes activity efficiency and effectiveness.  FIRST staff turnover, especially management, is disruptive.  MOF to adopt (institutionalize) reform initiatives but the “last mile” is out of FIRST control.  FIIRST proved to be flexible and responsive.  The FIRST MEL system is not functioning.  The outside context can present constraints and opportunities for FIRST (e.g., election).  FIRST can leverage the donor community to support reforms.  The MOF is a very difficult environment for reform so FIRST needs to navigate and mitigate constraints.  FIRST/USAID lacked a signed MOU.  The shift to IT solutions added to technical challenges of PFM reform.  Tunisia “brain drain” is ominous for MOF plans for digitalization and need for IT talent. IMF and KfW on budget support conditionalities in 2010.  MOF should hold a regular COPIL meeting to review FIRST Year 3 work plan and Sustainability Plan  FIRST should coach and support the younger staff at MOF. 58 (agents of change), planning for less MOF work at budget time, focusing on focal points and informal working groups, using “rewards” to incentivize (tablets, study tours), taking incremental “baby steps” to change, introducing “best fit” tools into MOF, supporting PMS for accountability, and setting up competition among counterparts for FIRST resources.  The lack of an MOU meant there was no formal project governance structure and SC/COPIL became irregular and infrequent, so there was less effective coordination and communication with MOF.  The shift to IT solutions in Year 2 increased complexity of interventions (e.g., UXP and upgraded fiscal information system)  The emphasis on IT solutions at MOF to digitalize information and services requires IT talent but young people going to EU for work, which slows down FIRST work (there are two focal point vacancies). 59 EQ 4 EQ 4: To what extent has FIRST succeeded in identifying, partnering, and coordina￾ting with various stake￾holders? What other partner-ships will strengthen FIRST and similar USAID/ Tunisia activities?  According to the IP contract, FIRST is to closely collaborate with MOF, and selectively with other GOT ministries, and also parliamentarians, private sector, civil society, and donor community, and coordinate with other USG agencies.  FIRST has now established effective working relations with MOF counterparts and through joint working with assigned staff. The SC/COPIL has been a good forum to resolve issues.  As for GOT outside MOF, FIRST has interacted with the Ministry of Major Reforms and Ministry of Development, Investment and International Cooperation (MDiCI).  FIRST has good communication with multilaterals EU, IMF, and World Bank but no relations with donor agencies.  FIRST had troublesome relations with subcontractor SOLIDAR but excellent relations with subcontractor PwC.  FIRST outreach to the private sector and civil society comprised engagement with the Academie des Banques et Finances (Tunisian Bankers Association training institute), the Millennium Challenge Corporation (MCC) as well as SOLIDAR Tunisie - a social justice nongovernmental organization on the winning bid for FIRST.  FIRST has limited its partnering to MOF, selected GOT Ministries, multilaterals, and subcontractors.  MOF going well, but FIRST lack of communication with MDiCI (to harmonize the role for TA), with donors (depends on their relations with USAID), and with organizations representing private sector and civil society (e.g., IACE and UTICA) is a missed opportunity.  FIRST should reach out to donors, representatives of private sector, civil society, and unions to build a broader constituency for supporting PFM reforms. EQ 5 EQ 5: To what extent are FIRST activities sustainable and contributing to Tunisia Govern-ment effective-ness (as per the J2SR 4.GC.1)?  FIRST is attempting to deliver 27 main activities or key results, of which 24 are ongoing at end of Year 2, so they must be completed in Year 3.  In addition, MOF needs to go the last mile and fully adopt and institutionalize these reform initiatives.  FIRST inputs (4Ts) are TA, training, tools, and technology.  TA comprises in-house experts and STTAs.  Tools comprise models, methodologies, manuals, user guides, modules, etc.  Training – FIRST delivered 15 different courses for 1,428 persons for a total of 2,348 training days.  Technology spending was US$1.12 million and included 20 pieces of software.  FIRST is advancing improvements in public (taxpayer) services, capacity-building for civil servants, building capability for independent tax policy analysis and policy formulation, enhancing efficiencies in revenue mobilization, and budget management. But credibility of government commitment to policies will depend on the new government in 2020.  It is impossible to say to what extent FIRST is contributing to GOT effectiveness because there has been no measurement of the effects of FIRST contributions on Tunisian government effectiveness.  FIRST should complete all activities in its Year 3 work plan in order to achieve key results.  FIRST should encourage and support MOF in fully adopting reform initiatives (i.e., institutionalization).  FIRST should discuss and agree its Sustainability Plan (in Year 3 work plan with USAID and with MOF), ideally in SC/COPIL forum.  FIRST Sustainability Plan should include MOF activities 60  TA sustainability was enhanced via enshrining interventions in legislation and regulations (e.g., fiscal code for targeted auditing and budget support conditionalities in Budget Law).  Tools sustainability was enhanced by the creation of a virtual resource library that accumulates all tools; this was handed over to partner mid-2020.  Training sustainability was enhanced by putting material into the library, and Training of Trainers to train new recruits and for refresher courses.  Technology sustainability was enhanced by security measures (to prevent theft), CIMF budget for maintenance and repair to keep equipment functioning, perpetual licenses for software (minimize recurrent costs), and keeping software codes in-house.  FIRST has a sustainability plan in the draft Year 3 Work Plan.  Through the 4 Ts, FIRST has enhanced public (taxpayer) services, built independence, strengthened policy framework, and directly contributed to mobilizing (tax) revenues and budget management. necessary to ‘institutionalize” reforms. 61 ANNEX VII: EVALUATION TEAM MEMBERS’ QUALIFICATIONS AND ROLES Denis Dunn – Team Leader Mr. Dunn has a BA in Economics and an MA in Planning (Regional Economic Development). He has 38 years’ professional experience in private sector development with 26 years in Africa, Middle East, Balkans, Caribbean, Eastern Europe, and near/central Asia for projects and assignments funded by USAID, World Bank, DFID, EU, Danida, and CIDA. Mr. Dunn’s primary expertise is in private sector engagement/MSME development (economic growth) with additional experience in governance (public administration reform and public financial management), capacity-building, and monitoring and evaluation. Dr. Mehdi Bin Brahem – Evaluation Specialist Mehdi Ben Braham is a Doctor in Economics, graduated from Paris Dauphine University. He is currently Associate Professor at ESSEC and researcher at the Laboratory of Economics and Industrial Management at the Polytechnic School of Tunisia. He conducts research in several areas such as the labor market, poverty, and job creation. He also carried out consulting missions to several Tunisian ministries (Prime Ministry, Ministries of Social Affairs, Youth and Sport, and Health) and international organizations (UNDP, ILO, World Bank, USAID, GIZ, and UNICEF) through field surveys and the evaluation of projects funded by international organizations. He has also collaborated with several ministries (MAS, MDICI, MEFP, etc.) in the framework of planning processes and public policy programming for anti-poverty programs. He actively participates in the “Euromed” network through regular training sessions for civil society in Tunisia, which recently led him to advocate with the European Commission to take economic and social rights into account in the Tunisian-European agreements. 62 ANNEX VIII: TECHNICAL NOTES81 Tax compliance management strategy. The tax compliance strategy aims to reduce the tax gap (the difference between the true fiscal capacity and the real collected tax) by improving voluntary compliance with fiscal duties. It does not cover organized fraud, which usually includes the production of falsified documents and/or legal arrangements, and which must be combatted by the application of criminal penalties. The proposed strategy is based on raising and maintaining tax compliance at as high a level as possible. It corresponds to the strategy generally followed by modern tax administrations. It is likely to be applicable regardless of the national context and culture. It was adapted for Tunisia based on the conclusions of the TADAT assessment of March 2018, supplemented by an in-depth analysis of the identified strengths and weaknesses conducted during a series of workshops. It is based on the following four pillars: (1) Measuring taxpayer compliance for each obligation and each taxpayer segment. Implementing a tax compliance strategy is based on risk analysis. This implies that compliance is measured by using indicators that measure the rates of the respect of each obligation, rather than raw data. These so-called “compliance” indicators are also used to measure the impact of mitigation activities undertaken by the administration. They measure both the compliance of taxpayers and the performance of the administration. They must therefore be interpreted taking into account their interaction and supplemented by other indicators called “management” indicators that measure the activity of the administration, the quality of the work, and their efficiency. Compliance indicators reflect the evolution of compliance, the improvement of service performance, or both. The establishment of a statistical system for measuring compliance is therefore an indispensable element of the strategy, and conditions its success. (2) Giving priority to meet future obligations rather than paying the previous fiscal debt. The implementation of a compliance strategy must not lead the administration to waive the collection of the taxes that have been evaded, or to stop fighting fraud and tax evasion. However, it must not lead to the condemnation of future compliance. The tax control and collection policy must therefore reconcile these two contradictory requirements, while respecting the principle of equality between citizens in terms of taxation. In concrete terms, it must therefore reserve coercion and sanction for the most serious cases and recidivism, and promote a return to compliance whenever the information held, such as the history of tax behavior and the commitments made by taxpayers collectively or individually, allows it to reasonably expect full compliance with all current and future duties. In terms of collection, this will result in granting settlement plans. In terms of tax control case selection, this will result in priority being given to targeted methods (punctual or targeted verification). (3) Tax administration response is adapted to taxpayer behavior. The proposed strategy is to make taxpayers or their representatives aware of their responsibilities by inviting them initially to correct for themselves the failures observed, then progressively using the offer of service, threats, and then coercion. Coercion may be legislative or administrative (reinforced and thorough auditing, offensive recovery, tax, and penal sanctions). (4) Achieving a balance between service and audit. Service activities include all actions to promote compliance: education, prevention, information, assistance, and facilitation of procedures. Audit activities include all actions aimed at correcting and dissuading tax noncompliance: application of sanctions, in-depth tax audit, and forced collection. The borderline between these two notions is hazy, and tax administrations tend more and more to reorient their control activities toward the service. Unified exchange platform (UXP) is proven, commercially available data exchange software. 81 From FIRST FY2 Work Plan in Annex 2, September 2018. 63 Currently the basis for all e-government functions in Estonia, UXP has been deployed in eight countries globally as a tool for streamlining data sharing between government institutions. UXP is highly secure because it utilizes strong encryption; it is reliable and easy to deploy and implement. The Estonian firm Cybernetica, owner of the UXP software, completed a scoping mission in Tunisia in July 2018 and concluded that UXP can be readily implemented given the level of the existing IT platforms at partner institutions and the high level of competency of IT staff at the MOF and other stakeholder institutions. UXP will be used to improve tax administration through the implementation of relevant applications that use data exchange in order to (1) provide more timely information, (2) increase the capacity of the tax administration to perform risk-based audits, (3) improve transparency in fiscal matters in order to improve fiscal civism, (4) estimate the components of the conformity tax gap in a more accurate and timely fashion in order to better target the opportunities for increased revenues, (5) improve data sharing and harmonization between GOT institutions leading to more efficiency, and (6) improve the capacity to inform better decision-making in tax administration by providing decision makers with more timely and efficient access to information. There will be three workstreams to the implementation of UXP in Year 2 of the FIRST project: (1) procurement and installation of the necessary hardware and software at partner institutions and training of IT staff on the use and maintenance of UXP, (2) implementation of a legal framework for data exchange through MOUs and formal agreements to share data between institutions, and (3) development of specific applications to improve tax administration and fiscal policy through secure data exchange. Upgrade CIMF capacity to support improved tax administration. The FIRST project will support upgrades to CIMF capabilities in three core areas: (1) improving the taxpayer registry, (2) online registration of taxpayers, and (3) improving the taxpayer account. These initiatives are part of the broader upgrading of the MOF’s IT infrastructure and are directly aligned with FIRST’s objective to improve tax administration. ACSS rollout. Developed with technical assistance from USAID under the TCP Project, the audit criteria selection system (ACSS) needs to be refined. ACSS will be an important tool for centralizing audit programming. It will be used by a headquarters unit responsible for a nationwide audit. First Project will also help DGI in the rollout of through technical and ICT assistance and a corresponding training program covering audit, collection, and other aspects of compliance to better educate auditors in regional directorates regarding the revenue-raising opportunities of enhanced risk-based audit selection and ensure a more consistent approach to audit selection throughout Tunisia. Medium-term fiscal framework. A more comprehensive approach to building analytical capacity will be undertaken within the context of a medium-term approach to budget development, through the development of a medium-term fiscal framework (MTFF) and a medium-term expenditure framework (MTEF). In Year 1, FIRST assisted the MOF by developing an initial MTFF model that will allow the ministry to produce medium-term forecasts of budgetary and macroeconomic impacts of tax and fiscal policies. In Year 2, we will work with the MOF to refine the MTFF model and to analyze the model’s outputs to inform policy decisions. Tax expenditures support. In anticipation of the adoption of the proposed new organic budget law by the Tunisian General Assembly, a working group under the guidance of the Directorate of Tax Advantages (DGAFF) is performing an analysis of current law tax expenditures to be consolidated into a single document called “Tax Expenditure Report: RDP.” Under the proposed law, this report must be submitted with the budget law. To date, the DGAFF work has focused on the specification of a reference tax system (needed to establish the basis for identifying what are considered to be “tax expenditures” rather than “normal preferences” that are inherent in the tax system design. Revenue losses that accrue due to identified tax expenditures are highly visible potential targets for increasing revenues, and past IMF advice has focused attention on the need to control tax expenditures as part of Tunisia’s long-term fiscal strategy. Previous IMF comments to the MOF have expressed a view that the scope of tax 64 expenditures identified by DGAFF may be too narrow. This affects the set of preferences that are considered potential targets for modification/elimination; designating a preference as part of the reference tax system could effectively shield it from consideration as a potential source of revenue that could be secured by repealing or modifying it. In short, if a preference becomes part of the tax expenditure list, it becomes a target. This happens normally in virtually all countries once viable tax expenditure analysis programs are established. DGAFF has given considerable thought to the identification of tax expenditures and based on its own analysis has moved forward with generating preliminary estimates of the estimated revenue losses that currently identified tax expenditures produce. DGAFF has also noted that IMF criticism on the scope of their tax expenditure work may well be attributable to Tunisia’s reliance on a tax system designed in line with French taxation concepts that at their core lead should lead to a tax system that includes more preferences than would normally be considered as part of the reference system by the IMF. FIRST will support the DGAFF in the continued development of the tax expenditure report by providing a short-term technical expert to lead the initial development and application of a tax expenditure analysis framework that more clearly evaluates the appropriateness of the methodological basis being used for the current report (considering its origins derived from French tax concepts), as well as identifies areas for further development going forward. It is anticipated that the advisor will provide guidance to DGGY about alternative measures of tax expenditure efficiency and effectiveness so that they may be integrated into future tax expenditure reports. Computable General Equilibrium (CGE) model. In Year 1, FIRST worked with the MOF’s Fiscal Analysis Unit (FAU) to develop a multi-sectoral CGE model. This model is a key analytical tool that will be used to estimate how key indicators of the Tunisian economy (prices, growth, employment, trade, investment, consumption, etc.) would be expected to react in response to specific changes in economic and fiscal policies (e.g., VAT, excises, customs duties, PIT, and CIT). CGE models, first developed in the 1970s, generally permit a more robust quantitative assessment of fiscal policy changes than can be achieved through other analytic frameworks. In April 2018, MOF officials were trained in Nabeul on the basics of CGE modeling. In Year 2, FIRST will provide the MOF with GAMS software and will continue training FAU staff in the use of the CGE models for more effective policy analysis. Electronic Tax Filing and Payment. To streamline the tax filing and payment system, we will support the DGI as it continues its development and deployment of e-filing, e-payment, and other e-government services. E-filing will significantly reduce the taxpayer burden through electronic taxpayer services. In Year 2, FIRST will procure a server for CIMF to provide distance training on e-filing under the current system. We will also assist the MOF in designing an upgraded e-filing system that is user- friendly and can reach a broad base of taxpayers in order to improve fiscal civism through voluntary compliance consistent with the compliance management strategy. Tax gap analysis. The MOF needs a firm understanding of how tax administration capacity and the tax law environment itself adversely impact tax collection. Understanding precisely the size and the nature of the tax gap will help the MOF target its strategies for closing the tax gap and will help measure progress toward increasing tax revenues. In Year 1, FIRST assisted DGELF in quantifying potential revenue collection by source for Personal Income Tax (PIT) and Value Added Tax (VAT). In Year 2, we will assist in quantifying the tax gap for Corporate Income Tax (CIT). As the UXP data exchange platform is implemented, we will help the MOF develop an application to automatically calculate the tax gap from different sources using data exchange. 65 ANNEX IX: DISCLOSURE OF CONFLICT OF INTEREST INFORMATION CAN BE OBTAINED THROUGH OF CUSTODIAN RECORD WITH IBTCI. 66 ANNEX X: FIRST COMMUNICATION MATERIALS VIDEOS – For Ministry 1) Budget Law Amnesty Campaign 1: Published on Ministry Facebook Page and website: 69,000 views + Broadcast on national television https://drive.google.com/file/d/1bOZ1T6TLZ39ySGrKxP6pkyQPCBTG4rby/view?usp=sharing 2) Budget Law Amnesty Campaign 2: Published on Ministry Facebook Page and website: 20,000 views https://drive.google.com/file/d/1Ehlkg-Ig-ZfVEuDHtjirWs8w8ArLLAEO/view?usp=sharing 3) Digital Payment Services Campaign 1: Published on Ministry Facebook Page and website: 16,000 views https://drive.google.com/file/d/1IK5sCB3A-Ojs7oYwgt3u6g-f_wRkydAz/view?usp=sharing 4) Digital Payment Services Campaign 2: Published on Ministry Facebook Page and website: 5,000 views https://drive.google.com/file/d/1HiMRpwzx6wMWh2DlX_BYBkG3FlQapAPV/view?usp=sharing 5) Celebrating women in leading roles at the Ministry: Published on Ministry Facebook Page and screened at USAID event: 10, 000 views https://drive.google.com/file/d/17fFZMuo4ibObkTbB3EDDntjTz9vvls-G/view?usp=sharing 6) Customer Service Training: Published on Ministry Facebook Page and website: 5,000 views https://drive.google.com/file/d/1esRjHWZQKA6hXOwHZfoePpI_awoD6A1s/view?usp=sharing 7) Promoting the new De-cashing initiative: To be published https://drive.google.com/file/d/1fifPuzJ5PLWMdR5PpFxaRh0oyIS1WPWR/view?usp=sharing 8) USAID participation at Enterprise Day Event: For USAID use. FIRST designed and provided booth and communication support https://drive.google.com/file/d/1854-3CCjZkvASd4HA3ndTn6DVgW3_u0E/view?usp=sharing RADIO – For Ministry 1) Budget Law Amnesty Campaign: 30-second spot broadcast on five radio station nationwide https://drive.google.com/file/d/1NQAW8AKn99TNe1EfL6AQSexQ_gURLNqE/view?usp=sharing 2) De-cashing Campaign: Three 40-second spots broadcast on five radio stations nationwide https://drive.google.com/file/d/1ieX5-UButzw1untIyeJuJSYXP8RJvvRL/view?usp=sharing https://drive.google.com/file/d/1yQHJJiNnbwb1u4RVFe783wyc0az5FOrs/view?usp=sharing https://drive.google.com/file/d/11sYJM3km5MSvQYUSpyLkW71ob0ETRUqD/view?usp=sharing PUBLICATIONS – For Ministry 1) Budget Law Amnesty Campaign: Online Flyers and banners on Ministry website and Facebook page https://drive.google.com/file/d/1XQ9TZ_BgyQ4yo10LQS2xPRCtIAWm_7xg/view?usp=sharing 2) De-cashing campaign: Posters for online and offline at tax offices https://drive.google.com/file/d/1xwi0KsDyUbPNuINJkDaSHG3gqicZzx50/view?usp=sharing 3) Comic book: Civic education campaign to raise awareness of school children on taxes, will be distributed at schools and online 67 https://drive.google.com/file/d/1Nb5PJLr_7Tg-Z3yeRkOBB1wVBrjupKBb/view?usp=sharing 4) Taxpayer Guide: To be distributed online and at tax offices (pending final validation) https://drive.google.com/file/d/1R93XmIYYCsEUxfbtfShiDG9ldM8MR_2v/view?usp=sharing SUCCESS STORY VIDEOS – For USAID 1) Digital Strategy https://drive.google.com/file/d/1h9cwF3Kllf81LpXXhX-Kbel4wS44IzZ8/view?usp=sharing 2) UXP – Data Exchange https://drive.google.com/file/d/1zxykxzJBSkFSSSqr8OXzgtmRe4fnt_hj/view?usp=sharing 3) Tax Administration Capacity-Building https://drive.google.com/file/d/1DfIuH_u_8OdEs30ywXu9Ev47mVeTpSCe/view?usp=sharing 4) Tax Policy – Tax Expenditures https://drive.google.com/file/d/1YUNxWvcMsYWYDAqhdWFtSvd19Z6iSiOn/view?usp=sharing 5) Tax Policy – Fiscal Incidence https://drive.google.com/file/d/1EiEtPiCnOCDzkr_Q4knJhw3rIUgpPWfz/view?usp=sharing SUCCESS STORIES – For USAID 1) Success stories in standard USAID snapshot format https://drive.google.com/drive/folders/1AYZ02Z-lxtgzj-si3ZpOou4Kymibfl3a?usp=sharing 68 ANNEX XI: CAPACITY DEVELOPMENT: SIX KEY MESSAGES82 Consider organizations as “open systems.” • They are embedded in a context. • They get inputs or resources. • They use their capacity to process these inputs to outputs (products and services). • Their performance and change prospects depend on the context, the inputs and the capacity – all have to be considered. Focus on products and services. • Look at the outputs of the organization(s) – product and services. What are they? What should and can they realistically become? • Capacity development must result in specific changes in outputs. • Avoid, initially, focusing on CD support elements (training, consultants, etc.). Explore the context and the inputs – and then go inside! • Before looking inside the organization, explore structural and institutional drivers of and constraints to change in the context. • Look for stakeholders – they make and break change! • Consider the inputs and resources. • Then, finally, look at the rest of the “inside” elements – now there is a chance to understand why a system works as it does, rather than just seeing why it does not work. Dig deeper to get a solid diagnosis. • Look for both formal and informal aspects of organizations. • Look for both the “functional-rational” dimension and the “political” dimensions of organizations the latter factors can have both positive and negative aspects. • Do not assume that organizations are only driven by functional-rational considerations. • Do not assume they are only driven by negative self-interest, internal politics, and power. • Be pragmatic – not everything needs to be known, or written down, at a certain point in time. Change and capacity development are mostly domestic processes. • External factors are often powerful drivers of capacity or change – but committed insiders must lead. • “Political” factors (e.g., power to push changes through or stakeholder pressure) are often more important than “functional-rational” factors (e.g., legal mandates or organizational structure). • A change strategy is required. • Aid agencies can contribute to capacity development, but cannot drive the process. Roles of and instrument for donors. • Make sure that the partner can and will lead! • Assist the partner to define realistic CD targets in terms of changes in organizational outputs. • Develop the partner relationship to build on trust and a shared view of key constraints on and opportunities for capacity development, both inside and outside the organization(s). • Finally, play a catalytic role, engage to build up demand for change, provide access to knowledge, pilot different approaches, and facilitate dialogue between domestic stakeholders. 82 Institutional Assessment and Capacity Development: Why, what and how? EuropeAid. September 2005, p. 7. 69 ANNEX XII: TUNISIA FIRST KEY RESULTS – CUMULATIVE YEAR 1 & YEAR 283 Key Result Relevance to FIRST Objectives Cross-cutting Assessment of MOF missions and alignment with current results framework Support for management by objectives reform to improve MOF efficiency Capacity-building of MOF staff: trained 1,428 staff (736 females, 692 males) MOF staff acquire skills to improve performance Tax administration Tax Assessment Diagnostic Assessment Tool (TADAT) report Detailed assessment of tax administration areas to improve Fiscal civism compliance strategy Strategy and indicators to improve voluntary tax compliance Introduction of targeted audit procedure Enables to recover VAT taxes and to improve fiscal civism Training and procedures on tax collection best practices Procedures to improve tax collection Training and procedures on VAT refund best practices Procedures to target VAT refunds Technical and material assistance in setting up the medium enterprise tax office The Tunisian tax administration is able to provide focused services to medium-sized enterprises Installation of the unified exchange platform for electronic data exchange connecting 8 institutions (CIMF, DGI, DGD, ATTT, CNAM, CNSS, RNE, and CPF) Government institutions are able to securely share electronic information to improve efficiency in tax administration and budget making Technical assistance for fiscal information system upgrade Priority web services developed (e-filing, electronic taxpayer registration, improved taxpayer database) Customer service training for tax collection agency staff Tax collection agencies are better able to assist taxpayers, leading to increased compliance E-filing training and online course development Businesses are better able to file electronically leading to increased revenues Fiscal Policy Development of micro-simulation models (VAT, PIT, and CIT) Ability to calculate tax gaps resulting from changes in tax structures and rates Development of a medium-term fiscal framework (MTFF) model Ability to estimate the medium-term (3 to 5 years) impacts of fiscal policy changes Development of a computable general equilibrium (CGE) model Ability to estimate macroeconomic and sectoral impacts of fiscal policies 83 Prepared by FIRST in September 2019. 70 Key Result Relevance to FIRST Objectives Tax expenditures analysis and report MOF is able to produce an annual tax expenditures report and compute foregone revenues from tax expenditures Tax incidence analysis, methodology, and report Analysis of which segments of the population and businesses bear the burdens of specific fiscal policies Communications New MOF website online User-friendly website providing valuable information to taxpayers Taxpayer guide User-friendly guide to taxpayers consolidating in one document all the tax obligations for Tunisian citizens and businesses Support for media campaigns (de-cashing, tax amnesty) Raising public awareness on specific features of the budget laws Internal webcasting system Improving internal communications at MOF by virtually connecting four offices in Tunis and eight regional offices Communication strategies for the MOF (internal and external) Strategies to improve the efficiency of the MOF through enhanced and strategic communication Support for setting up a communications unit at MOF Communications unit able to centralize communication functions and to implement strategies Assessment of MOF call center Recommendations for the improvement of the call center so that it is better able to serve citizens and businesses Other technical assistance Post-clearance audit procedures for customs Implementation of a post-clearance audit unit at Tunisian customs enabling “pre-cleared” companies to benefit from streamlined customs clearance processes Analysis of Doing Business Survey “paying taxes” indicator Strategy for improving Tunisia’s score on the “Paying Taxes” indicator of the Doing Business survey Support for FATCA implementation Support to MOF in setting up cybersecurity protocols and assistance to Tunisian Bankers Association on FATCA compliance requirements 71 ANNEX XIII: FIRST ORGANOGRAM 72 ANNEX XIV: FIRST CONSULTANTS AND ASSIGNMENTS Local/Expat Dates Assignment Expat Oct. 2017 Project start-up, local recruit Expat Nov. 2017 Political Economy Analysis (PEA) Expat Nov. 2017 Gender strategy Expat February & July 2018 MTFF model development Local March 2018 E-filing training Local March 2018 E-filing training Local March 2018 E-filing training Expat March 2018 TADAT Expat March 2018 TADAT Expat March 2018 TADAT Expat March–April 2018 Electronic data exchange Expat March 2018 Mobile App development Expat Nov. 2017, April & Dec. 2018, June 2019 M&E Expat May & June 2018 Fiscal civism compliance strategy Expat July–August 2018 Internal communications assessment Expat Oct. 2018 Tax expenditures report Expat Aug. 2018 & Jan. 2019 Post clearance audit for customs Expat Aug. 2018 Communications strategy Expat Dec. 2018 & April 2019 DME operations assessment Expat Dec. 2018 Assessment of taxpayer data base Expat March & April 2019 Enforced collection procedures & training Expat March 2019 VAT refund process best practices Expat April & May 2019 Targeted audit training Expat Feb. 2018 Legal review of the fiscal code Local Feb.–June 2019 Training on MTFF modeling Expat April & May 2019 Data visualization Expat May 2019 Micro-simulation model user guides Expat May 2019 Strategy for implementing. Comms. Unit Expat June 2019 Institutional assessment of call center Expat July 2019 Technical assessment of call center Expat Aug. 2019 Development of IT tools Local July–Sept. 2019 Legal framework for data exchange Local June–Sept. 2019 Paying taxes indicator of Doing Business Local June–Sept. 2019 Tax incidence study Local Sept.–Dec. 2019 Optimization of energy pricing Expat Sept.–Dec. 2019 Optimization of energy pricing Local Sept.–Dec. 2019 Tobacco taxation study Local Sept.–Dec. 2019 Tobacco taxation study Local Sept.–Dec. 2019 Special treasury funds study Expat Sept.–Dec. 2019 Special treasury funds study Expat Oct. 2019 Tax administration indicators Expat Oct. 2019 Strategic planning workshop Expat Nov. 2019 Fiscal policy analysis capacity-building 73 ANNEX XV: FIRST PROGRESS SUMMARY (SOURCE: YEAR 3 WORK PLAN TABLE 1) (Main) Activity Progress to date (activities carried out) % completion Cross-cutting activities Support for management by objectives (GBO) reform  Assessment of current performance management framework 40% Support for enhancing cybersecurity and compliance with FATCA  Assistance to the DGI in preparing a work plan for FATCA compliance  Training plan for Tunisian financial institutions on FATCA compliance 10% Objective 1: Improve the efficiency, transparency, and cost of fiscal administration. Implement UXP data exchange  Servers installed  UXP uploaded  Test environment functional  Support for the preparation of MOUs and the drafting of regulations to fully enable data exchange 60% Upgrade fiscal information system  Scoping report completed  Technical specifications for priority applications completed  Software development firm selected to assist CIMF with the development 50% Set up risk analysis system and tools to encourage compliance  Identification of indicators of risk of noncompliance  E-filing training  Review of the legislation on penalties  Introduction of Paradox software to clean up the taxpayer registry 30% Create fiscal civism compliance strategy  TADAT assessment  Fiscal civism compliance strategy prepared  Working groups at MOF started strategy implementation (indicators, priority actions) 50% Support for the new DME (Directorate of Medium-Sized Enterprises)  Strategy for a model DME  Acquisition of IT and office equipment for the DME (DGI and DGCPR)  Training of DME staff on best practices 60% Facilitate e-filing training  E-filing training for accountants  Acquisition of a dedicated server at CIMF to facilitate e-filing training  E-filing online course prepared 50% Facilitate customer service training  Training modules on customer service developed  Training of nearly 1,000 DGCPR agents in five regions (Tunis, Sousse, Sfax, Djerba, and Tabarka) 40% Support issue and targeted audits  Strategy for issue and targeted audits  Training of DGI staff on targeted audits  Changes to the fiscal procedures manual (CDPF) to allow targeted audits  Introduction targeted audits in the 2020 finance law 80% Objective 2: Enhance the capacity to develop and deliver tax policy and other fiscal reforms. 74 (Main) Activity Progress to date (activities carried out) % completion Update and expand micro￾simulation models  VAT, PIT, and CIT micro-simulation models updated  Model user guides prepared  MOF staff trained on the use of the micro-simulation models 80% Develop tax incidence and redistributive effects model and analysis  Tax incidence methodology prepared  Draft tax incidence model developed 20% Strengthen capacity of MOF to interpret tax gap analysis  Tax gap estimations for VAT, PIT, and CIT using micro￾simulation models  Training of MOF staff on micro-simulation models 70% Support the development and publication of a tax expenditures report  Analysis of the current tax expenditures  Review of the draft tax expenditures report  Technical assistance to the MOF for the preparation of the 2020 tax expenditures report 80% Develop the medium-term fiscal framework (MTFF) model and train MOF staff it its use  Development of a draft MTFF model  Implementation of the EViews software to support MTFF  Training of MOF staff on MTFF model  Training of MOF staff on econometrics principles  Preparation of a methodology to use the MTFF model 60% Strengthen CGE modeling capability at DGRE  Acquisition of computers and GAMS software for DGRE  Training on CGE modeling  Development of applications using CGE models 70% Provide technical assistance to help Tunisia improve its “Paying Taxes” indicator score on the Doing Business survey  Analysis of the drivers behind Tunisian ranking on the Paying Taxes indicator  Recommendations for improving the ranking 70% Objective 3: Improve communications, engagement, and consultation on priority reforms. Help establish a formal communications unit within the MOF  Strategy developed for the creation of a communications unit at MOF  Decree for the creation of the communications unit drafted 50% Improve internal communications  Strategy for improving internal communications  Implementation of a video-conferencing system at MOF 50% Facilitate recurring survey of taxpayers  Baseline survey conducted (2018)  Results of initial survey analyzed  2019 survey prepared 50% Draft and distribute proactive media products  Production of taxpayer pocket guide  Media campaign on interest amnesty in the 2019 finance law  Media campaign on de-cashing initiative 70% Engage with key stakeholders  Preparation of a youth essay competition  Engagement with the private sector (Enterprise days) 40% Implement new MOF website  New website redesigned  Website transferred to CIMF  Website content updated 80% Upgrade the MOF call center  Organizational assessment of the call center  Technical assessment of the call center  Recommendations for call center improvement and training 30% 75 ANNEX XVI: FIRST IMPLEMENTATION EFFICIENCY FACTORS Internal Positive Negative Design  Why, what, who are described in detail in the contract  Design still relevant in context  Duration too short for meaningful PFM and CD  How – weak participatory approach in Year 1 so FIRST languished and in Year 2 some deficiencies Assessment  TADAT excellent diagnostic tool for assessing weaknesses in fiscal system  Due to MOF apprehension, TADAT was delayed  FIRST team focus on TADAT in Year 1 contributed to dearth of results Planning  Built on predecessor TCP, backstopping support from contractor Home Office  Avoiding consensus speeds up delivery  Did not fully engage partner = weak “ownership”  Documents fail to outline approach or strategy Office  Functional office premises  Not embedded at MOF so less communication, coordination, and on-site “supervision” Staffing  Technically competent, professional experts, strong COP in Year 2  Year 1, engagement with Minister and Advisors but not DGs (working level counterparts)  50 percent turnover – substantial departures end 2018/early 2019 STTA  Many international consultants  Beneficiaries “learned a lot”  Some training was insufficient and not practical  Some lacked MENA knowledge and language skills Tools  Derived from “best practice”  Need “best fit,” – many tools had to be adapted Technology  IT solutions – 20 software pieces  Accompanying hardware  Legislative barriers, concern for data control, and security delayed operationalization Flexibility  FIRST added emerging priorities and dropped “sensitive” items  Adjustments to activity scheduling  Shifting of resources MEL  AMELP is comprehensive  Some indicators deficient and no data system  No process indicators for main activities Reporting  Monthly, quarterly, annual  MOF focal point does not receive reports  Documents fail to elaborate challenges External Positive Negative Election  New government might support structural changes (reform-minded)  Cannot push for reform in election year International organizations  Put conditionalities in budget support  Long time to include into Budget Law and get policy shifts MOF – champions  Can fast-track priority items if strong commitment – “drivers of change”  Lack of vision, strategy  Leadership not reform-minded MOF – staffing  Adjust pace/sequencing in delivery  Working groups, focal points, dedicated staff, counterparts/DGs  MOF staff busy especially at budget time  Some focal point positions vacant MOF – final mile  Rewards – study tour, computer tablets  Performance management systems  Difficult to operationalize (fully adopt) initiatives MOF – appetite  Receptive to incremental approach  Modest reforms only  No support for profound structural reforms  Pockets of resistance to change MOF – traditional  Benchmarking and best practice tools  Experts as “agents of change”  Antiquated systems, archaic structure (silos)  Hierarchical, old-school management Complexity  FIRST team technical competencies and experience; STTA level of effort  Reform is technically difficult – many issues  Resistance to change Governance  Steering Committee resolves issues  COP direct access to Sec-General  SC not official because there is no MoU  Infrequent meetings diminish communications and coordination Stakeholders  Civil society, private sector, and unions can lobby for reform (advocacy)  Powerful forces and vested interests block or undermine reform efforts Talent  Opportunities for young people to help digitalize MOF  “Brain drain” as young people move to EU for work 76 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523