DISCLAIMER: This report is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents of this report are the sole responsibility of The Kaizen Company, LLC and do not necessarily reflect the views of USAID or the United States Government. USAID MONITORING, EVALUATION, AND LEARNING ACTIVITY PHASE 1 EVALUATION: DOCUMENT REVIEW RELATED TO THE FRPFM ACTIVITY August 19, 2020 TABLE OF CONTENTS 1. QUESTION ONE 1 2. QUESTION TWO 5 3. QUESTION THREE 7 4. QUESTIONS FOUR AND FIVE TOGETHER 9 5. QUESTION SIX 10 ANNEXES 14 ANNEX 1: SELECTED LITERATURE REVIEW 14 ANNEX 2: PROJECT STRUCTURE AND SIGNIFICANT ACCOMPLISHMENT BY COMPONENT AND YEAR 29 ANNEX 3: INDICATOR TRACKING TABLE 39 ACRONYMS & ABBREVIATIONS AMELP Activity Monitoring, Evaluation, and Learning Plan CPEP Certified PPP Professional CYPRESS Capacity Performance, Results, Sustainability DO Development Objective EU European Union FGD Focus Group Discussion FRPFM Financial Reform and Public Financial Management GBD General Budget Department GDP Gross Domestic Product GFMIS Government Financial Management Information Systems GOJ Government of Jordan GRB Gender-Responsive Budgeting GST Goods and Services Tax IMF International Monetary Fund IPSAS International Public Sector Accounting Standards IR Intermediate Result ISTD JD Income and Sales Tax Department Jordanian dinar KII Key Informant Interview LPFM Leadership in Public Financial Management M&E Monitoring and Evaluation MEL Monitoring, Evaluation, and Learning MELA Monitoring, Evaluation, and Learning Activity MOF Ministry of Finance PEFA Public Expenditure Financial Assessment PFM Public Financial Management PFMA Public Financial Management Activity PI Performance Indicator PPP Public-Private Partnerships ROB Results-Oriented Budget SEPD Studies and Economic Policy Directorate TADAT Tax Administration Diagnostic Assessment Tool USAID United States Agency for International Development 1 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV JORDAN MONITORING, EVALUATION, AND LEARNING ACTIVITY (MELA) Phase 1 Evaluation Document Review Related to the FRPFM Activity1 Final Draft Report August 15, 2020 1. QUESTION ONE What is the macro situation in the below areas before the COVID-19 crisis compared to when USAID FRPFM started supporting the Ministry of Finance (MOF) in Jordan? ▪ Revenue performance ▪ Budget efficiency and transparency ▪ Fiscal sustainability ▪ Structural reforms ▪ Institutional maturity 1.1 MACROECONOMIC-FISCAL SITUATION In recent years and, in particular, during the period of the Fiscal Reform and Public Financial Management (FRPFM) project, Jordan has made progress reforming its economy. It has maintained macroeconomic stability, trade balances have improved, the Jordanian Dinar has remained relatively stable, inflation has been subdued, the financial sector remains sound, and the business climate has improved. At the same time, Jordan has faced significant challenges. Conflicts in nearby Jordan and Iraq have disrupted trade, and nearly 1.3 million Syrian refugees—representing about 15 percent of Jordan’s population—have strained public services and infrastructure. Volatile energy prices and supplies have adversely affected the economy. Recent data from the IMF summarize the situation.2 Real (i.e., inflation adjusted) Gross Domestic Product (GDP) growth slowed from 2.6 percent in 2015 to 1.9 percent in 2018. During the same period inflation declined from 2.5 percent to 1.8 percent. Exports rebounded from a decline in 2015 to 5.5 percent growth in 2018. GDP growth, however, has been insufficient to absorb the expanding labor market such that per-capita GDP growth has been negative since 2009. Exports and tourism have been on the rebound, in part due to favorable world prices for potash and phosphates and preferential access to U.S. and European Union (EU) markets. In addition, lower energy prices and resumption of gas imports from Egypt have lowered the import bill. Of particular note, Jordan was among the top improvers in the World Bank’s 2020 Doing Business Report3 (p. 1). Overall, the assessments made by the IMF in its Article IV consultation in 2020, and earlier by the European Union (EU) as part of its PEFA assessment, remain intact.4 The EU concluded that there was continuing and gradual improvement in public financial management (PFM) and that there were good prospects for continuing PFM improvement, while “efforts will be needed to make tax collection more efficient, internal controls less burdensome, and to reorient internal and external audit work to contribute more effectively to improving the efficiency of service delivery.” This was the case in 2017 and remains the case in 2020. The IMF concluded that “fiscal consolidation (has) also proved difficult to maintain, amid persistent fiscal slippages and tax administration deficiencies, and public debt was not reduced.” In this context the Jordanian 1 The evaluation team is comprised of the following members: The team was contracted by Kaizen Company to conduct this external final examination of USAID Jordan’s Fiscal Reform and Public Financial Management (FRPFM) project as part of the mission’s Monitoring, Evaluation, and Learning Activity (MELA). 2 See IMF. Staff Report for the 2020 Article IV Consultation and Report for an Extended Arrangement under the Extended Fund Facility. March 11, 2020. 3 https://openknowledge.worldbank.org/bitstream/handle/10986/32436/9781464814402.pdf 4 IMF, op. cit., pp. 1-4. European Union. Jordan: PEFA Assessment 2016, Feb. 23, 2017, pp. 12-13. 2 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV authorities have requested an extension of the Extended Fund Facility at 270 percent of Jordan’s quota ($1.3 billion). 1.2 FRPFM IN THE CONTEXT OF THE MEL PLAN The most relevant available data for evaluating FRPFM’s performance across these five project areas over the period 2015 (base year) to the present are found in the project’s monitoring, evaluation, and learning (MEL) plan. Indicators for the MEL plan included those specified in the April 2016 contract, as well as others proposed by the contractor and approved annually by the mission. The most recent yearly data are provided in the October 2019 MEL plan in addition to Quarter Two of the Year 5 Progress Report (See Annex 3: Indicator Tracking Table.) As it becomes clear in the discussion below, these data are imperfect for evaluating progress across the project’s four components. To provide context, it is worth noting the International Monetary Fund’s (IMF) assessment for the current state of affairs in public financial management in Jordan, which took into account activities of FRPMF: On the revenue side, decisive actions are needed to address tax administration weaknesses, related to inaccurate registration, filing and audit deficiencies, sizable tax arrears, weak compliance risk management, substandard access and use of information for audit and enforcement purposes, and extensive revenue leakage from smuggling and tax arbitration and transfer pricing practices in special economic zones. Further efforts to rationalize tax exemptions, revisit the property taxation framework, and realign exercises will be needed.5 The IMF’s assessment is important because for many of the fundamental functions in PFM— taxpayer registration, filing, auditing, tax arrears, compliance risk management, revenue leakages, and tax policy—the MEL plan and other activity tracked by the project have not clearly measured the status and changes in tax administration and other aspects of public financial management in Jordan. A key purpose of Phases 1 and 2 of our evaluation is to mark the state of our current knowledge and its limitations about these issues. There are 40 indicators in Figure 4 in the October 2019 MEL plan (pp. 27-35). We represent these here in a simpler version as shown in Annex 3. The sections below review how the project succeeded in terms of the MEL indicators for each program area. These indicators are important because they are the only performance measures recorded over time by the contractor during project implementation which is designed to measure project goals. (A) OVERALL PERFORMANCE The first five indicators (numbered 0.1 to 0.5 in Annex 3) attempt to capture project performance at a high level, defined as the Mission’s Intermediate Result IR 1.4: Fiscal Stability and Public Financial Management Improved. IR 1.4 is itself one of four intermediate results under the mission’s Development Objective (DO) 1 Results Framework (DO 1): Broad-based, inclusive economic development accelerated.6 Indicator 0.1 in Annex 3 is one of the highest-level indicators (net government debt as a percent of GDP). Its base value in 2015 was 83.2 percent, and its target value was to decrease to 60 percent by project end. As of Year 3, the most recent year for which data are available, this indicator had deteriorated, and exceeded 89% which is nowhere near the target. The target for indicator 0.2 — fiscal deficit as a percent of GDP — was to improve from -3.5 percent (base) to 0 percent at project end. By Year 3, there had been some improvement, but it was still below target. Indicator 0.4 — ratio of tax collections to GDP — was targeted to increase from 15.9 percent (base) to 20 percent. By Year 3 (most recent year data available), it had worsened and fallen to 15.1 percent. Indicator 0.5 (a broader measure of domestic revenue to GDP) was to increase from 22.2 percent (base) and by Year 3 it stood at 23.2 percent. 5 IMF, Article IV, op. cit., p. 9, and comments made to the evaluation team by FRPFM project officials. 6 See p. 9 of Deloitte’s contract. 3 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV No targets are provided for indicator 0.3, a measure of private-to-public sector funds, but the Year 4 Activity Monitoring, Evaluation, and Learning Plan (AMELP) document mentioned that these numbers are TBD (page 41). A preliminary implication of these “high level” indicators is that the project has had little if any impact on some of the most fundamental PFM targets: debt levels, deficits, and tax collections. At the same time, one must recognize that these are broad indicators, and it is possible that anticipated positive project impacts were overtaken by countervailing impacts. Component 1: Revenue Performance The MEL plan provides seven indicators related to revenue performance, all bearing on tax administration. The two most important, in the sense of increasing tax revenue, are 1.1 (tax arrears to tax collection) and 1.7 (stop filers to the number of taxpayers). Indicator 1.1 was targeted to decrease from 55.5 percent (base) to 49 percent. As of Year 2, the most recent year available, it was at 52 percent. Indicator 1.7 was to decrease from 35 percent (2016) to 20 percent. As of Year 3 it had worsened by increasing to 36 percent. In short, there was little or no improvement in these two key indicators. It is noteworthy that there was progress for “online” activity. These are for the percentage of tax revenue paid on-line (Indicator 1.3), the percentage of tax returns submitted electronically (Indicator 1.5), and the percentage of tax refunds paid on￾line (Indicator 1.6). It is not stated to what extent, if any, these changes led to net increases in tax revenue or to net reductions in costs of tax administration. There was a small decrease in time required to file a corporate tax return, but the reduction remains below target and increased at Year 3 compared with Year 2. Indicator 1.2 concerns the number of tax measures recommended, which was on target except for Year 2. The evaluation team recognizes that number counts are easy to tabulate, but that the true value of such an indicator depends on the individual components counted. Without more information as to what is being counted, it is difficult to evaluate what this indicator fundamentally measures, such as in new revenue collections.7 Component 2: Budget Efficiency and Transparency 11 indicators are given for budget efficiency and transparency, but these draw no clear picture of improvement. A summary of these 11 indicators is as follows: ▪ Indicator 2.1 (percentage of Government of Jordan (GOJ) institutions fully implementing IPSAS): The target is 100 percent, but as of Year 4, no progress is evident. ▪ Indicator 2.2 (percentage of total public expenditure to GDP): no targets are provided. ▪ Indicator 2.3 (score in the Open Budget Survey): The actual value (at 63 in Year 3) is approaching the target of 65. ▪ Indicator 2.4 (value of Public-Private Partnerships (PPP) transactions recommended): The actual value of $700 million in Year 4 substantially exceeds the target of $250 million. The evaluation team notes that “public-private partnerships” represent an activity peripheral to budget efficiency and transparency (as would be included under Public Expenditure Financial Assessment (PEFA) pillar 2). A better fit would be pillar 3, “management of assets and liabilities,” as referenced by the EU.8 ▪ Indicators 2.5 and 2.6: Little progress is evident. The MOF was close to fully implementing Government Financial Management Information Systems (GFMIS) at the project’s start for the general budget. However, independent budgetary institutions do not appear to be using GFMIS as of Year 4.9 7 Upon reviewing the draft evaluation report, FRPFM officials noted that at least some of these tax measure recommendations follow from IMF requirements. 8 See EU, op. cit., p. 12. 9 Without disputing the MEL plan results, FRPFM officials stated upon reviewing the draft report that “independent budgetary institutions that were of ‘commercial’ nature were fully integrated through the GU portal—they are on full accrual accounting and should not be integrated into GFMIS fully; remaining 4 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ▪ Indicator 2.7: The target for the whole-of-government M&E framework was achieved in Year 4. This means that 100 percent of GOJ line agencies include a narrative linked to the national strategy that can be measured as part of the M&E framework. ▪ Indicator 2.8: FRPFM almost fully achieved its total for “gender responsive budgeting” by Year 2. This means that most GOJ institutions “articulate the linkage between financial plans (budgets) and gender and youth outcomes in their budget requests.” As stated in the MEL plan (p. 74), this refers to ministries and departments that use gender indicators to disaggregate data by gender for beneficiaries and employees. Thus, there is better tracking of budgetary results by gender. However, this does not mean that budgetary expenses and revenues and the allocation of budgetary resources have been changed to promote gender equity. ▪ Indicator 2.9 (number of financial transparency measures implemented): On target (but see comment above for indicator 1.2 regarding the difficulty in interpreting the meaning of “counting numbers of measures.”) ▪ Indicator 2.10 (number of pre-feasibility studies by the PPP unit): Above target (but see comment for indicator 1.2 regarding the value of such a measure). ▪ Indicator 2.11 (number of PPP transactions recommended to the PPP Council): Above target (See comment for indicator 2.4.). Component 3: Fiscal Sustainability The six indicators for fiscal sustainability are as follows: ▪ Indicator 3.1 (number of laws, etc.): Above target. (See comment for indicator 1.2 regarding the value of such a measure.) ▪ Indicators 3.2 and 3.3 relate to “out-turns”: For expenditures and for total budget revenues, targets for both show gradual declines over time (with an unexpectedly large jump for revenues in Year 4). Actual values for both were consistently too high. There are two indicators tracked by the PEFA (see discussion for question 3, and also presented in Annex 1, section 6). Aggregate expenditure out-turns already had an excellent score at the project’s start (A); revenue out-turns were a modest C+ at project start. ▪ Indicator 3.4 (governorate staff with increased PFM capacity): The target was to achieve 70 percent with increased knowledge and understanding of PFM. The FRPFM Indicator Table (appendix 1 in the final report (p. 41)) shows that there was no increase in the level of knowledge and understanding of public financial management for governorate staff through June 30, 2020. ▪ Indicators 3.5 and 3.6 pertain to the value of grants disbursed to local research organizations, and number of reports, etc., produced by these organizations. Both are on target, but the value of expenditures at Year 2 on grants exceeded the target by 41%. However, there is no indication about the quality of the reports, the sustainability in funding for local research organizations, or the impact such reports have on PFM in Jordan. Component 4: Exit Strategy 11 indicators are offered for Component 4, which is a general category that bears on the ability of the GOJ to carry on in PFM reform once FRPFM ends. This was a foundation of the contract, which states, (p. 19), “In sum, with the close of this activity, the GOJ will be able to carry on its fiscal and public financial management responsibilities independently, excluding external shocks such as increased regional instability or widespread financial crisis.” Many of these indicators GUs were being integrated either through merger with existing departments or through actual connection to the system.” 5 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV pertain to training in general or to Deloitte’s proprietary Capacity Performance, Results, Sustainability (CYPRESS), in particular. A summary follows: ▪ Indicator 4.1 (GOJ staff with increased capacity in PFM): Close to target except at Y3. However, it is not evident how this training will lead to improvement in more fundamental PFM reforms (e.g., as in the PEFA or Tax Administration Diagnostic Assessment (TADAT) tool). ▪ Indicator 4.2 (GOJ directorates that can use CYPRESS): Close to target, but no enhancement at Y4. See comment for indicator 4.1. ▪ Indicator 4.3 (increased knowledge of gender equality principles): Above target until Year 4 (afterwards, the value drops to zero; the evaluation team suspects a data problem). See comment for indicator 2.8. ▪ Indicators 4.4 - 4.7: These indicators measure person-hours of training for each of the four components. In two cases there are no targets (4.5, 4.7); for 4.4 and 4.6 actual values are substantially under target. The target of 150,000 person-hours of training was a mandatory target in the contract (p. 13). ▪ Indicators 4.8 and 4.9 concern CYPRESS: Both are near or above target. See comments for indicator 4.1. ▪ Indicator 4.10 (community meetings on gender equality): No progress. See comment for indicator 2.8. ▪ Indicator 4.11 (number of legal instruments for gender equality): On target. See comment for indicator 2.8. 2. QUESTION TWO Assess the overall effectiveness of FRPFM’s approach, specifically, whether and how did the project achieve its intended outcomes till now? What are the main challenges? Has FRPFM been reacting to these challenges as required? Are there any recommendations based on best practices? The desk review for Phase 1 has relied primarily on project documents in drawing preliminary evaluation conclusions about FRPFM and recommendations for the follow-on PFM project. The analysis of the MEL plan (see question 1) showed that the project made little, if no progress, against high-order bench-marks (e.g., for debt, deficits, tax revenues collected). In principle, the MEL plan is the standard for measuring project performance, and is developed at the project’s beginning in a way to carefully measure and track the “anticipated results” that are stated in the contract.10 The evaluation team recognizes that there may be imperfections in the MEL plan, and that some measures are not under the direct control of the project. It would be unrealistic to expect project activities to have an immediate impact on such measures. In part to compensate for this, the evaluation team also studied the project’s design, its activities, and its accomplishments “on their own merits.” FRPFM’s overall approach reasonably follows many of the guides to public financial management. One of the most accessible and readable guides has been produced by USAID through the Office of Economic Policy.11 This guide was written by Deloitte Consulting, LLP under contract to USAID as a task in the Leadership in Public Financial Management (LPFM) contract. The four components of FRPFM and how to approach them conform well in many ways to guidance provided in this handbook. In particular, the guide notes that “there is no singular formula for PFM reforms that can guarantee success. Nevertheless, better knowledge of existing PFM systems and processes as well as standards can improve the results.” In short and as a starting point, FRPFM begins from a sound structure. Measuring performance is challenging and all available evidence should be considered. The MEL plan is one source of 10 See the signed contract between Deloitte Consulting, LLP and USAID, pp. 13-19, April 1, 2016. 11 See USAID, Guide to Public Financial Management, Revision 1, April 2015. Also see the summary of this guide in Annex 1. 6 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV information, but other data should be reviewed, as well. To this end the evaluation team reviewed the project’s structure and accomplishments as reported in annual work-plans and annual reports. The basic structure for FRPFM is as represented in Box 1 (see Annex 2). The basic structure is straightforward. The four components are defined at the “one digit” level. For example, Component 1, defined as “revenue performance improved through effective tax policy and administration,” is “1.” At the two-digit level under Component 1 are two subcomponents (tax administration, 1.1, and tax policy, 1.2). Each subcomponent has two or more subdivisions at the “three-digit” level, and each three-digit classification can have numerous tasks or activities at the “four-digit” level. Tasks and activities at the four-digit level number in the hundreds: 168 for Year 1; 185 for Year 2; 198 for Year 3; and 139 for Year 4. These elements are useful for project management, but do not provide much perspective for an evaluation. For example, some of the “deliverables or milestones” defined at the four-digit level include “recommendations provided,” “impact to GBD communications strategy,” and “improve Oracle process.” These may help a team leader manage work-flow, but say little in themselves to external performance evaluators. However, the three-digit elements can, in principle, be helpful for evaluative purposes since most of these three-digit units reflect important PFM functions. The “anticipated results” called for in the contract could logically fit within this level of detail.12 However, tracking progress at the three-digit level is not possible because the annual reports do not present results at this level.13 The evaluation team used its own judgement to decide how to track project accomplishments. To generate a stock-taking of project accomplishments, the evaluation team culled what it believes were the most important project accomplishments by year. These are shown in Box 2 (see Annex 2), which uses the contractor’s own words with little revising. The evaluation team assigned to each activity what it believed to be the most relevant design area at the three-digit (sometimes two-digit) level. Even this analysis, however, does not draw a clear picture of accomplishments that would reflect essential PFM results (such as stated in the contract's anticipated results). The reason is that in most cases (with exceptions for some indicators in the MEL plan) there is no statement of final objective or milestones. In other words, the evaluation team’s exercise in stock-taking as a way to better elucidate what was not captured in the MEL plan comes up short. Some examples suffice to show this. Consider the three-digit activity 1.1.5: “optimize risk-based tax audit.” The evaluation team found three outputs or outcomes for 1.1.5—two during Year 2 and one during Year 3 (see “significant accomplishments” in Box 2). In Year 2 the project provided a recommendation to create a database for Goods and Services Tax (GST)-exempt taxpayers, and conducted a strategy regarding corporate income taxpayers with sales revenue exceeding JD 1 million that were not registered. The project also organized workshops for the Income and Sales Tax Department (ISTD) on audit risk management. In Year 3 the project recommended using risk-based practices for payment of GST refunds. The evaluation team would agree that these would appear to be worthwhile activities. However, the annual reports do not indicate targets or measures for these activities. In general, one could make similar determinations for most three-digit level activities. The evaluation team conducted key informant interviews with all current component leads. In each interview, the evaluation team probed the component lead’s understanding of his or her component down to the three-digit (functional) level. In only a few cases did a Team Leader explain project results over time in terms of targets hit (or missed). An exception was for taxpayer services under Component 1 (1.1.4; see Box 2 that shows results for e-services). These taxpayer services are also in the MEL plan as indicators 1.5 and 1.6 in Annex 3. This 12 The evaluation team notes, however, that this was not done. There are 39 anticipated results stated in the contract. A few of these are in the MEL plan, but many are not and are not explicitly tracked in the annual reports. For this reason, the evaluation team will attempt to obtain further information about the anticipated results during the field research in Phase 2. 13 Moreover, the annual reports do not track progress in terms of the “anticipated results” in the contract. 7 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV exception, however, underscores one of the evaluation team’s important findings: if performance was not tracked in the MEL plan, it was not tracked. For many of the functional (three-digit) work areas, the contractor provided no systematic tracking of its work. For most of the three-digit functional areas there are no targets and no milestones. The contractor did not explain the connection between its project design and the “anticipated results” called for in the contract. Finally, there is no elucidated correspondence between the contractor’s functional design (i.e., three-digit functions) and the MEL plan. In short, the evaluation team is left with little besides the MEL plan to assess progress. What the evaluation team believes is that for many anticipated results, there was no well￾structured monitoring framework to track performance and that instead a “best effort” approach (“level of effort”) was used to implement FRPFM, with limited attention to monitoring. This was a “time and materials” contract and payment was not based on performance.14 Much of the contractor’s effort could have been at a good-to-high level of performance. However, neither the MEL plan nor a detailed stock-taking of project accomplishments from the annual reports allows the evaluation team to confirm this. The team believes there are ways to consistently and accurately monitor a public financial management project such as FRPFM that follow international practice. We discuss these under Question Three. 3. QUESTION THREE Compared with the reform activities in other countries and international best practice, what are the main recommendations that need to be taken into consideration in the upcoming Public Financial Management and Administration project? (Please provide general as well as specific recommendations related to each area, as follows): ▪ Revenue performance ▪ Budget efficiency and transparency ▪ Public expenditure management ▪ Structural reforms ▪ Fiscal sustainability ▪ GOJ institutional capacity and human capability to sustain reforms It is likely that the follow-on project will continue with numerous activities initiated by FRPFM.15 What is needed in the new project is a sound performance framework that can guide and monitor activities. Since FRPFM did not use such a framework, the new project lacks a well￾grounded gauge regarding the status of the PFM functions that FRPFM intended to address. In all areas of FRPFM activity, the evaluation team asked Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs) what needed to be done now that FRPFM is ending. In almost all areas there is work to do, reinforcing the team’s conclusion that many MEL targets were not attained and that many “anticipated results” called for in the contract are incomplete. The evaluation team will have more to say on this following completion of KIIs and FGDs. In addition, the evaluation team recommends that the new project assess the current status of PFM to better identify specific interventions to focus on. To this end, the evaluation team recommends that a framework structured along the lines of the existing PEFA methodology be used. The PEFA for Jordan is managed and funded by the European Union (EU), and uses standards and procedures accepted broadly by the international community. There are important reasons for adopting such a methodology. PEFA is widely accepted by the international community and has a track record for Jordan, with PEFAs completed in 2011 and 2016. The PEFA framework consists of seven pillars with 31 performance indicators (PIs). These cover all essential elements of a public financial management system, including strategic 14 See Deloitte contract, op. cit., p. 5. 15 Field research from Phase 2 will provide further information on recommendations based on information from the KIIs and FGDs. 8 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV planning, medium-term expenditure frameworks, annual budgeting, revenue management, control, accounting, reporting, monitoring and evaluation, and audit. Most if not all activities undertaken by FRPFM could be incorporated into PEFA’s 31 performance indicators. The new project could use a design structure similar to FRPFM with components, subcomponents, and more detailed functional or activity elements. However, in order to identify the most appropriate interventions and for informative monitoring and evaluation, these activities must clearly correspond to PEFA-like indicators. The new project may deem that not all aspects of PEFA are most appropriate for its design, and could modify this framework to incorporate other indicators, such as from the Tax Administration Diagnostic Assessment (TADAT) tool. TADAT focuses on tax administration, and its indicators could be used for tax administration activities. FRPFM did employ several PEFA-related indicators, for example, indicators 3.2 and 3.3 shown in Annex 3, measuring expenditure and revenue out-turns. The evaluation team provides the following illustrations as examples as to how the new project might structure itself for improved interventions and to follow stronger monitoring and evaluation in each program area. Component 1: Revenue Performance As an example for revenue performance, performance indicator 19 (PI-19, “revenue administration”) in the PEFA provides four subindicators that bear directly on activities that were conducted in FRPFM: taxpayer information, compliance risk, compliance audit, and arrears monitoring. The overall score for PI-19 in 2016 was a weak “C” (with A highest and D lowest). It would be highly informative to know how this grade may have changed, if at all, through activities of FRPFM. Knowing how—and why—this indicator changed would provide valuable intelligence in tailoring subsequent interventions. The MEL plan through indicators 0.4 (tax collections to GDP) and 1.1 (tax arrears to collections) (see Annex 3) suggests little progress was made and it is apparent that activities related to PI-19 would be good candidates for the new project. Component 2: Budget Efficiency and Transparency There are numerous performance indicators for budget in the PEFA. For example, PI-5 concerns “budget documentation.” The evaluation team learned from its desk study review of FRPFM that developing an organic budget law was achieved in Year 1 (see results for two-digit category 2.2 in Box 2). What the team cannot determine from available documentation is how this mattered. PI-5 measures why developing such a law is important, and focuses, for example, on basic budget elements such as these: forecast of the fiscal deficit or surplus; previous year’s budget out-turn presented in the same format as the budget proposal; current fiscal year’s budget presented in the same format as the budget proposal; and aggregated budget data for both revenue and expenditure according to the main needs of the classifications used. The two MEL plan indicators used by FRPFM (3.2, 3.3 in Annex 3) related to PI-5 deteriorated and would be relevant focus areas for the new project. Component 3: Public Expenditure Management Subcomponent 2.2 in FRPFM addressed increased GOJ adoption of public-private partnerships. The evaluation team’s stock-taking exercise identified considerable progress in this subcomponent, such as in identifying projects, conducting feasibility studies, and establishing a PPP pipeline. The MEL plan, through indicator 2.4, shows that the “value of PPP transactions recommended to the PPP council” was considerably above target. PEFA’s PI-11 (public investment management) bears directly on the types of activities conducted in subcomponent 2.2. It is important to know whether the scores changed because in themselves these targets in the MEL plan are arbitrary. The overall grade for PI-11 in 2016 was poor, at D+, leaving the evaluation team with the question of “Did FRPFM develop institutional changes to sustainably increase these scores?” Component 4: Structural Reforms 9 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV FRPFM’s Component 3 focused on fiscal sustainability through structural reforms. How far did the project get? From the MEL plan, little progress was made at the governorate level (subcomponent 3.2).16 What about for subcomponent 3.1 (improved fiscal policy formulation and execution)? Our stock-taking shows that considerable effort went into building a macro-fiscal database, developing economic models, undertaking research, and so on. In the interviews the evaluation team heard numerous informal, qualitative comments that suggested the same. However, there was little to suggest this work was sustainable, and certainly no evidence of concise tracking of progress. PEFA’s PI-14 covers one aspect of this (macro-economic and fiscal forecasting). The overall grade in 2016 was a modest C+. Did this improve from FRPFM’s considerable effort in this domain? From the documentation available to the evaluation team, it cannot be determined. However, intelligence from PI-14 would hold part of the answer and could suggest interventions for improvement. Component 5: Fiscal Sustainability Results-oriented budgets (ROB) were the focus of FRPFM’s subcomponent 2.1 (stronger GOJ auditing practices and results-oriented budgeting). A sound ROB provides valuable information to policy makers as to the impact and sustainability of government actions. None of the interviews provided evidence that any progress was made in ROB, and there is no indicator in the MEL plan to track it. However, PEFA’s PI-8 addresses these questions through measures of performance for service delivery, performance planning, whether performance was achieved, resources available for service delivery, and how well performance is evaluated. Jordan’s overall score in 2016 was a weak C. Did performance improve through FRPFM in ROB? We do not know, but a measure such as PI-8 would help answer this question in the follow-on activity. Component 6: GOJ Institutional Capacity and Human Capability to Sustain Reform FRPFM’s Component 4 in its exit strategy concentrates attention on institutionalizing training (4.1.3) and in applying its CYPRESS maturity planning (4.1.5). FRPFM was to be the capstone of PFM assistance to Jordan, such that the government could carry on independently at the close of the activity (See citation above in question 1). How far did the project actually go? The accomplishments sections in the annual reports show some of this (e.g., for IPSAS in Year 3). Training results reported in the MEL plan provide a number of person-hours in training for the project’s four components (see indicators 4.1 – 4.9 in Annex 1). However, actual training fell far short of targets. Several project officials stated that these targets were “way too high,” even though these were specifically identified in the contract (p. 13) and the contractor was no stranger to Jordan, having implemented Fiscal Reform 1 (2006 – 2009). Nonetheless, the new project should build on this training and institutionalize such training, either within the government of Jordan or through a local university. Continuing to monitor institutional maturity would also be recommended, although CYPRESS itself may or may not be the approach to take. This is a proprietary tool of Deloitte that may not be relevant or even desirable for the new project. The evaluation team believes that incorporating a PEFA (and TADAT) like approach to performance would be highly useful and cost-effective. By comparison, the 2016 PEFA was conducted by a three-person team over a period of about four months. Establishing such a framework from the project’s beginning—and maintaining it throughout the life of the project— would provide guidance in the choice of activities and in monitoring performance in the context of well-established international norms. 4. QUESTIONS FOUR AND FIVE TOGETHER 16 The evaluation team has subsequently learned that the project provided technical support enabling the governorate of Irbid to conduct budgetary outreach with the GBD, and that the project has developed budget guidelines for the governorates. The impacts of these activities are not clear, and none of them found their way into the contractor’s indicator reporting. 10 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV Question 4 Which interventions under FRPFM, in case any, can be expected to sustain over time? Why and how? Question 5 Looking at the work done during the previous years by USAID’s FRPFM project on institutional development, capacity building and administrative reform; was it beneficial and sufficient to achieve the intended results and sustainability? Are there any recommendations based on the best practices to improve those efforts? The responses to questions 1, 2, and 3 shed considerable light on the evaluation team’s response to questions 4 and 5. Analysis of the first three questions shows that results from the MEL plan alone indicate modest overall project impacts. In addition, the contractor’s reporting through annual reports does not in most cases permit the evaluation team to discern which institutional changes put in place were having important effects that would likely continue to be effective in the future. The reasons for this are noted in detail in the responses above. This is unfortunate, given that established and informative measures for PFM are available and have been used in Jordan for the past decade. However, the evaluation team’s stock-taking exercise does suggest that numerous activities under FRPFM appear to be productive, are likely to be sustainable, and should be encouraged and supported in the follow-on project. The following is an illustrative list of activities that were making notable progress and would be good candidates for continuation. It is not a comprehensive list and is one the evaluation team will add to during in-depth field research in Phase 2. ▪ E-services, such for electronic filing, electronic payments, and electronic refunds (1.1.4 in the stock-taking exercise). The increased use of e-services is impressive, appears to be sustainable, and should be supported in the follow-on project. ▪ Gender responsive budgeting (2.1.5). FRPFM has initiated Gender-Responsive Budgeting (GRB) across GOJ ministries. To date, the impact of GRB appears minimal (e.g., so far there are no apparent programmatic or budgetary implications), but minimal reporting across ministries has been initiated. ▪ PPP certification and training (2.3.1). The large gains in PPP pipeline growth and PPP project development and approval can likely be sustained through the PPP unit established by FRPFM and the PPP training and certification achieved. The follow-on PFM project could continue to support this work.17 ▪ The substantial work on the macro-fiscal database (3.1.3), macro models (3.1.4), and other capacity building in the Studies and Economic Policy Directorate (SEPD) could and should continue through the staff training initiated by FRPFM. Although training-level results were substantially under the targets (see MEL indicators 4.3 – 4.7), the concept is valid and should continue. ▪ More general capacity building across the MOF and into other ministries as pertains to PFM should continue. This includes at the governorate level. ▪ FRPFM placed considerable emphasis on the contractor’s propriety CYPRESS approach to institutional capacity building. On the basis of project reporting, good progress appears to have been achieved as to the “maturity state” of the SEPD, PPP unit, and ISTD. The follow-on project should carefully review these results and assess whether CYPRESS or some other approach would be relevant to the follow-on project. 5. QUESTION SIX What are the main reasons for deviations from submitted design and work plans, and have the deviations changed the intended outcomes? Include: areas, activities and approaches that have been less effective. 17 The evaluation team has learned that no one was fully certified for the Certified PPP Professional (CPEP). Many took the foundation course, but few did part 2. 11 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV The overall design of FRPFM remained relatively constant throughout project implementation. The four components and their subcomponents down to the three-digit level did not change much. There were modifications at the three-digit level (see Box 1 in Annex 2) for 2.1.3 and 2.1.4 (audit), 2.1.5 (gender responsive budgeting), 2.1.6 (results-oriented budget), 2.3.3 (PPP), and 3.3.3 (PPD). The Department of Lands and Survey was introduced in the second year, and work with the Department of Customs and the Ministry of Telecommunications were added in the fourth year as Component 5. These modifications, however, appear minor. A more important question, however, concerns the absence of well-defined project objectives beyond the general guidance provided in the contractor’s statement of work and as specified in the “anticipated results.” Several work areas discussed above (questions 4 and 5) are promising areas for continuation. But more generally, much remains to be done. To set the discussion, the evaluation team reviewed all of the anticipated results called for in the contract (pp. 13 – 19). There are 39 anticipated results, as shown below (by component). We indicate in parentheses for each result whether the evaluation team concludes that the result was attained, partly attained, unclear (but probably not attained), or not attained. Many of these were not tracked by the contractor, and our conclusions relied in part on information gathered in the KIIs and FGDs.18 Component 1 ▪ New income tax law implemented (yes) ▪ Increased tax revenues as a percent of GDP and revenue mobilization enhanced through increasing the number of tax registrations and active tax payers, and reducing the number of stop-filers (partially attained) ▪ Increased collections or reduced value of outstanding tax arrears (partially attained) ▪ Improved use of risk-based audits (yes) ▪ Reduced variance between forecast and actual revenues (no) Component 2 ▪ Increased number of performance audits performed to international audit standards (not clear)19 ▪ Improved quality of internal controls within line ministries (not clear) ▪ Fully executed internal audits within key line ministries. By key line ministries, USAID is referring to those ministries that have the largest budget lines. This includes but is not limited to the Ministry of Education, Ministry of Health, Ministry of Public Works, and Ministry of the Interior (not clear) ▪ GOJ full implementation of IPSAS; GOJ fiscal accounts transitioned from cash basis to accrual basis (no)20 ▪ GBD fully fledged monitoring and evaluation system that is operational and run by GBD staff (not clear) ▪ Improved measures of public spending as a result of full adoption of Results-Oriented government across the GOJ (no) ▪ Full utilization of gender-responsive budgeting (yes) 18 Upon reviewing the draft report, FRPFM officials noted that “it is important to address in this question work plan changes and changes in project priorities, such as debt work was taken up by US Treasury, work on IPSAS had to be revised and reoriented because of lack of preparation of GoJ prior to implementation, IMF programs, changes in leadership, lack of commitment to PFM and DRM reform plans drafted by other donor projects (EU, etc.).” The evaluation team would not disagree with these, and adds other challenges faced in answering questions 4a, 4b, and 4c in phase 2. 19 The extent of work appears to be that FRPFM assisted so that internal audit policies and procedures would be in line with internal audit (IA) standards, with partial testing of IA procedures conducted on MOF Treasury Directorate. 20 The extent of work appears to be that FRPFM relaunched MOF’s road map as the official road to IPSAS implementation, and assisted the IPSAS PMO in applying IPSAS to test the PP&E policy using MOF as the test pilot. 12 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ▪ All GOJ agencies utilizing GFMIS for budget preparation (covering 100% of public expenditures) (no)21 ▪ More effective budget planning and more efficient implementation, leading to higher budget execution rates and reduced number of transfers between budget heading (partially attained) ▪ Improved transparency of budgets, policies, and procurement practices (partially attained) ▪ PPP unit is operational and business processes and functions established (HR, procurement, etc.) (partially attained)22 ▪ Increased number and value of PPP transactions completed or in pipeline. At a minimum, the PPP unit should leverage eight transactions with a total value of $250 million at the end of this activity (partially attained)23 Component 3 ▪ Institutionalize policy analysis and research within MOF (no) ▪ High quality analytical reports on fiscal policy produced by MOF’s own staff (no) ▪ Reliable and publicly available forecasts on economic conditions produced by MOF’s own staff (no) ▪ Adoption of the next Medium Term Debt Strategy (MTDS) (no) ▪ Institutionalize MTDS and increased GOJ capacity to conduct periodic reviews and updates (no) ▪ Reduced cost and risk of managing government debt (unclear) ▪ Diversification of debt management tools (unclear) ▪ Increased average maturity of Jordan’s domestic debt from two years currently to three years in the medium term (unclear) ▪ Reduced cash deficits in governorates (no) ▪ Improved service delivery and coverage of public services through efficiency gains and better resource utilization (no) ▪ Increased transparency of budgeting and expenditures plans at the local level (no) ▪ Increased financial and budgetary capacity to handle donor financing (unclear) ▪ Increased capacity to plan and execute decentralized budget components resulting from the adoption of the new decentralization law (no) ▪ High quality applied economic analysis produced outside of the GOJ (no) ▪ High quality applied gender analysis is it relates to economic issues produced and utilized to inform decision and policy making (no) ▪ Public policy decision or choices informed by external analysis and advocacy (no) ▪ At least three independent local research institutions will be sustainable and no longer require USAID funding for technical and financial capacity building (no) Component 4 ▪ GFMIS made fully sustainable by the GOJ, both operationally and financially (no) ▪ PPP unit within MOF fully staffed by MOF and capable of executing PPP transactions independently (part, moved to PMO) ▪ GBD fully fledged monitoring and evaluations system operational and run by GBD staff (no) 21 For general budget Institutions, 29 GUs were added in year 4 after achieving the target of 100% in year 3. For independent budget institutions, FRPFM found it too difficult to bring the independent budgetary units into the current GFMIS, e.g., due to different uses of cash and accrual accounting. 22 The old PPP law established the unit. FRPFM built its capacity, but none of the core team who were trained moved to the new unit at the PMO. 23 The PPP unit at the MOF closed on no PPP projects. FRPFM gave technical support on 13 projects. Three projects in the pre-application assessment Phase, and 10 projects that have been either put on hold or terminated as a result of not meeting PPP criteria. 13 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ▪ Government-led public-private dialogue for engagement on fiscal and budget performance (no) ▪ Develop and implement a testing mechanism based on a milestone plan that would biannually evaluate progress towards sustainability of the assistance provided to GOJ counterparts (no) The evaluation team learned through the interviews that several anticipated results were dropped from the project (three debt results for component 3), but otherwise the anticipated results remained as called for. Of the 39 anticipated results, six were attained, three partly attained, four not clear, three dropped, and 23 not attained. In short, this overview can help focus attention on at least 33 areas where the new project could pick-up where FRPFM ended. The sustainability of many of the accomplishments shown in Box 2 (Annex 2) are not apparent. The evaluation team did not have time (or documentation) to probe the sustainability of many of these, but a sampling gives an idea of the type of concerns that the team has in that achieving a project “deliverable” is not at all the same thing as hitting an MEL target or satisfying a contractual anticipated result. ▪ The project’s developing forms for ISTD (1.1.2). It is not evident whether ISTD developed the capacity to create such forms for itself. ▪ The project’s writing reports for ISTD regarding, for example, the general sales tax (1.1.5). Can ISTD now write such reports itself? ▪ The project’s drafting an income tax law for the MOF (1.2.2). The MOF must draft and take ownership to be sustainable and be capable to modify and update the law or items in the law. Is this now the case? ▪ The project’s completing public expenditure reviews for the GBD (2.1.6). Can GBD now do this itself? ▪ The project’s developing a fiscal decentralization white paper for the Council of Ministers. However, the project’s impact at the governorate level appears to be minimal, at best. Outside of a well-defined performance framework, such as could be provided by a PEFA-like structure, discerning deviations and their causes is difficult. The evaluation team believes this could have been done under FRPFM. The contractor’s scope of work provides informed guidance as to anticipated results. As an example, consider anticipated results for subcomponent 3.1 (improved fiscal policy formulation and execution; Deloitte contract, p. 17). There are eight anticipated results for 3.1. In considering just two of them, the second bullet identifies “high quality analytical reports on fiscal policy produced by MOF’s own staff.” How many of these were produced each year and at what level of quality? Does the staff have the capacity now to write these on their own? The evaluation team saw reporting on this question but found no way to assess performance improvement. The evaluation team’s conclusion is that the contractor’s scope of work was, in general, clear on expectations and as outlined in the anticipated results. A performance matrix could have tracked these year-by-year. However, without such methodical monitoring, it is difficult to determine whether and to what extent FRPFM achieved many of the anticipated results. Field work in Phase 2 will further address these gaps. 14 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ANNEXES ANNEX 1: SELECTED LITERATURE REVIEW The following sections provide background and context for the evaluation conducted in phases 1 and 2. Section 2 reviews Deloitte’s contract. Section 3 offers background on the government of Jordan’s public financial management reform strategies over the years 2010 – 2020. USAID’s involvement in fiscal reform in Jordan is summarized in section 4, while section 5 briefly summarizes USAID’s Guide to PFM—an essential document for any mission wishing to implement a fiscal reform project using sound international practice. Sections 6 – 8 summarize several other essential international documents, including the PEFA, Doing Business Reports, and TADAT. Sections 9 – 12 overview other important government of Jordan documents, and is followed by summaries of the IMF’s stand-by arrangements for Jordan. The last two sections summarize other relevant USAID experience in PFM from El Salvador and Tunisia. 1. SECTOR CONTEXT The key public financial management (PFM) organizations and stakeholders in Jordan are as follows: the Ministry of Finance, with policy and treasury responsibilities; the income and sales tax directorate, with income and sales tax responsibilities; the general budget department, with budgeting responsibilities; the audit bureau, with external audit responsibilities and currently also involved in internal audit functions; and the anti-corruption commission, an independent civil body that deals mainly with anti-financial corruption in the public and private sectors. The income and sales tax directorate and the general budget department report directly to the ministry of finance but operate semi autonomously. 2. USAID JORDAN FRPFM CONTRACT (2016) SCOPE OF WORK USAID/Jordan’s Fiscal Reform and Public Financial Management (FRPFM) project is a four￾and-a-half year activity implemented by Deloitte to end the cycle of donor dependence and pave the way for a donor “exit strategy” in relation to independent public financial management (PFM) in Jordan. It is valued at $35,868,470 and began implementation in April 2016 (end of project is September 2020). The following sections are summarized from the USAID/Jordan FRPFM scope of work to provide context for components and activities that will be evaluated. A) BACKGROUND IN JORDAN At the time of project award, Jordan had experienced several shocks related to disruptions in gas from Egypt, which caused an increase in imports from other countries, as well as an influx of Syrian refugees. Together these put a strain on the economy, slowing GDP growth and increasing the fiscal deficit. The public debt of Jordan increased at a rate of 12 percent annually from 2006-2014. Jordan was also confronted by a trade imbalance, due in part to low international prices for key commodities, the disruption of trade with Syria, and severed transit routes to Turkey, Lebanon, and Europe. It was within this context that USAID began to assist the government of Jordan in public financial management, including the Jordan Fiscal Reform I and II projects, and the Jordan Fiscal Reform Bridge Activity. The subject of the current evaluation, the Jordan Fiscal Reform and Public Financial Management Project, was also awarded within this context and in support of USAID/Jordan’s Country Development Cooperation Strategy (CDCS) 2013-2017, which focused on establishing sound PFM practices to preserve macroeconomic stability and support inclusive growth. 15 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV B) FRPFM COMPONENTS In assisting the government of Jordan to advance PFM reforms and expand PFM capacity, the following components were prioritized under FRPFM: 1. Revenue performance (taxpayer registration, services, audits, collections, management of arrears, taxpayer feedback, tax policy analysis, legal drafting, and tax statistics) 2. Budget efficiency and transparency (internal control, international public sector accounting standards (IPSAS), management of arrears, external and internal audits, results oriented budgeting, and gender responsive budgeting, government financial management information system (GFMIS) 3. Fiscal sustainability and structural reforms (fiscal forecasting and analysis, government debt management, subnational fiscal management, public-private dialogue on fiscal issues, grant management, and nongovernmental fiscal research) 4. Exit strategy (institutional maturity assessment and action planning, implementation letters, M&E, publicizing of progress, and donor coordination) 3. GOVERNMENT OF JORDAN’S PFM REFORM STRATEGIES A) GOVERNMENT OF JORDAN’S PFM REFORM STRATEGY, 2010 – 2013 Jordan’s Ministry of Finance (MOF) developed a new element of its strategic plan entitled “overarching financial Management Reform for Jordan’s Public Financial Management 2010– 2013,” which includes tables of performance indicators covering the MOF and its general budget department, income and sales tax directorate, and the audit bureau to be monitored based on the Public Expenditure and Financial Accountability (PEFA) methodology. Individual action plans of the stakeholders also include performance indicators to measure results. The MOF progress report on the strategic plan implementation produced in 2010 continued on a yearly basis throughout this program. Below are the main features of the sector stakeholders' strategies and their accompanying action plans for the period 2010–2013/2014 specifying what the government aimed to achieve in the PFM sector and how. ● Ministry of Finance Strategic Plan 2010–2013 objectives: draw up public financial policy to enhance financial stability and encourage economic growth; reduce public indebtedness; improve the efficiency of control of public funds; promote the level of transparency and disclosure; improve the level of services; and enhance the capacities of Ministry of Finance staff. ● Income and Sales Tax Directorate Strategic Plan 2010–2014 objectives: increase revenues; raise efficiency in managing the tax system to ensure that every taxpayer complies with tax obligations; raise voluntary compliance of taxpayers by increasing tax awareness, improving transparency, and providing quality taxpayer services; develop staff capability; and develop modern and effective information technology to raise efficiency in managing tax system processes. ● General Budget Department Strategic Plan 2010–2013 included among its major responsibilities: prepare the general budget; develop manpower tables, allocate funds to implement policy in accordance with national priorities; evaluate Government of Jordan's programs, projects, and activities; monitor the execution of the budget; and prepare regular analytical reports. 16 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ● Audit Bureau Strategic Plan 2010–2014 included the Audit Bureau’s duties: submit annual reports to the House of Representatives; monitor revenues, expenditures, trust accounts, advances, loans, settlements and warehouses; ensure spending of public funds is legal and effective among others. The Anti-Corruption Commission (ACC) began its work in 2008. The EU supported the drafting of the ACC Strategy 2008-2012. The strategy comprises several components such as strengthening the capacity of the ACC, simplifying the business environment, reforming the public sector, training of public officials, awareness raising and reviewing corruption legislation, and an action plan to implement the strategy has been made. B) GOVERNMENT OF JORDAN’S REFORM STRATEGY, 2014 – 2017 Jordan’s PFM Reform Strategy (2014-17) had four primary objectives: to ensure long-term aggregate fiscal discipline, to develop policy-based budgeting, to encourage economic growth and private sector investment, and to make the government more responsive to ordinary citizens. ● Ministry of Finance and its associated Department’s Strategic Plan 2014–2017 objectives: improve medium-term fiscal forecasting and planning, produce a new public debt management strategy, further extend the GFMIS to include procurement, widen the coverage of the Treasury Single Account, develop financial reporting in accordance with international standards, and improve internal financial control and internal audit throughout the government. ● General Budget Department (GBD) Strategic Plan 2014–2017 objectives: update the General Budget Law to be consistent with results-based budgeting and ensure full consideration of priorities before work begins on detailed budget submissions. Other initiatives include improving the classification and reporting of expenditure, including reclassifying some maintenance expenditure now treated as capital as recurrent, transferring the management of investment projects from Ministry of Planning and International Cooperation (MoPIC) to the responsible Ministries, developing the medium￾term budget framework covering both the main budget and the budgets of the 59 extra￾budgetary Government Units (GUs), undertaking in-depth reviews of important departmental budgets, and ensuring that annual reports by GUs include information about their underlying financial situation and the risks they face of operating losses, bad debts, and payment arrears. ● The Tax Department's Strategic Plan 2014–2017 objectives: improve their organizations in the interests of efficiency in collection and in communication with taxpayers, while the General Supplies Department (GSD) aimed to review the current legislation governing public procurement. The government had also sought the advice of the IMF during the currency of the Stand-By Arrangement (SBA) 2012-15, on a variety of aspects of tax collection, accounting, and financial reporting. These have pointed to the desirability of simplifying tax structures and reducing the extent of exemptions and special preferences, making tax audit less cumbersome and more productive, and consolidating financial reporting of the main budget and GUs consistently with international standards. C) GOVERNMENT OF JORDAN’S PFM REFORM STRATEGY, 2018 - 2020 Overall the PFM strategy was designed to support the National Goals which include: 1) Maintain fiscal and monetary stability and control the deficit; 2) Build an efficient and low-risk fiscal / 17 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV financial system; and 3) Improve the level of service delivered to citizens in an equitable manner. The 2018-2021 Strategy is broader and addresses elements that were not covered in the previous PFM reform programs. The Strategy is designed to address existing systemic shortcomings identified in the assessments. The expected outcomes under each section are fully aligned with the PEFA and TADAT indicators, and activities are aimed at addressing identified systemic shortcomings. Notably, the MOF also commits in this document, for the first time, to the production of annual progress reports. Activities are to be led by a specialist task force to address challenges associated with the budget deficit, the level of public debt as well as tax avoidance and tax evasion. The strategy includes an emphasis on measures to promote and support voluntary compliance as it relates to tax avoidance and evasion. Both received increased attention in recent years, as citizens note concerns of perceived increased tax pressure and cost of living. There is also an Action Plan accompanying the Strategy which prescribes in detail the process of the implementation in each thematic area. For each thematic area in the Strategy, activities are set for every year between 2018 and 2021 and the responsible government body or other relevant institution is identified. Clear oversight and monitoring mechanisms over the implementation of the Strategy are also envisaged. TABLE 1 Pillar 1: Transparency of Public Finances ▪ Key Intervention 1: Strengthen budget classification ▪ Key intervention 2: Broaden coverage of expenditure in financial reports ▪ Key intervention 3: Performance information for service delivery Pillar 2: Management of Assets and Liabilities ▪ Key intervention 1: Strengthen fiscal risk reporting ▪ Key intervention 2: Strengthen public investment management ▪ Key intervention 3: Enhance public asset management Pillar 3: Policy-based Fiscal Strategy and Budgeting ▪ Key intervention 1: Strengthening macroeconomic forecasting ▪ Key intervention 2: Enhance the fiscal strategy ▪ Key intervention 3: Improve consistency of budgets with previous year's estimates ▪ Key intervention 4: Improve budget preparation process ▪ Key intervention 5: Strengthen the legislative scrutiny of budgets Pillar 4: Predictability and Control in Budget Execution/Tax Administration ▪ POA 1: Integrity of the Registered Taxpayer Base ▪ POA 2: Effective Risk Management ▪ POA 3: Supporting Voluntary Compliance ▪ POA 4: Timely Filing of Tax Declarations ▪ POA 5: Timely Payment of Taxes ▪ POA 6: Accurate Reporting in Declarations ▪ POA 7: Effective Tax Dispute Resolution ▪ POA 8: Efficient Revenue Management ▪ POA 9: Accountability and Transparency ▪ POA 10: International Taxation 18 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV Pillar 5: Predictability and control in Budget Execution ▪ Key intervention 1: Improve information on commitment ceilings ▪ Key intervention 2: Significance of in-year budget adjustments ▪ Key intervention 3: Improve procurement complaints management ▪ Key intervention 4: Improve internal controls on non-salary expenditure ▪ Key Intervention 5: Improve internal audit Pillar 6: Accounting and Reporting ▪ Key intervention 1: Enhance financial data integrity ▪ Key intervention 2: Enhance in-year budget reports ▪ Key intervention 3: Enhance annual financial reports Pillar 7: External Scrutiny and Audit ▪ Key intervention 1: Strengthen external audit ▪ Key intervention 2: Enhance legislative scrutiny of audit reports 4. USAID INVOLVEMENT IN FISCAL REFORM AND PUBLIC FINANCIAL MANAGEMENT USAID has already provided significant assistance to Jordan through three other PFM projects: ● Jordan Fiscal Reform I Project (2006-2009) – focused on preparing a tax reform proposal and revising the tax code. Introduced improvements to tax administration and steps to implementing the GFMIS. o Achieved results in building capacity of the Income and Sales Tax Department and the General Budget Department (GBD) o Improved revenue collection and budgetary formulation ● Jordan Fiscal Reform II Project (2009-2014), DAI – focused on seven objectives to improve the efficient use of public resources, improve transparency, monitoring of government activities, and increased efficiency of trading across borders. Also focused on implementation of the GFMIS. o Success in customs, GFMIS expansion and utilization, Audit Bureau capacity and GOJ adoption of results-oriented budgeting ● Jordan Fiscal Reform Bridge Activity (2014-2016) – builds on Jordan Fiscal Reform II ● Jordan FRPFM crucial to achieving objectives in Jordan’s 2013-2017 CDCS, IR 4: Fiscal Stability and PFM Improved under DO 1: Broad-based Economic Development Accelerated 5. GUIDE TO PUBLIC FINANCIAL MANAGEMENT24 USAID’s Guide to Public Financial Management is designed to give USAID field officers a foundational understanding of PFM to empower them to incorporate elements of PFM￾strengthening activities into program and project design. Strong PFM systems are critical for the effective, transparent, and accountable use of public funds and underpin good governance. It is recommended that the Guide be used as a desk reference on the general parameters of good practices in each subject area of public financial management. The references and resources cited in each section can be used to guide users to further details and more extensive materials on each topic. Application of the information available and the references provided should allow 24 See USAID, Guide to Public Financial Management, Revision 1, April 2015. 19 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV for a high degree of rigor in implementation of the program cycle and effective and efficient management, monitoring, and evaluation of projects and programming in public financial management. The Guide is structured around the core elements of the PFM system, which are budget planning and preparation, budget execution, public sector accounting, audit and evaluation. The Guide’s annexes contain two case studies of specific reform programs and lessons learned, a listing of categories of expenditures and functions under the UN Classification of the Functions of Government (COFOG) and the IMF General Financial Statistics (GFS) classification schemes. And, it offers guidance on the selection of performance indicators for PFM-related activities along with some sample indicators. As stated in the Guide, a strong public financial management system is an essential necessary condition for economic growth and government efficiency and effectiveness. It is a key element in increasing accountability, transparency, and legitimacy in governance. PFM includes the mobilization of revenue, allocation of funds, expenditures, accounting, and monitoring and evaluation. Sound PFM systems are fundamental to the appropriate use of donor assistance and donor agreements. Successful implementation of PFM reforms depends on maintaining a focus on fiscal discipline, allocative efficiency, and operational efficiency. To truly embrace PFM reform governments must commit to transparency and accountability to the public and all stakeholders in addition to legal, technical, and capacity enhancements. PFM processes and outputs are structured around the budget cycle and can help ensure that public expenditures are well planned, executed, recorded, monitored, reported, and evaluated. The planning process should engage key stakeholders in a participatory manner and be realistic in goals, timeframes, resources, and other constraints. The budget cycle planning and development processes should start with a macro-economic framework, fiscal framework, and strategic plans that reflect the country’s own priorities and economic context. Key aspects of strong budget execution include a transparent and competitive procurement system, expenditure controls that require adherence to the budget, a sound cash management system, and a good accounting system to record and report on revenues, expenditures, assets, and liabilities. Finally, a robust and comprehensive system of audits, monitoring and evaluation, and reporting are necessary to inform future budgetary and programmatic decisions. PFM reform initiatives depend on the country’s interest and ability to take ownership of the reforms, build a strategy, and invest in human and institutional capacity development. Governments should be responsible for setting their own PFM strategies including the sequencing of reforms, implementation schedules, and the financial and human resources required with the support of donors. Reform strategies should not be driven by donors, although informed discussions between donors and the partner country government may influence funding, strategic elements, and the timing of reforms. The PFM system must be tailored to country-specific contexts; thus strategy, planning, and implementation will differ across and within countries. However, good functional practices in PFM remain important guarantors of fiscal discipline, allocative efficiency, and operational efficiency. There is no singular formula for PFM reforms that can guarantee success. Nevertheless, better knowledge of existing PFM systems and processes as well as standards and good practices can improve the results. The introduction of modern, efficient, and effective PFM systems linked to information technology systems through an IFMIS and its subsidiary components can help to improve governance and particularly e-governance by providing timely and verifiable information on fiscal and budgetary performance. Additionally, as such systems’ functionalities are enhanced, 20 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV programmatic performance data can be directly linked for budgeting, decision- making, and accountability purposes. If reform programs including IT are properly designed and sequenced, partner countries can advance toward both e-governance and good practice PFM systems more rapidly by allowing them to pass over intermediate reforms. The capacities, political will, political economy, and capabilities of partner countries are vital to such progress, and the introduction of IT-based PFM systems is unlikely to be sufficient to counter a lack of preparedness, resources, or infrastructure. An Integrated Phase Approach combining elements of the basics first, platform, and evolutionary approaches should be used to program the sequence of PFM reforms in countries. Care should additionally be taken to ensure the severability and sustainability of each incremental reform within phases to ensure incremental and sustainable progress toward good practices in PFM. 6. PUBLIC EXPENDITURE AND FINANCIAL ACCOUNTABILITY (PEFA) ASSESSMENT25 Most of the assistance provided in PFM in Jordan has come from USAID and the European Union, in addition to aid received from Japan, Germany, France, and Korea. The PEFA assessment, sponsored by the EU, provides a scorecard using the PEFA assessment methodology and reports against benchmarks to showcase progress (or lack thereof) towards PFM since the previous assessment conducted in 2011. It also utilizes new criteria provided in February 2016 by the PEFA partners to establish a new baseline. The assessment focuses mostly on budgetary central government, which is comprised of 43 bodies (ministries and ministry departments which have their separate chapters in the budget). In comparison to when the previous PEFA assessment was conducted in 2011, Jordan has been under much more strain because of conflict in neighboring countries, including in Syria and Iraq. Not only has this put a strain on the country’s economy, but it has also resulted in an increasing refugee population, impacting public services, at a time when tax revenue has decreased (from 21 percent of GDP in 2007 to 15.4 percent in 2015) due to the lowering of rates and sales tax exemptions. Summary of effort made and improvements: ● There is an initiative currently in progress under the public investment unit of MoPIC and GBD to improve the monitoring and evaluation of investment projects ● External and internal audits are beginning to focus on performance, as opposed to compliance, but still have a long way to go in order to assist in improving the efficient delivery of services. ● Enhancement of the GFMIS ● Publication of information and increased transparency ● Tax collection has been reorganized to increase revenue Areas of needed improvement: ● While key performance indicators have been specified for every department’s expenditure program, they are not currently outcome focused. Instead, they focus on output or activity level results, and the actions required are not well defined 25 See Jordan: PEFA Assessment 2016, final draft, Feb. 23, 2017. The report was implemented by AECOM and funded by the European Union. 22 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV 24. Procurement B B A B D 25. Internal controls on non-salary exp. A B A A 26. Internal audit C+ C C A B Pillar 6: Accounting and reporting 27. Financial data integrity D+ D D D B 28. In-year budget reports C+ A B C 29. Annual financial reports C+ A B C Pillar 7: External scrutiny and audit 30. External audit D+ D D B D 31. Legislative scrutiny of audit reports C D* C D* A Source: Jordan: PEFA Assessment 2016 7. WORLD BANK DOING BUSINESS REPORT26 According to the World Bank’s Doing Business Reports, Jordan’s rank increased from 118 in 2016 to 75 in 2020, while the “distance to the frontier” (DTF) measure rose from 73 to 78.8. These reflect overall improvements in Jordan’s business climate. 8. TAX ADMINISTRATION DIAGNOSTIC ASSESSMENT TOOL (TADAT), 201727 The Tax Administration Diagnostic Assessment Tool (TADAT) presents a performance assessment of the Jordanian tax administration system conducted in March 2016, the findings of which are summarized in Table 2 below. Across the nine performance outcome areas (POAs) and 28 indicators assessed by the TADAT, Jordan’s ISTD scored highest in the area of accountability and transparency (POA 9), which includes the public perception of integrity and publication of activities and results. It scored lowest in the area of effective risk management (POA 2), highlighting the need for ISTD to follow the compliance risk management standards as outlined by the IMF and OECD, and to produce a compliance improvement plan. TABLE 3. JORDAN TADAT STRENGTHS AND WEAKNESSES Strengths Weaknesses ● Wide range of information on the taxpayer register ● Sound institutional risk management ● Proactive approach to voluntary compliance and taxpayer services ● Engagement and openness with the community ● Low reliability of the taxpayer register ● Lack of strategic approach to compliance risk management ● High rate of tax arrears ● Lack of a formalized tax ruling system 26 See World Bank, Doing Business Reports for 2016 and 2020, at http://jsf.org/sites/default/files/Jordan%20in%20the%20Doing%20Business%20Report 1.pdf and at https://www.doingbusiness.org/content/dam/doingBusiness/country/j/jordan/JOR.pdf. 27 See Tax Administration Diagnostic Tool (TADAT), Performance Assessment Report for the Hashemite Kingdom of Jordan, March 2016. 23 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ● Availability and wide range of e￾services ● Centralized approach to audit risk criteria and case selection ● Independent and multi-tiered tax dispute resolution ● Well-functioning revenue accounting system ● Inefficient GST refund payment system ● Low rate of use of e-services ● Inadequate analysis and evaluation of tax administration operations. Source: Jordan TADAT, 2016. 9. OPEN GOVERNMENT PARTNERSHIP INITIATIVE ACTION PLAN (2018-2020) In efforts to be more transparent and ensure accountability, the Government of Jordan (GOJ) produced an action plan under the Open Government Partnership Initiative covering the period 2018-2020. It included five main priorities: 1. Enhancing partnership and dialogue between the public sector and civil society 2. Development and enhancement of the application of Government Open Data Policy 3. Fostering national dialogue to achieve political reform 4. Unification and development of the national Human Right’s violation’s complaints mechanism 5. Institutionalization of the enforcement measures for Access to Information Law As it relates to this evaluation, in previous efforts to be transparent and open, the GOJ launched an open dialog for six laws, including the draft Income Tax Law, to be discussed amongst government and civil society groups. Further, under a milestone for priority 1 above, the GOJ committed to adopting a set of voluntary and mandatory standards for the development of governance, including but not limited to the declaration of annual budgets, the submission of income tax returns, salaries, and the development of monitoring procedures. In addition, the GOJ established a unit for the Open Government within MOPIC in collaboration with the Organization for Economic Co-Operation and Development (OECD). The objective of this unit is to assist the ministry and related government and nongovernmental bodies to participate in the Open Government Initiative. 10. JORDAN ECONOMIC GROWTH PLAN (JEGP), 2018 – 2022 The JEGP 2018-2022 consists of economic, fiscal, and sectoral strategies, and it identifies interventions for each policy. It includes seventeen objectives to be achieved through integration and coordination for implementation of these strategies and policies (fiscal, monetary, investment, and general policies of the economic sectors; p. 27). Some of the fiscal policy bases as mentioned in the JEGP are related directly to FRPFM, such as to decrease the budget deficit and public debt relative to GDP, and to ensure that domestic revenues cover current expenditures, JEGP indicated that this requires the implementation of policies for both revenues and expenditures. Some of these policies are (p. 29): ● For expenditures: Maximizing PPP, reducing capital expenditures, maximizing development of public spending, increase transparency in expenditures, and decentralization in preparing the state budget. 24 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ● For revenues: linking increases in taxes with economic growth, implementing a simplified tax system, and using technology in tax collections and in financial procedures, 11. JORDAN VISION 2025 Jordan Vision 2025 consists of four pillars: Society, Business, Citizens, and the Government. To achieve economic prosperity, the vision requested from the GOJ is to work on seven principles, including some that are directly related to the FRPFM project (Jordan Vision, p. 9): developing financial policies, implementing decentralization, limited budget deficit, preserving financial and monetary stability, and support implementing public sector reform programs. Increasing GDP growth is one of the goals of Jordan Vision 2025, and it is also one of the goals of FRPFM. The vision mentioned that the new income tax law aims to improve revenues (p. 45), and highlights tax evasion and the challenges of increasing government revenue. In 2012 tax evasion reached 1.2 billion JD, while total tax losses reached 2.3 billion JD (p. 46). Jordan Vision 2025 states that financial reform remains an urgent priority, requiring improved tax collection mechanisms, application of the law, increased transparency, and accountability (p. 46). It envisions decreasing public debt as a percentage of GDP to reach 76 percent in 2016 and to continue decreasing to reach 67 percent in 2020 (p. 59). The financial reform program is supposed to cut expenditures and raise government revenues. 12. IMF EXTENDED STAND-BY ARRANGEMENT (AUGUST 2016) The IMF Stand-by Arrangement, which lasted from 2012-2015, focused on supporting the GOJ to increase tax revenues, reduce losses incurred by NEPCO, and prioritize public investment. According to the IMF, with the help of this arrangement, the GOJ was able to undertake significant reforms. Nonetheless, the public debt increased to over 90 percent of GDP, unemployment rose, and the current deficit was high. It was under these circumstances that the decision was made that the IMF granted an extended arrangement in August 2016 to support financial reforms crucial to achieving Jordan Vision 2025. The IMF approved $723 million to assist Jordan in its economic and financial reform efforts, under a three-year extended arrangement under the Extended Fund Facility. The arrangement will help the GOJ to: ● Undertake gradual fiscal consolidation to reduce public debt ● Reforms to promote job growth, which was seen as a problem particularly among youth and women ● Monetary and fiscal policies that anchor the exchange rate and increase the resiliency of the financial system and regulatory framework. It was also agreed that the GOJ would prioritize assistance to the electricity and water sectors, to the extent the sustainability of those sectors will support fiscal consolidation. As one of the most water-scarce countries in the world, and in light of additional pressures put on the Jordanian water system due to the Syrian refugee crisis, this will include the allocation of resources to support the Water Authority of Jordan’s (WAJ) capital investment needs. 13. IMF EXTENDED FUND FACILITY FOR JORDAN (MARCH 2020) The Executive Board of the International Monetary Fund (IMF) recently approved a 48-month arrangement under the Extended Fund Facility (EFF) with Jordan for an amount equivalent to SDR 926.37 million (about $1.3 billion or 270 percent of Jordan’s quota) to support the country’s economic and financial reform program. It also provides for spending to contain and treat COVID-19. This program is anchored by critical structural reforms designed to lower electricity 27 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV fostering public-private interchange through the youth population. Human Resources Reform: FPEMP supported the goal to build a stronger administration through a training department and improved human resource utilization. 15. FISCAL REFORM FOR A STRONG TUNISIA (FIRST)29 Fiscal Reform for a Strong Tunisia (FIRST) is a three-year USAID activity that began with an inception phase during September/October 2017. FIRST builds on USAID efforts to improve public financial management in Tunisia and follows on from the Public Financial Management (PFM), Information and Communications Technology (ICT), Competitiveness, and Tax and Customs Reform activities at the Ministry of Finance (MOF). FIRST’s goal is to provide technical assistance and capacity-building to strengthen fiscal foundations for sustainable and inclusive growth with an aim to improve conditions for business, investment, and job creation. FIRST has four objectives: ● Improve the efficiency, transparency, and cost of compliance of tax administration ● Enhance MOF capacity to develop and manage tax policy ● Enhance capacity to address other fiscal reform priorities as they emerge ● Improve communications, engagement, and consultation on priority reforms. USAID/Tunisia requested METAL to conduct a midterm performance evaluation of the FIRST activity. The evaluation purpose is to identify the extent to which FIRST is achieving intended results, as well as to assess the relevance, effectiveness, and efficiency of the approaches used to achieve USAID/Tunisia objectives. According to the Mid-Term Evaluation Report dated Dec 30,2019, the absence of an implementation strategy resulted in that the USAID and the MOF could not fully understand FIRST’s overall technical approach. Though the Tax Administration Diagnostic Assessment Tool (TADAT) was not a priority for the MOF, the comprehensive assessment nonetheless proved to be a worthwhile exercise for all concerned parties. However, the focus on a deep diagnosis and delays likely contributed to the tardiness of FIRST implementation in Year 1. In terms of the activity’s contribution, only three of 27 key results (main activities) were fully completed by the end of Year 2 and 24 are ongoing. FIRST is contributing to Government of Tunisia (GOT) effectiveness by enhancing taxpayer services, building independence, strengthening the policy framework, and directly contributing to mobilizing (tax) revenues and budget management, but the evaluation can’t tell to what extent because there has been no measurement of effects and few completed main activities thus far. FIRST delivery (except for TADAT) was lacking in Year 1 and, although activity accelerated in Year 2, there were few concrete accomplishments. In the short term, efforts to reform tax administration are more likely than tax policy to yield tangible results, while outreach efforts can rapidly make government more transparent and promote voluntary tax compliance. Little was done by FIRST to advance opportunities for women in the fiscal domain, despite FIRST’s gender strategy and the robustness of the FIRST’s attention to gender equality issues in the documentation review, KIIs, FGDs, and various analyses. And due to the challenging environment that FIRST is operating 29 See IBTCI, Midterm Evaluation of USAID Tunisia’s Fiscal Reform for a Strong Tunisia (FIRST), December 31, 2019. 28 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV in, completion of all FIRST activities is unlikely by mid-2020, especially the “last mile” by MOF for sustainability. The low quality of work-planning and reporting documents is hampering the MOF’s and USAID’s oversight role. 16. REFERENCES Rozner, Steve, Sharon Hester, Rod Ettrige and Mounir Bardawil. “Tax Administration Diagnostic Assessment Tool, Performance Assessment Report”. March 2016. https://www.tadat.org/assets/files/Jordan Final PAR 2016.pdf “The Strategy for PFM Reforms in Hashemite Kingdom of Jordan (2018-2021)” Jordan Ministry of Finance. Wiggins, John, David Biggs, and Omar Al-Bokairat. “Jordan 2017: Public Expenditure and Financial Accountability (PEFA).” Jordan 2017 | Public Expenditure and Financial Accountability (PEFA). AECOM. Accessed March 12, 2020. https://pefa.org/node/581 “Jordan: Request for an Extended Arrangement Under the Extended Fund Facility – Press Release; Staff Report; and Statement by the Executive Director for Jordan”. International Monetary Fund Country Report No. 16/295. September 2016. https://www.imf.org/external/pubs/ft/scr/2016/cr16295.pdf Doing Business 2020: Economy Profile: Jordan. World Bank. Accessed March 2020. https://www.doingbusiness.org/content/dam/doingBusiness/country/j/jordan/JOR.pdf “The Fourth National Action Plan 2018 – 2020 Under THE Open Government Partnership Initiative (OGP).” Ministry of Planning and International Cooperation. Jordan. 31 October 2018. Accessed March 10, 2020. https://www.opengovpartnership.org/wp￾content/uploads/2019/01/Jordan Action-Plan 2018-2020.pdf “Final Report June 2011-June 2017 USAID/DAI Fiscal Policy and Expenditure Management Program, El Salvador” Midterm Evaluation of Fiscal Reform for a Strong Tunisia (FIRST), December 30, 2019 Jordan Economic Growth Plan Jordan Vision 2020 Guide to Public Financial Management, Bureau of Economic Growth, Education and Environment, Offices of Economic Policy Capacity Building Division, Revision 1 April 2015. 2020 Article IV Consultation and Request for An Extended Arrangement Under the Extended Fund Facility—Press Releases; Staff Report; and Statement by the Alternate Executive Director for Jordan 29 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ANNEX 2: PROJECT STRUCTURE AND SIGNIFICANT ACCOMPLISHMENT BY COMPONENT AND YEAR BOX 1: PROJECT STRUCTURE The contractor used the analytical structure shown in this box diagram as the project design. The four components were called for in the contract. The subcomponents at the two-digit level (2.1, 2.2, etc.) for components two and three were also called for in the contract. The contractor provided all other subcomponents (and sub-subcomponents at the three-digit level). The contractor used this structure in its work-plans throughout, with changes as indicated in Box 1. However, although the contractor reported many accomplishments in the annual reports, none of these are identified by the contractor’s three-digit design shown in Box 1. (All three-digit attributions in Box 2 are provided by the evaluation team.) In addition, the evaluation team could not discern a systematic methodology for tracking the “anticipated results” called for in the contract; these are not tracked by the design presented in Box 1 nor are they, for the most part, tracked by the MEL plan. (See text for some examples.) COMPONENT 1: REVENUE PERFORMANCE IMPROVED THROUGH EFFECTIVE TAX POLICY AND ADMINISTRATION 1.1 Tax administration 1.1.1 Set up a tax administration performance improvement task force 1.1.2 Assist the ISTD to implement new income tax law 1.1.3 Streamline tax registration and ensure the accuracy of the registry 1.1.4 Develop taxpayer services 1.1.5 Optimize risk-based tax audit 1.1.6 Strengthen collection/fraud investigations and the management of arrears 1.1.7 Enable taxpayer feedback 1.1.8 Other cross-cutting activities 1.2 Tax policy 1.2.1 Improve tax policy analysis 1.2.2 Provide tax policy recommendations COMPONENT 2: BUDGET EFFICIENCY AND TRANSPARENCY IMPROVED 2.1 Stronger GOJ auditing practices and results-oriented government 2.1.1 Strengthen internal controls 2.1.2 Transition to accrual IPSAS 2.1.3 (Y1, Y2) Strengthen the Audit Bureau to perform external (supreme) audit 2.1.3 (Y3) Internal audit, inspection, and control 2.1.4 (Y1) Assist MOF to define and disseminate internal audit standards, tools, and training plans 2.1.4 (Y2, Y3) Internal audit, inspection, and control 2.1.5 (Y1) Institutionalize GRB within budget planning 2.1.5 (Y2) Gender budgeting 2.1.5 (Y3) ROB monitoring and evaluation, KPI alignment, and Jordan Vision 2025 2.1.6 (Y1) Develop whole of government M&E framework 2.1.6 (Y2) ROB, monitoring and evaluation, KPI alignment, and Jordan Vision 2025 2.2 More transparent PFM practices implemented 2.2.1 Position the GFMIS unit for long-term sustainability 2.2.2 (Y1) Revise the budget preparation workflow and fully implement the GFMIS budget preparation module 2.2.2 (Y2) Revise the budget planning and preparation (BPP) workflow and fully implement the Hyperion module 2.2.3 Support the rollout of GFMIS 30 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV 2.2.4 Support the decentralization initiative 2.3 Increased GOJ adoption of public-private partnerships 2.3.1 Operationalize the PPP unit 2.3.2 Build PPP pipeline 2.3.3 (Y1) Develop PPP projects 2.3.3 (Y2, Y3) Support PPP unit and contracting authorities in implementing PPPs COMPONENT 3: FISCAL SUSTAINABILITY INCREASED THROUGH STRUCTURAL REFORMS 3.1 Improved fiscal policy formulation and execution, and management of general government debt 3.1.1 Assess the data and tools of SEPD and provide solutions 3.1.2 Support the development and implementation of a medium-term debt management strategy 3.1.3 Help SEPD update and further develop the macro-fiscal database 3.1.4 Support the application of models to be used in policy analysis research 3.1.5 Help SEPD undertake research and produce policy papers, revive MTFF 3.1.6 Help in the restructuring and upgrading of SEPD 3.1.7 Build capacity of SEPD staff 3.1.8 Assist the MOF in addressing key fiscal and economic challenges 3.2 (Y1, Y2) Fiscal management at the governorate level 3.2 (Y3) Grants 3.2.1 Map roles and responsibility for subnational fiscal management 3.2.2 Support the implementation of the decentralization law 3.2.3 Conduct a nationwide capacity building program 3.2.4 Support peer learning across governorate jurisdictions 3.2.5 Assist in establishing a fiscal decentralization unit 3.3 (Y2, Y3) Increased public-private dialogue and outreach 3.3 (Y3) Fiscal management at the governorate level 3.3.1 Conduct landscape assessment to highlight current opportunities for PPD and build buy-in 3.3.2 Foundations for PPD 3.3.3 Engage a nongovernmental policy community in relevant fiscal research through grants under contract 3.3.4 Improved capacity to manage strategic communication 3.4 (Y1) New Activity: Department of Lands and Survey 3.4 (Y2) Improved risk at the Ministry of Finance and streamlining transactions at the DLS 3.4 (Y3) Increase public-private dialogue and outreach 3.4.1 Support the Department of Land and Survey 3.5 (Y3) Improved risk at the MOF and streamlining transactions at the DLS COMPONENT 4: THE EXIT STRATEGY 4.1 The exit strategy 4.1.1 Engage counterparts in regular maturity assessments 4.1.2 Present an exit strategy for discussion with USAID and counterparts 4.1.3 Strengthen and institutionalize training functions 4.1.4 Donor coordination 4.1.5 Design and implement CYPRESS strategic planning 4.2 Monitoring and evaluation 31 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV 4.2.1 General knowledge about the activity performance and progress toward the target outcomes to adopt ongoing improvements BOX 2: SIGNIFICANT ACCOMPLISHMENTS BY COMPONENT AND YEAR The evaluation team attempted to cull the most important accomplishments from the annual reports, those that went beyond “process” to reporting concrete outputs or outcomes, and referred to as “significant accomplishments” in this box. This is imperfect science and one might debate whether the team left out some that might be included (or included some that may not have been that significant). The evaluation team also made an effort to indicate which three￾digit project function to assign these to, as defined in Box 1. The contractor did not do this, which makes tracking contractor performance extremely difficult. On this basis, the MEL plan is the only tracking of indicators during the period of performance, and as discussed in the text. The contractor has provided no careful tracking of results by its design (i.e., at the three digit level), or alternatively, by the “anticipated results” called for in the contract. COMPONENT 1 Significant accomplishments in year 1: ● 1.1.8 Launched 13 CYPRESS projects at ISTD. The projects completed in Year One are: e￾payments, e-filing, risk management, e-refunds, and no-filers. ● 1.1.2 Assisted ISTD in developing the forms, guides, and designs of e-registration, PAYE, non-filer, refunds, and risk management projects. ● 1.1.4 Helped ISTD to activate the functions of the call center. Such effort led to an increase in the number of outgoing calls from almost zero to around 500 calls a day. Significant accomplishments in year 2 (combines tax administration and tax policy): ● 1.1.4 The results of FRPFM tax administration support are of great importance in year 2: o 72% of personal income tax (PIT) returns were submitted electronically in 2018, compared to 2% in 2017. o 84% of tax payments (by value) were made electronically in 2018 compared to less than 1% in 2017. o Over 31,000 refunds, 88% of refunds issued, were deposited directly into taxpayer bank accounts using the International Banking Account Number (IBAN) of each taxpayer, compared to zero in 2017. ● 1.1.8 The Project helped ISTD plan their own capacity development program by providing a training needs assessment (TNA). The needs assessment identified gaps between the knowledge, skills, and experiences of ISTD staff and the skills needed to meet directorate objectives. ● 1.1.5 The Project prepared an analysis of general sales tax (GST) audits to help ISTD better align revenue expectations with GST performance. The key finding was a recommendation to create a database of GST-exempt taxpayers (e.g. banks, constructions, currency exchange, as well as companies exempted via a cabinet decision). The Project also conducted a study of corporate income taxpayers (CIT) with sales revenues in excess of JOD 1 million, that were not registered for GST. The Project presented its findings to ISTD, pointing out the impact that various exemptions have on the revenue generated. ● 1.1.5 FRPFM organized workshops for ISTD management on audit risk assessment, risk management, collection best practices, compliance strategies, audit procedures and techniques. Significant accomplishments in year 3 (for tax administration): ● 1.1.6 The project supported the MoF in designing and implementing an action plan for improving tax compliance and combating tax evasion. As a follow-up from the FRPFM team, an integrated archiving system was tested and completed to ensure that all ISTD 32 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV documentation is secured. In addition, 16,500 notices were issued to non-filers in August 2019. ● 1.1.4 FRPFM has continued to support the ISTD in improving revenues and compliance through development of e-services (e-filing and e-payment) and compliance programs. Accordingly, ISTD showed a notable improvement in terms of filing rate and revenue between 2017-2018. During the first half of 2019, ISTD has already received 85% of the returns and 96% of the revenues that were received in 2018 for the 2017 tax year. Due to fiscal year filing, additional returns and revenues attributable to tax year 2018 will be received over the remaining six months of 2019. These numbers are not impacted by the income tax law which is effective beginning with tax year 2019 returns and revenues. ● 1.1.5 The project team recommended using risk-based practices for payment of GST refunds rather than 100% audit of refund claims. Another important improvement to the GST refund processing was the addition of a box on the GST return for requesting a GST refund, rather than a separate form. Significant accomplishments in year 1: ● 1.2.2 Developed and presented a draft income tax law proposal and recommendations to improve tax compliance, combat tax evasion, expand the tax base, and improve the structure of the tax system. ● 1.2.2 Developed a concept paper upon the request of the MoF for establishing a financial investigation unit (FIU) that would be tasked with identifying tax fraud cases and gathering financial intelligence. Significant accomplishments in year 3 (for tax policy): ● 1.2.1 Income tax collections for the first seven months of 2019 increased by 32% compared to 2.9% for the same period in 2018. ● 1.2.1 FRPFM reviewed the current GST Law and other indirect taxes and prepared recommendations that would help bring more transparency and fairness to the system and improve the performance of the GST, in line with recent international experience. COMPONENT 2 Budget Significant accomplishments in year 1: ● 2.2 Finalized a comprehensive Organic Budget Law to meet IMF structural benchmark. ● 2.1.5 Prepared a comprehensive report on GRB in Jordan. The report included an assessment of Gender Budgeting, and an assessment of Gender-relevant Indicators in the 2017 Budget law. Significant accomplishments in year 3: ● 2.1.6 The project performed two Public Expenditure Reviews (PER): (1) five programs in the labor and vocational education sector; (2) Ministry of Tourism and Antiquities. ● 2.2.4 In support of decentralization initiatives, the project developed a template and database for Governorates Budget Profile to enable regular monitoring and reporting of budget expenditure performance. ● 2.1.6 The project developed “ROB Handbook for Budget Analysts” documenting ROB methodology and process. The Handbook represented a compendium of essential knowledge, skills, processes, and procedures required for monitoring and evaluating a results-oriented budget. ● 2.2 In response to GBD request, the project commenced its assistance in updating GBD Strategic Plan 2020-2022. The project recommended a process for GBD strategic planning review, assessed status of performance information contained in the current 33 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV GBD Strategic Plan and outlined proposed new relevant indicators (activities to be continued in Year Four). ● 2.1.5 FRPFM provided GBD and 8 pilot ministries staff training on GRB framework and the methodology of calculating financial allocations on gender basis, and technical assistance in formulating a Memorandum of Understanding (MoU) to institutionalize the gender budgeting process in Jordan. Treasury Significant accomplishments in year 1: ● 2.2.3 Completed the roll-out of the GFMIS to the central government entities. ● 2.1.2 Completed an assessment of the GOJ’s readiness for implementing IPSAS ● 2.1.2 Reviewed the current chart of accounts (COA), conducted workshops on COA design best practices and presented recommendations. ● 2.1.2 Completed a technical assessment of GFMIS for IPSAS accrual accounting. ● 2.1.2 Provided on the job capacity building to the Ministry of Finance staff for IPSAS accounting policy and chart of accounts. Audit Significant accomplishments in year 1: ● 2.1.3 Developed a transition plan to move the Internal Control Directorate closer to undertaking an Internal Audit function PPP Significant accomplishments in year 1: ● 2.3.2 Added 8 approved and pre-approved expected viable projects to the PPP pipeline. ● 2.3.1 Supported the PPP Unit and line ministries in identifying over $3,000,000 in grants and soft loans to finance the preparation (feasibility and sustainability studies, and transaction advising) of potential PPP projects. ● 2.3.1 Assisted the PPP Unit in securing JD 600,000 for potential PPP project development in 2017 ● 2.3.1 Created three drafts of the new PPP guidelines based on the new PPP Law and Regulations, best practices, and input from the PPP Unit and other public (e.g., line ministries) and private (e.g., banks) sector stakeholders. ● 2.3.1 Completed and uploaded the newly developed PPP database on the MOF system and trained the PPP Unit staff in managing and populating the new database. ● 2.3.1 Completed the PPP certification training and examination through Level One of three (PPP Unit and contracting authorities). Significant accomplishments in year 2: ● 2.3.3 The Project provided ongoing technical support to eight transaction advisors who prepared project feasibility studies and/or advisory documents for tendering and implementing PPP projects. There were sixteen PPP Council and Council of Ministers approved projects, and four projects in the pre-application assessment phase. Eight projects were either put on hold or terminated as a result of not meeting PPP criteria; such as value for money, affordability, and sustainability. ● 2.3.1 The Project supported the PPP Unit in drafting the PPP Law, as well as in preparing PPP Guidelines in English and Arabic. 34 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ● 2.3.1 FRPFM provided several training workshops for the PPP Unit, other MoF departments (GBD, Debt Directorate), MoPIC (PIM), Jordan Investment Commission (JIC) and relevant contracting authorities. These activities included a two week “Introduction to PPP’s” workshop in April and September, a one week “PPP Project Financing” workshop in September, and a training of trainers (TOT) workshop for the PPP Unit in April. The workshops were attended by over 50 participants from over 10 GOJ institutions. ● 2.3.2 The Project has supported the PPP Unit in identifying and securing over JOD 6 million for the development of more than 14 projects. This project development funding has come from the GOJ budget, MDBs, and donors. Significant accomplishments in year 3: ● 2.3.3 The PPP Unit prepared a new draft PPP Law that was approved by the Council of Ministers in late September 2019. ● 2.3.3 Prepared a draft PPP FCCL manual and financial models. ● 2.3.3 In July 2019, the Project, in coordination with the IFC, provided MoF, MoPIC and contracting authorities a Level I and a Level II PPP Certification Exam. ● 2.3.3 FRPFM supported various contracting authorities (line ministries, local governments, and other government institutions), through the PPP Unit, in providing ongoing technical support on 13 projects that have completed or are preparing feasibility studies and/or documents for tendering and implementing PPPs. At the end of June, there were 16 projects approved (of which 2 were subsequently terminated) by the PPP Council and Council of Ministers, and 3 projects in the pre-application assessment phase. To date, 10 projects have been either put on hold or terminated as a result of not meeting PPP criteria, such as value for money, affordability, and sustainability, demonstrating discipline being upheld and compliance with the PPP Law. ● 2.3.2 The total capital value of all PPP’s currently in the active pipeline is estimated at close to JD 600 million without including the National Broadband Network project on which no value has been estimated considering the broad range of structuring options available with significantly different levels of capital investment. IPSAS Implementation Significant accomplishments in year 1: ● 2.1.2 The Project prepared an enhanced GFMIS Improvements/Accrual Accounting Roadmap and recommended an improved management structure ● 2.1.2 The first group’s completion of the ACCA IPSAS certification training. Significant accomplishments in year 3: ● 2.1.2 An amended IPSAS Implementation roadmap was reviewed and sent for approval. ● 2.1.2 The ACCA IPSAS certification online training was delivered by the IPSAS PMO with assistance from the Project. ● 2.1.2 The Project continued to develop and deliver IPSAS based accounting workshops for approximately 80 MoF participants. The 5 workshops covered financial statements, revenues, reporting entities concept as well as end of month procedures. These workshops were conducted over 16 sessions. ● 2.1.2 The Project is also assisting the GOJ to restate the 2018 cash-based financial reports on an accrual basis that complies with IPSAS as far as is practical in the context of data availability. Government Units 35 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV Significant accomplishments in year 3: ● 2.2 The GUs Portal was successfully rolled-out and implementation was complete. The 2018 financial data is on the GUs Portal in accordance to the 2018 GUs Budgets Law. In coordination with GBD, the MOF’s GUs Accounts Unit prepared and published the 2018 GUs Final accounts on the MoF website. GFMIS Significant accomplishments in year 3: ● 2.2.3 The project assisted in moving into production the DLS-GFMIS and Customs￾GFMIS interface. Internal Audit Significant accomplishments in year 3: ● 2.1.4 As a start to introducing the Internal Audit concept, the Project conducted a 3-day workshop covering International Internal audit standards, internal control and audit functions as well as applying practical internal audit cases on MOF processes. ● 2.1.4 The project developed a one-year implementation plan to launch MOF as an internal audit pilot site. COMPONENT 3 Significant accomplishments in year 1: Macroeconomic ● 3.1.3 Updated and upgraded the comprehensive Macro-Fiscal database to be used in policy research and fiscal policy formulation by the SEPD. ● 3.1.4 Supported SEPD in building a macro-economic model that was used in forecasting main economic indicators. ● 3.1.5 Completed the MTFF report that was used to establish government annual budgetary ceilings for the 2018 GOJ budget. ● 3.1.6 Supported the restructuring of the SEPD into 4 divisions with clear mandates ● 3.1.7 Provided training for SEPD staff on: (i) use of software to do econometric work (EViews); (ii) economic research and analysis; (iii) statistics; (iv) econometrics and forecasting; (v) policy analysis; and (vi) writing policy research papers, (vii) running the macro and fiscal simulation model. ● 3.1.5 Conducted several policy papers covering topics like challenges facing economic growth in Jordan, impact of proposed pension reforms, measuring the informal sector, and others. Fiscal Decentralization ● 3.2.1 Delivered the “Governorate Capacity Assessment Report” to the MoF, after conducting assessment to four governorates to evaluate their capacities in PFM, budgeting and local development. ● 3.2.1 Developed the Fiscal Decentralization White paper which was approved by the Cabinet of Ministers and was published on MoF website and distributed to all relevant stakeholders. Provide technical support to the General Budget Department on identifying the criteria and formula governorates capital budget ceilings for 2018. ● 3.2.3 Prepared the Governorate Budget Preparation Manual for 2018, a guidebook for the governorates on the provision and procedures for budget preparation. 36 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ● 3.2.3 Conducted three regional workshops with the executive councils covering all twelve governorates in budget preparation for fiscal year 2018. Public-Private Dialogue ● 3.3.4 Prepared an assessment of the ISTD and the PPP Units’ websites. Final set of recommendations were submitted to both institutions. Department of Lands and Survey ● 3.4 Completed the DLS business transformation assessment. ● 3.4 Completed the cybersecurity assessment report. Grants ● 3.3.3 Awarded two grants, one for Public Private Dialogue (PPD) and one for Promoting E-Services for the Income and Sales Tax Department (ISTD). Significant accomplishments in year 2: Fiscal Policy Analysis ● 3.1.4 and 3.1.5 The project supported the MOF with macroeconomic modeling used in the MTFF report. The project also worked to build SEPD staff capacity by providing support in preparing two policy research papers improving research analysis. The project also supported the MOF and the ISTD in their efforts to reform Jordan tax policy. Fiscal Decentralization ● 3.2.3 The project promoted important national discussion on fiscal decentralization by organizing the conference, “Way Forward for Fiscal Decentralization.” The conference promoted the dialogue between the central government and the governorates around critical questions of the nature and extent of decentralization in Jordan. ● 3.2.3 The project delivered a policy paper titled “Towards Improved Decentralization Governance and Accountability.” Public-Private Dialogue And Communications ● 3.3.3 The project provided technical guidance to the awarded Public Private Dialogue (PPD) grantee to conduct three public discussions on key macroeconomic issues. The project also provided technical guidance to the awarded grantee in developing communication strategy and material of the ISTD e-services campaign. ● 3.4 The Project worked with the Department of Land and Surveys (DLS) to develop a communications strategy to improve citizen information outreach around new e-services and other offerings. Grants ● 3.3.3 A new RFA for Public Expenditures Analysis on Public Programs Supporting Gender Equality in Jordan was issued. 8 applicants applied for this RFA.As of the writing of this report, the Technical Evaluation Committee (TEC) was in the final stages of the selection process. ● 3.3.3 On a second grant, USAID had approved support to a think tank to conduct fiscal and economic policy research. Risk Management And Business Transformation ● 3.4 In support of DLS business transformation operations that provide improved customer service, the project is working with DLS to establish a one-stop-shop at a pilot Land Registry Directorate. 38 | FRPFM EVALUATION AND ASSESSMENT PHASE 1 USAID.GOV ● 4.1.1 Developed the PFM Sustainability Framework that focuses on building the capacities of the MoF to perform strategic planning, manage learning, promote human resource development and to refine leadership capacities as well as improving core PFM functions. ● 4.1.3 Conducted the pilot Training of Trainers Program (TOTP) with the MoF Budget Execution directorates targeting 14 PFM professionals in four priority topics (Cash Management, Internal Audit, IPSAS and professional communication). ● 4.1.3 Trained 16 GFMIS staff on Oracle R12 E-Business Suite modules. ● 4.1.5 Implemented the CYPRESS strategic planning framework with the ISTD, SEPD, PPP Unit, Financial Institute and the Budget Execution Directorates. ● Conducted the work environment assessment through the Employees Satisfaction Survey at MoF, DLS, ISTD and GBD.