FINAL REPORT PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 0 - - - - - FINAL REPORT MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROGRAM November 04, 2012 November 2012 This document was produced for review by the United States Agency for International Development. It was prepared on behalf of The QED Group, LLC, under Iraq PERFORM contract number 267 M 00 09 00513. Performance Evaluation and Reporting for Results Management (PERFORM) Contract Number: GS-10F-0405M Order No. 267-M-00-09-00513-00 FINAL REPORT Mid-Term Performance Evaluation of USAID/Iraq Financial Development Program November 2012 Submitted by: The QED Group, LLC 1250 Eye Street, NW, Suite 1100 Washington, DC 20005, USA Tel: +1.202.521.1900 Fax: +1.202.521.1901 www.qedgroupllc.com Baghdad Office: Victor 2 Compound Quarter Al Qadisiyah, Section No 604 Street No 57, House No 3 (QED Villa) Tel: +964.780.956.6996 DISCLAIMER -– The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 1 1-EXECUTIVE SUMMARY AECOM and its subcontracting partners were awarded USAID/Iraq’s $53.3 million Financial Development Program (FDP). The purpose of this award is to develop the private financial sector so that it offers the best long-term prospects for providing credit needed to promote and sustain economic growth and job creation. Total obligated amount as of March 2012 was $31.6 million, with nearly half that amount already spent. The project’s activities are organized across its three main components:  Component 1: Enhance Advocacy Capacity of the Private Financial Sector  Component 2: Improve the Soundness of the Private Financial Sector Through Institution Building and Targeted Reforms  Component 3: Improve Quality and Availability of Finance and Business Education. As the project has reached its mid-term mark USAID has commissioned a Mid-Term Evaluation. This assignment will: provide an independent assessment of project’s performance; identify reasons why progress is or is not being made; and, recommend actions for USAID to take, either at the project level or with the Government of Iraq (GOI), in order to achieve the project objectives. As provided in the Scope of Work, the evaluation was conducted to answer four sets of key questions related to the three project components and to AECOM’s project management. The evaluation team’s findings and recommendations as they relate to these key questions are provided below. Component 1: Private Industry Associations - Does the Federation of Private Banks in Iraq have the proper focus and scale of operations? FDP provided technical and financial support for establishing the Federation and making it operational. The establishment of the Federation is mostly acknowledged by stakeholders (private banks, FDP and USAID) as a misadventure and counterproductive. FDP focused its efforts on registering the entity and making it operational but did not focus adequate efforts on making the Federation financially viable. For instance, there is no evidence that FDP conducted analysis of how the organization was going to raise revenue. A business plan should have been developed for the Federation at the outset. Federation does not appear to be an effective advocacy body for private banks. Four of the five private banks that were surveyed for this evaluation were united in their lack of appreciation for the organization. Members of the Federation are unaware of the financial position and operational capacity of the organization. Moreover, they are not in favor of financing Federation’s activities when FDP funding dries up. The evaluation team’s recommendations for FDP’s Component 1 activities are provided below:  FDP and USAID should cut and reassess Component 1 activities and deliverables.  FDP should not provide any financial support to the Federation or the IPBL.  FDP should help Federation establish a business plan to safeguard USG’s investment.  FDP should explore ways of engaging IPBL up until USAID decides on whether or not to continue supporting advocacy-related activities under Component 1. Component 2-Targeted Reforms In Component 2, FDP focused its efforts on developing targeted reforms and working with the CBI to establish a Retail Payment System (RPS) and a Credit Information Bureau (CIB). FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 2 Targeted Reforms - Is FDP now focused on a clear reform agenda, and if so, to what extent are these reforms appropriate priorities for the CBI, the Federation of Private Banks, and other stakeholders? Activities pursued here are geared towards implementing four to five targeted policy, legal, regulatory and administrative reforms that will quantitatively and qualitatively expand financial sector intermediations. So far, FDP has fulfilled the first deliverable (i.e., developing a list of prioritized reforms) and is now focused on the second deliverable (i.e., implementing four to five targeted reforms). The targeted reform agenda is indeed clear, but the contents are debatable. CBI, the key stakeholder, did not take part in the process of selecting those reforms, and the outcome, thus, does not reflect CBI’s priorities. Initially, when the reform areas were identified, FDP was working mainly wih the Legal Committee of the Federation. In July 2012, FDP decided to establish a working group (WG) that comprises of a wider group of private banks to implement the reforms. So private bank participation in development of these reforms has been a recent occurrence, and it iremains to be seen whether or not the list of selected reforms adequately reflects their priorities. From discussions the evaluation team had with a number of those banks, there is strong likelihood that the banks will revisit that list. The evaluation team’s recommendations for FDP’s targeted reforms activities are provided below:  As a matter of priority, FDP should engage CBI, preferably as member in the WG.  FDP should allow the WG to decide which areas should constitute the four or five targeted reforms.  FDP should agree with the WG on a road map towards implementing the chosen areas of reform, envisaging specific, short-term tasks for FDP.  Critical here is to impress upon the WG that they will be “championing” the targeted reforms efforts. Retail Payment System - How important is it that a Retail Payment System as originally envisioned by the project be implemented at this stage, now that the CBI has departed from the original plan? Establishment of a comprehensive RPS is a top priority for the CBI. RPS will improve soundness of the private financial sector by enhancing the infrastructure necessary for expansion of bank and non-bank financial institutions. FDP has performed well on RPS-related tasks, and the only delay is in the final stage, namely, issuing the Request for Proposals (RFP) for the National Switch. As per their own admission, CBI is responsible for that delay, however, now the process is back on track. The evaluation team does not consider the alterations that CBI made in the process to be a departure from the original plan, but rather an over cautious approach, not uncommon in governmental organizations. In view of the importance of the RPS to CBI and other stakeholders, the evaluation team recommends continuation of support by FDP and USAID. The tasks envisaged by FDP for the current and subsequent phase, up until the system goes live, are adequate. Identified below are evaluation team’s recommendations for additional areas of support:  FDP should assist CBI in launching an awareness campaign that will ready the market (the public as well as the banks) for growth in using non-cash payment services.  FDP should back CBI in other interventions that will increase use of non-cash instruments (e.g., a major step that would make a quick impact is changing the payroll system in as many governmental bodies as possible to electronic payment). Credit Information Bureau - What are the causes for lack of progress in the start-up of a Credit Information Bureau and what might be more effective approaches than those already employed in the project? FDP activities here are geared towards establishment of an effective and structured credit information bureau (CIB). FDP’s deliverables for CIB for years 1 and 2 are as follows: Iraqi driven credit bureau FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 3 business plan and model developed; legal and regulatory requirements for credit bureau identified and recommended changes drafted; and public education campaign on the role of the bureau launched. FDP has fulfilled the first two deliverables, while the third has been ready for some time and is pending approval from CBI. FDP maintains that there are neither delays nor lack of progress with regard to the establishment of CIB. This position is substantiated by contractual deliverables which have been met by FDP. However, the process should have been further ahead than where it stands now. FDP finalized the plan and model for CIB in July, 2011, and the recommendation was in line with what CBI was aiming for, namely, establishing the CIB within CBI, by upgrading the current Information Exchange Department. There was, thus, an early achievement of a milestone, and no opposition from the main stakeholder. Accordingly, it is fair to state that more concrete steps should have been accomplished by now, even if they were not mandatory. With regard to FDP support, evaluation team has the following recommendations:  FDP support for CBI Implementation should be delivered through one Long-Term Technical Assistance expert.  Any other support needed (legal or technical) should be on short-term and time-bound basis.  The “Milestone Plan for CIB Test and Production Environment” recently developed by FDP should be included in the Work Plan, as it contains specific planned deliverables culminating in full implementation.  CIB implementation plan should be closely monitored by FDP, on shorter intervals than is customary, to allow for intervention and corrective action in the event of delays caused by human or technical deficiencies. Component 3: Training and Curricula Development - Is the project making any progress in helping the business schools to set up effective business degree programs? In July 2011, FDP conducted a Skills Gap Analysis to identify core areas of weakness needing to be addressed in both the curricula of Iraqi business and finance university faculties and the skill sets of the core staff at private banks. The study formed the basis for FDP's future interventions carried out under Component 3 that involve delivering technical and financial assistance to Iraqi universities for: Association to Advance Collegiate Schools of Business (AACSB) accreditation; curricula development; and training. Here, FDP is delivering support to Iraqi universities by partnering them with international universities and providing grant funding. The table below provides information on the partnerships established. Foreign university Iraqi university University of Dubai, U.A.E. Mansour University College, Baghdad Shippensburg University, U.S. University of Baghdad, Baghdad James Madison University, U.S. Mustansiriyah University, Baghdad James Madison University, U.S. Salah Al Din University, Erb il The degree of buy-in demonstrated by the domestic partner universities to the evaluation team has been high across the board. Mansour University College and Mustansiriyah University are very receptive to pursuing reforms, and they consider AACSB accreditation their primary future objective. Similarly, Business and Administration faculty of Baghdad University is administering changes to upgrade its programs, such as establishment of a business council and offering students internships in banks. Mustansiriyah University considers it a major obstacle that their partner university does not want to visit FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 4 them anymore and would like FDP to address this issue. James Madison University’s reluctance to travel to Baghdad was due to the tragic incident that happened in June 2010 when the Associate Dean of the School of Business, American University of Cairo was killed in Baghdad. Overall, there is evidence that the domestic universities have started working towards establishing more effective business degree programs. The evaluation team’s recommendations for FDP’s Component 3 activities are provided below:  Establish milestones to measure progress in the short to medium-term as no business school will achieve accreditation standards by end of FDP i.e., 2015.  Continue delivering assistance to Iraqi partner universities until the end of the project.  Enable Iraqi universities to establish linkages with financial stakeholders so that their curricula are more responsive to the needs of the market.  Terminate the Mustansiriyah University/James Madison University partnership and replace with another foreign university that is willing to travel to Baghdad, Iraq. Project Management: AECOM-Is AECOM providing good project management? FDP is being implemented by AECOM and its subcontractors. The project has 53 employees working across the three main components. Each of the three components is led by a Director who is assisted by a Deputy Director and subject matter experts. The project has recruited six Iraqi officers as Deputy Directors and it is envisaged that they will take ownership of FDP activities and serve as Directors in either later Year 3 or early Year 4 of the program. AECOM’s performance in implementing FDP varies from one Component to another. In part, this is due to the diversity of activities that are being implemented under the different components. Overall, the approach adopted by the project has largely been deliverable-driven where the focus is on ensuring that AECOM satisfies its contractual obligations. At times, AECOM gave limited consideration to important issues like continued relevance, sustainability and efficiency of project activities.  Under Component 1, AECOM invested significant resources in establishing the Federation and ignored critical issues relating to selecting the correct structure and making it financially sustainable.  Under Component 2, RPS is the project’s success story where FDP performed very well. The project is on target on CIB, but there was ample opportunity to jump ahead of schedule that FDP did not take advantage of. Legal and regulatory reforms is the least successful activity as banks were not properly involved, and CBI was not involved.  Under Component 3, FDP has done a satisfactory job. The previous and the current Component Director continue to implement activities in an effective and participatory manner. The evaluation team’s recommendations with regard to AECOM’s project management are provided below:  Program planning and implementation should be demand-driven and should take into consideration objectives and priorities of key stakeholders.  AECOM should scale back activities where there are value for money concerns (e.g. Component 1).  Frequent changes in FDP management should be avoided.  AECOM should routinely revise/update capacity building workplan for Iraqi officers. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 5 2-TABLE OF CONTENTS 3-INTRODUCTION .........................................................................................................................................................10 3A-Purpose of evaluation..............................................................................................................................................10 3-B-Audience.................................................................................................................................................................10 4-BACKGROUND..........................................................................................................................................................10 5-METHODOLOGY........................................................................................................................................................11 5-A-Approach................................................................................................................................................................11 5-B-Data Collection Tools............................................................................................................................................11 5-C-Limitations..............................................................................................................................................................11 6-COMPONENT 1-PRIVATE INDUSTRY ASSOCIATION....................................................................................12 6-A-Does the Federation of Private Banks in Iraq have the proper focus and scale of operations? ........................12 6-A-1-Findings..........................................................................................................................................................12 6-A-2-Conclusions....................................................................................................................................................14 6-A-3-Recommendations .........................................................................................................................................16 7-COMPONENT 2-TARGETED REFORMS...............................................................................................................17 7-A-Is FDP now focused on a clear reform agenda, and if so, to what extent are these reforms appropriate priorities for the CBI, the Federation of Private Banks, and other stakeholders?......................................................17 7-A-1-Findings..........................................................................................................................................................17 7-A-2-Conclusions....................................................................................................................................................18 7-A-3-Recommendations .........................................................................................................................................19 7-B-How important is it that a Retail Payment System as originally envisioned by the project be implemented at this stage, now that the CBI has departed from the original plan? .............................................................................20 7-B-1-Findings ..........................................................................................................................................................20 7-B-2-Conclusions ....................................................................................................................................................21 7-B-3-Recommendations..........................................................................................................................................22 7-C-What are the causes for lack of progress in the start-up of a Credit Information Bureau and what might be more effective approaches than those already employed in the project?...................................................................22 7-C-1-Findings ..........................................................................................................................................................22 7-C-2-Conclusions ....................................................................................................................................................23 7-C-3-Recommendations..........................................................................................................................................24 FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 6 8-COMPONENT 3-TRAINING AND CURRICULA DEVELOPMENT ................................................................25 8-A-Is the project making any progress in helping the business schools to set up effective business degree programs?.......................................................................................................................................................................25 8-A-1-Findings..........................................................................................................................................................25 8-A-2-Conclusions....................................................................................................................................................28 8-A-3-Recommendations..............................................................................................................................................29 9-PROJECT MANAGEMENT: AECOM.....................................................................................................................30 9-A-Is AECOM providing good project management? .............................................................................................30 9-A-1-Findings..........................................................................................................................................................30 9-A-2-Conclusions....................................................................................................................................................31 9-A-3-Recommendations .........................................................................................................................................33 10-LESSONS LEARNED................................................................................................................................................34 FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 7 2-A-Glossary AACSB Association to Advance Collegiate Schools of Business ACH Automated Clearing House ACTED Agency for Technical Cooperation and Development Architectural Engineering, Consulting, Operations and AECOM Maintenance BSC Bank Services Center BTI Bankers Training Institute CBI Central Bank of Iraq CCN Labour cooperating country nationals CIB Credit Information Bureau COP Chief of Party CSD central securities depository FDP Financial Development Program Federation Federation of Iraqi Private Banks GDP Gross Domestic Product GoI Government of Iraq IFC International Finance Corporation IPBL Iraqi Private Bankers League IQD Iraqi Dinar IT Information Technology LLC Limited Liability Company LT Long Term LTTA Long Term Technical Assistance MBA Master of Business Administration MENA Middle East and North Africa MF Mortgage Finance MoF Ministry of Finance MoHESR Ministry of Higher Education and Scientific Research MOU Memorandum of Understanding NGO Non-Governmental Organization PERFORM Performance Evaluation and Reporting for Results Management PPP Public–Private Partnership PWC Price Waterhouse Coopers QED The QED Group, LLC RFP Request for Proposal RPS Iraqi Retail Payments System RPS Retail Payment System RPSI Retail Payments System Infrastructure FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 8 RTGS Real Time Gross Settlement STTA Short Term Technical Assistance TA Technical Assistance TBI Trade Bank International ToT Training of Trainers UAB Union of Arab Banks USAID United States Agency for International Development USG United States Government WB World Bank WG Working Group FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 9 3-INTRODUCTION AECOM and its subcontracting partners were awarded USAID/Iraq’s $53.3 million Financial Development Program (FDP). The purpose of this award is to develop the private financial sector so that it offers the best long-term prospects for providing credit needed to promote and sustain economic growth and job creation. This overarching objective is to be achieved through capacity building efforts, developing institutions that are currently missing from the financial sector architecture and addressing legal and regulatory constraints. The project interventions began in July 2010 and are planned to continue through August 2015. Total obligated amount as of March 2012 was $31.6 million, with nearly half that amount already spent. 3A-Purpose of evaluation As the project has reached its midterm mark USAID has commissioned a Mid-Term Evaluation. This evaluation will: provide an independent assessment of project’s performance; identify reasons why progress is or is not being made; and, recommend actions for USAID to take, either at the project level or with the Government of Iraq (GOI), in order to achieve the project objectives. 3-B-Audience The main audience for this evaluation report are USAID and FDP. The evaluation is meant to inform USAID on how successful the project has been in achieving its key objectives. The analysis presented here will also provide FDP with important advice on areas it needs to focus on to enhance its performance. In addition, the report can also be used by USAID and other US government agencies (such as US Treasury, etc.) as they plan and/or implement financial sector programming. 4-BACKGROUND Iraq’s banking sector is dominated by the three largest state-owned banks - Rafidain, Rasheed, and Trade Bank. Together, these three control over 90% of banking sector assets and over 70% of the loans in the banking system, the majority of which are non-performing. The state-owned banks have a monopoly on all public sector banking business, and do not play a major role in private sector financing. The country’s private financial system is small and underdeveloped and comprises over 40 private, Islamic, or foreign banks. The scale and scope of services offered by these banks is limited. In 2010, total credit to the private sector was about five percent of GDP which is significantly lower than the MENA regional average of 42%. Most of the private Iraqi banks are domestically-owned and account for only three percent of the total Iraqi banking sector deposits. FDP’s overall objective of developing the private financial sector assumes that dialogue and technical assistance for the Central Bank of Iraq (CBI) and other GOI institutions – coupled with advocacy support for private sector stakeholders – can facilitate adoption of a legal, regulatory, and institutional environment that will improve efficiency of the banking system. The project activities are organized across three main components: Component 1: Enhance Advocacy Capacity of the Private Financial Sector. Under this component, FDP has been implementing activities aimed at strengthening a private industry association(s) to serve its members through advocacy and training. FDP is supporting association(s) to serve as key interlocutors FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 10 with the GOI on policy, legal, administrative and regulatory reform and capacity development under the project. Component 2: Improve the Soundness of the Private Financial Sector Through Institution Building and Targeted Reforms. Under this component, the project is assisting with the establishment of a credit bureau, a bank training institute and a retail payments system. Component 3: Improve the Quality and Availability of Finance and Business Education. Under this component, FDP is supporting the development of modern business and finance curricula, faculty, and degree programs at select business schools/universities -- to provide a supply of qualified Iraqis to fill the gap in finance and business skills needed and to stimulate or support legal/regulatory/administrative reforms in the financial sector. 5-METHODOLOGY 5-A-Approach The methodology adopted to conduct this evaluation involved implementing the assignment in three stages between August and November 2012. Information on key stages of the evaluation and data collection tools utilized is briefly discussed below. Further details of the evaluation can be found in the Implementation Plan (attached as Annex 5-1). Stage 1 – Inception. During this initial stage the evaluation team conducted document review and prepared an implementation plan for the evaluation. Stage 2 – Data Collection and Analysis. During this stage, the evaluation team conducted fieldwork and developed detailed assessments and analysis of FDP’s activities and interventions. Stage 3 – Preparation of Evaluation Report. The focus of the team during this stage was to synthesize analysis and findings into an Evaluation Report. Main deliverables for this evaluation are: Draft and Final FDP Evaluation Implementation Plan; FDP Evaluation Report Outline; In-country USAID Initial, Mid-Term & Exit Briefings; and, Draft and Final FDP Evaluation Report. 5-B-Data Collection Tools The data collection tools for this mixed method evaluation are: key informant interviews; focus group interviews, direct observation and surveys. The evaluation team also developed an Evaluation Matrix that provides information about main takeaways from key stakeholders (attached as Annex 5-2). 5-C-Limitations Provided below are issues faced by the evaluation team in conducting the evaluation:  Lack of institutional memory due to high rates of turnover within FDP.  Key FDP stakeholder (mainly Federation) not forthcoming in sharing information about its finances and operations.  Difficulty in accessing some stakeholders due to security concerns. For instance, evaluation team could not visit Mustansiriyah University due to security concerns. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 11  Logistical challenges of coordinatingteam movements for internal and stakeholdermeetings in Baghdad. 6-COMPONENT 1-PRIVATE INDUSTRY ASSOCIATION 6-A-Does the Federation of Private Banks in Iraq have the proper focus and scale of operations? 6-A-1-Findings This section provides findings that will serve as evidence for evaluation team’s efforts towards answering the key Component 1 question. For this, the findings focus on two main organizations that provide advocacy support to the private banks operating in Iraq - the Federation of Iraqi Private Banks (Federation) and the Iraqi Private Bankers League (IPBL). Structure. FDP started work on developing the Federation around mid-2011, and the entity was legally established in April 2012 under Company Law no. 21. A total of thirteen banks invested 100 million Iraqi Dinars each in paid up capital to establish a for-profit Limited Liability Company (LLC). According to the law, the LLC structure allows the Federation to have between two and twenty five members. The IPBL was founded in May 2004 by 19 private banks, under a license from the Ministry of Planning and Development operating under the name of Iraqi Banking Center. Subsequently, the entity was licensed as an NGO. In compliance with the relevant by-laws, IPBL has created four committees and these are: Arbitration; Legal; Banking Studies; and, Banking Relations. Other ad hoc committees are established on an as needed basis. Composition of these committees includes: minimum of one or two Board Members; IPBL’s Managing Director; and, specialists from member banks. At the time when Federation was established, the NGO law was being rewritten and therefore, for around a year, the IPBL was out of status. This meant that existing NGOs had to reapply for registration under the new law. The new NGO law was introduced in February 2010, and IPBL received its certificate from the NGO Directorate in March 2012. Management. Federation management comprises of the General Assembly and the Managing Director. The General Assembly includes all Federation shareholders and is broadly responsible for reviewing and approving the organization’s strategies and activities in consultation with the Managing Director. The General Assembly members also participate in specialized committees, such as the Administrative Committee, Advisory Committee, and the Steering Committee of Foreign Affairs. The Managing Director is a full-time position appointed by the General Assembly. The assembly is also responsible for determining the Managing Director’s duties and responsibilities (attached as Annex 6-1) as per Article 123 of the Company Law (no. 21) of 1997. IPBL has a full-time Managing Director, Mr. Aziz Hassoun, who is assisted by six full-time staff. The organization has a Chairman, Mr. Adnan Al-Chalabi (Managing Director of Baghdad Bank) who works in concert with the Board of Directors. Membership. The Federation has thirteen members/shareholders and IPBL’s membership comprises all 32 private and Islamic banks, ten branches of foreign banks and two financial sector companies (information for Federation and IPBL membership attached as Annex 6-2). FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 12 Revenue Streams. According to the Work Plan provided by Federation, it plans to raise revenues by providing: legal/financial language translation unit; arbitration center; consultancy services; and capacity building for private banks. However, the evaluation team did not come across any evidence to suggest that the Federation has realized any income from these sources, and its only source of revenues are the General Assembly and grant funding from FDP. IPBL generates sufficient revenues to cover its expenses and is financially sustainable. It derives revenues from: annual membership subscriptions; book publishing; fees from training courses; and interest income generated from a deposit with North Bank. Advocacy Services. Provided below is a description of advocacy work being carried out by the Federation:  Training for bankers, NGOs, etc. Federation member banks interviewed informed the evaluation team that it has held eight courses and trained approximately 100 students. The Federation plans to send 12 bankers to a training course in Turkey in November 2012. The Federation has also been providing training support on Local Councils to ACTED (NGO). Information on a survey carried out by the evaluation Team of Federation training beneficiaries is provided as Annex 6-3.  Signed MOUs with international organizations (such as Union of Arab Bankers and Turkish Banks) to promote private Iraqi banks.  Participationinseveralconferences,andcurrentlyraisingsponsorshipsforanupcomingconferenceinBaghdad  Partnering arrangements with four universities. Universities informed the Evaluation Team that they have not progressed beyond signing an MOU with the Federation. Provided below is information on advocacy work carried out by IPBL:  Participation in World Bank programs  Partnership with Union of Arab Bankers and bank societies in Lebanon, Egypt and Jordan.  Participation in conferences  Consultation with CBI in private banks affairs.  Membership in the Development Fund of Ministry of Planning and invited regularly to sessions of all parliamentary committees dealing with the support of the private financial sector.  Contribution of efforts towards the establishment of a Bank Guarantee Scheme in Iraq which is presently under legislative review.  Training for bank staff including sponsorship of bankers’ participation in training abroad for capacity and business development.  Cooperation with USAID, CBI, Iraqi universities (curricula and syllabi development, lecturers). FDP Support. FDP has provided a substantial level of support to the Federation in both financial and technical terms.  Technical Assistance. FDP played a pivotal role in the establishment of the Federation in year two of the project. In addition, FDP developed advocacy training plans that were implemented through the Federation and also established linkages with regional and international organizations.  Financial Assistance. FDP provided four grants to FDP totalling $480,560. Details for these grants are provided in Annex 6-4. FDP did not include IPBL in its activities until two months ago when the project first contacted IPBL. They established a vision for cooperation, however, since then there has not been any progress. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 13 6-A-2-Conclusions In understanding the existing circumstances of the Federation, it is important to analyze reasons as to why it was established in the first place. According to USAID, IPBL being out-of-status on account of the NGO law being rewritten (as mentioned earlier) was used as leverage by FDP to register the Federation as a for-profit LLC company. Here, FDP management informed that if IPBL was not out of status they would probably work with them. It appears that the decision to establish a new association was mainly driven by FDP’s ambition to meet its contract deliverables related to providing support to at least one industry association. Provided below is information relating to Federation’s financial sustainability, management and competitor i.e., IPBL. Financial Sustainability. FDP set out to establish an association that would be financially sustainable. All efforts towards establishment of the entity were geared towards registering and operationalizing the entity. The evaluation team did not come across any evidence that FDP gave sufficient consideration to how the new association would be financially viable. The evaluation team has been unable to form a definitive opinion on the Federation’s revenue generation capacity. Main reason for this is that Federation was not forthcoming in sharing its financial statements. Financial information that was shared was incomplete, but did not provide any indication that the organization had generated any revenues. This experience was echoed by most of the stakeholders (such as, members/shareholders of the Federation, FDP and USAID) who said that they were kept in the dark with regards to the financial position of the company. Management. According to USAID and FDP stakeholders interviewed, the chosen LLC structure for the Federation provides a lot more power to the Managing Director than the NGO structure. Four out of the five private banks surveyed by the evaluation team were of the opinion that there are management issues within the Federation. They said that they would support a change in management and/or restructuring the organization. A General Assembly meeting was convened on 29th of September 2012, and it was decided then that the Managing Director of Federation would report back in thirty days to provide detailed information on the operational and financial position of the company. This information would then be used by the Federation member banks to decide upon the future of the Federation. In the days leading up to the General Assembly meeting, FDP management provided a letter of support to the Federation which the evaluation team believes was done to appease the Federation’s member banks and ward off attempts geared towards changing management or liquidating the organization. IPBL. An inadvertent outcome of establishing the Federation was that IPBL underwent a renaissance. Prospects of having a competing organization helped galvanize IPBL membership and they addressed governance issues that had held the association back. Currently, IPBL has established itself as a viable advocacy organization that has established a track record for supporting private banks. The organization is involved in a broad range of interventions that deal with financial sector development, policy reforms, etc. Establishment of the Federation is mostly acknowledged by stakeholders (private banks, FDP and USAID) as a misadventure and counterproductive. Evaluation team’s bank survey found that two of the five private banks questioned the reasons for establishing a competing body when an association (IPBL) was already in existence and that this move created friction and confusion amongst the banking community as well as GOI. Initially, FDP developed a negative assessment of the IPBL and used that as a basis to justify the establishment of the Federation. Even FDP management acknowledges that the FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 14 Federation has not been successful – in fact, recently, FDP severed all ties with the organization for a few months and plans to cut off its funding by end of October 2012.  Is the Federation and/or IPBL growing and thriving, in terms of membership and internal organization? Is there evidence of potential for financial self-sustainability? Over the last few months there has been rising dissatisfaction within the Federation member ranks with regards to the entity’s management and practices. Evaluation team’s interviews and survey of Federation members found that the promise of USAID funding was the main driver for them to join and become a shareholder/member in the organization. FDP focused on registering the entity and making it operational but did not give much thought to making the Federation financially viable. For instance, there is no evidence that FDP conducted analysis of how the organization was going to make money. This point takes on added significance given that the Federation was being registered as a for-profit company. A business plan should have been developed for the Federation at the outset. On the other hand, evidence indicates that IPBL’s membership is over three times that of the Federation and represents a larger cross-section of private financial sector stakeholders (in addition to private banks and Islamic banks the organization also represents foreign bank branches and finance sector companies). The organization is financially sustainable and meets its obligations by generating revenue from membership dues, book publishing, training courses and interest income.  What are the prospects for the Federation and/or IPBL to become an effective advocate for banking sector reform? Federation does not appear to be an effective advocacy body for private banks. Four of the five private banks that were surveyed for this evaluation were united in their lack of appreciation for the organization. Members of the Federation are unaware of the financial position and operational capacity of the organization. Moreover, they are not in favor of financing Federation’s activities when FDP funding dries up. Compounding this is the lack of Federation’s revenue-generating capacity, as it has been mostly financed by FDP grants and shareholders’ capital. Therefore, the future of Federation appears to be doubtful if it continues to operate in its current form. That said, all five banks surveyed by the Evaluation Team said that they would be receptive to the Federation refocusing its efforts towards working as a profitable concern and delivering business support services (e.g., trainings, etc.). Managing Director of Baghdad Bank informed that as an NGO the IPBL represents its members, whereas, as a for-profit (LLC) company, the Federation represents itself. Of the two organizations, IPBL appears to be better poised to represent interests of private banks. This is because IPBL: has more members; represents different types of financial organizations; is financially sustainable; and, is involved in government’s reform agenda.  It is a key assumption of the project that private industry associations can be an effective source of advocacy for reform contributing to better regulation, competition and efficiency in the banking sector. Therefore, is it apparent that the Federation and/or IPBL will become an effective advocate for banking sector reform? The evaluation team did not come across evidence that supports the assertion that an advocacy association representing the private banks only has been successful in bringing about meaningful change FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 15 in the financial sector. To put this into context, state-owned banks dominate the financial sector and control 90% of the banking sector assets, so, any meaningful advocacy will need to involve them. Therefore, the key assumption stated above that Component 1 activities are predicated upon needs to be questioned. There are serious issues surrounding the Federation’s future given that it is not financially sustainable. The situation will further complicate once FDP stops financially supporting the Federation as its shareholders might consider liquidating the organization. IPBL, on the other hand, has more members, is financially sustainable and is involved in a range of interventions geared towards policy reforms and financial sector development. However, this in itself does not readily lead to the conclusion that IPBL can play a vital role in influencing broad-based change in the country’s financial markets. Overall, IPBL is a better advocacy organization than the Federation, however, in the medium-term; its prospects for bringing about major reforms appear to be limited. 6-A-3-Recommendations  Cut and reassess Component 1 activities and deliverables. The project and USAID need to curtail spending on Component 1 and then decide whether or not to continue support of advocacy associations. This should be done in light of: the project’s experience with the Federation; the amount of money invested on Component 1 activities; and, more importantly, whether or not, advocacy is an effective means of influencing financial sector reform.  FDP should not provide any financial support to the Federation or the IPBL. FDP has already invested millions of dollars into supporting the Federation which is not an effective advocacy organization and/or financially sustainable. Given this, FDP should not provide financial resources to IPBL as there is a real risk that FDP might repeat the miss-steps associated with supporting the Federation. Therefore, the project should not provide financial support to industry associations.  FDP should help Federation establish a business plan. FDP can provide technical assistance to the Federation that will help the organization establish a clear business plan and thereby safeguard USG’s investment. Support here should be on a limited and short￾term basis and can help the organization: set operational and strategic priorities; explore ways in which to be financially sustainable; and, assess viable operation structures (e.g., streamline the existing operations, operate as an affiliate entity of the IPBL, etc.)  FDP should explore ways of engaging IPBL. The project can engage with IPBL up until USAID decides on whether or not to continue supporting advocacy-related activities under Component 1. Terms of this engagement should be limited i.e., short￾term technical assistance only. If it is agreed to pursue advocacy-related activities then, as a first step, FDP should consult with IPBL to understand from them what their needs and priorities are. Next, FDP should develop a time-bound action plan on ways in which support can be provided and secure an approval from USAID for the proposed activities. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 16 7-COMPONENT 2-TARGETED REFORMS 7-A-Is FDP now focused on a clear reform agenda, and if so, to what extent are these reforms appropriate priorities for the CBI, the Federation of Private Banks, and other stakeholders? 7-A-1-Findings Activities pursued under this sub-component are geared towards implementing four to five targeted policy, legal, regulatory and administrative reforms. The assumption here is that, once implemented, these reforms will quantitatively and qualitatively expand financial sector intermediations. For this, FDP is following a phased approach towards two main deliverables. The first deliverble is a prioritized reform agenda for financial sector reforms developed by associations and the second deliverable relates to supporting the implementation of four to five targeted policy, legal, regulatory and administrative reforms. FDP efforts here were led by the Legal Team and business/commercial banking experts were not involved. The First Deliverable. According to FDP, its legal team worked with private banks to identify policy, legal, regulatory, and administrative barriers and develop the prioritized agenda for financial sector development. The prioritized reform agenda was created based on assessments which covered the following relevant aspects of financial systems in Iraq: financial infrastructure, institutions, instruments, regulation, and Islamic finance. This was followed by FDP studying barriers facing private banks in developing their businesses, with a focus on legal and regulatory issues, which resulted in identification of different categories of barriers across several key areas. Then, the project held seven working sessions with private banks that culminated in a workshop “Creating an Agenda for Action” which was held on the 4th and 5th of April 2011. The workshop was meant to provide the private banks with an opportunity to identify and prioritize key issues. The workshop resulted in the development of a matrix (attached as Annex 7-1) that provided a summary of Prioritized Reforms. The summary was meant to serve as a foundation for actions relating to enhancing private banking in Iraq. FDP developed the first deliverable, namely, a list of 12 Prioritized Legal and Regulatory Reforms (attached as Annex 7-2) and the list was approved by USAID on August 4, 2011. The Second Deliverable. Next, FDP identified four to five targeted reforms for possible implementation. This list was developed in October 2011 and covers the following areas: standardize procedures/remove legal barriers in Mortgage Finance; provide support to establish Secured Transactions systems; provide support to development of Letters of Credits by private banks; provide support to development of Deposit Insurance; provide Support to Credit Reporting Policies and Procedures; and, provide Support to Retail Payment Systems. As per FDP quarterly reports, the reforms areas (above) were identified in conjunction with private banks and that, initially, involved working with the Legal Committee of the Federation. In July 2012, FDP decided to establish a working group (WG) comprised of a wider group of private banks as the vehicle to carry out and implement the reforms instead of working with the Federation’s Legal Committee. FDP advised the evaluation team that the reason behind this decision was to give the private banks a voice in development of the reforms (information on the Working Group’s composition is attached as Annex 7-3). It is important to note, that the working group has no representation from CBI. Also, feedback received FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 17 from two of the members of the Working Group (Ashur, and Iraqi ME banks) suggests that they have different views on what the list should include as targeted reforms. The list of target reforms was developed in October 2011 and the working group held its first meeting in September, 2012. FDP attributed the time gap to the following:  The departure of the Project’s previous Legal and Regulatory Advisor in January 2012, his replacement arriving in Baghdad in May, 2012.  While the legal team has worked on all areas of reform, since October 2011 its priority has been in the areas related to regulations governing Credit Information and Payment Services, but in year three the work will shift to the other four areas. 7-A-2-Conclusions FDP has fulfilled the first deliverable (list of prioritized reforms) and is now focused on the second, namely implementing four to five targeted reforms. The agenda is indeed clear, but the contents are debatable. As stated in Findings, CBI did not take part in the process of selecting those reforms, and the outcome, thus, does not reflect CBI’s priorities. Only recently are the private banks being adequately consulted and engaged in development of the targeted reforms and the furture will tell whether or not the list of reforms reflects their priorities. However, from discussions the evaluation team had with a number of those banks, there is strong likelihood that the banks will revisit that list.  Do the reforms that were identified in a market assessment conducted by FDP in 2011 have enough support, such that, with USAID assistance, they are likely to happen or are they so extreme that even strong advocacy from the private sector and good technical assistance to the CBI would not make a difference? The previous answer addressed the issue of support forr the reform agenda. As for the likelihood that the reforms are implemented, the evaluation team’s assessment of the final list of targeted reforms is not altogether positive. For starters, the identification was carried out by legal, rather than business/commercial banking experts who are more capable of identifying priority areas with good chances of success.. Inclusion of RPS and CIB has also resulted in holding back the process, as focus of the “reforms” team was dedicated to these two items. Following is a brief commentary on the List (excluding RPS and CIB):  Mortgage Finance (MF). Not cited as an area of priority by any of the five banks surveyed by the Evaluation Team. Besides, what Iraq needs is building millions of new housing units. Construction finance is what the country needs now; it will be many years before MF is really needed.  Secured Transactions. This is a low priority in the short term. The situation should change as the sector develops, but right now there are numerous and diverse deficiencies in registering and enforcing collateral, rendering the chances of success in tackling them slim.  Private banks getting a part of government letters of credit and trade business. This is a very high priority for private banks, however, there are no legal barriers stopping this from happening so it is not an area for legal or regulatory reform work. Reversal of the Ministry of Finance’s (MOF’s) current stance will require persistent advocacy and improvement in performance of private banks. (See Minutes of meeting with MOF attached as Annex 7-4).  Deposit Insurance. This is becoming a high priority especially after the bankcruptcy of private banks i.e., Basra and Warka bank. Also, such an initiative, if implemented, might help improve MOF’s FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 18 perception of private banks as high risk counter parties. However, FDP should coordinate with CBI as they have already started work on this.  Are there other options (public/private, formal/informal) that should be explored as a basis for supporting the reform effort? It is the evaluation team’s view that it would be more effective to properly engage the core stakeholders, namely CBI and the banks, before exploring other options; without them, this whole exercise would prove to be futile. The initiative of FDP for future involvement of other parties, primarily reform-minded entities which they have termed "Partners for Reform" is not a bad idea. However, given the fact that the said partners are neither GoI nor CBI related, and not banks, FDP should tread carefully, as this might risk repeating the painful experience of Federation vs. IPBL if the "true" stakeholders view the contemplated partnership as a way to exclude or even coerce them.  What are more tenable reform areas? Provided below is information for other more tenable areas of reform.  Banking intermediation in Iraq is amongst the weakest in the world, and what little lending there is, depends on collateral and not on credit assessment. Introduction of risk analysis and measures is imperative to move away from the narrow collateral based lending. A priority area of reform is suggesting CBI regulations aimed at making it mandatory that banks introduce risk management function and tools.  According to CBI, there is a large supply gap in housing, and, thus, a need for a wave of reconstruction in Iraq. Securing funding for large projects in that sector is a challenge, caused mainly by the inability of banks to lend those projects because they have no physical collateral to offer. FDP could assist CBI to issue regulations/instructions governing construction finance, laying out for banks the structures associated with providing loans in a percentage of “good” receivables owed to contractors. 7-A-3-Recommendations The legal team of FDP has exerted considerable effort which led to good quality work. However, it is the evaluation team’s opinion that this is not a measurement of success for activities being implemented here. Success should be measured by the impact the work has produced towards expanding bank and non-bank financial institutions and services. Therefore, our recommendations going forward are as follows:  First order of priority is for FDP to engage CBI, preferably as member in the WG, or through an established mode of constant consultation with the WG.  FDP should allow the WG to decide which areas should constitute the four or five targeted reforms whose implementation would “quantitatively and qualitatively expand financial sector intermediations”.  Next, FDP should agree with the WG on a road map towards implementing the chosen areas of reform, envisaging specific, short-term tasks for FDP. Finally, it is important to impress on the WG that they will be “championing” this effort and that FDP will support them. This is critical to achieve success. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 19 7-B-How important is it that a Retail Payment System as originally envisioned by the project be implemented at this stage, now that the CBI has departed from the original plan? 7-B-1-Findings The objective of this sub-component of Component 2 is the establishment of a comprehensive RPS, in the context of improving the soundness of the private financial sector by enhancing the infrastructure necessary for expansion of bank and non-bank financial institutions. Deliverables under the contract envisage the achievement of Phases 1 and 2, which include all tasks of research, study, identification of business model, issuance of instructions, and others, culminating in advising CBI on the selection of the National Switch provider and on monitoring and supervision of implementation. FDP has performed the tasks leading to those deliverables, and the only delay is in the final stage, namely, issuing the Request for Proposals (RFP) for the National Switch. As per their own admission, CBI is responsible for that delay, however, now the process is back on track. Current National System. The current payment system within CBI [Iraqi Payments System (IPS)] does not cater to retail payments, as it only covers real time gross settlement (RTGS), automated clearing (ACH), and central securities depository (CSD). CBI informed the evaluation team that the IPS needs to be upgraded to provide a totally secure and reliable Iraqi Payment Systems infrastructure encompassing both an Inter-Bank payments Infrastructure (Wholesale banking backbone) and a retail payments service (National Retail Banking and Payment Systems) Infrastructure, encompassing:  Full integration of payment systems between the banks and the CBI with a high degree of automation  Higher service reliability, availability and operational resilience for domestic and cross-border payment  Elimination of the riskier payment instruments especially checks  Control of fraud and money laundering risks and prevention of the financing of terrorism  Total transparency, to help overcome inefficiency and corruption  Reduction of use of cash (i.e. move towards a cash-less society) to avoid security risks and cash handling costs by developing a consumer-oriented retail payment system. This should include payroll and direct debit services mediated by the ACH as well as card and mobile payment systems. List of omissions and deficiencies that will be overcome by upgrading the currently installed versions of the IPS components to the latest available are provided in Annex 7-6. Montran is the firm which provides and supports IPS, and they have confirmed that the said upgrade could be completed in time for RPSI. FDP is now in the final stage leading to the issuance of the RFP for license award. CBI’s Steering Committee and its Tender Evaluation Sub-Committee are in place, and PWC has been selected as independent RFP Advisor (after competition with Protiviti) and has been authorized to issue the RFP. FDP expects such issuance to take place by mid November, the process up to license award to take 4 months, and implementation – conservatively- to take 18 to 20 months. The draft RFP envisages a phased implementation, which should result in providing early benefits and impact on the Iraqi financial sector, without waiting until the full implementation of the system. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 20 The approach adopted for RPSI implementation provides that a consortium is awarded a five year license to build and operate the system. This rids CBI of the burden of investment cost, transferring that risk to the prospective consortium who will need to carry this burden and be financially sustainable. Implementation Challenges. FDP worked closely with CBI as the main stakeholder. Provided below is information for two main challenges faced in implementing the RPSI: 1. In November, 2011, when most of the work running up to issuing an RFP for award of a five year license for RPSI had been completed, CBI brought the process to a halt. During meetings with the evaluation team, CBI acknowledged that it was the cause of what it termed a “justifiable delay”. The delay stemmed from two main concerns that CBI had. The first was a regulatory concern: when they became so close to launching a project entailing all kinds of retail transactions, including the “tricky” mobile payment system, CBI wished to probe more into regulatory issues. On that, FDP helped CBI develop tight regulations and instructions. The second concern was about possible criticism from other ministries and governmental entities regarding awarding a contract to a vendor who would have a monopoly on running the payment system. FDP addressed that concern by recommending to CBI a process for award, involving several government stakeholders through a Steering Committee, which would then appoint a subcommittee to act as the Tender Evaluation Committee. The recommendation went on to envisage the appointment of an independent Advisor (international firm) to evaluate the bids, recommend the winner to the Evaluation Committee, who would then take it to the full Steering Committee for endorsement. This recommendation was accepted by CBI, resulting in the project moving ahead again after it was halted for a long period of time. 2. The second hurdle pertained to the aspirations of Trade Bank International (TBI) to launch its own “national” switch. Taking the opportunity of the “freeze” by CBI, TBI announced the idea and started lobbying for it. However, both FDP and CBI made clear to TBI that there is no such thing as more than one National switch in a country, and CBI advised that the delays did not mean that there would be a change in the adopted business model (a consortium to be awarded the license for RPS). CBI’s position eliminated this challenge. CBI Commitment. After the removal of the challenges discussed above, CBI’s commitment to RPSI was cemented by an announcement in a highly publicized public forum in July 2012, and also put in writing via a letter signed by the Deputy Governor to USAID Mission Director that confirmed CBI’s commitment for supporting and supervising the project (letter attached as Annex 7-5). During our meetings with CBI, the evaluation team was advised first hand of this commitment. CBI officials made it very clear that developing RPS is on the top of their priority list, stating that it has “crossed the point of no return” towards implementation. FDP Support. In meetings with the evaluation team, CBI officials praised the support they received on this project from FDP. 7-B-2-Conclusions At this stage, and as was stated to us explicitly by CBI, RPS is the Bank’s top priority. We do not consider the alterations that CBI made in the process to be a departure from the original plan, but rather an over cautious approach, not uncommon in government organizations.  What is preventing the implementation of the Retail Payment System (as originally envisioned) and which future activities, if any in this area should be supported by FDP and USAID? FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 21 FDP did well to resolve the deadlock caused by the concerns which CBI had, by understanding the concerns, recommending solutions, and helping implement those solutions, thus paving way to CBI publicly announcing its commitment to the project.There is nothing preventing the implementation of RPS. As explained above, there were delays resulting from changes to the process, not to the concept or business model. 7-B-3-Recommendations In view of the importance of the RPS project to CBI and other stakeholders, the evaluation team recommends continuation of support by FDP and USAID. The tasks envisaged by FDP for the current and subsequent phase, up until the system goes live, are adequate. Provided below are recommendations for additional areas of support: 1. During meetings, CBI informed the evaluation team that it would like to launch an awareness campaign to ready the market (the public as well as the banks) for growth in using non cash payment services. The aim here is to assist the new company that will build and operate the RPS to achieve financial sustainability. FDP should assist CBI in this regard. 2. Iraq is almost a 100% cash society. This will not change merely by establishing the RPSI. While an awareness campaign is a must, it will need to be backed by other direct interventions that will increase the use of non cash instruments. A major step that would make a quick impact is changing the payroll system in as many government bodies as possible to electronic payment. 7-C-What are the causes for lack of progress in the start-up of a Credit Information Bureau and what might be more effective approaches than those already employed in the project? 7-C-1-Findings In the context of its efforts to enhance financial sector intermediation in Iraq, USAID paid special heed to the important enabler of availing and exchanging credit information on borrowers, through a properly structured credit information bureau (CIB). Deliverables for CIB for years 1 and 2 are as follows: Iraqi driven credit bureau business plan and model developed; legal and regulatory requirements for credit bureau identified and recommended changes drafted; and, public education campaign on the role of the bureau launched. FDP has fulfilled the first two deliverables, while the third has been ready for some time and is pending approval from CBI. Existing Credit Registry. The existing registry of credit information is handled by a department within CBI (Credit Information Exchange Department) which has 16 employees. The system is paper-based, not automated, and manually updated. Online upload and online inquiry functionality are not available. This can result in mistakes by banks’ staff when they manually fill the forms and/or by CBI staff when they manually re-load the information. Only banks are party to the system and non-bank financial institutions are not. While all banks are mandated to contribute portfolio data, CBI informed the evaluation team that this is not always the case and that the current system is unable to ensure that banks are in compliance, or detect those which are not. Updates (tapes or paper) are provided monthly by banks and are then manually uploaded by department staff into the database. This process consumes around three weeks of the month FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 22 and severely affects the turnaround time for credit enquiries. It takes enquiring banks five to ten days (sometimes more) to secure credit reports. Information in the database neither provides consolidated statistics nor credit histories. These deficiencies do not allow the lending banks to carry out a proper credit assessment. The lengthy process of enquiry and the doubts in the reliability of the information obtained, discourage banks from fully participating in the current system. All five banks surveyed confirmed to the evaluation team their appreciation of the importance of credit information and pledged support to any new system that will provide them with reliable data. Different Structures for CIBs: CIBs are differentiated by their ownership; there are generally three categories: 1. Public owned bureau, placed within the central bank (usually Supervision Department). 2. Private bureau, owned and operated by a private company, supervised by the central bank. 3. Public-private partnership (PPP). From the outset, CBI has only been interested in option 1 (above), and FDP’s Business Plan for CBI dated July, 2011 supported this position. The Business Plan concluded that this structure represents the most feasible approach for Iraq at this time and is the most likely to succeed. FDP Support FDP intends to persuade the CIB Project Manager to accept the newly prepared Road Map which envisages acquiring a new software bureau system by mid-October 2012. 7-C-2-Conclusions FDP did a lot of work under this area, however, the fact remains that despite the promising development of identifying the preferred structure in early 2011, there was not much progress towards the actual establishment of a CIB. FDP maintains that there are neither delays nor lack in progress. This position is substantiated by contractual deliverables which, as shown in Findings, have been met by FDP. However, from a professional perspective – contractual obligation aside- there is indeed cause for passing a judgment that the process should have been further ahead than where it stands now. It might be that at the time of launching the project, allowance for more time had been made for the finalization of the business plan and model, to cover such tasks as study, research, diagnosis, gap analysis, insights from neighboring countries, and other. What transpired though was that the plan and model were finalized by FDP in July, 2011, and the recommendation was in line with what CBI was aiming for, namely, establishing the CIB within CBI, by upgrading the current Information Exchange Department. There was, thus, an early achievement of a milestone, and no opposition from the main stakeholder. Accordingly, it is fair to state that more concrete steps should have been accomplished by now, even if they were not mandatory. Based upon discussions with FDP and CBI we have come up with some deductions regarding the perceived delay:  The turnover within FDP of the experts working on this area has been high. CBI (Supervision) pointed this out, and complained that with every new expert coming on board a lot of time was spent going through discussion of areas which had been previously covered. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 23  CBI’s halt to the implementation of RPSI, mentioned in section 7-B-1 Findings above, also appears to have affected CIB as well.  It appears that FDP let the Project Manager within CBI set the pace for the advancement of this area. The Project Manager within CBI is the Head of IT. Banking Supervision, the “business” side for such project within CBI, were not adequately involved. This rid the project of a strong advocate, as the focus of IT was on a single issue which had nothing to do with the essence of the initiative and its importance for the banking and financial sector, namely, insisting on developing the current IT 1 system as opposed to acquiring a new system. FDP’s preference was to acquire a new system designed to include a platform for new products and services such as the scorecard, risk alert, fraud detection, and other. The debate on this issue has taken a long time, during which FDP focused its efforts on convincing the Project Manager. When questioned about this, FDP’s response to the Evaluation Team was that they were afraid to approach the Deputy Governor as this might cause the Project Manager to later sabotage the process by not cooperating. The evaluation team believes that an early “professional escalation” of this issue would have saved a lot of time and effort and that FDP’s “fears” were overstated, given the prospective involvement of other functions within CBI in implementation. 7-C-3-Recommendations In view of the current status of financial intermediation in Iraq, the evaluation team concurs with the recommendation that CIB is established within CBI. This can be accomplished by upgrading the capacity of human resources in the current Credit Information, and IT Departments, and by acquiring a new bureau core system that is designed to meet current and future requirements. In this context, we recommend the following immediate steps: 1. A final resolution of the issue of whether to develop a CIB system in-house or to acquire a new system. The issue is clear and warrants that FDP should escalate the matter to the Deputy Governor in the event the Project Manager continues to hold back the process. 2. Once the CIB system issue (above) is resolved, FDP should secure sign-off from CBI on a time-bound action plan for the CIB project. This will ensure the Bank’s continued commitment. 3. FDP should explicitly demand the involvement of Banking Supervision – the “business” side within CBI- in the project. Otherwise, further delays would probably occur when they are involved at later stages. With regard to FDP support, evaluation team has the following recommendations:  FDP support for CBI Implementation should be delivered through one Long-Term Technical Assistance (LTTA) expert. The evaluation team is in favor of having the expert currently supervising this area (Mr. Robin Watson) to continue assisting in that capacity. This would ensure “ownership” within CBI and accelerate the learning process of the project team. 1 The Evaluation Team discussed this with FDP and asked them why the project did not work with the business side and, in the debate over the issue of the system, could this have been better managed if the issue had been escalated. FDP’s response was that in future they will involve the business side more. On the second issue, they stated that they feared “retaliation” from the Project Manger during implementation. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 24  Any other support needed (legal or technical) should be sought on a short-term basis for defined periods of time.  The “Milestone Plan for CIB Test and Production Environment” recently developed by FDP should be included in the Work Plan, as it contains specific planned deliverables culminating in full implementation.  The implementation plan for CIB should be closely monitored by FDP, on shorter intervals than is customary, to allow for intervention and corrective action in the event of delays caused by human or technical deficiencies. 8-COMPONENT 3-TRAINING AND CURRICULA DEVELOPMENT 8-A-Is the project making any progress in helping the business schools to set up effective business degree programs? 8-A-1-Findings In July 2011, FDP conducted a Skills Gap Analysis to identify core areas of weakness needing to be addressed in both the curricula of Iraqi business and finance university faculties and the skill sets of the core staff at private banks. The study noted that deficiencies exist in: English proficiency; technical (hard) skills in finance and banking; and, planning, communication and negotiation skills, leadership training and understanding of business ethics (soft skills). The study formed the basis for FDP's future interventions carried out under Component 3. Activities implemented under Component 3 are geared towards providing technical and financial assistance to Iraqi universities for: Association to Advance Collegiate Schools of Business (AACSB) accreditation; curricula development; and training. Accreditation: The AACSB is the worldwide leader in accreditation standards for business schools. Accreditation by AACSB suggests that the level of a business administration and/or accounting program is at par with international standards and in line with the needs of the financial sector. For accreditation, a university is required to fulfill 21 qualification criteria. Accreditation is a lengthy process and requires an extensive commitment of resources and time. At a high level, the eligibility procedures for an institution applying for accreditation for the first time are that: institution must be a member of AACSB international; institution must offer degree-granting programs in business or management; degree programs in business must be supported by appropriate administrative structures; all degree programs in business offered by institution at all locations need to be reviewed simultaneously; institution must demonstrate diversity in its business programs, institution and its administrators; faculty and students must operate ethically, etc. Further details on the eligibility procedures are provided as Annex 8-1. Once a program is accepted by AACSB the accreditation process will be renewed every ten years so as to ensure that the institution maintains a high level of quality of teaching/faculty. FDP is supporting the accreditation of four domestic universities by partnering them with three international universities. The table below provides information on the partnerships established here. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 25 Foreign university Iraqi university University of Dubai, U.A.E. Mansour University College, Baghdad Shippensburg University, U.S. University of Baghdad, Baghdad James Madison University, U.S. Mustansiriyah University, Baghdad James Madison University, U.S. Salah Al Din University, Erbil Information on progress realized by Iraqi universities is provided below:  Baghdad University, Mustansiriyah University and Mansour University College were granted membership status by AACSB in April 2012.  According to Dubai University, it will take around eight to ten years for Mansour University College to secure AACSB accreditation.  According to Shippensburg University, Baghdad University could achieve AACSB accreditation in five years.  Initially, the level of cooperation between James Madison University and Mustansiriyah University was high, however, all this changed following the tragic incident that happened in June 2010 when the Associate Dean of the School of Business, American University of Cairo was killed in Baghdad. Currently, for security reasons, James Madison University has decided not to travel to Baghdad anymore.  Accreditation process for Salah Al Din University in Erbil started in May 2012 and is still in its initial phase.  A fifth university, Basrah University (public), will join the program shortly. Further details on the partnerships between Iraqi and foreign universities is provided as Annex 8-1. Curricula Development. Upgrade of business curricula is one of the key requirements for AACSB accreditation. The Skills Gap Report noted that curricula development is needed so that students can acquire more technically-oriented, banking-related skills while, at the same time, balancing specialized courses with general education courses or experiences that build soft skills. All three deans of domestic universities informed the evaluation team that their faculty received guidance from FDP on upgrading their curricula (key informant interviews). For instance, a Training of Trainer (ToT) workshop on “Integrating Entrepreneurship Principles into Curriculum” was held in August 2012 (two sessions in Baghdad and Erbil) by a professor from University of South Florida. In total, 103 participants attended the workshop and resulted in the professor sending a Letter of Invitation to the Iraqi faculty for a one month study visit at University of South Florida to study development of Centers of Entrepreneurship, Centers of Career Development and Centers of Internship. All Iraqi universities contacted by the evaluation team informed that they were committed to upgrading their curricula but were, in some instances, constrained by government regulation. According to these regulations a university can change up to ten percent of its curriculum annually without securing Ministry of Higher Education and Scientific Research (MoHESR) approval. If a university wants to make changes FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 26 over and above ten percent it will need to secure approval from the MoHESR’s Curriculum Committee (public private partnership) which is a bureaucratic/lengthy process. FDP informed the evaluation team it they have decided not to play a direct role in the MoHESR Curriculum Committee as this might be detrimental to the objectives of the project. However, the project claims that it has built a solid relationship with MoHESR and that the ministry supports the reform process. This was recently confirmed by the ministry when it advised all Iraqi universities to apply for AACSB accreditation. Grant agreements signed with the foreign universities provide that they advise their Iraqi counterparts on important aspects of curricula development through: Training of Trainers (ToT) workshops; study tours; direct advice, etc. Training. Training support delivered under Component 3 is mainly provided through ToT workshops. A majority of the ToT workshop participants are faculty staff of universities, colleges and institutes across Iraq. A small percentage of the participants are from the public sector (e.g., MoHESR, etc.), chambers of commerce and NGOs. So far, FDP has organized four ToT workshops and one all-day meeting between Shippensburg and Baghdad Universities. Total enrolment for the four ToT sessions was 259 participants. All workshops were delivered by foreign university professors except for one that was conducted by an FDP Long-Term Technical Assistance (LTTA) expert. Usually, professors of international universities are retained by FDP and contracted on Short-Term Technical Assistance basis. Provided below is information for the four ToT sessions:  Monterey Institute of International Studies conducted a workshop on “Current Issues in Finance and Banking Education” in December 2011  Pittsburg State University conducted a workshop on “Current Developments in International Finance” in June 2012  University of South Florida conducted a workshop on “Integrating Entrepreneurship Principles into Curriculum” in August 2012.  FDP LTTA conducted a workshop on “Applied International Finance,” in September 2012. Interviews with a sample of 34 participants of all 4 ToT workshops show that the great majority of them were very satisfied or satisfied with the relevance of the training, the competence of the trainer, the 2 contents of the training and the provision of required soft skills (teaching techniques) . Survey results of 34 participants of ToT workshops organized by FDP are provided in the table below. 2 The surveys were conducted by 2 data collectors of the evalutation team between September 30, 2012 and October 3, 2012. Survey sample comprised of eight participants from each of the three ToT workshops conducted by foreign universities along with ten participants of the ToT workshop conducted by the FDP LTTA expert. All ToT participants were intereviewed over the phone. Data of each survey was collected and collated seperately. Details of the survey results can be found in Annex 8-2, Annex 8-3, Annex 8-4, Annex 8-5 and Annex 8-6. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 27 Table: ToT Participant Survey Results FDP informed the evaluation team that it has scheduled two additional workshops in October and November 2012. Financial Support. FDP provided seven grants to universities totaling $1,528,424. In addition, all seven grants have a significant grantee contribution portion. Grant level information is provided as Annex 8-7. The grant agreements expire between December 2012 and July 2014, i.e. before the end of the lifetime of FDP. 8-A-2-Conclusions Business schools have started setting up more effective business degree programs by implementing AACSB requirements. Here, the Iraqi universities are supported by the project and by their foreign partner universities. The universities have initiated work on broad-based reforms that are geared towards curricula, institutional and faculty development. In addition, the universities are also working towards making their programs more relevant by linking them with the business and financial sector through establishment of joint business advisory councils, centers of entrepreneurship, and internships. The degree of buy-in demonstrated by the domestic partner universities to the Evaluation Team has been high across the board. Mansour University College and Mustansiriyah University are very receptive to pursuing reforms, and they consider AACSB accreditation their primary future objective. Similarly, Business and Administration faculty of Baghdad University is administering changes to upgrade its programs, such as, establishment of a business council and offering students internships in banks. Mustansiriyah University considers it a major obstacle that their partner university does not want to visit them anymore and would like FDP to address this issue. James Madison University’s reluctance to travel to Baghdad was due to the tragic incident that happened in June 2010 when the Associate Dean of the School of Business, American University of Cairo was killed in Baghdad. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 28 Overall, there is evidence that the domestic universities have started working towards establishing more effective business degree programs.  Training for future employees of Iraq’s private banks is intended to be achieved through university training programs – such as have been the goal of partnership grants with foreign universities, crafted to develop curricula at local Iraqi universities. Will partnership grants to foreign universities lead to the development of more relevant Iraq-based certification or degree program(s) than existing academic offerings? Iraqi higher education programs in business and finance lack private sector orientation and are not in line with international standards. In the Skills Gap Report academic programs in business, finance and banking were even characterized by antiquated teaching pedagogies. For these reasons, Iraqi universities had to be partnered with reputable, accredited international universities in order to develop more relevant business and finance degree programs which would then help them through the AACSB accreditation process. Accreditation can only be achieved if institutionalized relevant business and finance programs are offered by the respective university. Iraqi partner universities have commenced to build the respective institutions such as the creation of joint business advisory councils, but there is no cooperation with other Iraqi institutions also active in business and financial education yet. We are of the opinion that partnering Iraqi business faculties with foreign, accredited universities is a promising strategy. By providing them a sustainable mechanism (through AACSB accreditation) more relevant Iraq-based certification or degree programs will be achieved.  Will the business schools meet accreditation standards, especially with regard to curriculum development? Only approximately 80 universities outside the U.S. have achieved AACSB accreditation with three universities in the U.A.E., two in Turkey, one each in Saudi Arabia and Egypt. Iraqi universities have gradually started to upgrade their curricula and to develop their business programs. Initially, their focus is to change up to ten percent of their curriculum which they can change annually without ministerial approval. Given the high AACSB standards and the low level of Iraqi higher education system in business and finance, none of the business schools will achieve accreditation during the lifetime of the project. University of Dubai indicated that AACSB accreditation for Mansour University College, with their help, can be achieved in eight to ten years, otherwise, it would take between 18 to 20 years. University of Shippensburg advised the evaluation team that it believes that AACSB accreditation for Baghdad University, with its assistance, is possible in five years. The evaluation team agrees with estimates provided by University of Dubai and considers position of University of Shippensburg to be too optimistic. The evaluation team has witnessed Iraqi universities instituting accreditation related reforms and making gradual progress, however, it is unlikely that accreditation standards will be achieved by 2015. 8-A-3-Recommendations  Establish milestones to measure progress in the short to medium term As no business school will achieve accreditation standards by 2015, provisional milestones should be used to measure progress. For instance, provisional milestones can be developed so as to measure progress against the 21 AACSB accreditation criteria. FDP can utilize these milestones to chart progress in all universities that are working towards accreditation. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 29  Continue delivering assistance to partner universities FDP should continue to financially support its partner universities until the end of the project. This will result in more reforms being instituted in the domestic universities and increase the possibility that progress realized is not reversed.  Establish linkages with financial stakeholders FDP should encourage universities offering business and finance programs that they establish linkages with financial sector stakeholders (e.g., BSC, IPBL, banks, etc.). This will enable the universities to be more responsive to the needs of the market.  Terminate the Mustansiriyah University/James Madison University partnership FDP should terminate the partnership between James Madison University and Mustansiriyah University and replace this university with another foreign university that is willing to travel to Baghdad, Iraq. 9-PROJECT MANAGEMENT: AECOM 9-A-Is AECOM providing good project management? 9-A-1-Findings This section provides findings that will serve as evidence for evaluation team’s efforts towards assessing how effectively and efficiently the project, as a whole, is being implemented. The findings focus on the following aspects of FDP: management; financial performance; and, capacity building of local staff. Management. FDP is being implemented by AECOM and it subcontractors that include: The Louis Berger Group; 4Points; Monetics; sYnergy; and Sallyport Global Services. In total, FDP has 53 employees working across the three main components and in various operational capacities. There are 28 officers working on the three components. Each of the three components is led by a Director who is assisted by a Deputy Director and subject matter experts. Table below provides information for FDP by employee type. FDP Employee Type Breakdown Component Long Expat Term Long Term Local STTA Expat Short Term Local Staff Total Component 1 4 6 0 1 11 Component 2 3 4 4 0 11 Component 3 2 3 1 0 6 Administration & Other 5 13 1 6 25 Total 14 26 6 7 53 FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 30 The program has experienced significant turnover across its key leadership positions (e.g., the project has had three COPs in 2012 and it took the project six months to replace the Director for Component 2). Project turnover information is provided as Annex 9-1. (e.g., the project has had three COPs in 2012 and it took the project six months to replace the Director for Component 2). Financial Performance. Overall, total cost incurred in implementation of FDP is $21.7 million. The table below provides estimate values for the costs incurred by FDP across each of the three components. Due to AECOM and USAID accounting requirements, it was not feasible to secure exact costs per component. Cost information utilized to arrive at these estimates includes AECOM3 fees and payments made to subcontractors4 . Findings related to costs incurred here are provided below and outputs related information for each component is provided in Annex 9-2. FDP spent:  $8,013,647 in order to deliver outputs associated with Component 1.  $6,925,897 in order to deliver outputs associated with Component 2.  $6,775,870 in order to deliver outputs associated with Component 3. Capacity Building of Local Staff. FDP has recruited six Iraqi officers as Deputy Directors and it is envisaged that these officers will take ownership of FDP activities and serve as Directors in either later Year 3 or early Year 4 of the program.The six Iraqi officers comprise the Iraqi Deputies Group. 9-A-2-Conclusions AECOM’s performance in implementing FDP varies from one Component to another. In part, this is due to the diversity of activities that are being implemented under the different components. Overall, the approach adopted by the project has largely been deliverable-driven where the focus is on ensuring that AECOM satisfies delivery of its contractual obligations. The downside of following such an approach is that, at times, AECOM gave limited consideration to important issues like continued relevance, sustainability and efficiency of project activity and deliverables. As mentioned earlier, the project has experienced a high rate of turnover that has: resulted in disruption and delays in program implementation; caused concern and confusion amongst key stakeholders (such as, GOI/CBI, Federation, IPBL, private banks, etc.); and, resulted in key management roles being unfilled for long periods of time. Provided below is information on activity implementation, cross-component synergy, communication, linkages and follow-through and monitoring and evaluation for FDP’s three components. 3 AECOM expenses, for each component, includes: Expat Labor; CCN Labor; Travel, Transportation, and Per Diem; Allowances; Other Direct Costs were distributed evenly across the components. 4 Subcontractors costs includes payments made to subcontractors as well as Other Direct Costs. Component Cost Breakout Component 1 0% 100% Component 1 Component 2 Component 3 AECOM $2,673,387 $954,040 $2,340,855 Subcontractor $4,859,700 $5,934,702 $2,906,591 Grants $480,560 $37,155 $1,528,424 FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 31 Activity Implementation  Under Component 1, AECOM invested significant resources in establishing the Federation and ignored critical issues relating to selecting the correct structure and making it financially sustainable.It appears that in order to satisfy their Component 1 deliverables, the project spent over eight million dollars to establish an advocacy association as a “for profit company” that does not have revenue generating capacity. FDP management agree that they did not give any consideration to whether or not this entity could generate revenue. This constitutes a major oversight on the part of AECOM.  Under Component 2, RPSI is the project’s success story where FDP performed very well. The project is on target on CIB, but there was ample opportunity to jump ahead of schedule that FDP did not take advantage of. Legal and regulatory reforms is the least successful activity as banks were not properly involved, and CBI stayed away.  Under Component 3, FDP has done a satisfactory job. The previous and the current Component Director continue to implement activities in an effective and participatory manner. Cross-Component Synergy Evaluation team observed evidence of FDP’s legal team delivering support across all three components of the project and providing advisory support on legal issues associated with establishment of the Federation (Component 1); related to Target Reforms, CIB, RPSI and BSC (Component 2); and, on entering into partnerships with universities (Component3). FDP documents referred to the Federation as serving as an important vehicle for cross-component cooperation. This involved Federation committees supporting key interventions implemented under Component 2 (such as, Target Reforms, RPSI and CIB). Evaluation team could not validate these claims as there was limited or no communication between the project and the Federation during the course of the evaluation. Linkages & Follow Through Provided here is information on appropriateness of FDP’s cooperation with project beneficiaries.  Under Component 1, FDP adopted an activity-based focus and broader strategic objectives were largely ignored. For instance, the project invested its resources towards the establishment of a new association (Federation) and concerns with regard to making the entity financially sustainable were ignored.  Under Component 2, FDP has decided upon pursuing a list of target reforms without GOI/CBI buy-in. As mentioned earlier, GOI/CBI is not a member of the Legal and Regulatory Task Force that is responsible for taking forward the target reforms agenda.  Under Component 3, when interviewed, the domestic and foreign universities were generally appreciative of the support they received from the project. Monitoring and Evaluation FDP has a monitoring and reporting system set up that routinely provides progress updates for its various activities for internal and external (USAID) clients. Evaluation team observed instances where there were weaknesses in the project’s ability to take course correction. For instance, under Component 1, FDP continued investing in the Federation when it was clear that the organization would run into problems as FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 32 it was not generating any revenue. The project ignored the opportunity to help support the organization to operate as a for-profit company.  Is there good communication between the COP and Component Team Leaders, and are they all working toward common objectives? Evaluation team found that there is effective communication among the FDP management team. According to FDP, over the last few months the project has instituted work planning sessions that help provide a better understanding of the project’s priorities and activities across components. The project admitted that, in the past, different components were not as clued in with regard to what was happening in other parts of FDP.  To what extent has the project positioned itself to make the transition from expat to Iraqi staff for Component Team Directors and Subject Matter Experts as described in their technical proposal? Has the project hired high quality Iraqi staff and worked to build their capacity so that they can play a more prominent role in Year 3? FDP has a plan to transfer key roles within its organization to Iraqi officers. Initially, the plan was to take effect in Year 3, however, now this has been delayed to either late Year 3 or Year 4. The project has been utilizing on-the-job-training to get the Iraqi officers up to speed and most of them are satisfied with this approach. The team of Iraqi officers felt confident that by the time the hand-off occurs they woud be in a position to lead the main activities. Based on interviews with FDP’s team of Iraqi officers, evaluation team is of the opinion that they are interested in taking over key management roles within FDP. As mentioned earlier, the project is working with them to ensure they are adequately trained to tackle technical and management issues. Evaluation team also found that FDP has positioned the Iraqi officers to to serve as the main interlocutors between the project and its key (Iraqi) stakeholders. This enables the project to build trust and local ownership for FDP activities. The Iraqi officers need a formalized capacity building program. In 2010, FDP had developed a capacity building work-plan for the Iraqi Deputies, however, since then it has not been updated, and more importantly most of the Deputies were not aware of its existence. 9-A-3-Recommendations  Program planning & implementation should be demand-driven In determining their work plans and deliverables, AECOM and USAID should ensure that they are taking into consideration objectives and priorities of key stakeholders. For that, AECOM should engage its stakeholders before deciding on what type/scale of support is to be delivered.  AECOM should scale back activities where there are value for money concerns AECOM incurred the most costs in implementing Component 1 activities i.e., supporting establishment of Federation. There are serious issues with regads to value for money given that over eight million dollars were spent to create an ineffective and unsustainable organization.  Frequent changes in FDP management should be avoided AECOM should work towards lowering the turnover rate experienced by the project. This can be done by tightening their candidate sourcing and selection criteria. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 33  AECOM should routinely revise/update capacity building workplan for Iraqi officers Updating and implementing the capacity building workplan for Iraqi officers will help when the mangement transfer happens. 10-LESSONS LEARNED  Implementing partners should work with project beneficiaries to routinely assess relevance, sustainability and efficiency of project outputs.  Securing political support and local ownership at an early stage is a critical requirement for developing and implementing financial sector reforms. This is a critical requirement for both USAID and its implementing partners.  Program planning and implementation should be carried out in a demand-driven manner so that objectives, needs and priorities of key stakeholders are always kept front and center by the implementing partner.  Implementing partners should note that effective monitoring and evaluation system not only requires objective and factual progress reporting but also requires mechanisms for course correction.  Implementing partners should develop and implement project outputs in a culturally sensitive manner. Where possible, implementing partners should utilize local experts in client-facing roles to build trust and ownership amongst project stakeholders. FINAL REPORT- PERFORMANCE EVALUATION OF USAID/IRAQ FINANCIAL DEVELOPMENT PROJECT 34