Submitted to: USAID/IRAQ Baghdad, Iraq Submitted by: International Business & Technical Consultants, Inc. 8614 Westwood Center Drive • Suite 400 • Vienna, VA 22182 • USA Telephone: 703-749-0100 • Facsimile: 703-749-0110 with International Science and Technology Institute, Inc. 1820 North Fort Myer Drive, Suite 600 Arlington, Virginia 22209 May 1, 2006 UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT MONITORING AND EVALUATION PERFORMANCE PROGRAM, PHASE-II (MEPP-II) FINAL INTEGRATED REPORT PUBLIC SECTOR CAPACITY BUILDING EVALUATION UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT MONITORING AND EVALUATION PERFORMANCE PROGRAM, PHASE-II (MEPP-II) FINAL INTEGRATED REPORT PUBLIC SECTOR CAPACITY BUILDING EVALUATION Submitted to: USAID/IRAQ Baghdad, Iraq Submitted by: International Business & Technical Consultants, Inc. 8614 Westwood Center Drive • Suite 400 • Vienna, VA 22182 • USA Telephone: 703-749-0100 • Facsimile: 703-749-0110 with International Science and Technology Institute, Inc. 1820 North Fort Myer Drive, Suite 600 Arlington, Virginia 22209 May 1, 2006 United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. i Table of Contents List of Acronyms ___________________________________________________________________iii I. Executive Summary __________________________________________________________ 1 II. Introduction_________________________________________________________________ 3 III. Background_________________________________________________________________ 4 IV. Methodology ________________________________________________________________ 5 V. Economic Governance Summary Report _________________________________________ 7 A. Ministries of Finance, Planning and Development Cooperation, and Labor and Social Affairs _____________________________________________________________________ 7 1. Introduction_________________________________________________________________ 7 2. Iraq Financial Management Information System ___________________________________ 8 a) Accomplishments ____________________________________________________________________ 8 b) Site Visit ___________________________________________________________________________ 8 c) Training ____________________________________________________________________________ 8 d) Users Meeting _______________________________________________________________________ 9 • Recommendation ____________________________________________________________________ 9 • Lessons Learned_____________________________________________________________________ 9 e) Challenges/Problems _________________________________________________________________ 9 • Recommendation ___________________________________________________________________ 10 • Lessons Learned____________________________________________________________________ 10 3. Central Bank of Iraq – Bank Supervision Training Program_________________________ 10 a) The Training Program ________________________________________________________________ 10 b) The Challenge of Underlying Cultural Behavior and Attitudes _________________________________ 11 • Recommendation ___________________________________________________________________ 11 • Lessons Learned____________________________________________________________________ 11 4. Social Safety Net____________________________________________________________ 11 a) Accomplishments ___________________________________________________________________ 11 b) Defining Capacity Building ____________________________________________________________ 12 • Recommendation ___________________________________________________________________ 12 • Lessons Learned____________________________________________________________________ 12 B. Ministry of Planning and Development Cooperation_______________________________ 12 1. Accomplishments___________________________________________________________ 13 C. Ministry of Electricity ________________________________________________________ 14 1. Overview of the Ministry Electricity ____________________________________________ 14 2. BearingPoint’s Assistance Programs and Implementation _________________________ 14 a) Tariffs and Cost-of-Service ____________________________________________________________ 15 • Recommendation ___________________________________________________________________ 15 b) Corporate Financial Group (CFG)_______________________________________________________ 15 c) Metering __________________________________________________________________________ 16 • Recommendation ___________________________________________________________________ 16 d) Procurement _______________________________________________________________________ 16 • Recommendation ___________________________________________________________________ 16 e) Human Resources and Capacity Building ________________________________________________ 16 • Recommendation ___________________________________________________________________ 17 United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. ii 3. Future Technical Assistance __________________________________________________ 17 4. Lessons Learned ___________________________________________________________ 17 VI. Local Governance Program (LGP) _____________________________________________ 18 A. Context ___________________________________________________________________ 18 1. Getting Started _____________________________________________________________________ 18 2. Measuring Results __________________________________________________________________ 18 3. Security Issues _____________________________________________________________________ 19 4. Expectations Met____________________________________________________________________ 19 • Recommendation ___________________________________________________________________ 20 • Lessons Learned____________________________________________________________________ 20 B. Local Councils _____________________________________________________________ 20 • Recommendation ___________________________________________________________________ 21 • Lessons Learned____________________________________________________________________ 21 C. Local Departments/Directorates _______________________________________________ 21 D. Civil Society organizations (CDP/DDA) _________________________________________ 23 • Recommendation ___________________________________________________________________ 23 • Lessons Learned____________________________________________________________________ 23 E. Rapid Response Grants (RRGs) _______________________________________________ 24 • Recommendation ___________________________________________________________________ 24 • Lessons Learned____________________________________________________________________ 24 VII. Conclusion ________________________________________________________________ 24 FINAL REPORT APPENDICES ________________________________________________________ 1 Appendix 1: Proposed Revised Scope Of Work _________________________________________ 4 Appendix 2: Economic Governance Programs Detailed Evaluation Report Ministry of Electricity ______________________________________________________________ 13 Appendix 3: Economic Governance Programs Detailed Evaluation Report Ministry of Finance and Ministry of Planning Development _______________________________ 40 Appendix 5: USAID-Funded Economic Governance-II Capacity Building Programs, March 10, 2006 Debrief Presentation ______________________________________________________________ 131 Appendix 6: USAID-Funded Economic Governance Capacity Building Programs, March 17, 2006 Debrief Presentation ______________________________________________________________ 159 United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. iii List of Acronyms BE BearingPoint BOT Build-Operate-Transfer BSD Bank Supervision Department CBI Central Bank of Iraq CDP/DDA Civic Dialogue Program/Democracy Dialogue Activity CFG Corporate Financial Group CPA Coalition Provisional Authority CPFF Cost-Plus-Fixed-Fee CSO Civil Society Organizations CTO Cognizant Technical Officer DAC District Advisory Council DG Democracy and Governance Office DoD Department of Defense EEHG Egyptian Electricity Holding Company EG Economic Growth Office EG I Economic Governance I EG II Economic Governance II FRS U.S. Federal Reserve System GC Interim Governing Council HR Human Resources IBTCI International Business & Technical Consultants, Inc. ICRSS Iraq Center for Research and Strategic Studies IFMIS Iraq Financial Management Information System IIACSS Independent institute for Administration & Civil Society Studies IMF International Monetary Fund IR Intermediate Results IRMO Iraq Reconstruction Management Office ISLGP Iraq Strengthening Local and Provincial Governance IULA International Union of Local Governments LGA Local Government Associations LGP I Local Governance I LGP II Local Governance II LGPO Local Government Powers Order LGT RTI Local Governance Team MAT Ministry Advisory Team MEPP II Monitoring and Evaluation Performance Project, Phase II MNSTC-I Multi-National Security Transition Command-Iraq MoE Ministry of Electricity MoF Ministry of Finance MoLSA Ministry of Labor and Social Affairs MoPDC Minister of Planning and Development Cooperation MW Mega Watts NAC Neighborhood Advisory Council NDI National Democratic Institute United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. iv List of Acronyms (continued) OHRA Office of Reconstruction and Humanitarian Assistance OSSA I Organizational Structure and Staffing Assessments I OSSA II Organizational Structure and Staffing Assessments II OYO LGP Option Year One PAU Policy Advisory Unit PC Provincial Council PCO Project Contract Office PIR Project Intermediate Results PMP Performance Management Plan PRT Provincial Reconstruction Team QOLs Quality of Life Surveys RF Results Framework RFP Request for Proposal RRG Rapid Response Grants RTI Research Triangle Institute SIGIR Special Inspector General for Iraq Reconstruction SOW Scope of Work SSN Social Safety Net Sub-IR Sub-Intermediate Results TAL Transitional Administrative Law UN United Nations UNDG-ITF United Nations Development Group Trust Fund UNICEF The United Nations Children's Fund USAID United States Agency for International Development VPN Virtual Private Network WB World Bank WB-ITF World Bank Iraq Trust Fund United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 1 Final Report Public Sector Capacity Building Evaluation I. Executive Summary The results of International Business & Technical Consultant, Inc.’s (IBTCI) assessment of BearingPoint’s Economic Governance II and RTI’s Local Governance I and II capacity building activities described in this report present a mixed picture of accomplishments, unresolved challenges, and unrealized potential. In the case of BearingPoint’s efforts aimed at the Ministry of Electricity (MoE), the substantial work carried out has not resulted in significant results because: available resources were insufficient; mistakes were made in initially focusing efforts on long-term instead of more immediate issues; and, political will among MoE officials and Iraqi leaders was absent. The key lesson learned is, given the size and complexity of the problems of the power industry in Iraq, for capacity building efforts to be effective a unified and coordinated approach developed and implemented with the full participation and support of the Government and MoE is essential. There should be an integrated program for broad-based technical assistance for the MoE, sponsored and funded in part by USAID, the Iraq Reconstruction Management Office (IRMO), and the Project Contract Office (PCO), with additional support from the World Bank, U.N., and other bilateral donors. Specific recommendations regarding BearingPoint’s capacity building efforts with the MoE are: • The IRMO, PCO, USAID, and other relevant agencies quickly develop a united position on tariff increases and call for action by the Government of Iraq in order for tariff increases to be reflected in the 2007 budget. • BearingPoint should develop and execute a detailed technical improvement plan to establish an efficient and effective billing and collection system within the MoE. • BearingPoint should initiate systematic evaluations of warehousing, inventory management and control, and procurement and prepare a detailed improvement program for each with defined budget requirements and established priorities. • BearingPoint should use the Organizational Structure and Staffing Assessments I and II as a guide to develop and execute improvement programs for MoE organizational development, staffing, and personnel issues. These programs should focus on cost reduction efforts, especially in terms of overstaffing. BearingPoint efforts focused on the Ministries of Finance, Planning and Development Cooperation, and Labor and Social Affairs were considerably more successful, although questions surfaced throughout the evaluation on just how much capacity building had occurred. Significant accomplishments and, in at least one instance, an extraordinary outcome were achieved as a result of the training and technical assistance provided. However, in the cases of United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 2 the Social Safety Net and Ministry of Planning activities it is difficult to say to what degree those accomplishments embody achievements in capacity building. The key lessons learned from capacity building efforts in these Ministries are USAID and BearingPoint need to do a better job of program design and planning. For example, the program planning and design process should incorporate elements that address understanding of underlying cultural attitudes and behaviors at the same time technical knowledge and skills are being imparted. USAID and its implementing partners should also anticipate and plan for tracking accomplishments that go beyond the technical results desired to the broader context of political reform and good governance. In addition, criteria and procedures to minimize potential abuses in selection of participants for training should be written into project documents and be in place before activities begin. Specific recommendations regarding BearingPoint’s capacity building efforts with the Ministries of Finance and Planning and Development Cooperation include: • Developing ways to systematically track occurrences, such as meetings of ministry committees and sub-committees, where significant outcomes beyond the targeted technical results are being achieved. • Closely monitoring the selection process for training activities to reduce the potential for the wrong person being chosen to participate. • Developing and incorporating activities that go beyond the technical requirements of training and address the fundamental issues of cultural transformation. • USAID should develop a clear definition of capacity building. The results of RTI’s work on LGP I and II have been muted by the lack of a law supporting decentralization. This, in turn, has undermined the sustainability of many of the capacity building efforts undertaken. RTI capacity building activities were also adversely affected by unavoidable changes resulting from Iraq’s Provincial Council elections. On the positive side, at least two “star performers” were found. The first was the overall response by the LGP to the lack of legal clarity needed to enable decentralization, while the second was the development of a solid waste initiative by the Baghdad LGP team. The key lesson learned from RTI capacity building efforts in LGP I and II is the limitations and adverse impacts caused by the absence of the legal framework to make it possible for decentralization to take place. For example, frustrations emerged when LGP training alerted council members to their limited role under the existing decentralization context. Council members understood that the LGP can only move forward with decentralization to the extent that the law allows. The same will be true for the new PRT concept unless the requisite enabling legislation is passed soon. A second key lesson learned from LGP I and II is the problems caused by inadequate program planning and design. In LGP I, too rapid build up of field staff led to problems because roles were not well defined and demand for the skills available was not yet in place. In addition, owing to the premature deployment of staff problems arose as a result of an inability to provide for their security once they arrived in the field. Specific recommendations regarding RTI’s capacity building in LGP I and II include: • RTI and USAID should ensure that field staff know, with as much specificity as possible, what is expected of them in the program strategy. Plans should count on providing security for a very long period and field staff must be informed of the risks involved. United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 3 • It is imperative that a clearly defined local government structure be established. RTI and USAID should work to ensure that it is in place or a strategy for achieving it has been developed and is being implemented. • USAID should hold fast to requiring results-oriented indicators from program implementers. • The oversight provisions of the Rapid Response Grants Manual and scoring system should be utilized and a final acceptance visit and sign-off be made. Finally, mention must be made regarding the extraordinarily difficult circumstances with which BearingPoint and RTI were confronted on a daily basis throughout the lives of their respective projects. Both RTI and BearingPoint had workers injured, killed, and/or under some kind of continuous threat and had activities constantly at risk and/or being forced to adjust and readjust their direction and focus. Given these factors, it is remarkable that anything of consequence has been accomplished, let alone the sometimes formidable achievements described in this evaluation. II. Introduction To assist Iraq’s transition to democracy, in 2003 USAID initiated a broad development strategy that included improving the effectiveness and building the legitimacy of Iraqi government officials and institutions. USAID started by providing support to the former Interim Iraqi Government and more recently has worked with the Iraqi Transitional Government to strengthen the capacity of its legislative, executive, and judicial branches. USAID continues to work to strengthen Iraq’s private sector, strengthening human and technical capabilities to bring about a more secure and sustainable economy. Through its Monitoring and Evaluation Performance Project, Phase II (MEPP II) contract with IBTCI, USAID requested an evaluation of the status of capacity building1 efforts in Iraq. The purpose of this evaluation was to find, describe, and verify successes and failures in USAID/Iraq’s capacity building efforts in two programs: Economic Governance II (EG II, from October 2004 to present) implemented by BearingPoint (BE); and Local Governance I and II (LGP I and LGP II, from May 2003 to present), implemented by the Research Triangle Institute (RTI). This evaluation was intended to identify and draw lessons learned for future program designs and its results will be directed towards the Mission’s Program, Democracy and Governance (DG), and Economic Growth (EG) Offices. It is also envisioned that the evaluation results might help the U.S. Government, Government of Iraq, and other donors – in part, through the Capacity Development Working Group – to plan and execute future capacity building efforts. 1 The definition of capacity building used in this evaluation is: training and technical assistance that provide new knowledge and skills or improve on existing knowledge and skills for individuals and institutions. Due to time and security-related constraints, emphasis during the course of the evaluation was placed on individual capacity building activities. For future consideration, an interesting definition of capacity building has been proposed in a recent Iraq Reconstruction Management Office document, “Promoting Ministerial Capacity Development.” The paper discusses institutional capacity, organizational capacity, and individual capacity as elements that must be “aligned” for capacity developed to be sustainable. See related discussion below, p. 10. United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 4 III. Background There are two key factors under which RTI and BE have been forced to operate: first, the enabling environment was severely constrained by the war and its aftermath and has been shifting constantly since then; and, second, the security situation, always a major issue, has grown worse with the continuing insurgency. In the first case, the structure of government was crippled by the widespread looting and destruction of ministry buildings and infrastructure that took place immediately after the cessation of hostilities. Ministry buildings were abandoned and many suffered significant damage that delayed future restoration and a return to operation. The structure of government has also changed in major ways several times and the legal underpinnings of government have been unclear and likewise shifting continuously. First, there was the Office of Reconstruction and Humanitarian Assistance (OHRA), which established the initial civil administration mechanism for the coalition forces. Next, in May 2003, OHRA was replaced by the Coalition Provisional Authority (CPA), which effectively became the government of Iraq. By June 2003, an Interim Governing Council (GC) had been formed and later, based on the November 15 Agreement,2 was charged with writing the Transitional Administrative Law (TAL), the legal mechanism for achieving the return to Iraqi sovereignty. The TAL was developed by the CPA in consultation with the GC. Final agreement on the wording of the TAL was achieved in early March 2004. The TAL provided procedures and dates for the transition to sovereignty through an election procedure that created an Interim National Assembly that would be responsible for writing a new constitution. The TAL set the date for the return to sovereignty for June 30, 2004, and on that day the CPA ceased to exist. The TAL also provided the legal basis for Iraqi governance until the new constitution was written by the Interim National Assembly and approved in a national referendum. During this period, clarification on Iraq’s legal structure was needed to achieve gender equality, protect minority rights, promote decentralization, and ensure de-Baathification. The TAL provided for these issues, and later through additional Annexes and Administrative Orders sought to guide the development process. The TAL set the stage for the final step in the post-war evolutionary process. Interim National Assembly elections were held, the new constitution was written and adopted, new elections for the Council of Representatives were completed and, most recently, the Council members were sworn in. Still ahead are the formation of a new government, adoption of new laws, a second referendum on constitutional changes, and new elections for provincial council members. Both the RTI and the BE projects do not operate in a secure environment. Within a generally worsening security situation, BE and RTI have experienced significant security events that have set back their projects and caused each to reconsider the way forward. A recent study on the insurgency points out that its strategy has been to “strike at U.S. and other aid projects to undermine Iraqi acceptance of the Multi-National Security Transition Command-Iraq (MNSTC-I) and the perceived legitimacy of the Iraqi government.”3 Seeking maximum effect, the 2 The November 15 Agreement is important because it redirected program activities to support the June 30 date for sovereignty. This is when council ‘refreshment’ began a vastly expanded Civic Dialogue Program and expanded local grant budgets. 3 Center for Strategic and International Studies, Arleigh A. Burke Chair in Strategy, Washington, D.C., 20006, “Iraq’s Evolving Insurgency: The Nature of Attacks and Patterns and Cycles in the Conflict,” Anthony H. Cordesman and Arleigh A. Burke, revised working draft, February 3, 2006. United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 5 insurgency has struck at major aid and government projects after completion and disrupted activities when they acquire visibility or have high levels of employment. Insurgents and terrorists also have often struck at projects when the security forces protecting workers and aid teams were no longer there, leading local populations to blame the Coalition or Government for failing to keep promises or provide the proper protection. In addition to limiting access to project beneficiaries, security considerations have exacted a high price in terms of deaths and injuries and budget resources. A report by the Special Inspector General for Iraq Reconstruction indicates that at least 276 civilians working on U.S. aid projects had been killed by March 31, 2005 and at least 2,582 had been wounded. According to the report, the number of contractors killed rose by 19 percent (to 44) in the first quarter of 2005. The report indicates that a direct result of this upsurge in the insurgency is that security costs of USAID funded projects rose from 4.2 percent of the total between March 2003 and February 2004 to 22 percent during the final nine months of 2004. IV. Methodology The IBTCI evaluation team began its work in Washington on February 6, 2006 to find, describe, and verify successes and failures in USAID/Iraq’s capacity building efforts in the EG II Program, implemented by BE, and LGP I and LGP II, implemented by RTI. The team consisted of Harold Lippman, James Douglas, and Frank Hersman. Each member was assigned a specific area of responsibility: James Douglas focused on BE’s activities with the Ministry of Electricity; Harold Lippman evaluated BE’s activities with the Ministry of Finance and Ministry of Planning; and, Frank Hersman – assisted by IBTCI/Baghdad monitoring and evaluation specialist, Harvey Herr – focused on RTI’s LGP I and II project activities. Harold Lippman also served as the team Leader. While in Washington, the team gathered and reviewed documents, met with USAID officials as well as with BE and RTI staff, and developed a workplan for the upcoming fieldwork in Iraq. Two members of the team deployed to Baghdad on February 13 (arriving February 15); the third team member deployed February 17 and arrived two days later. After conducting entrance meetings with USAID/Iraq’s Program Office, DG, and EG staff and BE and RTI personnel, the team commenced its work gathering information and scheduling interviews. However, for two of the three team members this effort was interrupted when, in a follow-on meeting (February 21) with the DG team leader, who had been out of the country during the entrance briefing, it was learned that the requirements specified in the original Scope of Work regarding RTI’s LGP I and II Project activities were not what had been intended. Two days later (February 23), the DG team leader and Program Office CTO provided a revised SOW for the RTI part of the evaluation and work effectively began on the LGP I and II capacity building activities. To compensate for the delay in getting started on this portion of the evaluation, seven additional days in-country were provided for the team member responsible for the RTI work and three days were added for the team leader. (See Annex 1 for the revised SOW) Over the ensuing three weeks, team members gathered and reviewed source documents and conducted on-site interviews of BE and RTI staff and Iraqi officials and beneficiaries. Source documents included, but were not limited to: BE and RTI monthly, quarterly, and annual reports; United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 6 project contracts and workplans; performance monitoring plans;4 internal assessments; relevant deliverables; and, USAID strategic planning, annual reporting, and evaluation documents. The team examined the concerned ministries’ core functions; assessed activities undertaken to address problems identified with the core functions; and determined the extent to which activities undertaken have impacted the core functions, identifying specific successes, failures, and/or problems encountered. While severely limited by the already deteriorated security situation – which worsened significantly in the aftermath of the February 22 bombing of the Shiite mosque in Samarra – team members made every effort to meet with project beneficiaries outside the International Zone. Meetings were arranged with current and former local council members who had received training from RTI and Ministry of Finance beneficiaries of training provided by BE were interviewed on site at the Central Bank and Iraq Financial Management Information (IFMIS) System Data Center. Another trip was arranged to observe a major meeting of IFMIS “users” – all of whom had undergone BE training – at the Ministry of Finance. The lead time necessary for organizing trips outside the International Zone was considerable. Attempts to interview provincial council members and the Baghdad provincial council legal committee failed on three occasions. The team, however, was able to speak to some council members by telephone. IBTCI’s Iraqi field monitors contributed significantly to the team’s efforts outside the International Zone in Baghdad and across Iraq. With the aid of small-scale training on serial questioning and probing techniques, one monitor used a list of questions expressly developed for the purpose of interviewing a small sample (10 of 200 participants) who had undergone Phase I IFMIS training provided by BE. This monitor also completed interviews with 14 civil service staff from five departments in Kirkuk who had received LGP training. The monitor conducted the interviews using a questionnaire developed for this purpose. The questionnaire included questions with categorical responses as well as ones that called for qualitative, open-ended responses. Another field monitor attended the FMIS users meeting and wrote a summary of the question￾and-answer proceeding. This field monitor was then tasked with validating early LGP grant infrastructure activity in Baghdad through interviews with local residents and city government staff. The third monitor interviewed local government officials in Baghdad who had undergone LGP I training. Finally, an unanticipated facet of the methodology evolved spontaneously in the field in response to the changes in the scope of work requested by the Mission. Mission staff were most accommodating in responding to requests, providing input on the number of activity areas to be examined, and clarifying the substance of what was wanted. This cooperation and collaborative approach also facilitated additional interim briefings beyond the mid-term and final ones provided for in the scope of work. This relationship was possible because USAID/Baghdad has a very high rotation rate, leaving current staff not as well informed on earlier project activities as they want to be. The evaluation provided an opportunity for them to use IBTCI to help fill the information gap. 4 BearingPoint did not have a performance monitoring plan for its EG II capacity building program, although RTI did. Thus, the team was only able to respond to the SOW evaluation question on this subject in terms of RTI’s LGP I efforts. United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 7 V. Economic Governance Summary Report In July 2003, USAID launched an effort to build the capacity of the Iraqi Government to manage the transition from a command to a market-driven economy. That effort, Economic Governance I (EG I), was implemented by BearingPoint and ended in 2004. It helped Iraqis develop policies, laws, and institutions that better regulate trade, commerce, and investment, while also providing support to ministries, the Central Bank, and other government organizations that are key parts of the economy. Building on the reforms and other accomplishments brought about by EG I, in September 2004 USAID put into place the Economic Governance II (EG II) project to help Iraq establish a policy￾enabling environment that fosters private sector led growth. Also implemented by BearingPoint, EG II’s overall objective is: To assist the Government in structuring and quickly implementing an operational legal, fiscal, regulatory & institutional framework within which policy can be formulated to foster a transparent and investment-friendly, market-based economy. The three to five year project has been providing technical assistance to all Iraqi line ministries and 32 other government bodies. EG II’s main program objectives are to: • rebuild a governmental system that is efficient, transparent, and corruption free; • introduce fiscal balance and monetary stability, discipline, and sustainability; • put into place a viable, well-regulated banking system that will facilitate investment, monetary stability, and the development of the economy; • implement a sound, transparent, and predictable commercial legal framework that fosters the rule of law and promotes confidence in the system; • introduce viable, functioning electrical, telecommunications, and government￾wide IT management systems; • introduce new pension and social safety net systems that will protect the most vulnerable individuals and allow the government to gradually eliminate state subsidies for food, energy, and electricity; • put into place clearer and more structured procedures to facilitate regulation of businesses and individuals dealing with the government; and • promote the participation of NGOs in policy formulation As of December 31, 2005, total expenditures incurred in EG II amounted to $63 million. Ministries of Finance, Planning and Development Cooperation, and Labor and Social Affairs 1. Introduction Given the enormity of the task called for in the original scope of work, i.e., evaluating the programs financed by USAID/Iraq in the Ministries of Finance and Planning and Development Cooperation, the Mission’s Economic Governance team suggested that IBTCI narrow the focus of its work to four areas of BearingPoint capacity building activity supported by EG II: • the Iraq Financial Management Information System (Ministry of Finance); United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 8 • the Bank Supervision Training Program (Central Bank of Iraq); • the Social Safety Net Program (Ministry of Finance and Ministry of Labor and Social Affairs); and • technical assistance provided to the Ministry of Planning and Development Cooperation. 2. Iraq Financial Management Information System The IFMIS is an integrated, automated, and internet-based information system that will replace the Government of Iraq’s outdated manual accounting system. The overall goal is to give the Ministry of Finance the ability to: • produce timely, accurate, and comprehensive financial reports; • manage cash expenditures across spending agencies; and • establish modern controls and management of cash resources, accounting processes, and budget execution in accordance with Iraqi law and international standards. Work on IFMIS began in 2003 under EG I and has continued under EG II. It is anticipated that all 182 sites will be fully installed by June 2006. IFMIS has been in test and pilot mode since early 2005 and is projected to be fully operational in time to perform basic budget execution/accounting functions for fiscal year 2007. a) Accomplishments Based on source documents reviewed, extensive interviews with BearingPoint advisors and senior Ministry of Finance officials, and on-site observation, IFMIS can clearly be regarded as a success. According to a January 2006 BearingPoint document, key accomplishments include: • completing the necessary infrastructure to support the new system, including refurbishing the Ministry of Finance computer and data center, procuring and installing hardware; and developing an internet network to connect the initial 54 Phase I sites; • installing, testing, and commencing live operations of IFMIS software in Phase I sites that represent more than 80 percent of Iraq’s federal budget expenditures; and • completing training of more than 250 IFMIS data entry and IT staff. b) Site Visit In addition to these accomplishments, an IBTCI evaluation team member visited the IFMIS Data Center to see how it was equipped and functioned. The two-room center has some 20 personal computers, several printers, and other requisite equipment. The computers are used for operational and training purposes. A handful were in use at the time of the visit and all of the rest appeared to be in good working order. “Help desk” staff described their functions and how they respond to questions from users. c) Training Because training was one of the most important tasks in the development of the IFMIS, an IBTCI Field Monitor interviewed ten of the 200 individuals who participated in BearingPoint supported Phase I courses and workshops. The sample was drawn from a list provided by the IFMIS Implementation Coordinator and structured interviews were conducted with the use of prepared questions (see Appendix 3, Annex 1) that were posed sequentially. The interview United States Agency for International Development Final Report Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 9 write-ups prepared by the Field Monitor show that the training activities were well-organized and imparted the desired knowledge and skills to participants. d) Users Meeting Among the two or three most striking situations encountered during this evaluation was the March 7, 2006 IFMIS “users” meeting at the Ministry of Finance. The meeting provided an opportunity for users to interact with IFMIS management in a non-stop, two-hour question-and￾answer session. The dozens of questions were almost entirely focused on operational problems and issues, such as the lack of electricity and confusion over data entry requirements. However, while the central purpose of the meeting was clearly realized, something more important also took place: capacity building efforts were giving rise to a broader good governance outcome well beyond their stated goals. Firstly, the simple fact that the meeting was occurring was itself noteworthy, since it would certainly have been an aberration under the previous regime. Secondly, the opportunity for face-to-face interaction provided the context for an extraordinary, spontaneous dialogue. Prompted by questions calling for the Ministry to provide additional resources for the user sites, a senior manager delivered a tutorial on good governance. He emphatically stated that only the Iraqi National Assembly can provide increased budget resources, concluding “we are not like the previous regime that could increase any ministry’s budget by decree.” He scolded those present for going abroad to conferences and training programs and using those occasions as personal junkets. He also made the point that “incompetent” staff in his own ministry and other ministries are causing harm. • Recommendation While one meeting certainly does not qualify as a sea change, that it occurred and is likely to be happening in other venues suggests that the significant impacts it demonstrates should be captured and put to further use. BearingPoint and/or USAID should consider developing ways to promote and systematically track such meetings and related undertakings. • Lessons Learned Significant accomplishments resulting from USAID-inspired capacity building activities are occurring in the settings where the interventions are taking place. For example, in several ministries Steering Committees and Sub-Committees have been established to coordinate capacity building and other assistance efforts. USAID and its implementing partners should anticipate and plan for such occurrences, which go beyond the specific results desired from a given capacity building activity to the broader context of political reform and good governance. e) Challenges/Problems Notwithstanding the positive results achieved in establishing the IFMIS, numerous challenges threaten its continued development. BearingPoint documents cite the following examples: • poor security and infrastructure hamper communication and automation efforts; • difficulties with internet connectivity and electricity shortages; • the technical, professional, and managerial capacity of Ministry of Finance staff is limited and these factors slow the implementation process tremendously; • very low level of computer literacy, combined with limited exposure to modern practices and techniques in all agencies, inhibit timely implementation; and • mid-level managers are resistant to change and must take ownership of IFMIS. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 10 In addition to these formidable challenges, IBTCI learned that significant problems had occurred in the selection of individuals to participate in IFMIS training. In the initial Phase I sessions that involved some 200 participants, a knowledgeable BearingPoint advisor estimated that about 75 of those selected were managers, supervisors, or others who were not going to actually use the knowledge and skills acquired from the training to enter the data into the IFMIS. This means that additional training – with all this implies in terms of added time and cost – had to be done to provide the appropriate people with the necessary knowledge and skills. BearingPoint advisors said this problem has also occurred elsewhere in the Ministry of Finance and other ministries. • Recommendation Training coordinators in ministries and BearingPoint should closely monitor the participant selection process and USAID/Iraq should assign oversight of this matter to a specific individual within the Mission. Consideration should also be given to having “overview” training courses for supervisors and managers, as well as conducting training activities in parts of Iraq, such as the North, which are safer and less costly than sending participants outside the country. Moreover, USAID and its implementing partners will need to exert continuous pressure on their Iraqi government counterparts to see that ongoing attention is being paid to reducing abuses in this area. • Lessons Learned In designing capacity building activities that include training elements, potential abuses in selection of participants should be anticipated. Criteria and procedures to minimize such possibilities should be written into project documents and be in place before the activities begin. 3. Central Bank of Iraq – Bank Supervision Training Program Under EG II, BearingPoint has continued efforts initiated in EG I to assist the Central Bank of Iraq (CBI) in becoming capable of executing monetary policy, maintaining price stability, and ensuring the safety and soundness of the banking sector. A key feature of these efforts has aimed at strengthening the CBI by building the capacity of its Bank Supervision Department (BSD). a) The Training Program As of January 2006, BearingPoint completed a two-year training program for the BSD designed to strengthen banking operations by developing an effective supervisory role within the CBI. BearingPoint organized an ongoing series of seminars, classroom courses, and on-the-job training activities to introduce modern, risk-based supervisory techniques to BSD personnel. The seminars, courses, and on-the-job training have covered diverse topics, including: • The Basle Committee’s 25 Core Principles for effective banking supervision; • International accounting and financial reporting standards; • Credit risk management; • Asset/liability and liquidity management; and • Bank financial analysis The effects of these training activities have been significant, as confirmed in interviews with senior CBI officials and four former participants (two men, two women). One senior official said the training provided CBI with knowledge that has helped make its staff more reliable and credible in dealings with neighboring country financial institutions. Another likewise mentioned United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 11 the positive effects of learning about international standards and concepts, but also pointed out there have been difficulties in applying that knowledge to Iraq’s special circumstances. The four former participants also expressed concern with relating what they had learned to the Iraqi environment, but emphasized that they were much enriched by the courses. In addition to helping to solidify everyday banking supervision operations and enabling staff to become familiar with and start to use international standards and best practices, the training program has prompted BSD to develop and conduct its own training for staff on a regular basis. For example, BSD has developed and taught seminars on the Basle Core Principles, accounting, and credit risk to CBI and private bank staff. BSD has also developed, distributed, and put into practice its own credit risk guidelines for the country’s commercial banks. b) The Challenge of Underlying Cultural Behavior and Attitudes In startling contrast to the notable success achieved in training BSD officials, these same officials and their staff have failed to use 20 desktop computers they received more than two years ago. According to the senior BearingPoint advisor who shared this anecdote, “the desktop computers are in a separate room and are rarely, if ever, used for their work.” An Iraqi￾American senior BearingPoint advisor confirmed this point, asserting that hardly any managers, supervisors, and other “bosses” use computers in their work. They look on computers as something “beneath them,” to be used solely to replace a typewriter. The culture of business and government, he noted, is mired in reliance on hand-written documents that goes back “hundreds of years.” Concepts of productivity and efficiency, he added, are not understood. BearingPoint/USAID capacity building efforts, he concluded, do not address these and other critically important underlying characteristics of Iraqi culture. • Recommendation In its capacity building efforts USAID and its implementing partners must develop and better incorporate activities that go beyond the specifics of training and address the fundamental issues of cultural transformation. • Lessons Learned Capacity building program planning and design should include elements that address reform of underlying cultural attitudes and behaviors at the same time the technical knowledge and skills are being imparted. Failure to do so increases the likelihood of partial success and/ or failure in the short term and substantially diminished prospects for long-term sustainability. 4. Social Safety Net Under EG II, BearingPoint has worked with the Ministry of Finance and Ministry of Labor and Social Affairs (MoLSA) to develop a new Social Safety Net (SSN) program to reduce poverty and mitigate the impact of major economic reforms on poor and vulnerable segments of the population. SSN implementation began in December 2005 with an initial target of lifting some 850,000 Iraqi families out of poverty by the end of 2006. The SSN is a means-tested program comprised of financial and non-financial benefits and services to low income families, the disabled, and the unemployed. Nearly 25 percent of Iraq’s people are eligible for SSN benefits. a) Accomplishments Based on a review of available source documents – most notably, monthly reports – and interviews with advisors, BearingPoint has clearly played a pivotal role in the SSN’s successful development and early implementation. The following monthly report excerpts are illustrative: United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 12 • Project advisors supplied a social protection bulletin that included unemployment statistics. A template tracking matrix was supplied, and statistics have been requested at the Governorate and national level, on a regular basis to monitor the growth in registrations of low income families, unemployed and disabled. (January 2006) • The Social Safety Net Law, prepared by Project advisors, was read two times successfully in the Transitional National Assembly plenary session, and is now awaiting final vote. (December 2005). • Prepared the first draft of Instructions and Standard Forms to facilitate a smooth transition from the existing to the new system. (November 2005) • Delivered the first draft of the strategy for a public education campaign to raise awareness among poor families of the Social Safety Net. (November 2005) While fully acknowledging the accomplishments referenced in the above excerpts, from the standpoint of this evaluation’s objective of assessing BearingPoint’s contribution to capacity building efforts, the record is less clear. Throughout the excerpts, in phrases such as, “project advisors supplied” and “prepared the first draft,” there is a definite sense of the advisors providing technical assistance to help get something done rather than build capacity. In an interview with a key BearingPoint advisor, the advisor maintained that all meetings with Iraqi counterparts, in effect, involved training because they were interactive and counterparts often “took ownership” of the discussion and/or ultimate product. However, efforts to arrange interviews with counterparts and/or beneficiaries (such as participants in training provided to MoLSA staff) to verify this assertion were unsuccessful. b) Defining Capacity Building This discussion ties directly into a basic question that arose at the very inception of this evaluation and reappeared continuously throughout it: what is meant by capacity building? BearingPoint and others seem to confuse tools, such as courses, seminars, workshops, and on￾the-job training that provide new knowledge and skills or improve on existing knowledge and skills with technical assistance that helps get something done. The latter, most typically, does not build capacity; the former virtually always does. In the excerpts above, for example, supplying a social protection bulletin or delivering the first draft of a strategy does not build capacity. The interchangeable use of these two definitional elements causes confusion in identifying and reporting accomplishments and capturing the real effects of an intervention. • Recommendation USAID should consider establishing separate categories for these definitional elements: one for capacity building that reflects new or improved knowledge and skills and the other for functional technical assistance provided by advisors and other experts. • Lessons Learned An agreed definition of capacity building should be part of all project documents, starting with RFPs and continuing on through contracts, SOWs, workplans, and other essential paperwork. Without a clear definition in place from the start, confusion is inevitable, with attendant consequences, for example, in being able to accurately and consistently describe what has been accomplished. Ministry of Planning and Development Cooperation The Mission Economic Growth Team requested that IBTCI look into BearingPoint efforts to provide advisory services requested by the Minister of Planning and Development Cooperation United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 13 (MoPDC) for the period June – December 2005. Two advisors – one for legal matters, the other for economic affairs – were assigned to this task, but only one of them was available at the time of the evaluation. The terms of reference for that individual’s position, senior legal advisor, stipulate that he was to advise and assist the Minister, upon request, in matters such as: • Commercial and trade law and policy; • Foreign investment; • Public procurement regulation; • Multilateral and bilateral financial organizations; • International donor and donors’ trust funds; and • Ad hoc matters assigned directly by the Minister. 1. Accomplishments Based on a review of monthly reports and discussions with the senior legal advisor, it is clear he and his economic advisor counterpart made significant contributions to the Minister of Planning, as indicated in the following monthly report excerpts: • At the request of the MoPDC, BearingPoint Advisors joined the Iraqi delegation to the ministerial level International Conference on Iraq, held in Brussels, Belgium, June 21- 22, 2005. Advisors assisted MoPDC on all matters involving its presentations on economic recovery and reconstruction, before and throughout the Conference. • The Advisors assisted in the preparation, review, and completion of the following: o Iraq’s National Development Strategy; o Minister’s keynote speech; o IRFFI donor meetings’ agenda, official statements, communiqués; and o Position papers on the World Bank Iraq Trust Fund (WB-ITF) and the United Nations Development Group Trust Fund (UNDG-ITF). (July 2005) • The Advisors participated in the preparation of background briefings and analytical materials for the MoPDC delegation that attended the World-Bank/IMF Annual Meetings in Washington, D.C. At the request of the Minister, reviewed and revised proposed draft MoPDC Letters of Intent, Letters of Agreement and Memoranda of Understanding. (September 2005) However, as with the Social Safety Net, and perhaps even more so in this case, there is little evidence that meaningful capacity building called for in EG II took place. Similar to the SSN advisor, the senior MoPDC legal advisor believes capacity building did take place in his work. He cited the example of his modeling important behavior and attitudes that are not part of the Iraqi bureaucratic culture, such as arriving on time, working a regular schedule throughout the day, and taking sick leave. However, he also explained that much of what he did for the Minister utilized knowledge and skills people at the Ministry did not have and the priority was to get things done. The setting and the requirements of his job in many respects were not conducive, for example, to on-the-job training, mentoring, or other capacity building tools. In many situations, he acknowledged, in responding to the Minister’s requests he crossed the fine line between serving as an advisor and being a MoPDC employee. This blurring of his role, he added, created difficulties with BearingPoint management and USAID – a situation he said was further complicated by related problems with other BearingPoint colleagues. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 14 Ministry of Electricity 1. Overview of the Ministry Electricity The goal of the Ministry of Electricity (MoE) is to provide sustainable and reliable electric power to its customers at just and reasonable cost-based rates, without the need for government subsidies. The MoE functions as a vertically integrated electric power company. That is, it maintains the generation capacity as well as the country-wide electric grid and operates central dispatch. Structurally, the Ministry includes separate legally constituted operating companies that function under a complex set of Iraqi laws and regulations. The MoE was reorganized in October 2003, establishing eight headquarters directorates, four common purpose directorates, and 20 operating companies (seven generating companies, six transmission companies, and seven distribution companies). As generating units are being refurbished and new units are being added, the total amount of installed generating capacity is rising, from approximately 9,000 Mega Watts (MW) in 2003, to approximately 11,500 MW today. However, actual generation has been relatively flat, in the range of about 3,000 to 3,500 MW, and on occasion, passing 4,000 MW of production. Thus, the percent of that installed capacity available is falling, not rising. At the same time, demand is rising and now approaches 8,000 MW, resulting in major power shortages throughout Iraq. This means that currently, the Ministry is only able to sustain power at a minimal level of service, even though its installed capacity is rising because of new generating units coming into service. At the same time, it is collecting minimal revenues because of extremely low tariffs and ineffective billing and collection practices. Additionally, the Ministry’s annual cost of service is rising dramatically, largely from the completely uncontrolled increase in staffing and payroll costs. Since 2003 there has been nearly a 25% increase in staffing, and an even greater increase in payroll costs, but no increase in productivity. The Ministry and the Government of Iraq must recognize that the electricity sector and services are not part of a program to be funded by the government, but rather are state-owned public enterprises that should be self-supporting and operated like any other commercial enterprise. The Ministry and the Government need to take much of the responsibility for the critical situation that exists in the electricity sector, and they need to make the difficult decisions required to introduce dramatic changes in the way MoE operates. 2. BearingPoint’s Assistance Programs and Implementation In the electricity sector, under Economic Governance I (EG I), BearingPoint focused on: 1) formulating, completing, and submitting a detailed ten-year master plan to the MoE; 2) identifying the legal and institutional reforms needed to establish an Iraqi electricity regulatory authority; and, 3) assisting in institutional capacity building and providing reports of the broad￾based technical assistance needs of the Ministry, such as the Organizational Structure and Staffing Assessments I and II (OSSA I and II). The assistance provided under EG I ended in June 2004. The Economic Governance II (EG II) program, initiated in October 2004 as successor to EG I, had a difficult operational beginning. One advisor was shot at, another adviser’s car was bombed, and BearingPoint’s electricity adviser was killed on the airport road to Baghdad. Consequently, most of the advisers moved to Amman for several months before returning in April 2005. Also, as a result of the security situation, there were and continue to be problems in obtaining the services of advisers with significant relevant power experience. In addition, there are continuing problems affecting the ability of the advisors to gain direct access to MoE staff. A United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 15 corollary problem was and continues to be a lack of inter-agency cooperation and communication on issues of mutual interest and concern regarding the Ministry. In the electricity sector, EG II pursued two goals: 1) restructuring of the electricity industry; and, 2) institutional capacity building. Based on USAID’s Request for Proposal and its subsequent award, the initial focus of EG II was on privatization and market reform, restructuring, and regulatory reform – all of which are goals that are not and will not be achievable for the foreseeable future. Furthermore, addressing critical institutional building needs identified in EG I, took a secondary role. For example, after OSSA I and II revealed serious problems facing the utility, such as inefficient and ineffective organization and overstaffing, a major technical assistance program should have been developed around the reports’ recommendations. The fact that there was no technical assistance follow up to OSSA I and II was a major mistake and reflected BearingPoint’s focus on long term goals over the critical, immediate problems MoE faced. In addition, the needs assessments carried out under EG I, which identified improvements needed in institutional development such as warehousing/inventory control and reductions in personnel, were never followed under EG II. Furthermore, although there were significant resources devoted to privatization, restructuring, and regulatory issues that have produced substantive documentation and calls for action, the resultant recommendations have not been accepted by the Government. With the appointment of a new Minister in June 2005, and staff changes within BearingPoint, the EG II program focus shifted from regulatory reform to the technical areas (i.e. core functions) of: 1) tariffs and cost-of-service; 2) a corporate financial management group; 3) metering (a major component of billing and collections); and, 4) procurement. Also in June 2005, an additional technical area, human resources and capacity building, was included. a) Tariffs and Cost-of-Service BearingPoint conducted extensive and well-documented cost-of-service and related studies that justify major increases in tariffs (rates). The final report and tariff recommendations are under final review and should be released soon. Concerns about potential negative impacts were addressed effectively. Considering that there is a critical need for a substantial increase in tariffs, the Government must find the will to make critical tariff decisions. • Recommendation It is strongly recommended that the U.S. Department of State’s Iraq Reconstruction Management Office (IRMO), the U.S. Department of Defense’s Project Contract Office (PCO), USAID, and other pertinent agencies, quickly take a united position on tariff increases and call for Government action in order for tariff increases to be reflected in budget calculations for the 2007 budget year. b) Corporate Financial Group (CFG) BearingPoint focused on organizational development of the CFG as a functioning staff unit within the Ministry, with responsibilities for conducting financial analysis and reporting, formulating financial strategies, and providing assistance to senior management in their decision-making. To support the CFG staff, a joint BearingPoint/MoE decision was made to develop and implement three computer based financial management modules that were available in Arabic from the Egyptian Electric Holding Company. In March 2006, Ministry staff went to Egypt to undergo technical training, consistent with a defined systems implementation work plan developed by BearingPoint. There is a clearly defined implementation program for United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 16 organizational development of the CFG. Given these factors, it is reasonable to conclude that the CFG will be established as a functioning organization in the near future and the financial system modules obtained from Egypt will be used effectively. When this is accomplished, it will be a very important achievement for the Ministry. c) Metering Early in EG II, BearingPoint produced a brief, undated report on billing and collection issues that identified significant shortcomings in existing processes. Of these identified needs, BearingPoint focused on meters and related issues and left other billing and collection issues for later consideration. BearingPoint’s work program was and remains focused on conducting detailed surveys of damaged or missing meters, developing an information base on existing meters and customers, assessing metering options, and developing a meter replacement strategy. This is an ongoing technical activity, with final recommendations from BearingPoint anticipated in June of 2006; the meter survey will extend throughout the country with no closing date specified. It is estimated that approximately 30 percent of meters need replacing and meters need to be installed where there are none. Given that there are approximately 2.6 million registered customers, the need is for some 780,000 new meters. The actual meter replacement will be a major and very expensive project. Cost projections need to be made and potential funding sources identified. It should be recognized that even with a successful meter program, it is but one component of a total billing and collection process and other significant improvements are needed in order to establish an effective billing and collection program. • Recommendation BearingPoint should develop and execute a detailed technical improvement plan to establish an efficient and effective billing and collection system within the MoE, with integrated sub-systems of a meter program; a customer information system (for billing and accounting purposes); and revised collection systems. The organizational development model to follow is the one now being implemented for the CFG. d) Procurement Of the total EG II resources devoted to the Ministry, relatively few have been devoted to procurement issues. A number of training programs were provided that were of high quality, but they were also generic, i.e., they outlined general principles of procurement and concepts of international bidding. Purchasing and procurement is a key management function within the Ministry, since it involves the expenditures of hundreds of millions of dollars. As a general statement, too few resources have been devoted to this technical area for there to have been any measurable impact on improving MoE operating practices. • Recommendation BearingPoint should initiate systematic evaluations of warehousing and inventory management and control, and the related function of purchasing and procurement, and prepare a detailed improvement program for each with defined budget requirements and established priorities. e) Human Resources and Capacity Building In response to Ministry concerns, BearingPoint identified human resources (HR) and capacity building as a core function. The modified work program of June 2005 includes HR as a core function. BearingPoint’s assistance to the Ministry in this area is through support for the current Steering Committee and Sub-Committee structure as a means to effectively coordinate capacity building activities. HR and Capacity Building Sub-Committees have been established at the MoE to provide the basis for developing and coordinating future capacity building programs, regardless of the source of those programs. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 17 In October 2005, through IRMO, a Ministry Advisory Team (MAT) was established to provide an interface with the Government ministries, including the MoE and its Steering Committee. MAT members represent USAID, PCO, donor agencies, and the U.S. Military. The purpose of MAT is to insure that current and proposed technical projects are executed without conflict and duplication. MAT and the Steering Committee need structured participation by BearingPoint and other contractors in order to function effectively. Members of MAT and contractors, such as BearingPoint, need to establish close relations with the Ministries, with daily interaction between staff, in order to be effective. Effective technical assistance cannot be provided in isolation from those receiving the assistance. • Recommendation BearingPoint should use the OSSA I and II Reports as a guide to develop and execute improvement programs for MoE organizational development, staffing, and personnel issues. These programs should focus on cost reduction efforts, especially in terms of overstaffing. 3. Future Technical Assistance For the past several months, BearingPoint has had only one staff member assigned to provide technical assistance to the Ministry and his primary focus has been on the development of the CFG and the three financial modules that will be implemented in the very near future. What this means in practical terms is that BearingPoint is no longer providing technical assistance to the Ministry. Given the enormously complex problems faced by the Ministry, there should be major technical assistance programs in effect. In late 2005, BearingPoint identified critical technical assistance needs in the areas of finance, billing and collections, warehousing and inventory control, and procurement. Earlier in EG I, the OSSA I and II reports pointed to critical problems of overstaffing that need to be addressed through technical assistance. The Electricity Sector of MAT is formulating a technical assistance program for the Ministry that should result in a defined and coordinated program to effectively address these identified needs. 4. Lessons Learned A critical mass of resources must be devoted to providing capacity building assistance to the Ministry in order for such assistance to be effective, given the size and complexity of the problems of the power industry in Iraq. BearingPoint has done substantial work in its capacity building technical assistance program at the MoE. Yet, most of that work has not resulted in introducing significant changes at the Ministry for several reasons: available resources were insufficient; the original technical assistance under EG II focused more on long-term issues rather than issues of immediate concern; and, the very difficult environment in which BearingPoint has had to operate during the past two years. Until a unified and coordinated approach is formulated and executed, the efforts of individual donors and their partners will be largely ineffective, although there have been and may continue to be notable exceptions. There should be an integrated program for broad-based technical assistance to the MoE, developed, sponsored, and funded, in part by USAID, IRMO, and PCO, with additional support from the World Bank, U.N., and other bilateral donors participating in the rebuilding of Iraq. It goes without saying that the Government of Iraq and the Ministry need to be active, not passive participants in such a program. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 18 VI. Local Governance Program (LGP) A. Context The Local Governance Program (LGP), implemented by the Research Triangle Institute (RTI), was seen as part of the Coalition Provisional Authority’s (CPA) effort to seize the initiative in support of the ‘rush to sovereignty’ that resulted from the November 15, 2003 Agreement calling for Iraqi sovereignty by June 30, 2004. The Agreement formed the basis for the expanded Civic Dialogue Program/Democracy Dialogue Activity (CDP/DDA) that began in February 2004. A related activity was the council ‘refreshment’ that started in November 2003 and continued for several months. Both of these activities were intended to facilitate the handover of sovereignty and prepare for the elections of January 2005 and both affected how the LGP deployed its field staff. 1. Getting Started The LGP 1 program started May 1, 2003 and by December the international staffing reached 230, peaking in mid-March 2004 with more than 2,500 Iraqi staff. Not only was the build-up rapid, staff needed to be deployed in all but one (Al Anbar) of the 18 governorates, in addition to four regional offices. That LGP was able to deploy to all these locations by the end of December was in itself a major logistic achievement. However, by April 2004, as a result of the uprising in the South, the governorate teams were withdrawn to Kuwait. The uprising coincided with a reduced budget for LGP Option Year One (OYO), resulting in sizeable staff reductions. These events had an unsettling effect and the rapid swings in employment levels posed significant management problems. The management challenge for RTI and USAID was noteworthy as few, if any, USAID contracts of such scale had ever been awarded. The scale in program costs and personnel was historic. With the rapid deployment that ensued, there was little time for team building and organizational development. Initial work plans and program implementation measures were based on assumptions, with limited supporting baseline fieldwork. Inevitably, in the rush to deploy staff reached the field without a clear understanding of what was expected of them. LGP governorate teams were dispatched to the field with only a vague understanding of their specific activities. In short, in a rush to provide assistance to Iraq and its citizens, little time was given to fully briefing staff on organizational planning and development before they were deployed. However, once in the field team leaders, who typically had vast experience coupled with advanced degrees, used the process of putting together workplans as a form of team-building. This process engaged the staff in council building, improving local service delivery, identifying focal points for introducing gender issues, and seeking out civil society organizations. There were also monthly team leader meetings convened to present new materials and to share lessons learned, although as security conditions worsened the holding of these meetings presented significant logistical challenges. 2. Measuring Results In an attempt to measure results from the activities envisioned by the contract agreement, RTI’s headquarters monitoring and evaluation section developed a Results Framework (RF) organized into four Intermediate Results (IRs), seven Project Intermediate Results (PIRs), and 32 Sub-Intermediate Results (Sub-IRs). The RF suggested a plethora of possible activities and identified numerous indicators. With the complex RF, measuring the indicators became a problem as there was no systematic feedback mechanism to inform management of possible changes needed. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 19 While the USAID CTOs and RTI understood that remedial action was needed, events overtook the need to find a solution. Following the uprising in the South and the teams’ withdrawal to Kuwait, and in response to a management acknowledgement that a new RF and work plan were needed, a work plan for OYO was put together incorporating specific activities and indicators related to their new IRs. This, in effect, became the LGP I performance management plan (PMP). However, by the time the work plan was approved, the OYO reduction meant that staff decreases continued and there proved to be insufficient staff to implement the PMP. 3. Security Issues Security considerations often prevented LGP I teams from meeting with their local government counterparts. A lack of armored vehicles limited mobility causing staff to wait until security personnel and vehicles were available before they could embark on a site visit. Early in the project no armored vehicles were available, plus there were no inserts in the shrapnel vests, rendering them ineffective. Nevertheless, some field staff accepted the risk and traveled widely; however, following the April uprising more restrictions were rightly placed on staff travel. Simultaneously, there were a number of security incidents specific to LGP, including a hostage taken, a team pinned down by small arms fire for 18 hours, roadside bombs, ambushes, and threats from local militia. A car bombing of the Italian compound that killed 27 in Nassiriyah occurred next door to the LGP office, which was destroyed. 4. Expectations Met With a few exceptions, LGP I met the expectations set forth in the contract agreement despite the consequences of the “refreshment” and “rush to sovereignty” decisions, the deteriorating security situation, and a host of other problems encountered in the often chaotic conditions in the field. Under LGP I significant background information was systematically collected, training modules and materials were developed and consolidated into six modules that were used throughout the country, and handbooks and toolkits were prepared to support the development of Local Government Associations (LGA). The development of local council Charters in Baghdad and Hilla were exemplary achievements that continue to influence the course of local government code development. Records of LGP I achievements are described in voluminous weekly and monthly reports. At a greatly reduced amount of funding LGP II segued from LGP I, keeping some of the same staff and activities and building on the latter’s experience. LGP II activities are focused on: • promoting policy reform in support of local governance; • supporting clarification of the roles and responsibilities of different levels of government; • promoting increased efficiency of local service delivery; • assisting in the development of regularized mechanisms of citizen participation in governmental decision-making processes; and • capturing learning through systematic study and reflection. LGP II efforts examined as part of this evaluation include activities attendant to the introduction of the Provincial Reconstruction Team (PRT) concept by the U.S. Ambassador last August, which became manifest in October 2005. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 20 • Recommendation Decentralization is likely to take a very long time. RTI and USAID should ensure that field staff know what is expected of them in the program strategy with as much specificity as possible. This should include holding quarterly meetings with LGP team leaders and others as necessary to provide information and address issues that arose during that period. This will ensure that information is shared uniformly with all concerned. • Lessons Learned Too rapid build up of field staff can lead to frustration when roles are not well defined and the demand for the skills is not yet in place. The deployment of field staff may have been out in front of the demand and the ability of the program and coalition forces to secure them in the field. The untimely withdrawal of field staff for security reasons lessened LGP’s credibility with those it was trying to reach and hampered RTI’s ability to achieve some intended results. B. Local Councils In partial compliance with the contract deliverable of providing a framework for decentralization of authority to local government structures based on collaborative development, LGP set the stage for participation in the development of councils throughout Iraq. According to the RTI May 2003 monthly report, Dr. Johnson worked with CPA and Iraqi leaders to develop an interim structure of government, to represent the population in the Baghdad metropolitan area. Dr. Johnson participated in the meetings of a working group composed of USAID staff, CPA staff and Baghdad City Department Heads, aimed at designing an interim governance structure. The Baghdad initiative led to the establishment of 88 (now 104) neighborhood councils, beladiya or district councils, and the City Council.5 These were built on a caucus structure with the lower levels providing representatives to the higher ones. The caucus model was used widely by the LGP, but was not the only model employed. LGP also helped increase the number of women members on the councils, and in ethnically diverse Kirkuk there was an effort to provide balanced ethnic representation on the councils. Other LGP teams developed election procedures that were applied in the election of local councils and in many instances the LGP worked with councils that had already been formed by the initial multi-national forces. In the final LGP I report RTI stated that it had developed/assisted 750 councils, consisting of 437 neighborhood councils, 195 sub-district councils, 96 district councils, and 16 provincial councils. IBTCI evaluators could not determine whether this is correct, but it is fair to say that all the LGP teams focused on developing or working with councils as a main priority. However, RTI’s statement that it “trained and mentored” more than 20,000 council members appears to be exaggerated.6 .Simple mathematics suggests that this may be incorrect, since it would mean that an average of 26 members from each council received training. It is more likely that a smaller number of council members were trained on more than one occasion. Training for 5 The Baghdad city council formed by caucus has now been removed by the elected Provincial Council, which has assumed its powers. Additionally, the Provincial Council removed its mayor and established an Essential Services Committee that looks after the City. This has been highly contentious, with the former City Council seeking a reversal of the order for its removal. The initial court ruling in February 2006 has been in favor of the City Council; the Provincial Council has 30 days to appeal. 6 By the end of LGP 1 in March 2004, the annual report states: “To date, LGP has trained more than 9,000 council members, of whom approximately 5 percent are female, on their roles and responsibilities.” In later presentations the number was reduced to 2,000. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 21 council members currently includes modules on meeting procedures, codes of conduct, budgeting, council-executive duties, citizen participation, and transparency. The Baghdad Council has produced a Council Procedure Manual and developed charters for the City, provincial and regional councils.7 A common problem that emerged with the development of the councils was a lack of legal clarity on their roles and responsibilities. This provided the impetus for the development of the Baghdad charters. This lack of clarity grew as an important issue as the LGP became more deeply engaged with the councils and local directorates. The CPA was looked to as the provider of this clarity, but it failed to do so. When Local Government Powers Order (LGPO) 71 finally emerged, it likewise failed to provide the clarity being sought. RTI monthly reports refer to the frustration over this issue; for example, in the May 2004 report a local council official notes that it is not clear who has control over the governorate budget and expenditure plans – the central government, the governorate council, or the governors. This clarity issue remains of fundamental importance because there is currently limited enabling legislation regarding decentralization. LGP has supported advocacy efforts by Iraqi Local Governance Associations (LGA) and provincial councils to bring this issue to the attention of appropriate authorities such as the Council of Representatives (CoR). The CoR recently held a conference lead by the Baghdad Provincial Council where the latter put forward its draft Provincial Law as a proposed model. This national conference is to be followed by regional conferences with the same purpose. Clear enabling legislation to bring about decentralization is essential for the new PRTs to be successful. Moving ahead without it is likely to lead to frustration, given the problems that were encountered in the past over clarity of roles. • Recommendation RTI should help to form a Provincial Government that would liaise with the CoR on issues such as decentralization. The U.S. Government should also directly lobby the CoR to push for support for decentralization so that it is not left to chance. • Lessons Learned Frustrations emerged with the LGP when council training alerted members to their limited roles under the existing decentralization context. Council members understood what was lacking in LGPO 71 and that the LGP can only move forward with decentralization to the extent that the law allows. The same will be true for the new PRT concept. C. Local Departments/Directorates Early involvement by LGP with the local departments/directorates was based on initial assumptions about their roles and responsibilities.8 In Babil, the LGP team reported in August 2003 that they “continued working with al-Hillah’s 54 municipal departments to establish a methodology to identify priority budget needs regarding public service delivery and potential external fiscal resource allocation. A three-part workshop is planned for August 14, 2003 for all 7 According to the July Monthly Report, in Baghdad alone there were 1,500 council members when all levels of advisory councils were considered. 8 Local departments/directorates are the representation of the central ministries at the local level. Local department/directorate staff work for the ministries and not for the local council or mayor. Councils are advisory, not legislative with respect to the departments/directorates. The latter prepare budget estimates for the local jurisdiction that are forwarded to the ministries and eventually to the Ministry of Planning and Ministry of Finance for approval. Citizen input in the budget process is rare. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 22 54 local departments to introduce them to participatory democratic processes, transparency, and accountability.” At the time there was lack of clarity regarding the intergovernmental fiscal system and local government’s capacities. Also in August 2003 in Basrah, the LGP Urban Finance Specialist implemented an organizational budget and financial management survey of the 73 government departments and institutions operating in the governorate. These results were used to determine the technical assistance needed to build the capacity of these local government institutions and to assist in the preparation of their 2004 budget requests. A final report based on the survey results was presented in the workshop. This then led to activities with the Ministry of Finance in Basrah and the 73 departments streamlining the distribution of and accountability for funds allocated by the MoF. The Basrah survey was subsequently developed into a nationwide study of government departments. By September 2004, the LGP undertook an assessment of local government institutions throughout Iraq. In all, the Director Generals of more than 1,900 local institutions were interviewed. Information was collected on their views of possible future decentralization, organization structure and staffing, personnel issues, conditions of service, levels of service provision since the war, budgeting and revenue collection, capital projects and planning, asset management, and audits. The information was compiled in a report and information on each institution was entered into a query system. From these efforts, the LGP gained a deeper understanding of the local government departments/directorates, which helped in pressing for clarification in the laws and administrative procedures underpinning decentralization. By October 2004, there was growing frustration with the CPA because it seemed to be operating at cross purposes to LGP decentralization efforts. In terse language, the RTI report for that month states: There were no CPA policy decrees this month that adversely affected the current work of the LGT [local governance team]. The fact that the local departments still believe they need authorization from national government ministries for program and financial decisions hampers LGT’s ability to push them toward decentralization and taking responsibility for local affairs. This will take a great deal of work to change mind sets at both the national and local levels, in coordination with a strong push from the national and local CPA. Perceptions of CPA’s attitudes eventually led RTI to press for strengthening its Policy Reform Team, which was attempting to work with a reluctant CPA governance team. This effort gathered strength as recognition of the need for legal and administrative clarity that supported decentralization became apparent. This work continues into LGP II, which provides for a much stronger role for developing the necessary legal code to enable decentralization. Training of departments/directorates staff occurred frequently in all but the most difficult governorates. Opportunities ranged from basic public management to highly technical training in software applications, such as AutoCAD, GIS, and STAAD (used in structural engineering). IBTCI evaluators located 14 staff in Kirkuk who had received such LGP training. They recalled the training and spoke highly of it, although some noted that they were not yet able to apply what they had learned due to security concerns with relevant infrastructure (roads and bridges). United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 23 D. Civil Society organizations (CDP/DDA) In the base year, LGP I intended to help create and develop Civil Society Organizations (CSOs) by providing them technical assistance. CSOs were seen as institutions that would encourage transparency in and demand services from local government. To support this effort, LGP I set about making an inventory and assessment of all CSOs throughout Iraq. To do this, regional monitoring and evaluation officers, combined with local Iraqi staff, were used to cover the major urban areas in their regions. Information on CSOs was systematically collected from January – March, 2004 with the use of a questionnaire designed to collect basic contact information, characteristics of the organization, governance and leadership, resources and finances, and training needs. A report was issued but, more importantly, a database and query system were developed in English and Arabic that permitted search and access to the CSOs. More than 1,000 CSOs were identified and became the baseline for CSO activity. Subsequently, the Government of Iraq began to officially register CSOs/NGOs in the country. Before the end of the LGP 1 base year, the program focus had shifted in response to the November 15, 2003 sovereignty agreement. The role in the development of civil society became one of stimulating civic participation and understanding in the run-up to sovereignty and the January 2005 elections. CSO activity then became the Civic Dialogue Program/Democracy Dialogue Activity (CDP/DDA). Program objectives were to inculcate in the participants the underlying principles and practices of democracy. The CDP/DDA program was closely monitored by USAID, since the CPA wanted firm evidence that the population was being reached with the democracy message. Toward this end, monitoring instruments were designed to collect information on the number of participants in attendance at events. For most events a small sample of attendees were interviewed using a structured questionnaire. Teams were trained to enter the data into computers and forward the information to headquarters for analysis and entry into the database. This data collection method is the basis for the estimate that more than 750,000 Iraqis were reached throughout Iraq in approximately 22,000 events. Information obtained from individual attendees included before and after impressions of the issues dealt with during the event. These data were then used to assess the impact the events had on attendees’ points of view. An analysis of these data appears in the LGP I document, “Civic Dialogue Program Performance Assessment,” dated November 2004. The evaluation noted that the CDP/DDA tended to reach a gender balanced audience, but one that was better educated than average and a bit older. It noted that the dialogues most likely to change opinion were the role of media in democracy, gender equality, and elements of free and fair elections. Dialogue themes that stimulated most engagement were civic participation, empowerment and advocacy, the role of media in democracy, and the constitution. Younger and less educated participants were found to be more easily swayed. The evidence, in short, shows that the CDP/DDA stimulated political awareness, encouraged participation in civic issues, and had a positive effect in stimulating discussion based on results-oriented data. • Recommendation USAID should hold fast to requiring results-oriented indicators from RTI. • Lessons Learned Results oriented data can be collected and analyzed when thought through and agreed by implementers. The CDP/DDA program provides an example of how program management information can be systematically collected and used in monthly and weekly program reporting.. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 24 E. Rapid Response Grants (RRGs) RRGs were a cornerstone of LGP I. Grants occurred in every governorate and were well reported. A database system was created early in LGP I, providing continuing figures on Grant amounts and completion. The purpose of the IBTCI evaluation of the RRG Program was to determine if the claimed benefits were as described and if the Program was conducted in accordance with the USAID-approved RTI Grants Manual. The scope of the evaluation embraced the following items: • Did grants meet the requirements in the Grants Manual; • Determine if the grant write-up was a priority project for support using the scoring criteria in the manual; • Note if quality control measures were taken, such as required site inspections of the work in progress and relationships with the relevant government agency and community; and • Were communications established with the media and were attributions made regarding USAID support of the project. Upon inspection of the grant files there was no evidence that grants were selected based on the scoring system set forth in the Grants Manual (Section 3.2). This may suggest that this element should be removed from the Grants Manual when contractors are bidding on emergency repair work. The number of bids received for the project was not listed in one file and the winning bid was often the lowest bid, with little consideration for the quality of work based on past performance. In many instances site inspection reports were not signed. The discussions the IBTCI Field Monitor had with officials of the Sanitation Department indicated that the work performed pursuant to all four grants was of high quality, met a priority need, and assisted in restoring basic services to the citizens in the particular service area. It appears that in all cases the designated USAID CTO signed off on the grants on a timely basis. The files reviewed do not evidence any efforts, except in one case, to coordinate the RRG with other donor activities, such as military programs. Any deficiencies in the project were largely those related to RTI Grants Manual record keeping requirements. • Recommendation The RRG worked well and was well documented. It is recommended that the oversight provisions of the Grants Manual and scoring system be utilized and a final acceptance visit and sign-off be made under what is now called the “Learning by Doing” Grants component under LGP II. • Lessons Learned RRGs provided important leverage for LGP I in the absence of budget line item support for particular types of activity, such as improving and repairing infrastructure. The Grants Manual was not followed to the letter, but resulting validated grants were of an acceptable quality. The scoring system in the grants was not used, but this tool should not be discarded since it can help add to transparency in the selection process. VII. Conclusion The results of IBTCI’s assessment of BearingPoint’s Economic Governance II and RTI’s Local Governance I and II capacity building activities described in this report present a mixed picture United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 25 of accomplishments, unresolved challenges, and unrealized potential. In the case of BearingPoint’s efforts aimed at the Ministry of Electricity, the substantial work carried out has not resulted in significant results because available resources were insufficient, mistakes were made in initially focusing efforts on long-term instead of more immediate issues, and political will among MoE officials and Iraqi political leaders was absent. Regarding activities focused on the Ministries of Finance, Planning and Development Cooperation, and Labor and Social Affairs, BearingPoint efforts were considerably more successful, although questions surfaced throughout the evaluation on just how much capacity building had occurred. Significant accomplishments and, in at least one instance, an extraordinary outcome were achieved as a result of the training and technical assistance provided. However, in the cases of the Social Safety Net and Ministry of Planning activities it is difficult to say to what degree those accomplishments embody achievements in capacity building. For RTI’s work on LGP I and II, the results of the training have been muted by the lack of a law supporting decentralization and unavoidable changes resulting from several Provincial Council elections, which led to officials being replaced by others who had little capacity to carry out their functions or deliver basic services. The lack of a decentralization law and a permanent government undermined the sustainability of many of the capacity building efforts undertaken by the LGP. It should be noted ,however, that two “star performers” were found. The first was the overall response by the LGP to the lack of legal clarity needed to enable decentralization. This response included: establishment of a Policy Reform Team; development of a matrix proposing assignments of responsibilities, based on local assessments, for specific functions across all levels of government; elevating RTI decentralization advice to the Ministry level; and, Ministry acceptance of a pilot decentralization program. The second was the development of a solid waste initiative by the Baghdad LGP team. The Baghdad LGP served as coordinator and advocate for the initiative and also provided strategic grants, management training, and study tours for Amanat solid waste professionals. Many solid waste problems remain, but LGP support has helped the Amanat achieve a meaningful result based on transparency, coordination, and training. Finally, no summing up statement in this case would be complete without mention of the extraordinarily difficult circumstances with which BearingPoint and RTI were confronted on a daily basis throughout the lives of their respective projects. Throughout the period covered by the evaluation, both RTI and BearingPoint had workers injured, killed, and/or under some kind of stress or threat causing these organizations to constantly adjust and readjust their direction and focus. Given these factors, it is truly remarkable that anything of consequence has been accomplished, let alone the sometimes formidable achievements described in this evaluation. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 1 UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT MONITORING AND EVALUATION PERFORMANCE PROGRAM, PHASE-II (MEPP-II) FINAL REPORT APPENDICES PUBLIC SECTOR CAPACITY BUILDING EVALUATION Submitted to: USAID/IRAQ Baghdad, Iraq Submitted by: International Business & Technical Consultants, Inc. 8614 Westwood Center Drive • Suite 400 • Vienna, VA 22182 • USA Telephone: 703-749-0100 • Facsimile: 703-749-0110 with International Science and Technology Institute, Inc. 1820 North Fort Myer Drive, Suite 600 Arlington, Virginia 22209 May 1, 2006 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 2 Final Report Appendices Table of Contents Appendix 1: Proposed Revised Scope Of Work _________________________________________ 4 Appendix 2: Economic Governance Programs Detailed Evaluation Report, Ministry of Electricity ______________________________________________________________ 13 Appendix 3: Economic Governance Programs Detailed Evaluation Report, Ministry of Finance and Ministry of Planning Development _______________________________ 40 Appendix 4: Local Governance Program (LGP) Detailed Evaluation Report __________________ 66 Appendix 5: USAID-Funded Economic Governance-II Capacity Building Programs, March 10, 2006 Debrief Presentation _________________________________________________ 131 Appendix 6: USAID-Funded Economic Governance Capacity Building Programs, March 17, 2006 Debrief Presentation _________________________________________________ 159 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 3 APPENDIX 1: EVALUATION OF CAPACITY BUILDING ACTIVITIES TASK-A (MARCH 2006) PROPOSED REVISED SCOPE OF WORK United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 4 Appendix 1: Proposed Revised Scope Of Work Table of Contents I. Background ………………………………………………………………………… 5 II. Statement of Work ………………………………………………………………... 5 A. Purpose ………………………………………………………………………………… 6 B. Evaluation ………………………………………………………………………………… 6 C. Methodology ………………………………………………………………………………… 7 III. Team Composition ………………………………………………………………… 8 IV. Schedule/Timeframe ………………………………………………………………… 8 V. Deliverables …………………………………………………………………………. 9 VI. Level of Effort …………………………………………………………………………. 10 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 5 APPENDIX 1: PROPOSED REVISED SCOPE OF WORK I. Background To assist Iraq’s transition to democracy, USAID initiated a broad development strategy that includes improving the effectiveness and building the legitimacy of Iraqi government officials and institutions. USAID has provided support to the former Interim Iraqi Government, and is now working with the current Iraqi Transitional Government to strengthen the capacity of the legislative, executive and judicial branches of government. It is also providing capacity development to Iraq’s private sector. 9 Through IBTCI’s Monitoring and Evaluation Performance Project, Phase II (MEPP II), USAID wishes to evaluate its success in building capacity through its development programs from program inception in Iraq until now. The proposed evaluation will: Identify “Capacity Building Development Assistance Tools” retrospectively from USAID’s program, projects and activities from inception of USAID work in Iraq • Identify contexts for future application of these “Capacity Building Development Assistance Tools”. Evaluation results can help the US Government, the Government of Iraq and other donors, partly through the Capacity Development Working Group, to plan and execute future capacity building efforts. This evaluation will focus on two of USAID’s implementing partner programs - BearingPoint’s Economic Governance Program and RTI’s Local Governance Program. These implementing partners provide virtually essential capacity building assistance to the Ministries of Finance, Electricity, Economic Planning, Municipalities and Public Works. Such assistance is central to USAID development strategy. Moreover, it is delivered throughout Iraq at the national and local levels of government. II. Statement of Work Evaluate the programs financed by USAID/Iraq of the following implementing partners: Bearing Point – Economic Governance II Program (Ministries of Finance, Electricity and Planning). RTI - Local Governance Program (LGP I and LGP II) 9 / Definition of capacity development: “Capacity development is the strengthening of human and technical capabilities to support a society in its development of a more secure and sustainable economy, government, and infrastructure.” (Source: October 24, 2005 Report #SEGIR-05-022: Managing Sustainment for Iraq Relief and Reconstruction Fund Programs.” United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 6 o Promote increased efficiency of local service delivery (Pilot Projects in Service Delivery – Ministry of Municipalities and Public Works) o Building the Capacities of Local Administrators. Purpose The purpose of this evaluation is to find, describe and verify successes and failures in USAID/Iraq’s capacity building efforts in these two programs. It is also to provide lessons learned for future program designs. Evaluation results are directed towards the Mission’s Program, Democracy and Governance and Economic Growth Offices. It may also assist the Strategic Objective Teams in their efforts to design the Mission’s Results Framework. B. Evaluation Questions The questions to be addressed by the evaluation are: 1. For each of the client Ministries of Finance, Electricity and Planning for Bearing Point; and Ministry of Municipalities and Public Works, and local governments for RTI • What Ministry/local government functions or problems were addressed with capacity building activities? • What capacity building tools such as on-the-job training, ad hoc advice and formal training were used to induce improvements? • Which indicators have been used to measure improvements? 2. Which of the following core functions were addressed with which capacity building tools such as on-the-job training, ad hoc advice and formal training: i) strategic planning and policy development, ii) fiscal management, iii) personnel management, iv) leadership and communications, v) information technology, vi) administrative rules and procedures, and vii) other critical technical skills for the ministries, organizations and local governments? 3. Did the contractors meet the requirements established in their Scopes of Work? Have they produced the required outputs on schedule? 4. What were the main/most important capacity building outputs or achievements obtain from USAID investment? What are the impacts in terms of the effectiveness of local government capacities in delivering services to the citizens? What are the main failures that the contractors experienced? When possible, findings should be presented in terms of the core functions listed above, and in relation to relevant and applied indicators. 5. What additional indicators are being used or could be used to measure impacts and success of these programs, regardless of the timeframe in which they were developed? 6. Are there any data to support and demonstrate attribution to USAID’s investment for capacity building achievements in the sectors evaluated? 7. What specific or contextual reasons explain star performing activities on the one hand and non-performers on the other? What general lessons are evident from both star performers and non-performers? 8. Out of the lessons learned, what are the pre-requisites and/or the key assumptions for the different capacity building activities to be effective? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 7 9. Summarize lessons learned and suggest levels of USAID/Iraq involvement in managing projects. 10. Compare effectiveness of “up-front indicator development though explicit institutional analysis with indicators developed during contractor operational activity. 11. Assess effectiveness of RTI’s assistance to improve local governments administrative capacity to carry out their administrative functions. Are the skills acquired during the training applied in undertaking their responsibilities? To the extent possible, determine where those trained, especially under LGP1, are now and what are they doing? C. Methodology The point of departure for the methodology should be: • Implementing Partner lists of capacity building successes, failures, • Implementing Partner lists of indicators in use and proposed for use, • Implementing Partner lists of contacts that the evaluation Team can use to begin verifying Implementing Partner capacity building results The following discussion is offered for clarification of the evaluation questions and possible method options. The definition of Capacity Building presented above should be reviewed for accuracy and relevance in Iraq. Question No. 1 requires that the evaluators explore whether activities were tied to solve problems and if indicators were established. Trend graphs can be used to get a picture of performance. Evaluators should review and analyze Scopes of works and work plans from implementers to obtain the required information. Question No. 2 might be addressed by a matrix of activity linkage to the listed core functions. Again linkage may be defined as implementation of the activity would have a demonstrable causal effect on the performance of the function. For RTI specifically, additionally to the core function areas, analyze the capacity building activities in terms of women in development, citizen participation, policy advocacy and conflict management. Question No. 3 might include an analysis of contractor performance by comparing the implementation of activities against the contract required deliverables. An analysis of variance of actual implementation against the work plan timetable should be prepared. Question No. 4 could be addressed by analyzing results of questions 1 through 3. Question No. 5 is a process of finding out or defining indicators s to measure program’s impacts and success, based on the information gathered and through an analysis made by the evaluators. Question No. 6 raises the issue of whether the causal linkage of activities to results can be demonstrated by data. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 8 Question No. 7 is a summary of the implications supported by the findings of questions No. 1 – 6 above. Which activities merit further support, which might be dropped, and what are the lessons learned for future design activities? Question No. 8 – pre-requisites or key assumptions might be presented separately for successes and failures Question No. 9 - examples might be used to illustrate conclusions related to use of “up-front” and “in-process” indicator development. III. Team Composition • Evaluator with experience in energy projects and public utility services • Evaluator with experience in national government finance and planning functions • Evaluator experienced in local government finance and service delivery management • 3 Iraqi evaluation monitors IV. Schedule/Timeframe • Pre-travel interviews, research, review of documentation (SOWs, work plans, reports of implementers) and planning (5 days). A work plan and methodology of work should be submitted to USAID before deployment of consultants to Iraq. (February 6 - 10, 2005) • Travel (2 days) (February 13-15, 2006) Hal Lippman, Jim Douglas; (February 17-19, 2006) Frank Hersman • Initial meeting with USAID, Team Preparation Meeting/planning (TPM) with Iraqi Field Monitors and USAID staff (2 days). (February 16-18, 2006) • Baghdad interviews, field research and data consolidation (23 days for Hersman, 18 days for Lippman, 16 days for Douglas). (between February 18 – March 17, 2006) • Mid-Term debriefing for USAID technical staff and Program Office with initial findings (1 day). (on or about March 6, 2006) • Final results debriefing of Mission Management/Technical Staff (1 day)(March 11, 2006) • Drafting of report with findings and recommendations (5 days each for Douglas [March 7 -11, 2006] and Hersman [ March 13 – 17, 2006]); 7 days for Lippman, March 18-24) • Travel (2 days) (Douglas, March 12 - 13, 2006; Lippman, March 15 – 16, 2006, Hersman March 18-19, 2006) • USAID provides comments (7 days) (March 25-31, 2006) • Final Report preparation (4 days Evaluation Team Leader; 1 day each for the other 2 team members (April 1 - 4, 2006) United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 9 V. Deliverables Work Plan, including a methodology before deployment. (February 10, 2005) Mid-term Briefing for Technical Offices (Economic Growth and Governance) and Program Office (On or about March 6, 2006) 1. Debriefing on final results to Technical Offices, Program Office and Senior Management prior to departure (March 11, 2006)* 2. Draft report submitted. (March 24, 2006) 3. Outline of Final Report • Executive Summary of findings and interpretations/recommendations • Methodology • Findings (including graphs, tables and visual presentations of findings) • Summary and Conclusions • Recommendations • Annexes containing list of contacts and other relevant data • Submission of Final Report (April 4, 2006) *In addition to the March 11 debriefing, an updated debriefing may be offered by the team member responsible for the RTI portion of the evaluation on or about March 16, 2006. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 10 VI. Level of Effort Activity/Schedule # Total Person Days Expats # days # Iraqi Monitor s # day s 1. E xpat 2. Iraq i Pre-travel research, interviews, planning review of documentation (SOWs, work plans, reports of Implementers), and preparation and submission of a work plan and methodology for evaluation 3 5 0 0 15 0 Travel 3 2 0 0 6 0 Initial meeting with USAID, Team Preparation Meeting/planning with Iraqi evaluators and USAID customers 3 2 3 2 6 6 Field research and data consolidation 3 D-16 H-23 L-18 3 12- 14 57 36-42 Mid-term Briefing for USAID Technical Offices and Program Office 3 1 0 0 3 0 Additional consultations, research or meetings 3 1 3 1 3 3 Drafting of findings and recommendations 3 D-5 H-5 L-7 3 2 17 6 Debriefing of final results for USAID 3 1 0 0 3 0 Travel 3 2 0 0 6 0 Final Report preparation 3 D–1 H-1 L-4 0 0 6 0 TOTAL 3 D-36 H-43 L-43 3 17- 19 122 51-57 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 11 Appendix 2: USAID-Funded Economic Governance-II Capacity Building Programs Ministry of Electricity Detailed Report United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 12 UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT MONITORING AND EVALUATION PERFORMANCE PROGRAM PHASE-II (MEPP-II) FINAL DETAILED EVALUATION REPORT USAID-FUNDED ECONOMIC GOVERNANCE-II CAPACITY BUILDING PROGRAMS MINISTRY OF ELECTRICITY Submitted to: USAID/IRAQ Baghdad, Iraq Submitted by: James Douglas International Business & Technical Consultants, Inc. 8614 Westwood Center Drive • Suite 400 • Vienna, VA 22182 • USA Telephone: 703-749-0100 • Facsimile: 703-749-0110 May 1, 2006 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 13 Appendix 2: Economic Governance Programs Detailed Evaluation Report - Ministry of Electricity Table of Contents I. Executive Summary …………………………………………………………………. 14 A. Tariffs and Cost-of-Service ………………………………………………………………. 15 B. Corporate Financial Group (CFG) …………………………………………………….. 15 C. Metering …………………………………………………………………………………. 15 D. Procurement …………………………………………………………………………………. 16 II. Introduction …………………………………………………………………………. 16 III. Methodology …………………………………………………………………………. 17 IV. Ministry of Electricity – Background Information ……………………………… 17 V. BearingPoint’s Programs and Contract Implementation …………………….. 19 VI. The Framework of Core Functions (Technical Areas) …………………….. 20 VII. Core Function: Financial Management ……………………………………… 21 A. Findings ………………………………………………………………………………… 22 B. Conclusions ………………………………………………………………………………… 23 C. Recommendations ……………………………………………………………………….. 23 VIII. Core Function: Procurement ………………………………………………………… 23 A. Findings …………………………………………………………………………………. 23 B. Conclusions …………………………………………………………………………………. 24 C. Recommendations ……………………………………………………………………….. 24 IX. Core Function: Metering ………………………………………………………… 24 A. Findings ………………………………………………………………………………….. 25 B. Conclusions ………………………………………………………………………………….. 25 C. Recommendations ………………………………………………………………………… 26 X. Core Function: Tariffs/Cost-of-Service ……………………………………….. 26 A. Findings ……………………………………………………………………………………………. 26 B. Conclusions ………………………………………………………………………………….. 28 C. Recommendations ………………………………………………………………………… 28 XI. Core Function: Human Resources (HR) and Capacity Building ……………… 28 XII. Future Potential Capacity Building Programs ……………………………….. 29 XIII. Lessons Learned …………………………………………………………………... 30 XIV. BearingPoint Current Workplans ………………………………………………… 31 A. Legend …………………………………………………………………………………………….. 32 B. Financial Management Systems and the CFG …………………………………………….. 33 C. Tariffs and Cost-of-Service ………………………………………………………………… 35 D. Metering Programs ………………………………………………………………………….. 36 E. Procurement Programs ………………………………………………………………………….. 37 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 14 ECONOMIC GOVERNANCE PROGRAMS DETAILED EVALUATION REPORT – MINISTRY OF ELECTRICITY I. Executive Summary In the electric sector, Economic Governance I (“EG I”) focused on: (1) formulating a detailed ten-year master plan, completed and submitted to the Ministry of Electricity (Ministry or MoE); (2) identifying the legal and institutional reform needed to establish an Iraqi electric regulatory authority; and (3) institutional building, providing reports of the broad-based technical assistance needs of the Ministry, Organizational Structure and Staffing Assessment (“OSSA I & II”). EG I ended in June of 2004. Economic Governance II (“EG II”) initiated in October of 2004, as successor to EG I, had a difficult operational beginning. One advisor was shot at, another adviser’s car was bombed, and BearingPoint’s electric adviser was killed on the airport road to Baghdad. As a result, most of the advisers moved operations to Amman for several months before returning in April of 2005. There were, and continue to be problems of obtaining advisers with significant relevant power experience. There are continuing problems of having direct access to the Ministry staff. A corollary problem was, and remains, a lack of inter-agency cooperation and communication on issues of mutual interest and concern regarding the Ministry. In the electric sector, EG II pursued two goals: (1) restructuring of the electric industry and (2) institutional capacity building. Based on USAID’s Request for Proposal and the subsequent contract, the initial focus of EG II was on pursuing goals of privatization and market reform, restructuring, and regulatory reform. These goals are not achievable in the foreseeable future. Addressing critical institutional needs identified in EG I took a secondary role. OSSA I and II addressed major problem issues facing the utility: inefficient and ineffective organization and huge overstaffing and related personnel issues. Implementing OSSA I and II recommendations would be difficult in the best of circumstances; a major technical assistance program should have been developed around these recommendations. The fact that there was no technical assistance follow up to OSSA I and II was a major mistake. The broad based need assessments of EG I identified many improvement needs in institutional development that were never followed in EG II, such as warehousing /inventory control, organizational improvements, and overstaffing and related personnel issues. Although the significant resources devoted to privatization, restructuring, and regulatory issues may have produced substantive documents and recommendations, the recommendations have not been accepted by the Government. With the appointment of a new Minister in June of 2005, and staff changes within BearingPoint, the EG II work program focus shifted from regulatory reform to the technical areas (i.e., core functions) of: (1) tariffs and cost of service; (2) a corporate financial management group; (3) metering (a major component of billing and collections); and (4) procurement. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 15 A. Tariffs and Cost-of-Service BearingPoint conducted extensive and well documented cost-of-service and related studies that justify major increases in tariffs (rates). The final report and tariff recommendations are under final review and should be released soon. Concerns about potential negative impacts were addressed effectively. There is a critical need for a substantial increase in tariffs. The government must find the will to make critical tariff decisions. It is strongly recommended that IRMO, USAID, UN, and the World Bank, quickly take a united position on tariff increases, which may involve a compromise tariff, and essentially demand government action. B. Corporate Financial Group (CFG) BearingPoint focused on the organizational development of the CFG as a functioning staff unit within the Ministry, with responsibilities for financial analysis and reporting, for formulating financial strategies, and for providing assistance to senior management in decision-making. To support the CFG staff, a joint (BearingPoint/Ministry) decision was made to develop and implement three computer based financial management modules that were available, in Arabic, from the Egyptian Electric Holding Company. Currently, Ministry staff is in Egypt undergoing technical system training; consistent with the defined systems implementation work plan developed by BearingPoint. There is a clearly defined implementation program for organizational development of the CFG. It is a reasonable conclusion that the CFG will be established as a functioning organization in the near future and the financial system modules obtained from Egypt will be used effectively. This will be a very important achievement within the Ministry, but it is yet to be accomplished. C. Metering Early in EG II, BearingPoint produced a brief report on Billing and Collection issues, which identified significant shortcomings in existing processes. Of these identified needs, BearingPoint focused on meters and related issues and left other billing and collection issues for later consideration. BearingPoint’s work program was, and currently remains, to conduct detailed surveys of damaged or missing meters, develop an information base on existing meters and customers, assess and formulate recommendations on metering options, develop a metering strategy, and establish a metering replacement strategy. This is an ongoing technical activity, with final recommendations from BearingPoint anticipated in June of 2006; the meter survey will extend throughout the country with no closing date specified. It is estimated that approximately 30% of meters need replacing, and meters need to be installed where there are now no meters. Given that there are approximately 2.6 million registered customers, the need is for more than 780,000 new meters. The actual meter replacement will be a major project and very expensive. Cost projections need to be made and potential funding sources identified. It should be recognized that even with a successful meter program, it is but one component of a total billing and collection process. Other significant improvements are needed in order to establish an effective billing and collection program. The improvements could include a customer information system, required for customer billing and accounting purposes, and revised bill collection programs. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 16 D. Procurement Of the total EG II resources devoted to the Ministry, relatively few resources have been devoted to procurement issues. A number of training programs were provided which were quality products, but they were also generic, i.e., outlining general principles of procurement and concepts of international bidding. Purchasing and procurement is a key management function within the Ministry; it involves the expenditures of hundreds of millions of dollars. As a general statement, too few resources were devoted to this technical area to have any measurable impact on improving operating practices within the Ministry. For the past several months, BearingPoint has had only one staff member assigned to provide technical assistance to the Ministry and his primary focus has been on the development of the CFG and the three financial modules that will be implemented in the very near future. What this means in practical terms is that BearingPoint is no longer providing technical assistance to the Ministry. This Report first provides background information on the Ministry and on BearingPoint’s technical assistance (capacity building) programs under EG II. The report is then organized by core function (technical area) reviewed. A brief profile of each core function is provided, followed by findings, conclusions, and recommendations, appropriate to the respective core function. The final two sections identify: (1) critical areas of importance to the Ministry that should be addressed by further technical assistance programs; and (2) an overview statement of lessons learned from this evaluation. II. Introduction This Evaluation Report on Technical Assistance to the Electric Sector (“Report”), conducted pursuant to the Monitoring and Evaluation Performance Program, Phase II (“MEPP II”), provides the evaluation of BearingPoint’s current capacity building programs as those programs relate and support the Ministry of Electricity (“MoE” or “Ministry”), as conducted pursuant to Economic Governance II (“EG II”). It should be recognized that BearingPoint’s work programs in the electricity sector have more of the characteristics of technical assistance or management consulting that they do to “capacity building” training programs as traditionally used within USAID. BearingPoint’s efforts in electricity have involved comprehensive and complex studies, system design and development programs, and technical assistance in organizational development. Training programs have been relatively incidental to the total effort. Also, at the outset it should be recognized that starting in late 2004 and early 2005, the MoE began assuming more and more responsibility for defining the technical areas for assistance and guiding the direction of that assistance. The original Request for Proposal (“RFP”) identifies four technical areas for assistance: (1) management; (2) planning and asset management; (3) regulatory; and (4) financial and accounting. The current contract, as modified, identifies these technical areas, with a slightly different focus and modifies the tasks and sub-tasks, while the actual BearingPoint work program shifts, or redirects, both the areas of technical assistance and related tasks. There are significant differences. This evaluation is based on the technical areas as currently defined, and the tasks currently or recently undertaken. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 17 This Report first provides background information on the Ministry and on BearingPoint’s Capacity Building Program for the Ministry, pursuant to USAID-Funded Economic Governance II Project. This Report is then organized by core function. The first core function, or technical area, reviewed is the framework of Ministry specific core functions (technical areas) and a review of the technical areas which have received capacity building assistance from BearingPoint assistance during EG II, but are no longer considered active. The remaining five issues addressed are the five current technical areas, or Ministry core functions, receiving capacity building assistance from BearingPoint: Financial Management; Procurement; Metering; Tariffs/Cost-of-Service; and Human Resources (HR). In this Report each core function is summarized, followed by findings, conclusions, and recommendations, as may be appropriate to the technical area. The Report concludes by identifying future/potential technical assistance programs and a summary of lessons learned from this evaluation. III. Methodology Methodology used was Electric Sector specific, Core Function specific, and Technical Assistance specific. The universe of BearingPoint documents was reviewed, selecting documents relevant to electricity, core function specific, and related technical assistance activities. These reports were reviewed in depth and discussions where held with appropriate BearingPoint or former BearingPoint staff. Such review and discussions lead to the findings presented in this report. Review of the findings lead to the conclusions and a review of the conclusions lead to the recommendations contained herein. The Scope of Work, Evaluation of Capacity Building Activities, Task A, identified a series of eight questions that should be addressed within this work effort. These eight questions are addressed in Annex A. of this report: (Annex A - Summary Response to Evaluation Questions). IV. Ministry of Electricity – Background Information The goal of the Ministry can be stated simply: to provide sustainable and reliable electric power to its customers at just (non-discriminatory) and reasonable (affordable) cost-based rates, without the need for government subsidies. The Ministry of Electricity functions as a vertically integrated electric power company. It maintains the country-wide electric grid and operates central dispatch. It has separate legally constituted operating companies, and operates under a complex set of Iraqi laws and regulations. The MoE was reorganized in October 2003, establishing eight headquarters directorates; four common purpose directorates (Central Maintenance & Workshops; Technical Services; Transmission Projects; and Production Projects; plus a Security unit); and 20 operating companies, including seven generating companies, six transmission companies, and seven distribution companies. This reorganization increased both operating costs and labor costs significantly. According to OSSA I and OSSA II reports, in 2003, there were approximately 38,000 employees within the Ministry, including approximately 2,000 security guards. Reliable but undocumented reports indicate that the number has now grown to nearly 50,000 employees, including an unspecified number of “electric police” (“security staff”), which are budgeted in the Ministry of Defense, and “ghost” employees, those who are on the payroll but are never at work. That is nearly a 25% growth in staffing, without any noticeable increase in productivity! Actual United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 18 payroll costs have increased at a greater rate because the CPA restructured a complex low￾wage-with-allowances system into a flat-wage structure at a much higher cost level. Per BearingPoint’s 10-Year Master Plan, dated July 2004, in 2003, the installed generating capacity approached 9,000 MW of power. From a recent report from the Ministry, dated 12 February, 2006, the installed generating capacity had increased to 11,687 MW of electric power, but available capacity (actual production) was only 3,541 MW, which included 351 MW of purchased power from Iran and Turkey. Installed capacity includes generating units that have been recently refurbished and new units recently installed and brought on-line, as well as the older units. Iraqi generating installed capacity and available capacity, on 12 February, 2006, by type of generation, is outlined as follows: Installed Available Type of Generation Capacity Capacity Steam 5,015 MW 1,496 MW Gas 4,159 MW 1,096 MW Hydro 2,513 MW 596 MW Source: Ministry of Electricity, National Dispatch Center, Daily Unit Production Report for February 12, 2006 As generating units are being refurbished and new units are being added, the total amount of installed capacity is rising, but actual generation has been relatively flat. As reported in the March 8, 2006 US State Department’s Iraq Weekly Status Report, generation since January 2004 has been in a range of about 3,000 to 3,500 MW, and on occasion, passing 4,000 MW of production, so the percent available of that installed capacity is falling, not rising. At the same time, daily demand for electric power is high and growing at a rapid rate; daily demand is estimated at 7,000+MW, resulting in major electric shortages and power outages throughout the country10. In other words, the Ministry can not sustain reliable power at even a minimal level of service, even though its installed capacity is rising because of new generating units coming into service. It’s a paradox that there can be rising generating capacity and no increase in production! At the same time, it is collecting minimal revenues, because of extremely low tariffs and shortcomings in their billing and collection programs. If tariffs were raised dramatically, they would still be the lowest in the Middle East. At the same time the Ministry’s annual cost of service is rising dramatically, largely from the completely uncontrolled increase in staffing and payroll costs. Reorganization of the Ministry in 2003 resulted in increased costs, not reduced costs. Operational costs have risen dramatically. Some of the new generating plants are gas fired, but because of a lack of gas fuel, heavy fuel oil is used, dramatically reducing power output capacity and increasing plant maintenance costs. Some of the new power plants were installed to run on diesel and with fuel shortages. Diesel is being imported at very high prices (the government is paying for these costs out of the Ministry of Finance Budget). The Ministry and the government must recognize that the Ministry is not a government program to be funded by the government, but a government owned public enterprise that should be self- 10 Special Inspector General for Iraq Reconstruction ( SIGIR), January 2006 Report to Congress. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 19 supporting and operated like any other commercial enterprise. In the long run, profits should be generated and the government should receive those profits. The Ministry and the government need to take much of the responsibility for the critical situation that exists in the electric sector. The Ministry and the government need to make the difficult decisions required to introduce dramatic changes to the way MoE operates; otherwise technical assistance can not prevent further disintegration of service. That is what is happening; electrical service is disintegrating in Iraq. V. BearingPoint’s Programs and Contract Implementation In October 2004, BearingPoint initiated its three year EG II program to support economic reforms in Iraq. EG II is the successor to Economic Governance I (“EG I”), which ended in June 2004. As stated by BearingPoint, the overall objective in the Electric Sector was: A commercially viable sector with improved regulation. 11 Another BearingPoint report expanded on the objective of establishing a viable sector, stating: develop and help establish a restructuring plan and market model for the electricity sector that facilitates private sector participation, clearly separating policy, regulation and operations…12 The term “restructuring” has significantly different meaning than the term “reorganization.” Restructuring means changing roles and functions within the electric sector and changing the role played by the MoE. Reorganization refers to substantive changes within the Ministry, generally made for purposes of increased efficiency and effectiveness. Late in the course of EG I, BearingPoint produced a document titled, Organizational Structure and Staffing Assessment for the Ministry of Electricity (OSSA I). This report provided a critical evaluation of organizational and staffing shortcomings within the Ministry. Primary conclusions were that the Ministry was very poorly organized and greatly overstaffed. The report provided a broad set of recommendations for reform in the areas of reorganization and staffing. The report provided a number of insights, including the identification of the broad reforms to the MoE that need to occur over a period of time and in an orderly sequence, as outlined below: Program Activity Restoration recovery from war damage/looting Modernization recovery from major lag in technology/management Governance elimination of crime and corruption Unbundling separation of work processes and accounts Stable Enterprise with stability, establish regulatory body and: - Competitive market - Restructuring - Private Power - Privatization 11 USAID/Iraq Economic Governance II Project, BearingPoint, presentation to Electric Steering Committee, 21 December 2004. 12 BearingPoint, Overview of USAID Assistance to Iraq on Economic Governance, 12/13/2004. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 20 The report then noted: This sequence is not rigid, and different parties would define stages in different ways, but the general flow of reform is consistent with global experience. Skipping stages or processes would imperil subsequent stages.”(Emphasis added). BearingPoint later released OSSA II, a report which expanded on the recommendations and the proposed action programs of OSSA I. While the reports were submitted to USAID, the recommendations did not translate into any technical assistance programs by BearingPoint. OSSA I and OSSA II addressed the major problem issues facing the utility: the lack of efficient and effective organization and huge overstaffing and related personnel issues. Implementing these recommendations would have been difficult in the best of circumstances; a whole technical assistance program should have been developed around these recommendations. The fact that there was no follow up of OSSA I and II by BearingPoint was a major mistake. Under the EG II contract between USAID and BearingPoint, the initial primary focus and resources of EG II were on legislation (new electric laws), regulation, and restructuring of the electric sector, rather than on improving the operations of the Ministry. Related objectives were: (1) separate policy (Ministry), regulation (independent regulator), and operations (operating companies); (2) unbundle Ministry operations; (3) establish new corporate entities; (4) develop new market models; (5) establish new institutions; and (6) establish a privatization strategy. As BearingPoint reported to USAID in various reports over the past year, while this focus on restructuring and market reform produced a series of solid well thought through reports and a series of substantial recommendations, none of these recommendations have been accepted or implemented by the Iraq government. Over the course of this EG II effort, there have been eight broad technical areas which have been the focus or subject of technical assistance by BearingPoint, of which five are currently active. The inactive technical areas relate to restructuring and market reform, electricity laws, and a regulatory scheme. In the contract between USAID and BearingPoint the basic technical areas and related tasks were identified in broad terms, allowing BearingPoint to change the technical areas and their work program with USAID concurrence, but without requiring contract modification. Last year’s changes in focus were made in response to views of the current Minister of Electricity. These changes were stated in progress reports in late 2005 to USAID. VI. The Framework of Core Functions (Technical Areas) Core Ministry functions, termed “technical areas” in USAID’s original RFP, have changed over time from the original RFP, to the most recently amended USAID Contract with BearingPoint, to their present assistance program.13 Before examining BearingPoint’s assistance program to the Ministry, it is important to clarify core functions (technical areas) to which this assistance has been and is being directed, and examine the basis for those changes of technical areas, summarized as follows: 13 In this context, assistance programs are the “Capacity Building Development Tools” identified in Scope of Work, Evaluation of Capacity Building Activities, Task A, IBTCI Consortium, March 5, 2006. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 21 As Amended (Award No. 267-C-00-04-00405-00) Original USAID RFP /Latest Contract Work Program Management Planning and Asset Management Regulatory Finance and Accounting (Modification No. 05) Work Program Original Focus Work Program Current Focus Legislative and Regulatory Restructuring Operations - Metering - Billing and Collections - Financial/MIS Financial Management Procurement Metering Tariffs HR/ Capacity Building The current Minister of Electricity is focusing on critical operating needs and those Ministry functions that directly support operations, on the reasonable rationale that the limited resources that are, or may be available, should not be devoted to the longer term initiatives recommended by BearingPoint, i.e., regulatory programs, Electricity Law, and private sector market initiatives (Independent Power Producers, Build-Operate-Transfer (BOT) contracts, etc.). Therefore, these initiatives are on hold for the foreseeable future. Also it should be recognized that implementing these reforms will require coordinated action by other ministries, the Prime Minister’s office, as well as enabling legislation. To facilitate capacity building within the MoE, BearingPoint and the Ministry, early in 2005, re￾established a Steering Committee14, which had originally been formed several years previously, developed and implemented to coordinate and assist in the implementation of technical assistance programs. Under the Steering Committee, Sub-Committees were established, made up of staff of the MoE, and organized functionally, i.e., a sub-committee for each core function or technical area, to which BearingPoint was providing assistance. As the focus on core functions changed over time, the structure of the subcommittees changed accordingly. Currently there are five core functions that BearingPoint and the Ministry consider active, although only two (Financial Management and Metering (to a very limited extent), are actively supported with technical assistance at this time. These five areas are reviewed and evaluated in the following sections. VII. Core Function: Financial Management The objective: the Ministry and its operating companies need effective financial tools to support policy and fiscal strategy development, to provide the information base for effective decision￾making by MoE management. After discussions between BearingPoint and the Ministry, the 14 Per interviews with former BearingPoint staff, originally an Executive Work Group was established, with a subcommittee called the Executive Reengineering Group; during the time between EG I and EG II these groups became inactive, attributable to the lack of any mentoring by BearingPoint. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 22 decision was made to develop and implement three computer based financial management modules that were available in Arabic from the Egyptian Electricity Holding Company (EEHC). These financial modules should not be confused with the integrated FMIS system under development for the Ministry of Finance, nor are they intended to replace the existing internal budget and accounting system of the MoE. In conjunction with the development of these financial modules, BearingPoint focused on the organization and functions of a Corporate Financial Group (CFG). The unit would be responsible for reporting, financial analysis, and financial strategy development for the MoE. Defined responsibilities include: • Provide assistance to MoE senior management in decision making, with special attention on financial analysis of proposed policies, programs, and major projects. • Prepare forecasts for Ministry’s operations and review operating and financial performance, comparing actual to forecast. • Maintain the three computer-based Financial Management modules. A. Findings Under BearingPoint guidance, 60 MoE staff have been selected and are now (March, 2006), in groups of 20, participating in a three week training program in Egypt covering the new systems. In addition, three trainers will undergo a three month training-for-trainers program in Egypt, at the conclusion of the initial training. This training program is currently in progress and BearingPoint’s advisor is based in Egypt throughout this training period. Two members of the Steering Committee will attend for 10 days to better understand the program and survey the quality of training. The CFG organization will consist of three functional units: Cash Management & Financial Forecast; Financial Analysis & Studies; and Financial Planning. Clear duties and responsibilities have been defined. In carrying out their respective work programs, staff will utilize the three financial models obtained from EEHC, which are: • Cash Management Model • Financial Planning Model • Loan Tracking Model A functioning CFG office will be established when all the trained staff has returned from Egypt. BearingPoint, in addition to assistance in obtaining these models and training staff, is also assisting in the organization of the CFG office, developing useful work programs for the CFG, and providing follow-up assistance as needed. Current plans of BearingPoint are to assist the MoE in establishing clear policies and procedures for budget development and execution with the Ministry, first focusing on the operating budget, and then developing programs to budget for capital projects. BearingPoint has recently completed, but has yet to submit to the MoE, a report on budget processes and requirements. This report identifies significant shortcomings in present budget practices within the Ministry, reviews and describes briefly the sound principles underlying a modern budget system, and outlines the initial steps that the Ministry should follow to strengthen and improve its budget program. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 23 B. Conclusions This technical assistance program has, or soon will, resulted in MoE establishing a functioning organization, the CFG, with a defined organizational structure, staffing, assigned duties and responsibilities, and a comprehensive work program. The organization will have acquired three computer based systems to support their analyses and reporting. Staff of this new unit, CFG, will have been trained, along with three trainers who have the responsibility to establish an on￾going training program. If the CFG comes together and functions as intended, it will be a major accomplishment of both BearingPoint and the MoE. C. Recommendations In reference to budget processes, it should be recognized that developing and implementing a modern budget system within the MoE is no small undertaking and, regardless of how important such a program may be, BearingPoint does not have current plans to develop and implement such a program. It is recommended that a budget technical assistance project be established. In the future, to the extent that effective budget polices and practices are established, there is the need to establish a formal organizational entity within the Ministry responsible for budgeting. It appears that the best organizational location for such an entity would be a functional entity within CFG. Capacity Building assistance would be needed for such organizational development. VIII.Core Function: Procurement The objective: the Ministry needs to improve the efficiency and effectiveness of its purchasing and procurement activities. The MoE needs effective audit and accounting controls over purchasing procedures to guard against potential graft. It needs to establish effective policies and procedures to prepare and execute large complex international tenders for facilities (power plants, high voltage transmission lines, and so forth), equipment, and services (including consulting services). BearingPoint has provided limited technical assistance in this functional area, relative to its other technical assistance programs to the MoE. A. Findings BearingPoint developed and provided two training programs to MoE staff: (1) Private Sector Participation, two four-day courses, to 15 people each, in November 2005; and (2) International Tendering & Procurement, two courses, to 20 people each, also in November 2005. There was also a one day program given to MoE staff on legal/ contract issues. The Evaluator reviewed the BearingPoint Training Programs and determined that these courses were generic to the extent that they were not directly tied to existing practices and procedures of the Ministry. BearingPoint has not had any staff devoted to Procurement training or related activities since the training programs were completed in late 2005, and anticipates no resources will be devoted to this technical area during the balance of their contract. The tender development program either for management contracts, or for the development of IPPs or BOT/BOO are not being activity pursued by the Ministry. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 24 B. Conclusions Of the EG II technical assistance provided to the MoE, relatively little resources have been devoted to the Procurement area. Much of the resources that were devoted to the procurement function pursued activities unrelated to existing practices and purchasing issues of the Ministry. The few training programs that were given were quality products, but they were also generic, i.e., outlining general principles of procurement, of concepts of international competitive bidding, or for contracts for private sector participation in the power sector. To the extent that assistance was given for issuing specific tenders, e.g., a management contract for billing and collection, there has been no detailed follow-up, and no indication from the Ministry that such private sector contracts will be pursued. No BearingPoint resources are currently being devoted to the Procurement area, and none are anticipated for the balance of this year. Purchasing and Procurement is a key management function of the Ministry; it involves the expenditures of hundreds of millions of dollars a year. As a general statement, too few resources were devoted to this technical area to have any measurable impact on improving operating practices for purchasing within the Ministry. C. Recommendation There is the need to conduct a thorough assessment of the existing practices and procedures, including organizational, staffing, and operational issues, of the procurement function within the Ministry, identifying shortcomings and defining an improvement program. One of the primary focuses of such an assessment should be on the effectiveness of existing audit and accounting procedures to protect against graft and other illegal activities. Such an assessment should provide the basis for establishing a broad based assistance program, for which allocation of funds needs to be considered by USAID. IX. Core Function: Metering Early in the EG II program, BearingPoint produced an excellent, but brief, Report on Billing and Collection, undated, that identified significant shortcoming to existing billing and collection processes, including meters and meter reading, the billing system, revenue collection, customer service, and overall administration of the processes. The report outlined a series of needed improvements, including but not limited to: • Consolidation and updating billing master files; replace billing system; development of an advanced Customer Information System • Modernization of cash offices (collection centers) • New bill collection procedures • New meters and related systems to replace damaged or broken meters. Of these identified needs, BearingPoint assistance in EG II focused on meters and related system issues, and left the other billing and collection issues for the future. BearingPoint estimates that as much as 30 percent of the electric meters may be missing or damaged. Given 2.6 million registered customers; there is a need to develop a realistic United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 25 replacement plan for approximately 780,000 meters, plus an unspecified number of new meters for new connections. The actual meter replacement will be a major project and very expensive. Cost projections need to be made and potential funding sources identified. BearingPoint’s work program in this core function was and is to conduct surveys of existing meters and identify problems, as well as obtaining information or data about the customer i.e., name address, and related data. This data should provide the basis for establishing reliable customer information, essential for proper billing and collection functions. As part of the metering program, BearingPoint and the MoE have developed extensive information on differing options for metering technology, have developed a metering strategy, and are formulating a meter replacement plan. A. Findings All existing meters, some having been installed as early as 1966, utilize electromechanical technology which has a rotating disk supported on jewel and pivot or magnetic bearings. The predominant type used in Iraq uses magnetic bearings. A BearingPoint survey concluded that the metering systems proved to be in good condition. Good industry practices calls for meters to be removed from service at a maximum of 15 years, refurbished, tested, and recalibrated before being reused. BearingPoint recommended that an effective maintenance program be established that follows industry practice and is working with the metering sub-committee to establish such a program. In replacing the large number of damaged or malfunctioning meters, BearingPoint recommended that future meter purchases be electronic solid state (SS) meters. SS metering offer a range of configurations, from a simple single rate meter to multiple tariff meters, with and without remote reading facilities. BearingPoint provided the MoE with broad based set of findings and recommendations relative to the selection of SS meters. As part of their analyses, BearingPoint considered the use of prepayment meters and recommended a pilot evaluation program be established. BearingPoint noted that pre-payment metering is best suited to the current Iraq IT infrastructure. Pre-payment metering is a method of collecting revenue from customers before they use the electricity. They involve customers paying for a token to be issued or a card or key to be programmed to instruct the meter to allow electricity to be used up to the value of the purchase. B. Conclusions BearingPoint has conducted extensive research on electric meters and formulated sound recommendations for modernization programs. However, it must be recognized that money is in very short supply. It’s of little value to gain an in-depth knowledge and understanding of leading-edge meter technology if there is no money to acquire and install new meters. After BearingPoint completed an assessment of billing and collections processes, it was a major mistake to focus only on metering issues and ignore the other areas of needed improvements to the billing and collections systems (customer information system and collection procedures and practices). United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 26 C. Recommendations Actions to date have resulted in identifying the appropriate meter technology and in implementing training programs; however a final report is yet to be released. To move forward with an actual meter replacement program, there is the need to clarify the costs, define a specific replacement program, and secure funding for that program. Finally, an effective metering strategy must be integrated into an overall improvement program of the billing and collection processes, of which metering is, or should be, an integral part. Major improvements to the total billing and collection systems are of critical importance in generating needed additional revenues. X. Core Function: Tariffs/Cost-of-Service A “tariff” means the rate charged individual customers. Tariffs vary by customer class, because tariffs are, or should be, cost-based, and the cost of electric service varies by customer class. Cost-based means that the tariff should reflect the actual production and service delivery costs. A tariff, besides defining rates, should define clearly the terms and conditions of service, i.e., rules for connection and disconnection, deposit fees, minimum fees, customer rights, and like matters. Tariffs are a critical core function because of the impact they have on revenues generated. Tariffs should not be viewed in isolation of two other key issues: (1) effective billing and collection processes, to insure that all customers are billed and pay their bills; and (2) effective and efficient operations and management of the electric service, to insure that customers are not paying for unneeded or unjustified costs. BearingPoint has focused on billing and collection issues, which is addressed in another Core Function, at least as it relates to meters. A. Findings BearingPoint has conducted a series of cost of service and cost allocation studies, on the basis of which tentative tariff or rate changes have been developed. The final report on tariffs is in the final stage of preparation. The cost amounts used reflect operating costs and do not include capital costs, which is a separate matter. The critical issue addressed was that while the tariffs (rates) applied in 2002 generated sufficient revenues to cover operating expenses in 2002, the cost of providing service has risen dramatically in recent years, resulting in major operating deficit, requiring an increasing amount of government subsidies, since rates have not been increased. This critical situation can not be long sustained. BearingPoint will certainly recommended major increases in tariffs. Their recommendations, as drafted, are well documented. Potential concerns about negative impacts of such tariff increases have been addressed in meetings with the Ministry, with extensive documentation justifying the proposed increases. To provide a clear understanding of the problem poised by the sharply increased costs of electric service, BearingPoint provided recent cost data, outlined as follows: United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 27 Year Expenses (1,000ID) 2003 191,044,223 2004 299,667,737 2005 468,280,540 2006 (Budget) 641,588,496 BearingPoint has conducted a number of related studies or analyses, including: (1) rate comparisons with other countries; (2) collection rates; (3) demand limiters to reduce consumption; (4) existing subsidies in existing tariffs; (5) affordability (ability to pay); (6) price elasticity of demand (estimating the relationship between prices (rates) and consumption (electric use); and (7) studies related to customers not billed (broken meters or illegal connections). These studies are under final review by BearingPoint before their release as part of the tariff recommendations. Rate comparisons, as surveyed by BearingPoint, reflect the following (2004 prices in US cents/kwh): Country Rate • Jordan 5.70 • Yemen 5.30 • Syria 2.00 • Saudi Arabia 1.70 • Iran 1.20 • Iraq .18 As part of their Tariff work program, BearingPoint developed and delivered training sessions on Cost of Service and Tariff to MoE employees and assisted in the establishment of a Tariff Department. MoE adopted BearingPoint recommendations to re-start billings after the war; billings were resumed in December of 2003. Recent collection rates as a percent of amount billed are: Year 2003 for the months billed, 24%; Year 2004, 64%; first six months of 2005, 80%. Current estimates indicate a moderate improvement in collection rates. Pre-war collection rates were reported to be approximately 95%. In response to a request by MoE, BearingPoint studied the economic feasibility of installing demand limiting devises on customer lines as an effective means to reduce consumption. This was a limited study that defined issues but made no recommendations. The overall cost of placing these devises on the 2.1 million customer connections, was estimated at $107 million dollars. The unanswered key question is whether the cost savings or benefits on demand reduction would be greater than the cost of installing and maintaining the demand limiting devices. From BearingPoint’s perspective, a much better option to installing demand limiting devises would be to increase the tariff (rate or price) significantly enough to encourage energy conservation and reduce demand. The affordability study, as conducted by BearingPoint, concluded that rates could be raised significantly without, in most cases, exceeding affordability benchmarks (measurements of a United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 28 percentage of average household income). To the extent that there is a problem, it could be handled in two ways: (1) a subsidized life line rate, or (2) a government welfare subsidy that includes an electricity element in its structure. B. Conclusions BearingPoint has conducted substantive well documented cost of service studies, on the basis of which they have developed a reasonable tariff structure. The tariff structure appears sound and well thought through. As currently drafted, BearingPoint does not address the question of whether the current costs are reasonable or excessive, because of operational inefficiencies, overstaffing, waste, poor organization and management. The customer should not have to pay for overstaffing, waste, and ineffective organization. C. Recommendations While the various tariff studies developed by BearingPoint are sound, there was no review of terms and conditions of service, which are an organic part of a tariff. There should be a review of the terms and conditions of service and formulation of a clear set of recommendations. As part of those terms and condition, provisions should be made for a minimum billing fee assessed to all connected customers. To an extent, that would address the problem of customers who may receive service but are not billed due to faulty meters. Recommendations relative to terms and conditions may be included in the final report on tariffs. The issue of illegal connections needs to be approached separately, through the development of an effective enforcement program. As noted above, the draft cost of service studies made by BearingPoint assume that costs are reasonable, or they are what they are. There was no review of costs for reasonableness. One of the primary roles of an independent regulator is to question costs and reject unreasonable costs. Absent an independent regulator, the Minister of Electricity carries the responsibility to insure that costs are reasonable, and to implement effective cost control and cost containment measures. Clearly in this environment this effort would be a very demanding and difficult undertaking. Clearly this is an area for major capacity building assistance that is very important to the future of the Ministry. In the meantime, there is a critical need to raise tariffs (rates) significantly. It is strongly recommended that IRMA, USAID, UN, and the World Bank quickly take a united position on tariff increases, which may involve some compromise, and essentially demand that rates be raised by the government. It should be noted that the need to critically review costs is separate from the need to raise tariffs. When that is accomplished, to the extent it is accomplished, then the major issue of subsidies received from other government ministries can be addressed. Besides generating revenues, tariff increases are an indirect, but appropriate, means to limit consumption – another worthwhile goal. XI. Core Function: Human Resources (HR) and Capacity Building In response to Ministry concerns, BearingPoint identified HR (Human Resources) and Capacity Building as core Ministry functions for capacity building assistance. The modified work program of June 2005 includes this HR function as a core function. However, no BearingPoint technical resources have been, or are being, devoted to this function. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 29 The BearingPoint assessment reports (OSSA I & II), prepared in EG I, identified major problems of overstaffing, a top-heavy structure of grades and position titles, and a lack of career development programs, and related issues of concern. EG II technical assistance to the electricity sector did not establish any programs devoted to an HR core function. BearingPoint’s assistance to the Ministry in this area is through supporting the current Steering Committee and Sub-Committee structure as a means to effectively coordinate Capacity Building activities, including the HR and Capacity Building Sub-Committee. The primary purpose of establishing a HR and Capacity Building Sub-Committee at the MoE is to provide the basis for developing and coordinating potential future capacity building programs, regardless of the source of those programs. In October 2005, MAT (Ministry Advisory Team) was established to provide effective interface with the Ministries, including the Ministry of Electricity’s Steering Committee. MAT members for Electricity represent IRMO, USAID, PCO, donor agencies, and the US Military. The purpose of MAT is to insure that current and proposed technical projects are executed without conflict and duplication. MAT- Electricity and the Steering Committee need the active participation by BearingPoint and/ or other contractors in order to function effectively. Members of MAT and contractors, such as BearingPoint, need to establish close working relations with the Ministries, and with daily interactions among staff, in order to be effective. Effective technical assistance can’t be provided when it’s isolated from those receiving the assistance. XII. Future/Potential Capacity Building Programs In its presentation to the State Department/IRMO, in December 2005, BearingPoint identified potential future work in the following areas: • Finance – Unbundling accounts of operating Directorates – Moving to international accounting standards – Preparing an Asset Register – Training in accounting and finance • Billing and Collections – Developing a new billing system and revising collection procedures • Warehousing, assets – Developing a new asset management system – Developing new systems for warehousing, logistics, and asset management • Procurement – Developing a new procurement and tendering system – Implementing tender(s) for new generating capacity These programs are designed to improve areas that are in critical need of improvement. In developing and implementing improvements to billings and collection systems, warehousing and asset management, and procurement, a significant component should be the development of effective accounting, audit, and operating procedures to adequately protect the Ministry and its customers from graft and illegal activities that could be occurring. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 30 It is recommended that another technical area of assistance be added to this list: Organization and Staffing. There is the need to review the existing organizational structure of the Directorates, to develop and establish a more efficient and effective operations and management structure. There is a critical need to address staffing issues in a systematic way. “Productivity can not be improved effectively as long as there is over employment.” (Page 26, BearingPoint, Organizational Structure and Staffing Assessment (OSSA I), undated). There is uncontrollable staff growth that must be managed effectively. XIII. Lessons Learned It is important at the beginning of any technical assistance program define consistent realistic goals and objectives. In this case there were really two goals: 1) to provide a long term vision for restructuring the electric sector through new electric statutes, a new regulatory body, and privatization of functions; and 2) to improve core functions of the MoE to improve operations and management. Given the current critical situation in the electric power delivery system in Iraq, the second goal should have received the highest priority, as the only realistic priority for technical assistance to the government. Clearly priorities, goals and objectives were not defined before technical assistance was initiated. A critical mass of resources must be devoted to providing capacity building assistance to the Ministry in order for such assistance to be effective, given the size and complexity of the problems of the power industry in Iraq. In the power sector, there are critical technical assistance (capacity building) needs. Sufficient resources are not available to deal effectively with those needs. Certainly the resources currently available to USAID are insufficient to deal effectively with those needs. BearingPoint has done substantial work in its capacity building technical assistance program at the MoE. Yet most of that work had not resulted in introducing significant changes at the Ministry, largely because the available resources were insufficient for effectiveness, given the enormous complex problems faced by the Ministry. Additionally, the original technical assistance under EG II focused more on long-term issues, like regulation and privatization, than on issues of immediate concern, like poor organization, overstaffing, and lack of effective inventory control and management. There should be an integrated program for broad-based technical assistance to the Ministry, developed, sponsored, and funded, in part by IRMO, with additional support and funding from other US government agencies, the World Bank, the UN, and individual donor countries participating in the rebuilding of Iraq. Until a unified and coordinated approach is formulated and executed, the efforts of individual agencies will be largely ineffective, although there may be, and have been, notable exceptions. It would not be unreasonable for USAID and IRMO to have the leadership role in developing an integrated capacity building assistance program. It goes without saying that the Iraqi government and the Ministry should be active participants in such a program, not passive recipients of any program that is developed. BearingPoint has established, and is supporting, a mechanism to introduce change to the MoE, in a very practical way, by establishing an Iraqi Steering Committee with associated Sub￾Committees that focus on technical areas to which assistance has been and continues to be provided. These sub-committees, with outside capacity building assistance from BearingPoint, or others, could be expanded to address other critical issues and form the coordinating entities United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 31 to introduce change throughout the Ministry. The Steering Committee provides overall direction to the Sub-Committees. The Sub-Committees meet twice a week and the Steering Committee once a month. Key officials of the MoE participate in these committees. They form a positive force for introducing change and should be both encouraged and supported. Currently sub￾committees include: Financial Management, Tariffs, Electric Metering, Procurement, and HR￾Capacity Building. A full time BearingPoint staff, and perhaps an IRMO staff member, representing MAT-Electricity, should be assigned as a mentor to insure this capacity building activity (sub-committee programs) at the Ministry is sustained, strengthened, and expanded. The Ministry and the government need to take much of the responsibility for the critical situation that exists in the electric sector. The Ministry and the government need to make the difficult decisions required to introduce dramatic changes to the way MoE operates; otherwise technical assistance can not prevent further disintegration of service. That is what is happening; electrical service is disintegrating in Iraq. The Ministry and the government should recognize that the Ministry is not a government program to be funded by the government, but functions as a public service provider operated by the government that should be self-supporting and operated effectively and efficiently. In the long run, profits should be generated and the government should receive those profits. Either the Minister has management autonomy, i.e., the ability to make management-decisions, or he does not. If he does not, he needs to obtain or be granted that authority. If he does, then he needs to exercise that authority. He serves, or should serve, as the chief executive officer of the organization and be responsible for effective and efficient governance of its operations. In the last analysis, the electricity crisis is an Iraqi problem that must be solved through Iraqi leadership. XIV. BearingPoint Current Workplans A. Legend B. Financial Management Systems and the CFG C. Tariffs and Cost-of- Service D. Metering Programs E. Procurement Programs United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 32 A. Legend Current workplan Original workplan Previous workplan (ie timing has changed more than once) Time to complete the entire Task Text Denotes text that has changes since previous report Denotes time passed Denotes Task completed BE support role only, MoE has primary responsibility Text Denotes deleted task Delayed 8 Task delayed Deleted 8 Task deleted On Hold 8 Task on hold On time 9 Task ongoing, on time Early 9 Task completed or started early New Task 9 Task added since previous report Done 9 Task completed since previous report SC Steering Committee SC/F Finance Subcommittee SC/M Metering Subcommittee SC/P Procurement Subcommittee SC/T Tariffs Subcommittee MoE Ministry of Electricity CFG Corporate Finance Group EEHC Egyptian Electricity Holding Company BE BearingPoint COT Composite Option for Training United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 33 B. Financial Management Systems and the CFG Progress E No. Area Activities Tasks Sub-Tasks Responsible Completion Workplan Comments/Risks since last Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun E.5.1 Continue training programs for ME staff in the following areas: IAS [TBD] Jun-06 Corporate finance training provided for CFG accounting, IT skills, team work, planning, project management; Included under E.5.2 below E.5.2 Assist the ME with the implementation of a financial management & [TBD] Jun-06 Hassan Kasseba has left the project for personal reasons Management Information IT systems; the initial focus to be on installing three modules (Financial Planning, Cash Management, Loan Tracking); Establish a Corporate Finance Group within the Ministry to improve Barker financial planning & analysis; Implementation plan Draft detailed implementation plan for the Finance work Kasseba W4 Aug 05 Completed W4 Aug Implementation plan approved by Steering Committee MoE W2 Oct 05 Translation submitted. No comments received Implement 3 financial management models Iraqize models [TBD] W1 Dec 05 SC/F claim it will take 1 year to complete. Reviewing options Identify counterparts in MoE Kasseba W1 Oct 05 SC/F doing it with IT assistance Done 9 Go through the models with counterparts [TBD] W4 Nov 05 SC/F taking longer than expected to understand the models Map models to MoE accounting information [TBD] W4 Dec 05 Ongoing. Will occur during training in Egypt Revise models to match MoE information MoE W4 Dec 05 Changes may be fundamental - MoE uses Govt accounting Map models to Iraqi accounting terms [TBD] W4 Dec 05 Iraqization team to travel to Egypt Revise models to match Iraqi accounting terms MoE W4 Dec 05 Will occur during training Populate models [TBD] W4 Feb 06 Delay in Iraqization means this cannot start in time Collect base year data MoE W4 Jan 06 Data collection form submitted to SC/F Initial run and checking of models [TBD] W2 Feb 06 Will occur during train the trainer course in Egypt Models running with base data [TBD] W4 Feb 06 Test in Egypt and on return to Iraq Refine models [TBD] W2 Apr 06 Initial models based on "what we can get" Identify additional data needs [TBD] W1 Mar 06 Transition to corporate style financial reporting Report on transitioning models to IAS [TBD] W3 Mar 06 Implementation is a task for the CFG Comments on the report MoE W1 Apr 06 Final Report submitted [TBD] W2 Apr 06 Maintain models [TBD] W4 Apr 06 A key task of the CFG Develop data collection & reporting system [TBD] W4 Feb 06 Ministry buy in of the system MoE W2 Mar 06 Need to convince other depts to co-operate Establish reporting system within CFG [TBD] W4 Apr 06 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 34 C. Financial Management Systems and the CFG (cont.) Progress E No. Area Activities Tasks Sub-Tasks Responsible Completion Workplan Comments/Risks since last Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Establish Corporate Finance Group (CFG) Decision on organisation structure Kasseba W1 Sep 05 Agreed at 8 September SC meeting Submit Arabic version of options paper Kasseba W2 Aug 05 Completed W2 Aug Subcommittee reviews options paper MoE W3 Aug 05 Completed W2 Aug Decision taken by Steering Committee MoE W1 Sep 05 Drafted MoU for Minister to sign Formally establish Barker W4 Dec 05 Still awaiting a hard copy of the Minister's approval Identify staff for CFG/training Kasseba W1 Oct 05 SC/F prepared list from 1,000 applications Ministerial decree establishing CFG MoE W3 Oct 05 Minister has approved the MoU for all the Finance work Staff transferred to CFG Barker W4 Dec 05 SC/F has selected 60 staff. Delaying formal transfer Specify resource requirements Barker W3 Oct 05 List provided to SC/F Allocate resources to CFG MoE W4 Dec 05 Awaiting formal allocation of staff Delayed 8 Initial CFG Group meeting/workshop Barker W2 Dec 05 rescheduled for 3 March Workplan Barker W3 Oct 05 Draft workplans for each CFG unit Barker W3 Oct 05 Submitted to SC/F Transition Barker W4 Apr 05 Awaiting formal establishment by MoE Staff, activity transition to new CFG Barker W4 Apr 05 Ongoing support Barker W4 Jun 05 To ensure sustainability Provide ongoing support to establish CFG Barker W4 Jun 05 Training in Corporate Finance & 3 Models Preparation [TBD] W1 Nov 05 Agree approach to training MoE W1 Sep 05 Agreed training in Egypt & 3 trainers for sustainability USAID approval for budget USAID W2 Oct 05 Approved Draft syllabus [TBD] W3 Oct 05 Main syllabus agreed, Trainers to be completed by EEHC Delayed 8 Agree package / negotiate contract with Egypt BE Contracts W1 Dec 05 contract signed Prepare training materials EEHC W4 Dec 05 final version expected late February Select "trainee trainers" [TBD] W4 Oct 05 expect to be named late February Implementation [TBD] W3 Apr 06 Delays agreeing with EEHC Training of Trainers EEHC W1 Apr 06 Delays agreeing with EEHC Training CFG staff EEHC W3 Apr 06 Delays agreeing with EEHC Provide budget assistance Specify MoE's requirements MoE W3 Aug 05 Agreed broad outline with SC Agree Workplan Barker W3 Oct 05 Reviewing 2005 Budget Adviser mobilises Barker W4 Sep 05 Arrived W4 Sep 05 Provide assistance with 2006 budget Barker W4 Oct 05 Not required by the MoE - 2006 Budget already completed Review budget processes Barker W2 Nov 05 Draft Manual prepared, being translated Establish new budget processes within CFG Barker W4 Jun 06 SC/F needs time to review new manual United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 35 D. Tariffs and Cost-of- Service Progress E No. Area Activities Tasks Sub-Tasks Responsible Completion Workplan 2006 Comments/Risks since last Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun E.5.4 Provide policy advice & guidance to the ME on financial analysis of the MinistKrause W2 Dec 05 Bob Krause completes his tour W2 Dec. Insufficient budget cost centers, cost of service, inter-company transfers, etc.; to extend Update Cost of Service Study Collect remaining 2004/2005 cost data from MoE Krause, MoE W4 Aug 05 Completed Complete cost of service analysis Krause W3 Sep 05 Completed Develop tariff recommendations (G, T, D) Krause W2 Oct 05 Completed for 2005. Minister now wants for 2006 budget data Develop 2006 tariff recommendations Krause W3 Oct 05 Completed but gaps discovered in the budget data Draft report submitted to MoE Krause W3 Oct 05 Draft 2006 tariff analysis included Comments received MoE W1 Nov 05 MoE provided missing data Delayed 8 Final report submitted to MoE Krause W2 Dec 05 Analysing new data before submitting final report Delayed 8 Tariff recommendations to the Minister Krause W2 Dec 05 Current Government will not increase tariffs On Hold 8 Follow up with MoE to progress new tariffs Krause ? Current Government will not increase tariffs Develop Co-ordinated Subsidy Policy Recalculated cost of fuel subsidy ($6 bn) W2 Oct 05 On Hold 8 Feedback on briefing note submitted 1st August Minister ? Minister passed the paper back to SC without comment On Hold 8 Draft more detailed subsidy options paper Krause ? On hold On Hold 8 Discuss with MoF Krause ? On hold On Hold 8 Discuss with Social Security Krause ? On hold On Hold 8 Discuss with other stakeholders Krause ? On hold On Hold 8 Submit report to MoE Krause ? On hold On Hold 8 Comments from MoE MoE ? On hold On Hold 8 Finalise report including agreed subsidy program Krause ? On hold On Hold 8 Prepare Cabinet paper, other necessary submissions Krause ? On hold On Hold 8 Assist MoE to sell to other parts of Government Krause ? On hold On Hold 8 Electricity subsidy plan agreed Government ? On hold On Hold 8 Prepare Implementation Plan Krause ? On hold On Hold 8 Advise on implementation/set up the subsidy Krause ? On hold Affordability Analysis Household Survey data received Government W4 Sep 05 Data provided (before publication) W4 September Analyse data Krause W2 Oct 05 Completed Draft report on affordability, private generators, other Krause W2 Oct 05 Completed Submit report to MoE Krause W2 Oct 05 Report submitted Comments received MoE W2 Nov 05 Verbal comments received from SC/T Final report Krause W3 Nov 05 Completed Feed into Tariff & Subsidy analyses Krause W2 Nov 05 Completed Cost of Service Study Program One analysis completed, detailed note to follow. Identify gaps, weaknesses in MoE cost data Krause W3 Oct 05 Completed, report submitted Discuss options with MoE Krause W2 Oct 05 Completed early Identify resources available Krause W4 Oct 05 Included in report Draft recommendations Krause W2 Nov 05 Included in report Present recommendations to MoE Krause W2 Nov 05 Comments received MoE W2 Nov 05 Finalise and agree program Krause W2 Dec 05 Early 9 Support MoE in seeking funding for studies Krause W1 Dec 05 Adviser completes his tour W2 December Early 9 Additional Formal and Informal Training Krause W1 Dec 05 Cost of Service to be included in CFG Workshop on 8 Dec Study on Demand Limiting Meters Request by SC - limiting power use leaves more to go around Analysis of customer data Krause W1 Oct 05 Data already collected through other parts of the work Calculate the voltage cut off level Krause W1 Oct 05 To maximise customer coverage with current low generation Identify meter types Matthews W2 Sep 05 Completed Draft Feasibility Study Krause W2 Oct 05 Completed Done 9 Comments from SC Krause W1 Dec 05 No comments received despite follow up Done 9 Final report Krause W1 Dec 05 To be covered in final compilation report United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 36 E. Metering Programs Progress E No. Area Activities Tasks Sub-Tasks Responsible Completion Workplan 2006 Comments/Risks since last Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun E.5.2.1Provide policy advice and guidance to the ME on improving Matthews Jun-06 metering & introducing new meter technology; Meter Population Survey Pilot survey - Rusaffa W4 Jun 06 Risk - Survey not yet established formally as a MoE project PDAs: Delivery to Baghdad Jiminez W3 Sep 05 Delivered on 18 September. Being checked PDAs: Upload application, billing data Jiminez W3 Sep 05 Completed Application: Translate screens Jiminez W3 Sep 05 Completed Application: test Arabic version MoE W2 Oct 05 Translation completed, formatting screens Application: Finalise in Arabic Jiminez W2 Oct 05 Final fixes being done W2 Oct Application: provide latest billing data MoE W4 Aug 05 Completed (Baghdad only) Application: Revise data for uploading Matthews W1 Sep 05 Completed (Baghdad only) Application: Upload data to PDAs for survey Jiminez W3 Oct 05 Awaiting final Arabic version of application Enumerators: Final selection of individuals MoE W4 Aug 05 SC/M selected 50 On time 9 Enumerators: Training in survey techniques MoE W4 Dec 05 Further 26 PDA's deliverd training to follow Enumerators: Training in the application Matthews W3 Oct 05 Completed 24 October 2005 Survey Plan: Draft detailed Plan Matthews W3 Sep 05 Completed and submitted to SC/M Survey Matthews W4 May 06 SC/M committed to start on 27 November. Actual 6 December Compile data Jiminez W1 Jun 06 Awaiting survey Update MoE billing files MoE W2 Jun 06 Awaiting survey Draft Priority Metering Action Plan Matthews W2 Jun 06 Which ones repaired first, what technology Comments received MoE W3 Jun 06 Finalise report Matthews W4 Jun 06 Pilot survey - Karkh W4 Jul 06 Risk - Survey not yet established formally as a MoE project PDAs: Upload application, billing data Jiminez W3 Sep 05 Completed Application: provide latest billing data MoE W1 Sep 05 Provided with Rusafa Application: Revise data for uploading Matthews W3 Sep 05 Completed early with Rusafa Application: Upload data to PDAs for survey Jiminez W3 Oct 05 Same time as Rusafa Delayed 8 Enumerators: Final selection of individuals MoE W2 Dec 05 SC/M to select karkh enumerators Delayed 8 Enumerators: Training in survey techniques MoE W2 Feb 06 Still awaiting selection of trainees Enumerators: Training in the application Matthews W3 Oct 05 MoE trainers trained. PDAs included in general survey training Survey Plan: Draft detailed Plan Matthews W3 Sep 05 Completed and submitted to SC/M Survey Matthews W4 Jun 06 Awaiting completion of 1st batch of training to start Compile data Jiminez W1 Jul 06 Awaiting survey Update MoE billing files MoE W2 Jul 06 Awaiting survey Draft Priority Metering Action Plan Matthews W2 Jul 06 Comments received MoE W3 Jul 06 Finalise report Matthews W4 Jul 06 Extend Survey to next districts 2008 To wait until Rusafa and Karkh are proceeding smoothly Select next districts to be surveyed MoE W1 Feb 06 SC/P wants to include Salahaddin & Basrah Target districts agree to survey MoE W2 Feb 06 Identify enumerators MoE W1 Mar 06 Obtain billing data MoE W1 Oct 05 SC/M provided data for all Iraq. Cleaning it up Prepare billing data for uploading Matthews W4 Jan 06 Upload data to PDAs Jiminez W1 Feb 06 Prepare Survey Plan Matthews W2 Feb 06 Train enumerators MoE W4 Feb 06 Survey MoE W1 Jul 06 Establish new meter technology Awaiting confirmation that SC/M has made arrangements Ministry approves draft strategy Minister W4 Oct 05 Unlikely to hear from Minister. Going ahead anyway Study Tour of meter manufacturers MoE W4 Jan 06 Abdul Amir reduced time and scope planned trip in March Finalise tour dates Matthews W1 Nov 05 SC agreed January to cover Egypt and Algeria as recommended Obtain USAID/Ministry approvals Matthews W4 Dec 05 Still awaiting a new approval from the Minister Draft Action Plan for introducing new technology Matthews W2 Feb 06 Comments received on draft MoE W3 Feb 06 Finalise report Matthews W4 Feb 06 Feed into Meter Survey Action Plans Matthews W2 Apr 06 Delayed to match timing of Survey United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 37 F. Procurement Programs E No. Area Activities Tasks Sub-Tasks Responsible Completion Workplan Comments/Risks Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun E.2.2 Development of a skill set & training in international contracting & Fijalkowski Jan-06 procurement. Provide training & advice to the Kurdish Ministry of Industry & to a new procurement organization in Kirkuk on procurement & tendering best practices Procurement training for the MoE Janusz Fijalkowski left early. MoE has requested a 3rd course Prepare training materials Fijalkowski W1 Aug 05 Completely redone (see below) Confirm dates of courses MoE W1 Oct 05 Completed. SC/F agreed new dates 1st Course: Private Sector Participation Kupisz W3 Nov 05 SC/P has requested more courses for 2006 Identify trainees MoE W2 Sep 05 Completed Arrange MoE Training Centre Fijalkowski W1 Aug 05 Completed Prepare new material for second course Kupisz W2 Oct 05 Completed Deliver training Kupisz W3 Nov 05 2 courses, 15 people each. 2nd course 13-16 November 9 2nd Course: International Tendering & Procurement Kupisz W4 Nov 05 SC/P has requested more courses for 2006 Identify trainees MoE W3 Aug 05 Completed Arrange MoE Training Centre Fijalkowski W1 Aug 05 MoE to rearrange when new date agreed Prepare material on contracts Lockhart W4 Aug 05 David Lockhart delivering a day on legal/contract issues Prepare new material for second course Kupisz W3 Oct 05 Completed 9 Deliver training Kupisz W4 Nov 05 2 courses, 30 people each. 2nd course 20-23 November Other procurement training requested by USAID W4 Jan 06 Awaiting confirmation of dates by USAID Prepare Training Material Kupisz W1 Nov 05 Same as MoE course but squeezed in to 2 days Deliver training to PCC in Kirkuk Kupisz W4 Jan 06 2 day course, based on MoE course 2 Deliver training to Ministry of Industry in Suleymaniyah Kupisz W4 Jan 06 2 day course, based on MoE course 2, 30 trainees Deliver training to Ministry of Industry in Erbil Kupisz W4 Jan 06 2x2 day course, based on MoE course 2, 70 trainees E.2.6 Assist the ME in issuing a management contract to the private sector Kupisz May-06 Minister agreed on 1st August. Previous attempt unsuccessful to take over billing and collections in one or more districts Advisory role only, no implementation Pilot districts: Salahaddin & Basra Provided indicative timetable and financial model to SC/P 9 Due diligence MoE W4 Nov 05 BE input limited to advice on implementation Submit Issues Paper Kupisz W2 Aug 05 Issues, options and recommendations. Agree scope & objectives of the tender MoE W2 Oct 05 Arabic version submitted to SC/P W1 Oct 9 Collect information MoE W4 Nov 05 Met Salahaddin, Basrah Agree structure of tender MoE W1 Nov 05 Tender process discussed during Salahaddin meeting Marketing MoE W2 Jan 06 Need approval from Abdul Ameer 9 Draft Advertisement MoE W4 Nov 05 Drafts completed, awaiting confirmation from AA 8 Advertise MoE W1 Jan 06 Scheduled for 4 January 8 Draft Request for Proposals MoE W2 Dec 05 SC/P completing draft 8 Expressions of interest submitted Bidders W2 Feb 06 Scheduled for 8 February 8 Agree shortlisted bidders MoE W3 Feb 06 Await expressions of interest 8 Transaction Documents MoE W2 Feb 06 Former DG of Legal will be working with SC/P 8 Draft Information Memorandum MoE W2 Feb 06 Outline submitted to the SC/P. Scheduled for mid-Feb 8 Draft Tender Rules Kupisz W2 Feb 06 1st draft submitted. Scheduled for mid-Feb 8 Draft Management Contract MoE Legal W2 Feb 06 Contents page provided - SC/P has not started Draft Confidentiality Agreement MoE Legal W3 Nov 05 We provided draft (in Arabic) to SC/P. They need to review Comments on drafts MoE W1 Dec 05 Being done informally by SC/P as they go Finalise MoE W2 Dec 05 9 Bid MoE W2 Apr 06 Timetable drafted. Less due diligence time allowed 8 Issue Confidentiality Agreement to shortlist MoE W3 Feb 06 After shortlist selected 8 Sign Confidentiality Agreement Bidders W4 Feb 06 Scheduled for 22 February 8 Issue draft Transaction Documents MoE W1 Mar 06 Scheduled for 1 March 8 Sign Tender Rules Bidders W2 Mar 06 Not allowed to participate unless they agree to Tender Rules 9 Bidder due diligence Bidders W1 Apr 06 Starts when Tender Rules signed 8 Submit comments on draft Management Contract Bidders W3 Mar 06 Issue final Management Contract MoE W4 Mar 06 Finalise during bid to minimise negotiations 9 Bids submitted Bidders W2 Apr 06 Scheduled for 12 April 9 Evaluation & Closing MoE W2 Jun 06 9 Evaluate bids MoE W4 Apr 06 9 Agree winning bidder(s) MoE W4 Apr 06 Scheduled for 26 April 9 Final negotiation & Closing MoE W4 May 06 If contract agreed before bid submitted. Scheduled for 26 May 9 Hand over MoE W2 Jun 06 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 38 Appendix 3: USAID-Funded Economic Governance-II Capacity Building Programs Ministry of Finance and Ministry of Planning Detailed Report United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 39 UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT MONITORING AND EVALUATION PERFORMANCE PROGRAM PHASE-II (MEPP-II) FINAL DETAILED EVALUATION REPORT USAID-FUNDED ECONOMIC GOVERNANCE-II CAPACITY BUILDING PROGRAMS MINISTRY OF FINANCE AND MINISTRY OF PLANNING Submitted to: USAID/IRAQ Baghdad, Iraq Submitted by: Harold Lippman International Business & Technical Consultants, Inc. 8614 Westwood Center Drive • Suite 400 • Vienna, VA 22182 • USA Telephone: 703-749-0100 • Facsimile: 703-749-0110 May 1, 2006 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 40 Appendix 3: Economic Governance Programs Detailed Evaluation Report – Ministry of Finance and Ministry of Planning Development Table of Contents I. Background …………………………………………………………………… … 41 II. Introduction …………………………………………………………………… … 41 III. Iraq Financial Management Information System …………………………… 42 A. Accomplishments ……………………………………………………………………. 42 B. Site Visit ……………………………………………………………………………… 43 C. Training ……………………………………………………………………………… 43 D. Users Meeting ……………………………………………………………………………… 43 • Recommendation ……………………………………………………………………. 44 • Lessons Learned ……………………………………………………………………. 44 E. Challenges/Problems ……………………………………………………………………. 45 • Recommendation ……………………………………………………………………. 45 • Lessons Learned ……………………………………………………………………. 46 IV. Central Bank of Iraq – Bank Supervision Training Program …………. 46 A. The Training Program …………………………………………………………………….. 46 B. The Challenge of Underlying Cultural Behavior and Attitudes …………………….. 47 • Recommendation ………………………………………………………………..…… 47 • Lessons Learned ………………………………………………………………..…… 48 V. Social Safety Net ………………………………………………………………. 48 A. Accomplishments ……………………………………………………………………… 48 B. Defining Capacity Building ……………………………………………………………. 49 • Recommendation ……………………………………………………………………… 50 • Lessons Learned ……………………………………………………………………… 50 VI. Ministry of Planning and Development Cooperation …………………… 50 A. Accomplishments ……………………………………………………………………… 51 VII. Conclusion ………………………………………………………………………... 52 Annex 1: FMIS Interviews ………………………………………………………………... 53 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 41 ECONOMIC GOVERNANCE PROGRAMS DETAILED EVALUATION REPORT - MINISTRY OF FINANCE (MoF) AND MINISTRY OF PLANNING DEVELOPMENT I. BACKGROUND In July 2003, USAID launched an effort to build the capacity of the Iraqi Government to manage the transition from a command to a market-driven economy. That effort, Economic Governance I (EG I), was implemented by BearingPoint (BE) and ended in 2004. It helped Iraqis develop policies, laws, and institutions that better regulate trade, commerce, and investment, while also providing support to ministries, the Central Bank, and other government organizations that are key parts of the economy. Building on the reforms and other accomplishments brought about by EG I, in September 2004 USAID put into place the Economic Governance II (EG II) project to help Iraq establish a policy￾enabling environment that fosters private sector led growth. Also implemented by BearingPoint, EG II’s overall objective is: To assist the Government in structuring and quickly implementing an operational legal, fiscal, regulatory & institutional framework within which policy can be formulated to foster a transparent and investment-friendly, market-based economy. The three to five year project has been providing technical assistance to all Iraqi line ministries and 32 other government bodies. EG’s main program objectives are to: 1) rebuild a governmental system that is efficient, transparent, and corruption free; 2) introduce fiscal balance and monetary stability, discipline, and sustainability; 3) put into place a viable, well-regulated banking system that will facilitate investment, monetary stability, and the development of the economy; 4) implement a sound, transparent, and predictable commercial legal framework that fosters the rule of law and promotes confidence in the system; 5) introduce viable, functioning electrical, telecommunications, and government-wide IT management systems; 6) introduce new pension and social safety net systems that will protect the most vulnerable individuals and allow the government to gradually eliminate state subsidies for food, energy, and electricity; 7) put into place clearer and more structured procedures to facilitate regulation of businesses and individuals dealing with the government; and 8) promote the participation of NGOs in policy formulation As of December 31, 2005, total expenditures incurred in EG II amounted to $63 million. II. INTRODUCTION Given the enormity of the task called for in the original scope of work, i.e., evaluating the programs financed by USAID/Iraq in the Ministries of Finance and Planning, the United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 42 Mission’s Economic Governance team suggested that the IBTCI evaluation team narrow the focus of its work to four areas of BearingPoint capacity building activity supported by EG II: 1) the development of the Iraq Financial Management Information System (Ministry of Finance); 2) the Bank Supervision Training Program (Central Bank of Iraq); 3) the establishment of a Social Safety Net Program(Ministry of Finance and Ministry of Labor and Social Affairs); and, 4) technical assistance provided to the Ministry of Planning. III. IRAQ FINANCIAL MANAGEMENT INFORMATION SYSTEM The Iraq Financial Management Information System (IFMIS) is an integrated, automated, and internet-based information system that will replace the Government of Iraq’s outdated manual accounting system. The overall goal is to give the Ministry of Finance the ability to: • produce timely, accurate, and comprehensive financial reports; • manage cash expenditures across spending agencies; and, • establish modern controls and management of cash resources, accounting processes, and budget execution in accordance with Iraqi law and international standards. IFMIS is a key requirement of the December 2005 Stand-By Agreement with the International Monetary Fund, which when fully implemented will allow Iraq to reduce its commercial debt burden from $14 billion to $3 billion. Work on IFMIS began in 2003 under EG I and has continued under EG II. The project was organized into two phases aimed at establishing 182 IFMIS sites across the country and in ministries and other “spending” agencies in Baghdad. In Phase I, which ended in August 2005, 54 sites were “fully installed;” i.e., personnel trained, equipment purchased and operating, internet connection functional, and data input transmitted directly into the central database at the Ministry of Finance. Under Phase II, which began in September 2005, the remaining 128 sites are in various stages of development. It is anticipated that all 182 sites will be fully installed by June 2006. IFMIS has been in test and pilot mode since early 2005 and is projected to be fully operational in time to perform basic budget execution/accounting functions for fiscal year 2007 (as required by the IMF Stand-By Agreement). A. Accomplishments Based on source documents reviewed, extensive interviews with BearingPoint advisors and senior Ministry of Finance officials, and on-site observation, IFMIS can clearly be regarded as a success thus far. According to a January 2006 BearingPoint document, key accomplishments include: • completing the necessary infrastructure to support the new system, including refurbishing the Ministry of Finance main computer and data center, procuring and installing the hardware; and developing an internet virtual private network (VPN) to connect the initial 54 Phase I sites; United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 43 • installing, testing, and commencing live operations of IFMIS software in Phase I sites that represent more than 80 percent of Iraq’s federal budget expenditures; • completing an initial draft of a national chart of accounts, which complies with international budget accounting and financial reporting principles, required to be ready for use in the 2007 budget year; and • completing training of more than 250 IFMIS data entry and IT staff. B. Site Visit In addition to these reported accomplishments, the IBTCI evaluation team member responsible for this activity area visited the IFMIS data center to see first-hand how it was equipped and functioned and to talk with the Director General, IFMIS Implementation Coordinator, and staff. The two-room data center is equipped with some 20 personal computers, several printers, and other requisite equipment. The computers are used for operational and training purposes. A handful was in use at the time of the visit and all of the rest appeared to be in good working order. The rooms housing the computers and printers are equipped with large air conditioners with their own alternative power sources. “Help desk” staff described their functions and how they responded to questions from users in the field. They said one of the most common problems they encounter is users – either because of continuing difficulty with basic computer procedures, gaps in power, and/or other factors – failing to open e-mails containing important information in a timely manner. C. Training Because training was one of the most important tasks in the development of the IFMIS, an IBTCI Field Monitor interviewed ten of the 200 individuals who participated in BearingPoint supported Phase I basic computer and data entry courses and workshops. The sample was drawn from a list of participants provided by the IFMIS Implementation Specialist and the structured interviews were conducted with the use of a list of questions (see Annex 1) that were posed sequentially. The interview write-ups prepared by the Field Monitor show that the training activities were well￾organized and effective. Interviewees said they had learned what they needed to carry out their IFMIS functions and had acquired knowledge and skills – especially, computer basics – they had not possessed previously. They pointed out that handouts were provided and that these materials were used frequently once they were back in their work settings. They also emphasized that IFMIS trainers followed-up with them after the training sessions and help desk staff were readily available to answer questions once they returned to their work environments. They all reported there were no problems – e.g., instructors not showing up, training materials being unavailable, sessions failing to start and/or end on time – with the training and gave high ratings on the results in evaluation forms filled out at the end of the workshops. D. Users Meeting Among the two or three most striking situations encountered during this evaluation was the March 7, 2006 IFMIS “users” meeting at the Ministry of Finance. The conference room was filled to overflowing with some 150 attendees (more than 90 of whom were women) from IFMIS activity sites in Baghdad and throughout the country. The meeting provided an opportunity for users (virtually all of whom had undergone IFMIS training) to interact with IFMIS management in United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 44 a face-to-face setting. The Deputy Minister of Finance, joined by three Director Generals, the BearingPoint senior advisor, and the IFMIS Implementation Coordinator chaired the non-stop, two-hour question-and-answer session. The dozens of questions – the majority of which were posed by women – were almost entirely focused on operational problems and issues, such as the lack of electricity, difficulty accessing the system, and confusion over data entry requirements. IFMIS leaders seated on a raised platform at the front of the room responded to all of the questions, as the audience listened intently. While the central purpose of the meeting was clearly realized, something more important was also taking place: capacity building efforts were giving rise to a broader good governance outcome well beyond their stated goals. In the first place, the simple fact that such a meeting (the second, preceded by a similar one in November 2005) was occurring was itself noteworthy. Such an undertaking, conducted as it was in an open, participatory manner, would certainly have been an aberration under the previous regime. Secondly, the opportunity for face-to-face interaction provided the context for an extraordinary, spontaneous dialogue. Prompted by a series of questions calling for the Ministry of Finance to provide additional resources for IFMIS data centers, one of the Director Generals took the podium and proceeded to deliver a minutes-long, hard-edged tutorial on good governance and political reform. He emphatically stated that only the Iraqi National Assembly can provide such increased budget resources, concluding “we are not like the previous regime that could increase any ministry’s budget by decree.” He scolded those present (and others who were not there) for receiving funds to go abroad to conferences and training programs and using those occasions as personal junkets. He also made the point that “incompetent” staff in his own and other ministries are causing harm and stressed the need for IFMIS users to “operate properly” because of Iraq’s responsibilities to international organizations like the IMF. • Recommendation While one meeting certainly does not qualify as a sea change that it happened and is likely to be happening in other venues suggests the thought that the significant impacts it demonstrates should be captured and put to further use. BearingPoint and/or USAID should consider developing ways to promote and systematically track such meetings and related undertakings (e.g., regular meetings of Steering Committee that have been established in a number of ministries). A local Iraqi, for example, could be assigned the task of observing and writing up summaries of such occurrences on a regular basis. • Lessons Learned Significant accomplishments resulting from USAID-inspired capacity building activities are occurring in the settings where the interventions are taking place. For example, in several ministries Steering Committees and Sub-Committees have been established to coordinate capacity building and other assistance efforts. USAID and its implementing partners should anticipate and plan for such occurrences, which go beyond the specific results desired from a given capacity building activity to the broader context of political reform and good governance. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 45 E. Challenges/Problems Notwithstanding the positive results achieved in establishing the IFMIS, numerous challenges pose a threat to its continuing development and sustainability. BearingPoint documents cite the following examples: • poor security and infrastructure hamper communication, access to sites, and automation efforts; • difficulties with internet connectivity and electricity shortages; • the technical, professional, and managerial capacity of Ministry of Finance staff is limited and these factors slow the implementation process tremendously; • very low level of computer literacy, combined with limited exposure to modern practices and techniques in all agencies, inhibit timely implementation; • mid-level managers are resistant to change and must take ownership of IFMIS implementation; and • lack of automated banking, payroll and debt management systems limit the extent of IFMIS reforms within the Government of Iraq. In addition to these formidable challenges, IBTCI learned that significant problems had occurred in the selection of individuals to participate in IFMIS training. In the initial Phase I sessions that involved some 200 participants, a knowledgeable BearingPoint advisor estimated that about 75 of those selected were managers, supervisors, or others who were not going to actually use the knowledge and skills acquired from the training to enter the data into the IFMIS. According to BearingPoint sources, these abuses reflect several factors: • bureaucratic cultural attitudes – e.g., managers/supervisors believe they must know how to do everything their staff does in order to maintain the latter’s respect; • the desire to take the advantage of an opportunity to travel outside Iraq after decades of restrictions on being able to do so; and, • sizeable per diem from BearingPoint, in addition to one from the Iraqi government. The result of this abuse means that additional training – with all that this implies in terms of added time and cost – had to be done to provide the appropriate people with the necessary knowledge and skills. BearingPoint advisors further pointed out that this problem has not been confined solely to IFMIS; it has occurred elsewhere in the Ministry of Finance and other ministries, as well. A BearingPoint advisor explained that a deputy minister had recently scrubbed an impending training activity after learning that one of the proposed participants was a security guard related to another high ranking official. • Recommendation Training coordinators in ministries and BearingPoint should closely monitor the participant selection process and USAID/Iraq should assign oversight of this matter to a specific individual within the Mission. Consideration should also be given to having “overview” training courses for supervisors and managers, as well as conducting training activities in parts of Iraq, such as the North, which are safe and less costly than sending participants outside the country. Moreover, USAID and its implementing partners will need to exert continuous pressure on their Iraqi United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 46 government counterparts to see that ongoing attention is being paid to reducing abuses in this area. • Lessons Learned In designing capacity building activities that include training elements, potential abuses in selection of participants should be anticipated. Criteria and procedures to minimize such possibilities should be written into project documents and be in place before the activities begin. IV. CENTRAL BANK OF IRAQ – BANK SUPERVISION TRAINING PROGRAM Under EG II, BearingPoint has continued efforts initiated in EG I to assist the Central Bank of Iraq (CBI) in becoming capable of executing monetary policy, maintaining price stability, and ensuring the safety and soundness of the country’s banking sector. As operated under the previous regime, CBI was the money-printing arm of the government. It was not independent and had no enforcement powers or ability to carry out monetary policy. A key feature of these BearingPoint efforts has aimed at strengthening the supervisory role of the CBI by building the capacity of its Bank Supervision Department (BSD). The BSD was created in 1994 following the establishment of private banks in the previous year. All banks, whether private or state-owned, are subject to regular supervision by the BSD. The BSD has five divisions: inspection, for on-site examination work; audit, for off-site monitoring; research and study, for research-related issues; risk, for collecting credit information on loans; and, administration, for general administrative work. BSD has about 70 staff based in Baghdad and 10 each in supervision offices in Basra and Mosul. A. The Training Program As of January 2006, BearingPoint completed a two-year training program for the BSD designed to strengthen banking operations by developing an effective supervisory role within the CBI. In coordination with the International Monetary Fund (IMF) and U.S. Federal Reserve System (FRS), BearingPoint organized an ongoing series of seminars, classroom courses, and on-the￾job training activities to introduce modern, risk-based supervisory techniques to BSD personnel. The seminars, courses, and on-the-job training have covered diverse topics, including: • The Basle Committee’s 25 Core Principles for Effective Banking Supervision; • International Accounting and Financial Reporting Standards; • Credit Risk Management; • Asset/Liability and Liquidity Management; and • Bank Financial Analysis The effects of these training activities have been significant, as confirmed in interviews at CBI with the Deputy Governor, Director General of the Banking Supervision Department, and four (two men, two women) former participants. The Deputy Governor said the training provided CBI with knowledge that has helped make its staff more reliable and credible in dealings with neighboring country financial institutions. The Director General likewise mentioned the positive effects of learning about international standards and concepts, but also pointed out there have been difficulties in applying that knowledge to Iraq’s special circumstances. The four former United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 47 participants, who had attended one or more sessions in Bahrain, Cairo, Amman, and Istanbul, also expressed some concern with relating what they had learned to the Iraqi environment, but emphasized that they were much enriched by the courses. One of women said the training has helped her feel “much more professional,” a sentiment shared by the other, who nodded in agreement. In addition to helping to solidify everyday banking supervision operations and enabling staff to become familiar with and start to use international standards and best practices, the training program has prompted BSD to develop and conduct its own training for staff on a regular basis. For example, BSD has developed and taught seminars on the Basle Core Principles, accounting, and credit risk to CBI staff as well as private banks. BSD has also developed, distributed, and put into practice its own credit risk guidelines for the country’s commercial banks. Most recently, work has been underway to strengthen its examination manuals by incorporating the information provided over the course of the ongoing training effort. Also notable is the training program’s practice of selecting participants for future courses on the basis of what they had learned in prior sessions. Such was the case in the recent IMF/FRS Bank Financial Analysis Seminar in Amman. Participants were selected from among the program’s advanced students based on their demonstrated knowledge of key concepts, terminology, practices and procedures. B. The Challenge of Underlying Cultural Behavior and Attitudes In startling contrast to the notable success achieved in training CBI officials in the Banking Supervision Department, these same officials and their staff have failed to use 20 desktop computers they received more than two years ago. According to the senior BearingPoint advisor who shared this anecdote, “the desktop computers are in a separate room and are rarely, if ever, used for their work.” An Iraqi-American consultant serving as a BearingPoint senior advisor affirmed this point, asserting that 99 percent of managers, supervisors, and other “bosses” do not use computers in their work. They look on computers as something “beneath them,” to be used solely to replace a typewriter. The culture of business and government, he added, is mired in reliance on hand￾written documents that goes back hundreds of years to the Ottoman period. Concepts of productivity and efficiency, he added, are not understood. BearingPoint/USAID capacity building efforts, he concluded, do not address these and other critically important underlying characteristics of Iraqi culture. • Recommendation According to the Iraqi-American advisor, if USAID and other donors do not address these key underlying cultural attitudes, “when the present pressure for change goes away, the black hole will simply close up again,” as it has so many times over the years. Accordingly, in its capacity building efforts USAID and its implementing partners must develop and better incorporate activities that go beyond the specifics of training and address the fundamental issues of cultural transformation. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 48 • Lessons Learned Capacity building program planning and design should include elements that address reform of underlying cultural attitudes and behaviors at the same time the technical knowledge and skills are being imparted. Failure to do so increases the likelihood of partial success and/ or failure in the short term and substantially diminished prospects for long-term sustainability. V. SOCIAL SAFETY NET Under EG II, BearingPoint worked with the Ministry of Finance and Ministry of Labor and Social Affairs (MoLSA) over the latter part of last year to develop and implement a new Social Safety Net (SSN) program to reduce poverty and mitigate the impact of major economic reforms on poor and vulnerable segments of the population. With 500 billion Iraqi dinars (approximately $340 million) included in the government’s 2006 budget, SSN implementation began in December 2005 with an initial target of lifting some 850,000 Iraqi families out of poverty by the end of this year. In contrast to food, fuel, and other subsidies provided by the previous regime, the SSN is a means-tested program comprised of financial and non-financial benefits and services to low income families, the disabled and the unemployed. It is estimated that nearly 25 percent of Iraq’s population live below the $1 per day individual poverty threshold and thus are eligible for SSN benefits. For unemployed Iraqis, SSN provides vocational and entrepreneurial training, career guidance, and help with job interview preparation to help them get back into the labor market. As of the end of January 2006, over 79,000 unemployed workers had registered with the Employment Centers operated by the Ministry of Finance, following a series of public announcements regarding the Social Safety Net program. Of that number, some 73,000 unemployed individuals were completely new registrants. With unemployment estimated at 40 percent, this dramatic surge in registrants is most encouraging. A. Accomplishments Based on a review of available source documents – most notably, monthly reports – and interviews with advisors, BearingPoint has clearly played a pivotal role in the SSN’s successful development and early implementation. The following monthly report excerpts are illustrative: • Project advisors supplied a social protection bulletin that included unemployment statistics. A template tracking matrix was supplied, and statistics have been requested at the Governorate and national level, on a regular basis to monitor the growth in registrations of low income families, unemployed and disabled. (January 2006) • Discussed an Emergency Warning Report produced for MOLSA. This covered the potential areas of weaknesses in the administration system….The Project team and MOLSA examined ways abuses could be deterred, minimized or detected. The advisor requested information on staffing numbers by Department and task in order to assess the capacity in processing new claimants and general administration. (January 2006) United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 49 • The Social Safety Net Law, prepared by USAID Project advisors, was read two times successfully in the Transitional National Assembly plenary session, and is now awaiting final vote. (December 2005). • Prepared the first draft of Instructions and Standard Forms for a smooth transition from the existing system to facilitate a larger coverage of social protection in Iraq. The Instructions and Standard Forms reflect the changes included in the amended Social Welfare law. (November 2005) • Delivered the first draft of the strategy for a public education campaign to raise awareness among poor families of the Social Safety Net. The campaign will also provide information regarding the legal provision and the administrative procedures of the new Social Safety Net system. (November 2005) • It was agreed with MoLSA that the Project will work closely with MoF and Ministry of Labor and Social Affairs (MoLSA) by: o providing assistance in drafting amendments to current law to provide the legal basis for this initiative; o undertaking necessary reforms for the registration of beneficiaries, to begin on 01 December 2005; o working with other donors, including the World Bank, to implement an IT system to accommodate the registration process; o coordinating with the GoI in a public outreach program to assist the public in accessing this program and its benefits; o building capacity of officials to structure and to administer the program, with training and development of procedures; o assisting the GoI in setting up a poverty monitoring system, based on COSIT surveys and analyses; and o regularly provide MoLSA with the necessary input for timely adjustment of the social safety net for the benefit of the poor and the vulnerable. (October 2005) While fully acknowledging the accomplishments referenced in the above excerpts, from the standpoint of this evaluation’s objective of assessing BearingPoint’s contribution to capacity building efforts, this record is less clear. Throughout the excerpts, in phrases such as, “project advisors supplied” and “prepared the first draft,” there is a definite sense of the advisors providing technical assistance to help get something done rather than build capacity. In an interview with a key BearingPoint advisor, the advisor maintained that all meetings with Iraqi counterparts, in effect, involved training because they were interactive and counterparts often “took ownership” of the discussion and/or ultimate product. However, efforts to arrange interviews with counterparts and/or beneficiaries (such as participants in training provided to MoLSA staff) were unsuccessful. B. Defining Capacity Building This discussion ties directly into a basic question that arose at the very inception of this evaluation and reappeared continuously throughout it: what is meant by the term capacity building? BearingPoint, and others for that matter, seem to confuse tools, such as courses, seminars, workshops, and on-the-job training that provide new knowledge and skills or improve on existing knowledge and skills with technical assistance that helps get something done. The latter, most typically, does not build capacity; the former virtually always does. In the excerpts above, for example, supplying a social protection bulletin or delivering the first draft of a strategy United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 50 does not build capacity. The interchangeable use of these two definitional elements causes confusion in identifying and reporting accomplishments and capturing the real effects of an intervention. • Recommendation USAID should consider establishing separate categories for these definitional elements: one for capacity building that reflects new or improved knowledge and skills and the other for functional technical assistance provided by advisors and other experts. • Lessons Learned An agreed definition of capacity building should be part of all project documents, starting with RFPs and continuing on through contracts, SOWs, workplans, and other essential paperwork. Without a clear definition in place from the start, confusion is inevitable, with attendant consequences, for example, in being able to accurately and consistently describe what has been accomplished. VI. MINISTRY OF PLANNING AND DEVELOPMENT COOPERATION The Mission Economic Growth Team requested that IBTCI look into BearingPoint efforts to provide advisory services requested by the Minister of Planning and Development Cooperation (MoPDC) for the period June – December 2005. Two BearingPoint advisors – one for legal matters, the other for economic affairs – were assigned to this task, but only one of them was available at the time of the evaluation. The terms of reference for that individual’s position, senior legal advisor, stipulate that he was to advise and assist the Minister, upon request, in matters such as: • Commercial and trade law and policy; • Foreign investment; • Public procurement regulation; • Multilateral and bilateral financial organizations; • International donor and donors’ trust funds; and • Ad hoc matters assigned directly by the Minister. Specific tasks called for in the terms of reference included advising on and/or assisting with: • The National Development Strategy and related submissions to the international donor community; • Ministry of Planning institutional strengthening and training in public procurement regulation; • Commercial and trade law and policy formulation and implementation; • Foreign investment legislative and regulatory development; and • Liaison with other government ministries, institutions, and entities. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 51 A. Accomplishments Based on a review of monthly reports and discussions with the senior legal advisor, it is clear he and his economic advisor counterpart made significant contributions to the Minister of Planning, as indicated in the following monthly report excerpts: • At the request of the MoPDC, BearingPoint Legal and Economic Advisors joined the Iraqi delegation to the ministerial level International Conference on Iraq, held in Brussels, Belgium, 21-22 June 2005. Jointly with Iraqi Ministry officials, and in direct consultation with donor representatives in Baghdad, the Advisors assisted MOPDC on all matters involving its presentations on economic recovery and reconstruction, both before and throughout the Conference. The Advisors’ technical assistance enabled MoPDC to prepare timely and in proper form the following for the Conference: (i) Iraq’s Strategic Vision for National Development; (ii) Minister’s speech; (iii) Briefs on donors’ feedback to the Speeches and Vision Statement; and, (iv) PowerPoint presentations. (June 2005) • The Advisors assisted in the preparation, review, and completion of the following: o Iraq’s National Development Strategy; o Workshop PowerPoint presentation on MoPDC’s Donor Coordination Unit and Donor Assistance Database; o Minister’s keynote speech; o IRFF I Donor Meetings’ agenda, official statements, communiqués; o Position Papers on the World Bank Iraq Trust Fund (WB-ITF) and the United Nations Development Group Trust Fund (UNDG-ITF); and o Various briefings and memoranda to the Minister. (July 2005) • The Advisors participated in the preparation of background briefings and analytical materials for the MoPDC delegation that attended the World-Bank IMF Annual Meetings in Washington, DC. At the request of the Minister, they reviewed, advised on, and revised several proposed draft MoPDC Letters of Intent, Letters of Agreement and Memoranda of Understanding. They also participated in several Baghdad Donor Core Group and Sector Working Group meetings (including on rule of law, education, health, and official briefings to senior donor representatives). (September 2005) However, as with the Social Safety Net, and perhaps even more so in this case, there is little evidence that meaningful capacity building called for in EG II took place. Similar to the SSN advisor, the senior MoPDC legal advisor believes capacity building did take place in his work. He cited the example of his modeling important behavior and attitudes that are not part of the Iraqi bureaucratic culture, such as arriving on time, working a regular schedule throughout the day, and taking sick leave. He also pointed out that the Minister’s senior professional and support staff with whom he worked learned from observing him doing his job; for example, how to develop an agenda, what goes in a memo, how to bullet points in a document, and the differences between research, legislative, and action memos. On the other hand, he explained, much of what he did for the Minister utilized knowledge and skills people at the Ministry did not have and the priority was to get things done, often under tight timeframes. The setting and the requirements of his job in many respects were not United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 52 conducive, for example, to on-the-job training, mentoring, or other capacity building tools. In many situations, he acknowledged, in responding to the Minister’s requests as called for in his terms of reference. Over time he crossed the fine line between serving as an advisor and being a MoPDC employee. This blurring of his role, he added, created difficulties with BearingPoint management and USAID – a situation he said was further complicated by related problems with other BearingPoint colleagues. VII. CONCLUSION The results of IBTCI’s assessment of BearingPoint’s EG II efforts focused on the Ministries of Finance, Planning and Development Cooperation, and Labor and Social Affairs present a mixed picture of accomplishments and questions on just how much capacity building had occurred. Significant accomplishments were achieved in the training and technical assistance provided to IFMIS users and Central Bank of Iraq Supervision Department officials and staff. Moreover, in at least one instance – the extraordinary IFMIS users meeting at the Ministry of Finance – an outcome far beyond the anticipated results was achieved when principles of good governance and political reform surfaced in the question-and-answer session. However, in the cases of the Social Safety Net and Ministry of Planning, while positive results were realized from BearingPoint efforts, it is difficult to say to what degree those accomplishments were a reflection of capacity building. In many respects, the BearingPoint advisers working in these areas focused on seeing to it that things got done, even if that meant doing it themselves. Much of what they did utilized knowledge and skills that staff at the concerned ministries did not possess and the pressure on them to get things done, often under tight timeframes, limited the opportunities to do on-the-job training, mentoring, or other capacity building activities. Finally, the evaluation also surfaced a number of important issues that need to be addressed in the course of developing future capacity building programs. First, is the absence of elements of capacity building that address critically important underlying characteristics of Iraqi culture at the same time the technical knowledge and skills are being imparted. The omission of these elements increases the likelihood of partial success and/ or failure in the short term and substantially diminished prospects for long-term sustainability of capacity building efforts. Second, is the basic question of what is meant by capacity building. BearingPoint and USAID seem to confuse tools, such as courses, seminars, workshops, and on-the-job training that provide new knowledge and skills or improve on existing knowledge and skills with technical assistance that helps get something done. The interchangeable use of these two definitional elements causes confusion in identifying and reporting accomplishments and capturing the real effects of capacity building efforts. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 53 Annex 1: FMIS Interviews I. FMIS Interview Questions 1. Name of interviewee, job title (including name of Ministry and Department), gender. 2. Date and place of training session. 3. Length of training session, number of instructors, number of participants. 4. What was the purpose of the training? 5. Were there written materials given out at the training? Have they used those materials in carrying out their FMIS functions and, if yes, how? 6. How has what they learned in the training helped them in carrying out their FMIS functions? Ask for examples, such as, I can enter data into the computer. 7. Did the training provide all the information they need to carry out their FMIS functions? If not, what was missing in the training? 8. Did they learn anything more from the training that they were not able to do before? 9. Were there any problems with the training? For example: • was the training session longer or shorter than scheduled; • did all the instructors show up and did they do a good job; • did all the scheduled participants show up; • did any participants leave early; • were important training materials unavailable (the videotape machine did not work); • did translators arrive on time and were they capable? 10. Did they fill out an evaluation form after the training was completed? 11. Has anyone contacted them to find out how they are using what they learned in the training and whether or not they need additional training? 12. Do you need additional training to be better able to carry out your FMIS functions? If so, please specify. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 54 II. FMIS Interview # 1 (3/2/06): Nawal Ibraheem Mansour Al Shammari 1. Name of interviewee, Job title ( including name of Ministry and Department), gender Nawal Ibraheem Mansour AL Shammari, Accountant manager, Ministry of Water Resources 3. Date and place of training session. February 2005, Amman 4. Length of training session, number of instructors, number of participants. One week, one instructor, 18 participants 5. What was the purpose of the training? Application system, financial management, free balance 6. Were there written materials given out at the training? Have they used those materials in carrying out their FMIS functions and, if yes, how? Yes, I received materials and used it with examples resolved later. There were also pens and papers. I bought a system for the records using the auto and manual so I needn’t any manuals. 7. How has what they learned in they training helped them in carrying out their FMIS functions. Ask for examples, such as, I can enter data into the computer. I got great benefit from that as it suits our country and I am eager to have the second phase of that system. 8. Did the training provide all the information they need to carry out their FMIS functions? If not, what was the missing in the training? The information we received is perfect as we can sort out reports and enter them into the computer. When I have relevant questions, I find someone to help. I ask about everything, I upload tables, I sometimes find problems during application. 8. Did they learn anything from the training that they were not able to do before? When I had the training, I was aware of everything, the only thing I noticed is that all the ministries do their task at one time. All information comes at once as if they are one unit. The nice thing is that we know now each other; we ask each other, that is really encouraging. 9. Where there any problems with the training? For example: • was the training session longer or shorter than scheduled: It was so intensified, with work time from morning till the afternoon. levels vary among participants, also in the computer skills. Application started here in the first month . United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 55 • did all the instructors show up and did they do a good Job Yes, he show up and did a great job • Did all the scheduled participants show up Yes, all of them. • Did any participants leave early No, as a car was coming to take all of us. • Were important training materials unavailable (the videotape machine did not work)? Everything was available; the electricity in Baghdad was a horrible problem 10. Did they fill out an evaluation form after the training was completed? Yes, all of us filled out those forms positively as people love to have more knowledge from such useful trainings. 11. Has any one contacted them to find out how they are using what they learned in the training and whether or not they need additional training? Yes, they were in charge of the room they allocated to us with a supervisor. They took our phone numbers. We often come here if we have any questions to ask. I asked some questions but no one asked us. They were so competent, so patient with no knowledge people. They used to encourage us to ask. I am ready to apply this system, and I knew there is another training which I think I need to increase my knowledge Note: • I think you have to classify the participants according to their computer skill • I got recognition from Ministry of Finance while the Ministry I belong to did not pay any attention to me. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 56 III. FMIS Interview #2 (3/2/06): Eman Ibraheem Mahmoud 1. Name of interviewee, Job title ( including name of Ministry and Department), gender Eman Ibraheem Mahmoud, Accountant Manager, Ministry of Justice. 2. Date and place of training session. The first one on Sep 2004, in Baghdad, and the second on Dec. 2004 in Amman 3. Length of training session, number of instructors, number of participants. One week in Baghdad, one week in Amman, one instructor, 18 participants 4. What was the purpose of the training? The purpose of the training is to study the FMIS program, especially that we did not use such program in Iraq before, and this is the first time that this good program enter the Ministries of Iraq. 5. Were there written materials given out at the training? Have they used those materials in carrying out their FMIS functions and, if yes, how? Yes, I received materials, I used to use them during the training and when I first imply the program but consequently, I stopped using it because I no longer need it, I imply everything everyday so I save the steps by heart. 6. How has what they learned in they training helped them in carrying out their FMIS functions. Ask for examples, such as, I can enter data into the computer. Yes, if I wouldn’t have the training, I wouldn’t be able to use this program properly, for example I can now sort out reports and enter tables in to the program which makes the work easier. 7. Did the training provide all the information they need to carry out their FMIS functions? If not, what was the missing in the training? The Training gave us appropriate information but from the practical point of view, I mean when I want to imply something, some times some issues may appear which will lead to certain question, so I call the trainer in the data center and ask him for an answer, for example, once I added extra field, and I did not how to delete it, so I call the trainer, and he explained the steps for me, and I manage to continue my work. 8. Did they learn anything from the training that they were not able to do before? Yes, for example, I did not know how to use the internet before the training, but the trainer taught me how to use it, and also he opened an email account for me and gave me its password. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 57 9. Where there any problems with the training? For example: • was the training session longer or shorter than scheduled: No, not at all, because we were happy to have the training in Amman, as no power may cut off, no explosions, no other responsibilities, no work; no kids need to be feed up, only the training. In the contrary the training in Baghdad faced several difficulties like the streets are blocked sometimes and we went late to the training cause of the traffic. • did all the instructors show up and did they do a good Job Yes, they were so efficient and skillful, and till now, they are ready to help, explain, and answer any question related to work. • Did all the scheduled participants show up Yes, all of them. • Did any participants leave early No, in fact we were so embarrassed even to take any permission to leave unless they tell us to take a break. • Were important training materials unavailable (the videotape machine did not work)? Everything was prepared 10. Did they fill out an evaluation form after the training was completed? Yes, I did 11. Has any one contacted them to find out how they are using what they learned in the training and whether or not they need additional training? Yes, our trainers in the data center always contact us, and we are also keeping in touch with them by phone calls and emails if there are any misunderstandings. 12. Do you need additional training to be better to carry out your FMIS functions? If so, please specify. Only in case there is a second stage in the program. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 58 IV. FMIS Interview #3 (3/2/06): Zena Adeeb 1. Name of interviewee, Job title ( including name of Ministry and Department), gender Zena Adeeb, IT ( Engineer ) Ministry of Finance, Data center Dep. 2. Date and place of training session. Apr. 2005 in Amman. 3. Length of training session, number of instructors, number of participants. One week, one instructor, 30 participants 4. What was the purpose of the training? As I am an IT, it was a general idea about in put records, expenditure voucher and Journal voucher 5. Were there written materials given out at the training? Have they used those materials in carrying out their FMIS functions and, if yes, how? Yes, I received materials, I used them only during the training cause I am an IT. I did not work on that program. 6. How has what they learned in they training helped them in carrying out their FMIS functions. Ask for examples, such as, I can enter data into the computer. Yes, As I am an IT, I know now the signs that I did not know before. 7. Did the training provide all the information they need to carry out their FMIS functions? If not, what was the missing in the training? Yes, for me I was able to carry out my home work with out any problems it was very simple but regarding to my colloquies in the training, they are used to imply the journal voucher, before there were no time limited in entering the data but now they are obliged in entering data before the closing date . 8. Did they learn anything from the training that they were not able to do before? Yes, of course, for example, before the training when I saw some staff working on that program I did not understand what they are doing, and I did not know what certain signs means, but after the training I understand all the signs in the free balance accurately. 9. Where there any problems with the training? For example: • was the training session longer or shorter than scheduled: No, it goes according to the scheduled, but in some how there was some pressure on us due to the long hours that we spend in the training and get shorts break. • did all the instructors show up and did they do a good Job Yes, he was so good and patient. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 59 • Did all the scheduled participants show up Yes, all of them except only one who came in the 2nd day by the GMC, cause he was not able to catch the airplane. • Did any participants leave early No, we only had a one hour break usually for lunch; we were always in a hurry to reach the training hall before the time end. • Were important training materials unavailable (the videotape machine did not work)? No, everything was there and well connected with all the computers, everything was working. 10. Did they fill out an evaluation form after the training was completed? Yes. 11. Has any one contacted them to find out how they are using what they learned in the training and whether or not they need additional training? Me as an IT no, cause no one suppose to contact me. But there were some emails including some questions arrived and the help desk helped in solving problems. 12. Do you need additional training to be better to carry out your FMIS functions? If so, please specify. No, this in not my specialize, Note: The good thing in that program lies in letting the users of that program be aware that they are responsible for any mistake happened in their in put or in giving their user name or password to their friends and in case in entering wrong data, we are as an IT could recognize the wrong records and identify to which ministry is belong and specify the department. It’s a good program which refused any fault. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 60 IV. FMIS Interview #4 (3/3/06): Rokzan Thurya Rushdi 1. Name of interviewee, Job title ( including name of Ministry and Department), gender Rokzan Thurya Rushdi, Accountant, Ministry of Labor 2. Date and place of training session. Baghdad, Dec. 2004, & Amman Jan.2005 3. Length of training session, number of instructors, number of participants. One week, one instructor, 18 participants in Baghdad, one week, one instructor 16 participants in Amman . 4. What was the purpose of the training? Control the State budget throughout the free balance program. 5. Were there written materials given out at the training? Have they used those materials in carrying out their FMIS functions and, if yes, how? Yes, I received them and got benefit of them during the training, for Ex. In the beginning when I want to pull out a certain voucher I refer to those materials to read the steps that I need to do, and we imply the program after 2 month of the training. 6. How has what they learned in they training helped them in carrying out their FMIS functions. Ask for examples, such as, I can enter data into the computer. Got great benefit from that FMIS program, especially that I was good in manual, but when I learned how to carryout the program in the computer I recognized that is more easier and much better, I can now enter journal voucher and work on the free balance in an accurate way. 7. Did the training provide all the information they need to carry out their FMIS functions? If not, what was the missing in the training? Yes, with much expansion, as we took all the necessary information and much more, all the issues that we need in carrying out the program. 8. Did they learn anything from the training that they were not able to do before? Yes, the computer cause I was not able to use the computer before, Mr. Ammar the instructor taught me everything. And now I can use the computer properly, and I am not the only one who did not know how to use the computer before the training, but many other , and now we learned lots on the computer 9. Where there any problems with the training? For example: • was the training session longer or shorter than scheduled: No, everything was arranged according to the agenda. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 61 • did all the instructors show up and did they do a good Job Yes, more than excellent, Mr. Ammar " the instructor" did his best to let us know how to work on the FMIS program. • Did all the scheduled participants show up No, not all of them, only one her ticket was available but she did not attends to the airport and she did not come to Amman later. • Did any participants leave early No, this case not happened at all. • Were important training materials unavailable (the videotape machine did not work)? Everything was available, and there were spars, such as there were 2 white boards and 2 slide show, when one board was full they use the other one. 10. Did they fill out an evaluation form after the training was completed? Yes, it was useful and very good training. 11. Has any one contacted them to find out how they are using what they learned in the training and whether or not they need additional training? Yes, Mr. Ammar " the instructor" always asked if we have any questions or things that we do not know how to deal with, and he send a person his name is Moa'yed who always come to our Min. and ask about the implication of the program and we always comes here " to the data center " especially that in our Min. our internet is not working now so we used to come here to do our work. 12. Do you need additional training to be better able to carry out your FMIS functions? If so, please specify. Yes, I wish that, I mean I wish to take the 2nd part of the program. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 62 V. FMIS Interview # 5: Hayder Sabeeh Hussein Name of interviewee, Job title ( including name of Ministry and Department), gender Hayder Sabeeh Husseein, Assistants Judge, High Judicial Council 2. Date and place of training session. Baghdad Feb 2005 & Amman Mar.2005 3. Length of training session, number of instructors, number of participants. One week in Baghdad one instructor, 6 participants & one week in Amman , one instructor, 12 participants 4. What was the purpose of the training? Studying the FMIS " the Free Balance" program, to know how to input the expense's vouchers instead of hand working. 5. Were there written materials given out at the training? Have they used those materials in carrying out their FMIS functions and, if yes, how? Yes, I received materials, It contains full explanation about FMIS program, from the first step till the end, for example, if I entered wrong data, I could bring back the wrong record again and do the changes on it. 6. How has what they learned in they training helped them in carrying out their FMIS functions. Ask for examples, such as, I can enter data into the computer. I got 100% benefit of FMIS program, as we entered the calculation of the year 2005 on the same program, and know we are entering the calculation of the year 2006, this program regarded as a checking program cause we imply the hand work manually. 7. Did the training provide all the information they need to carry out their FMIS functions? If not, what was the missing in the training? Regarding to my work, what I learned in the training was enough as I can know use the program, pull out reports, and enter the hand work manually by the computer. 8. Did they learn anything from the training that they were not able to do before? No, I only got benefit from the FMIS program as I know very well how to use the computer before. 9. Where there any problems with the training? For example: • was the training session longer or shorter than scheduled? No, it was just like the schedule. did all the instructors show up and did they do a good Job? Yes, he was excellent and we got so much benefit of him. • Did all the scheduled participants show up? Yes, all of them. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 63 • Did any participants leave early? No, • Were important training materials unavailable (the videotape machine did not work)? No, everything was well prepared. 10. Did they fill out an evaluation form after the training was completed? Yes. I did 11. Has any one contacted them to find out how they are using what they learned in the training and whether or not they need additional training? Yes, we were contacting each other I mean the " datacenter" staff, each time when we had a problem or facing any obstacles, for ex. There was some amendments on the mathematical numbers of the year 2006 according to those of the year 2005, so we came here and save the new numbers. 12. Do you need additional training to be better to carry out your FMIS functions? If so, please specify. For the time being, No, unless if there is some changes on the program, in that case we all need to know that changes. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 64 APPENDIX 4: USAID-FUNDED ECONOMIC GOVERNANCE-II CAPACITY BUILDING PROGRAMS LOCAL GOVERNANCE PROGRAM (LGP) DETAILED REPORT United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 65 UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT MONITORING AND EVALUATION PERFORMANCE PROGRAM PHASE-II (MEPP-II) FINAL DETAILED REPORT USAID-FUNDED ECONOMIC GOVERNANCE-II CAPACITY BUILDING PROGRAMS LOCAL GOVERNANCE PROGRAM (LGP) Submitted to: USAID/IRAQ Baghdad, Iraq Submitted by: Frank Hersman Harvey Herr International Business & Technical Consultants, Inc. 8614 Westwood Center Drive • Suite 400 • Vienna, VA 22182 • USA Telephone: 703-749-0100 • Facsimile: 703-749-0110 May 1, 2006 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 66 Appendix 4: Local Governance Program (LGP) Detailed Evaluation Report Table of Contents I. Introduction …………………………………………………………………………… 67 II. LGP Scope of Work and Deliverables ……………………………………… 67 A. Functional Area 1 ……………………………………………………………….. 68 (Activity Research, Analysis, Design, Monitoring and Evaluation) B. Functional Area 2 ……………………………………………………………….. 69 (Building the Capacities of Sub-national Administrations) C. Functional Area 3 ……………………………………………………………….. 73 (Building the Capacities of Civic Institutions to Participate in Political Processes; including being prepared, if directed to provide Support for Sub-national Elections). D. Functional Area 4 ……………………………………………………………….. 731 (Provision/Administration of Rapid Response Grants and Technical Assistance) III. LGP and the Formation of Councils ………………………………………………. 75 IV. Development of a Legal Basis for Iraqi Local Government Organizations - Powers and Responsibilities………………………………… 81 V. Conclusions and Recommendations ……………………………………… 89 A. Compliance With Contract Deliverables ………………….………………………………... 90 B. Reporting Requirements ……………………………………………………………… 92 C. Security ………………………………………………………………………………… 95 Annex 1: Thi Qar Governorate: Local Governance & Management Systems ……. 97 Annex 2: Training and Development Initiatives - Option Year One …………….. 102 Annex 3: Evaluation Example from Al Muthanna ………………………………………. 103 Annex 4: Questionnaire Administered in Kirkuk to Assess Training Provided ….. 104 Annex 5: Provincial Council Composition Achieved through Caucus in Maysan .. 108 Annex 6: Council Activity and Evaluation in Diwayniyah ………………………. 109 Annex 7: Neighborhood/District/City/Governorate Councils ………………………. 111 Annex 8: LGP 1 Deliverables, Reports, Monitoring ……………………………….. 112 Annex 9: USAID Iraq Local Governance Project/ Mid-Term Evaluation Questionnaire ………………………………………………… 114 Annex 10: Table of Events …………………………………………………………………. 127 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 67 LOCAL GOVERNANCE PROGRAM (LGP) DETAILED EVALUATION REPORT I. Introduction With funding from USAID, Research Triangle Institute (RTI), a North Carolina consulting company entered into two contracts to assist in restoring basic services to the people of Iraq and to develop the local government institutions necessary to devolve and decentralize the strong central government that had been formed by Saddam Hussein and the Baathist Party. The first contract, titled “Iraq Local Institutions Support and Development Program” (herein after referred to as LGP 1), was signed on April 11, 2003, but made effective on March 26, 2003. The amount of this first contract was $167,973,016 and was a Cost-Plus-Fixed-Fee (CPFF) contract. The contract provides for programs to be carried out during a base year and three option years. LGP 1 exercised option year one (LGP 1 OYO) in March of 2004, and was granted a 45-day extension period beyond the second year in April 2005. Total expenditure for LGP 1 and LGP 1 OYO plus extension were reported as $241,910,757.15 This amount included total grants funding of $14,368,010. Out of the total project expenditures approximately 23.4% were expended in Iraq. The balance was expended outside of Iraq, however some of those expenditures would have been for equipment used inside of Iraq (e.g., armored vehicles). The second contract, titled “Iraq Strengthening Local and Provincial Governance” (ISLPG, hereinafter referred to as LGP 2) was signed on April 26, 2005 with an initial award of $89,999,968 and became effective on May 9, 2005. This is also a CPFF contract with 2 base years and 3 option years. The LOE of this contract was later significantly modified to support the new direction the US Ambassador wished to take in Iraq. These two parts of the same contract are referred to as LGP 2-1 and LGP 2-2 to distinguish this change. This evaluation covers LGP 1, LGP 1 (OYO) and LGP 2-1. The “Scope of Work, Evaluation of Capacity Building Activities, Task A” was prepared based on conversations with a US-based consultant employed by IBTCI and the responsible officials in the Democracy and Governance Division in Baghdad, Iraq The SOW was given final approval on February 16, 2006 and is attached as Appendix 1 of the Final Report Appendices. II. LGP Scope of Work and Deliverables The Scope of Work of LGP 1 (Section C of contract, pages 3-4) requires the contractor to provide services in the following four functional areas: 1. Activity Research, Analysis, Design, Monitoring and Evaluation 2. Building the Capacities of Sub-national Administrations 3. Building the Capacities of Civic Institutions to Participate in Political Processes; including being prepared, if directed to provide Support for Sub-national Elections 15 RTI (June, 2005), ‘LOCAL GOVERNANCE PROGRAM FINAL FINANCIAL REPORT. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 68 4. Provision/Administration of Rapid Response Grants and Technical Assistance With respect to the four functional areas specified, the evaluation found general compliance as noted below A. Functional Area 1 (Activity Research, Analysis, Design, Monitoring and Evaluation) In their initial PMP, LGP defined a series of Quality of Life Surveys (QOLs) that were to be used to monitor the IR. Four QOL were completed. The QOL surveys were national household surveys implemented around a stratified cluster sample design (based on Iraqi Census data, and an earlier UNICEF Multiple Integrated Cluster Survey design). Survey implementation utilized local Iraqi partners.16 Questionnaires prepared for the QOL surveys were arranged around the LGP project IRs. A standard set of core questions continued through the four rounds of the survey. The standard questions sought to inform the LGP about recognition of local councils and officials, service levels, attitudes on the effectiveness of local government institutions, preparedness for elections, attitudes towards decentralization, knowledge of CSOs, and membership in groups and associations. Other questions were intended to help understand where communities looked for guidance and support, and how they identified themselves with respect to the nation of Iraq. Core background questions sought to track changes in household income levels, employment, property ownership, and basic demographics. In addition, each of the QOL added additional thematic questions designed to inform LGP activities.17 Results of the QOL surveys were prepared and released to USAID. It remains unclear whether the reports were used to modify LGP program activities, or how widely they were distributed to field teams. The nature of nationwide surveys precludes making statements of attribution to LGP about observed changes in estimated service levels. Attribution in this case should be seen as very weak, and the result of the action of many players of which the LGP is but one. LGP 1 went on to conclude an inventory and assessment of Civil Society in Iraq18, and of local government departments and directorates19 in permissive areas. Both of these activities provided reports and query systems. The query systems were intended to be used by the field 16 Iraq Center for Research and Strategic Studies (ICRSS) and Independent institute for Administration & Civil Society Studies (IIACSS). 17 One general thematic area was the ‘State of Community in Iraq.’ Under this theme questions were asked about participation in civic activities, trust, solidarity and perceived future threats. QOL 2 looked at the food ration system to estimate the proportion of families registered in the system. QOL 3 included questions on family intentions to vote in the upcoming elections, and added questions on crime victimization, use of mortgages, self-definition of economic status, and a series of questions on empowerment and political activity. QOL 4 introduced questions related to victimization from terrorist acts, plus added questions about the current state of the ‘food basket’ distributed by the Ministry of Trade (in advance of monetization). 18 Civil Society Organizations (CSOs) in Iraq: Findings from an Assessment of Civil Society Organizations in Iraq, July 2004 19 Government Department/Directorate Organizational Assessments Report: ‘The Public Utilities’, November 2004 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 69 teams as baseline information for their local programs. Changes in the operating environment precluded use of these systems. LGP 1 designed a monitoring system for the Civic Dialogue Program/Democracy Dialogue Activities (CDP/DDA). This included the use of instruments to collect information on individual events (number attending, estimated sex ratio, type of participants, etc.) as well as a questionnaire aimed at a selection of participants. The event instruments were fed into a database system using laptop computers in the governorates. The system was applied throughout Iraq, but better in some areas than in others (the North and South Central were more effective recorders of events than were the Central and South regions). Data collected were analyzed and reported regularly as an Annex to the Democracy Matters (the weekly reports). The widely quoted estimate that 750,000 citizens participated in some 22,000 dialogue events is based on the reporting of events using the monitoring system. The evaluators did not find reason to doubt these estimates.20 Later an evaluation of the CDP/DDA program identified more precisely the characteristics of those attending the dialogue events and the impact it had on them. The evaluation showed that the dialogue events did manage to sway public opinion in a ‘positive’ way. It showed that certain event themes were better at this than others.21 An important expectation from activities under functional area 1 was to “recommend options for the devolution of authority to the local level, including appropriate legal, fiscal, and regulatory frameworks.” This proved to be an enduring task tackled independently by the different LGP teams. Frustration began to emerge as the teams sought legal and structural clarity for the activities they wished the councils and local departments to undertake (Annex 1 Dhi Qar Diagnostics Report…). This resulted in LGP 1 applying more resources to policy reform creating first a Policy Advisory Unit staffed with TDY personnel, and then a Policy Review Team with a permanent staff and Ministry level access. This function has remained and segued into LGP 2 under the activity 1 of that contract. The importance of putting in place an enabling law that supports decentralization is more acute now than ever before. The current US strategy with the Provincial Reconstruction Teams (PRT) is underpinned on the premise that the authority for decentralization exists. However much remains to be done to convince the new government to support greater local autonomy through a new local government law. It appears that LGP 2 is aware of this need from the perspective of the councils and Ministries. It was less certain that enough attention was being paid to lobbying with the new government (Council of Representatives) to ensure passage of strong and clear enabling law. B. Functional Area 2 (Building the Capacities of Sub-national Administrations) There has been general compliance under this functional area. Not all areas in Iraq have achieved or remained in a permissive status. Initial Rapid Response Grants were used to shore up local administrative institutions (both local councils and local departments and directorates).22 20 It was not possible to verify specific events, but the paper trail evidenced in the weekly reports plus the evaluation of the CDP/DDA program bring confidence to these estimates. 21 USAID/RTI, LGP Civic Dialogue Program Performance Assessment, November 2004 22 See the section below on grants. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 70 1. Local Departments and Directorates All of the LGP teams in the field engaged proactively with the local departments/directorates of the central ministries. Primary objectives were the improvement of service delivery. Each LGP governorate team engaged in training developed independently. When technical experts were available to the teams these experts provided training and mentoring to those departments (e.g., Water Department, Sewerage, Sanitation, Public Finance, Law and Council Operations).23 Training developed around the expertise available locally. Each LGP Team (regional or governorate) developed capacity building activities for their areas. As an example from OYO, Baghdad planned 12 training and development initiatives (see Annex 2). These were a combination of study tours, public colloquia, workshops, computer training for the Audit Department, conference/workshop on Billing and Collection systems for Municipal Services, conference/workshop on Decentralization/Privatization of Municipal Services, and training for IT Department to run and maintain the financial management systems network. Basra worked effectively with the Treasury Department (see box below). This was a combination of grants, formal training, learning by doing and mentoring. There are many examples in LGP 1 such as this one. The weakness of LGP 1 was that it was not able to produce summary reports that captured the totality of what was being achieved.24 Report from the Public Finance Specialist, Basra Governance Team, USAID Local Governance Project The renovation of the Supreme Audit Board office in Basra will be finished by the end of March. It was CPA financed. Currently, the 46 audit staff is operating out of a small room in the Rafidian Bank building and has not been conducting audits as such. I have been working with the General Director, Supreme Audit Board, to develop an organizational chart, to develop a work plan to begin conducting audits and to develop a scheme for organizing the staff by sector in the new offices. I will also submit a grant to provide equipment and furnishings to create a productive working environment for the audit staff. To reinstitute the audit function, LGP will assist the Audit Board in conducting an audit of the $1 million cash advance provided to the electricity transmission and distribution offices in 2003. The Treasury Department has requested this audit of the Audit Board to begin mid-next week. For reference, representatives of the Supreme Audit Board in Baghdad will be visiting Basra on Thursday, March 11th. We will be meeting with them to discuss our program. The Basra Treasury office is functional. CPA has been working to find a better location for the Treasury but the current location is functional. We have been working with the Treasury since October 2003 mostly providing basic computer training courses, assistance in preparing monthly expenditure reports, small grant for furniture and equipment and working with departments to produce trial balance statements. We have been working to build the capacity of the Treasury Director to take on more responsibility for managing the government’s finances. We have weekly meetings at the Treasury with the Director to discuss issues and to offer our advice. We have been facilitating meetings with the departments to develop their 2004 spending plans to enable 23 Baghdad had experts in public finance, solid waste, sewerage and water; Basra (at different times) had experts in solid waste, electricity, sewerage and public finance. Basra made significant advancements in understanding and improving the operations of the Ministry of Finance and the Supreme Audit Board. In Baghdad the public finance expert worked to set up a computerized system of accounts, to process a Performance Audit, and to set up an Internal Control System; his work covered 21 departments/offices in the Amanat and extended to the 11 beladiyas (municipal areas within the Amanat). 24 See Annex 3 for an internal evaluation of activities with Al Muthanna’s departments. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 71 the Treasury to disburse the approved 2004 operating and capital funds. Basra LGP continues to work very closely with the Basra Treasury Directorate. How many sites have actually prepared a budget based on the RTI assistance/training? Basra LGP has been working closely with CPA/GST to assist the Basra governorate institutions to develop 2004 spending plans. The approved spending plans are required by Treasury to determine their monthly allocations. To date, we have assisted the following institutions in developing spending plans: 1. Basra local councils and Interim Provincial Council 2. Directorate of Education 3. Directorate of Agriculture 4. Roads and Bridges 5. Basra Airport and Iraqi Airways 6. Tax Offices 7. Health Directorate The spending plan package includes the following: A detailed 2004 monthly spending plan based on the chart of accounts, a staffing table for each institution, a budget summary providing an overview of the organization and what the budget provides and an organizational chart. The evaluators attempted to validate LGP 1 training done in Kirkuk. The team was able to identify the names of individuals and departments who received training with assistance from a former LGP team member. Information included a basic idea of the training provided. A questionnaire was prepared for use by one of our field staff who translated it into Arabic (see Annex 4). Our field staff, assisted by local LGP staff in Kirkuk who provided directions and introductions, was able to interview 14 staff from 5 departments in Kirkuk. This was the only opportunity we had to validate the training received. It is important to put the results of the validation in context. To do this we asked our field staff to provide a qualitative sense of how those interviewed see the current situation and the future. Iraqis feel themselves to be far behind other countries. They are eager to learn and to advance. They are waiting for the leadership a new government might provide (perhaps a reflection of the centrist and socialist indoctrination of the former regime). In the difficult living circumstances in Iraq when there is any opportunity to escape, even briefly, to another country for a training opportunity it will be sought after. For this reason the tendency is to elevate the value of training to high levels. Responses were therefore expected to be highly positive; given the context the results of the rankings should be discounted a point or two. In Kirkuk, those who received training in Microsoft Office were happiest. Next were the Directors who had taken the Management course or the Leadership course. The said that they felt more self confident in dealing with their staff, and now understood the importance of listening to staff comments and suggestions. Three rating questions were asked regarding how well the material was presented, the importance of the material to their work, and whether they would advise colleagues to receive the same training. Ratings were on a scale of 1 to 10 with 10 being the highest. Quality of ‘presentation’ scored 8.2, ‘importance’ scored 7.1, and ‘advising colleagues’ to take the training scored 9.5. The lower score for ‘importance’ reflects the comments that not all were applying what they had learned to their work. Some said this was because of security, or because they were waiting for a new government. Others said they were applying the material (accountants, and those who benefited from computer training). Nevertheless, all but one of those interviewed said it was helpful and they looked forward to further training. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 72 Only half of those interviewed were aware of the LGP program, but all knew of RTI. Just three knew of USAID. Trainees came from six offices and included junior and senior staff. One of the Directors trained supervised 750 people. Three of those interviewed said they had also received training from NDI or UNICEF. RTI initially offered its training by presenting a menu of courses that were available; the directorates then nominated three or four candidates to attend who were in positions where the training might be useful. Length of training depended on the course offered and varied from two days to ten days. All expressed gratitude to RTI for offering the training. Summarizing the evaluation, it is clear that LGP training had been provided to the directorates and that those who received the training were able to recall the experience. Those interviewed rated the training highly, but it was true that not all training had been applied in their work. Computer training and training in billing systems were apparently most successful. All interviewed indicated that they received training materials from LGP. 2. Councils In capacity building efforts for council members, it is clear that all of the LGP teams engaged the councils by providing formal training, learning by doing and mentoring. The LGP teams were involved at an early stage in establishing the councils. (See Annex 5 Provincial council composition in Maysan). In Baghdad this began in May, immediately after their entry in country. In the section below entitled “LGP and the Formation of Councils” develops fully how councils were formed, and provides a view of the problems encountered. It is clear from the accounts provided in the Weekly and Monthly Reports that all the councils engaged by the LGP received training of some sort. Most would have received training in how to run a meeting. Other councils were mentored to include the development of a Charter, or Council Rules of Procedure.25 More advanced councils were mentored in how to engage with local administration, how to establish committees and how to conduct public hearings. Evaluating the progress that the LGP was making in building the capacity of the councils as fully functioning institutions in a participatory democracy was troublesome. Attempts were made by individual LGP teams to develop indicators for this purpose (see Annex 6 “Council Activity and Evaluation in Diwayniyah” showing correspondence on this subject; and Annex 7 “Al Maymouna Qada Council report and evaluation sheet”). The lack of legal clarity detailing the authority of the councils was an issue for many of the LGP teams in the governorates. Council members themselves raised the issue (Babil, Kirkuk). Directors General of the civil administration still reported directly to their Ministries. This became a stumbling block for the LGP. In promoting decentralization they reached the limit of what the law could allow. They were advising councils to take on powers that they apparently did not possess. This is developed in the following section on “The Legal Basis for Decentralization.” 25 The Diwaniyah Rules of Procedure contain eight Articles: Council Meetings, Duties and Responsibilities of Members, Council Meeting Procedures, Committees Boards and Commissions, Relations with City Administration, Public Hearing Procedures, and Use of Rules. Reference is made to ‘Parliamentary Procedure at a Glance.’ United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 73 C. Functional Area 3 (Building the Capacities of Civic Institutions to Participate in Political Processes; including being prepared, if directed to provide Support for Sub-national Elections). As indicated above, LGP 1 concluded an inventory and assessment of Civil Society in Iraq. This was to be the basis for the intended work of the LGP with civil society organizations. Early in the project LGP coordinated with CAP programs to avoid overlap. Some of the CAP were working with neighborhood groups to bring projects to councils. By OYO, the CSO role had diminished and LGP civil society activities become focused on the Civic Dialogue Program/Democracy Dialogue Activities.26 These were designed to inform the public at large on the meaning of democracy, human rights, the TAL, gender equality, role of the media, the role of the military in a democratic society as well as other issues. The concept was to have an informed citizenry in the build up to sovereignty and before the elections of January 2005. This was a successful program that is well documented in its evaluation27 and in the annexes of the weekly reports. The LGP teams did engage in election activity for sub-national councils. Nassiriyah developed, with local officials, voter rolls based on food ration cards and other forms of identification, and used it successfully to elect district level council members. D. Functional Area 4 (Provision/Administration of Rapid Response Grants and Technical Assistance) Early in the RTI program the main emphasis was on restoring basic services to the Iraqi population centers which had been destroyed in the war effort or had been neglected as a consequence of inadequate funding owing to past wars Iraq had with Iran and with the Coalition Forces during the first Gulf War. Small scale repair projects and infrastructure projects pursuant to Rapid Response Grants (RRGs) were carried out that restored basic services to millions of Iraqis across the country. In total RTI expended about $14,000,000 in RRG grants during the Base Year and Option Year One. The RRG grants were considered to benefit the RTI Program in giving legitimacy to the local government councils that RTI was establishing and also to provide training for RTI staff in grants management that would provide future staff benefits not only to RTI but also to other donor and government agencies, and Iraqi contractors. Many illustrations of the beneficial impacts of the RRG grants are given in the weekly, monthly, annual and final reports RTI submitted to USAID to demonstrate the value of these small grants. The evaluators sought to determine if the claimed benefits of the RRG Program were as described and if the Grant Program was conducted in accordance with the USAID approved RTI Grants Manual. Because of the security constraints on travel it was decided to select four sanitation grants carried out in Baghdad to determine if they were effective in restoring basic services, complied with the provisions of the Grants Manual, and delivered the claimed benefits. 26 The USAID Community Action Program (CAP) continued to work with communities to develop community action groups (CAGs). The CAGs were seen as nascent formal civil society organizations. Promising CAGs were encouraged to become local NGOs and received additional training. 27 USAID/RTI, LGP Civic Dialogue Program Performance Assessment, November 2004 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 74 The evaluators reviewed grant files for the four sanitation grants to determine compliance with the Grants Manual and to determine how data were collected relating to other important factors, such as the estimated number of Iraqi citizens benefiting from the grants, jobs created and from procurement benefiting the local economy. An Iraqi engineer with a Masters Degree was hired to interview officials at the Baghdad Sanitation Department that would be able to render an opinion as to the quality of the work performed and if the project met an important priority. The scope of our evaluation included the following: 1) meeting the requirements of the Grants Manual to determine if the Grant write-up was indeed a priority project for support using the scoring criteria for accepting a project for support, 2) if quality control measures were taken, such as required site inspections of the work in progress, 3) relationships with the relevant government agency and the community, and 4) if communications were established with the media, and if attributions were made for USAID support of the project. It was important to know how the grant contributed to achieving the goals and short and long term strategies set forth in the RTI Work Plan, and whether there was transparency and fairness in the bidding procedures that were followed. The four Sanitation Projects were selected for review: 1. Karrada Drain Replacement Grant, Implementation started, 1/26/2004; completed 6/07/ 2004, $180,000, Project No. 161. 2. Improved Security Measure s for Municipal Solid Waste Vehicles, Implementation started, 1/26/2004; completed 2/23/2004, $171,450, Project No.192. 3. Kamaliya Sewage System Improvements, Implementation started 2/23/2004; completed 8/17/2004, $65,000, Project No. 147. 4. Kamaliya Pump Repair, Implementation started 11/04/2003, completed 1/04/2004, $20,000, Project No. 162 Upon inspection of these grant files , it was determined that there was no systematic ordering of the grant papers-the basic contract document was as likely to show up at the end , the beginning or in the middle of the grant file. In none of the files was evidence presented that the grant was selected based on the scoring system set forth in the Grants Manual, Section 3.2, perhaps suggesting that this element should be removed from the Grants Manual when contractors are bidding on emergency repair work.. The number of bids received for the project was not listed in one file, and the winning bid was often the lowest bid with little consideration of the quality of work based on past performance. In one case the bid was selected by a single reviewer, who then obtained a pro forma concurrence by the team leader. In many instances the site inspection reports were not signed. The official papers relating to the draw downs and final payments were not organized in one place in the file. There is nothing in the files to indicate whether the RRG followed a regional/national strategy as required by Section 7.5, if Grant Activity Completion Requirements were followed (Section 7.2.4, items 56 & 57) relating to Lessons Learned and Public Dissemination, or if the reporting requirements for reports to USAID were satisfied (Section 7.2.1) or the requirement in Section 7.1.6 for Separate Financial Record Keeping and Reporting was followed. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 75 The discussions by the Iraqi Field Monitor (hired by the Evaluation Team) with officials of the Sanitation Department indicated that the work performed pursuant to all four grants was of a high quality and met a priority need of the Department at that time. The grants did assist in restoring basic services to the citizens in the particular service area. It appears that in all grants, the USAID CTO signed off on the grants in a timely manner. The files reviewed do not evidence any efforts, except in one file, to coordinate the RRG with other donor activities, such as with military programs that have substantial programs and funding in the sanitation area. In conclusion there is no doubt that the four sanitation RRG projects served an important need in restoring basic service delivery to the citizens in this Baghdad Sanitation Department service area. Any deficiencies in the project were largely those relating to record keeping in compliance with the requirements of the RTI Grants Manual. Comprehensive data on the RRG grants program is set forth in the LGP Final Report, Rapid Response Grants, undated report, which covers 224 RRG grants totaling $17,287,975. This Report notes that all grant projects are cataloged in a “LGP National Grants Reporting System (GRS) which has been used to create “ innovative reports that were used as Grant Management Tools, and to make critical decisions and adjustments to grant activity at the National and Regional levels.” The Report notes that USAID was a recipient of reports generated by this system. Of the 223 approved grants only six are noted as incomplete and closed, all but one of which were closed for security reasons. The other project was closed because of time constraints. Twenty one grants were cancelled, all but one of which were cancelled because of security reasons. The remaining grant was cancelled because the necessary personnel left the project. When cancelled grants are considered, as shown in the final LGP 1 financial report, total expenditure on RRG amounted to $14,368,010. Clearly the overall Rapid Response Grant Program can justifiably be considered an outstanding success in restoring basic services across all geographical areas of Iraq; in restoring basic services delivery to millions of Iraqi citizens in all areas of their national life; and in the process trained many Iraqi nationals as Grant Administrators for the benefit of RTI and other Iraqi organizations and contractors. III. LGP and the Formation of Councils Each of the provincial LGP teams was involved in the development of local councils. The Local Governance Program Final Report, June 2005, states that 437 neighborhood (Hayy) councils, 195 sub-district (Nahiya) councils, 96 District (Qada’) councils, and 16 Provincial (Mahafada) councils were developed. It was not possible for the evaluators to verify these numbers. Consistent weekly and monthly reports, as well as the internal LGP evaluations, suggest that these numbers are substantially correct. Unclear is the extent of involvement that LGP had in the initial formation of provincial council and district councils. These councils tended to have been formed by the military groups that initially passed through the area, and were later the focus of CPA lead initiatives in support of council formation and refreshment. What is certain is that the LGP played a fundamental role in supporting the councils by providing or refurbishing United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 76 needed office space; but more fundamentally in the provision of training in the fundamentals of council operations (issues of Council training are developed below). Lower level council development at the neighborhood and sub-district level were a significant activity of the LGP. Each province, in addition to Baghdad, had a different strategy for council development shaped by the enabling environment presented to them. Babil’s (Hilla) strategy called for the development of neighborhood councils using Iraqi facilitators trained for the purpose and supported by a back office international specialist. They also mentored and trained extensively with their Provincial Council. Basra engaged from the top down developing first the Provincial Council, then moving to the district and sub-district councils (eventually they moved to support the development of a council for the Marsh Arabs who had been abused and isolated in the former regime). Al Muthanna developed an elaborate caucus procedure that ensured representation of key groups on the Provincial Council. They went on to encourage district and sub-district councils. Dhi-Qar involved the local administration in establishing voter rolls based on food ration cards to elect councils. All provinces, except in the Kurdish region, were actively developing and training local councils. The Kurdish regions had been ‘independent’ since 1992, and had developed their local government following Law 159 of 1969, although councils were dominated by the prevailing Kurdish political parties. The evaluation team was unable to contact many of the LGP staff employed during the early days of LGP 1 in its attempt to validate the dimensions of council activity. Fortunately, Mark Etherington, in his recent book covering his experiences in Wasit, validates many of the issues referred to in LGP 1 reporting from that period. Etherington was head of the CPA Governance Team in Wasit. Included under his authority would have been the activities of the RTI LGP 1 program, and these were mentioned in his book. What he has written informs the context of the LGP 1 program during its first year, and provides the context for council development. His text is highlighted where reference is made to issues that support later development of lessons learned. Etherington saw his role as ‘the seizure of civil initiative’.28 The mandate from the CPA was to seek the broadest possible popular support. “Where there was no civil society it should be created.”29 The idea was to begin the process of creating political awareness, by “re￾invigorating political debate. Councils should be established in towns and villages where there were none….There was a caveat---there should be no elections.”30 The CPA believed that it would not have been possible to put in place voter lists and the mechanisms necessary to conduct an election. It also believed that political party development was at a nascent level. The LGP team in Wasit preceded Mr. Etherington, and what Etherington found in his October 2003 tour of Al Kut would have been what the LGP team had seen.31 “What we had inherited 28 Mark Etherington (2005), “Revolt on the Tigris: The Al-Sadr Uprising and the Governing of Iraq”, Cornell University Press, Ithaca, New York, page 7 29 Op. cit page 22 30 Op. cit. page 22, the CPA, and Ambassador Bremer preferred the ‘caucus’ method for creating councils. Shia leader Al Sistani pushed for elections based on the food ration rolls. 31 The LGP September Monthly Report notes that RTI opened four new offices within the Wasit Governorate building and placed a 12-member LGT to assist and support the governor, his senior staff, and the city council. Before the LGT’s arrival, the council was paralyzed and had not developed accepted modus operandi. For example, council members had individually decided which districts they would United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 77 was institutionalized disorder: the old framework remained but the people who had given it reason and purpose had disappeared, and with them the discipline they instilled. All over the city public services had collapsed. The looting of public property had become a national sport. Cars, sewage and rubbish trucks, army equipment: most had simply been appropriated in the anarchy accompanying the Coalition’s advance, driven to the Iranian border and sold.” 32 “The Councils that Bremer had asked me to establish were self-evidently a method of broadening political representation, but these too were patchwork and their proposed powers opaque. How, finally, were these entities to interact?”33 Nevertheless he sets out to visit all the mayors and existing councils in the districts and sub-districts of Wasit. What he found was a mixture or ineffective councils. Some had been threatened and disbanded; others were the instruments of the local tribal authority, and others family oriented. By November the CPA were looking at dealing with 17 councils and had a staff of four internationals.34 Councils were in disarray. Some councils were of uncertain origin while others had been hastily formed by the Marine Corps. By October 2003, some of the initial governance infrastructure that had been developed as an initial interlocutor for the military was coming apart. “In the border town of Badrah, 100 km. north of Kut, the townspeople had again ejected their mayor, this time at gun-point. In four major towns there were no mayors at all. The ad hoc Council in the sub￾district of Hafriyah consisted only of members of a single family.”35 The governor of Wasit had been appointed by the Marine Corps before they departed the province in August. He (the governor) noted the absence of Mayors since the downfall of Saddam’s regime remarking to Etherington that “Each district should have had one, and each sub-district a Director, who discharged similar duties albeit on a smaller canvas…..how the resulting gaps were to be filled, and what powers were to be wielded by future incumbents, was unclear.”36 The governor yearned to (himself) appoint the missing mayors and directors in a bid to underwrite his tenuous appointment. “There had been no tradition of these (councils) before the war in the form in which we now understood them. The Marine Corps had established a few, in a necessarily ad hoc way, and in other cases groups of citizens had assumed the responsibility on their own initiative. Such people would invariably describe themselves as public-spirited, while those excluded would castigate them as a criminal band of thieves and opportunists.”37 The Marine Corps left behind represent, without consulting each other. In order to create a more representative and responsive city council, the LGP team facilitated the assignment of council members to the 10 districts and helped to establish 9 subcommittees. The LGT also began working to establish advisory councils at the neighborhood and district levels. A 10-member District Advisory Council has been established for each district. In addition, 30 locations have been identified and assessments are under way to select 10 buildings to refurbish and house the 10 DACs. For the first time, the city council members began discussing and identifying problems faced by their respective neighborhoods and districts, thereby ending five months of inactivity. 32 Op. cit. page 54. 33 Op. cit page 78 34 Op. cit., page 99; here he apparently overlooks the fact that LGP had an active council team deployed. 35 Op. cit page 64, Hafriyah was later selected by MMWP as the location for a demonstration project on decentralization. 36 Op. cit page 75. 37 Op. cit, page 78 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 78 a ‘constitutional council’ assembled from a list of “800 names gathered from all corners of Wasit.”38 The council included religious leaders, political parties, union leaders, tribal chiefs and others. The council continued to meet after the departure of the Marines up to the arrival of the CPA team (and with the support of LGP 1). They were distinctly aware of the CPA plan to restructure councils across Wasit and were determined to resist this. However, the original council was dismissed; and a new council was to be selected in as ‘democratic’ a manner as possible. The Wasit plan was to select council members proportionately from the District and Sub-District Councils. This meant first that the District and Sub-District councils needed to be reviewed and the bad apples removed. Each of the District and sub-district councils would then nominate members for the Provincial Council. Once done, the Provincial Council met and elected a Chairman from among its members. LGP 1 played a supporting role in this process, providing the Wasit Provincial Council with a secretariat and helping to train and run the council.39 This process for selecting/electing councils varied from one governorate to the next; this process was generally referred to as a caucus. The 15 November 2003 agreement provided an exit strategy for the CPA and a timetable for the return to sovereignty for Iraq. It also had a profound effect of operations in the field. The agreement set a date of 30 June for the end of the CPA, and left uncertain how the work that had been begun with the councils would continue. In Wasit, plans were made for an advisory tribal council as well as a consultative group of political and religious parties that would consult with the Provincial Council. The Iraq Governing Council (established in June 2003 as a result of UN Security Council resolution 1483) gained recognition as the focal point for the transition to sovereignty. In support of the transition a three-month period began on 15 November 2003 that would commence a process of ‘refreshment’ for the councils aimed at ensuring they were ‘representative.’ UN Security Council resolution 1511 encouraged IGC to act within a reasonable time. Governors who may have been appointed were to be confirmed by council vote. Councils were “to be given new local powers commensurate with CPA’s intention to de-centralize Iraqi government.”40 At the same time additional funding was to be made available to the provincial teams. For the LGP 1 this was to be increased Grants funds; but this later proved incorrect. In the specific case of Wasit, having just completed a process to restructure its councils, this came at an awkward moment. Paraphrasing Etherington, the 15 November Agreement forced all the provinces into the same box when most of them were working with an ad hoc assortment of councils and assemblies that they were reluctant to tamper with. The purpose of the ‘refreshment’ was to help establish a “caucus process designed to produce an interim Iraqi government.” The CPA wanted to draw on the refreshed councils as the source of representatives to the interim government (called the provincial caucus system). In the weeks to follow, Ayatollah Al Sistani called for direct elections to the interim National Assembly (Constitutional Convention) requesting a UN mission to evaluate the possibility of elections. In the end, the caucus system was not used having been replaced by January 2005 elections. The UN mission and IFES, a USAID Implementing Partner which was working on promoting elections, found that food ration cards could be used to develop voter lists. 38 Op. cit page 117 39 Op. cit., page 127 40 Op. cit page133, these were the new powers eventually described in the LGPO 71. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 79 The development of councils was at the core of the CPA de-centralization plan, and was reflected in the early LGP 1 reports stressing council development.41 But Etherington notes “Fundamental problems remained. These Councils were at the very heart of our decentralization plans and would provide provincial adhesive during the stresses and strains of transition. Yet the order designed to enshrine their new local powers was delayed month after month.”42 Coupled with the 15 November Agreement was a strategic plan that envisioned a return of executive powers to the ministries.43 The IGC also did not want to see an erosion of its powers. In Wasit, Etherington saw that “increasingly muscular Ministries, buoyed up by the prospect of transition, began to block what were seen as CPA initiatives and appointments and contributed further to a general sense of anxious confusion. We were left in an invidious position: we had no choice but to exhort our Councils ever onward as the transition deadlines approached; yet were unable to supply the legislative foundation and salaries that alone might have confirmed our words and assuaged their fears. I do not believe that CPA Baghdad understood the damage this caused.” 44(Emphasis added). In Baghdad the LGP team was assembling the city council. Early on, Ambassador Bremmer announced that a priority project for LGP was to establish a new city council for Baghdad in order to serve the needs of the military, the CPA and USAID in discharging their program responsibilities in Baghdad. In prewar Iraq, the city council was composed of two-thirds central government department heads and one-third representatives that were selected by Ba’ath Party officials. Lacking any established precedents or election laws, LGP Baghdad designed a novel representation system for anointing city council representatives through a selection process (the caucus process) anchored in the selection of neighborhood representatives. The neighborhood representatives were the building blocks that formed the district councils. District council members were selected to form the Baghdad city council (the Amanat Council). The CPA also imposed restrictions on council membership related to balanced representation from tribes, and ethnic and religious groups. At the end of the selection process the city council had a membership with broad geographic and social representation. Representation was proportional at the district level and based on 1997 population census estimates. By July 30, 2003 88 NAC’s (Neighborhood Councils), 9 DAC’s (District Councils) (now 104) and the Baghdad City Council were formed. 45 These were built on a caucus structure with the lower 41 Restructuring local government initially meant developing a system of councils, rather than undoing the centrist control that Ministries had on local development. An LGP 1 contract deliverable asks for a decentralization framework. “The decentralization framework shall articulate political and administrative functions as well as intergovernmental finance recommendations as appropriate.” Later, when more was understood about how local departments functioned in the centrist system, the limitations of the enabling law were apparent. 42 Op cit., page 158. The order referred to is LGPO 71. 43 Op. cit., page 134. While this did not fully emerge the TAL in Article 56 provides only for coordination and review of federal ministry operations. While local fund raising is allowed, funds from local sources are not likely to be significant. Historically nearly Iraq’s entire budget is funded from Oil revenues. 44 Op. cit., page 159 45 The Baghdad city council formed by caucus has now been removed by the elected Provincial Council which has assumed its powers, removed its mayor and established an Essential Services Committee that looks after the City. This has been highly contentious with the former City Council seeking a reversal of United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 80 levels providing representatives to the higher levels. The caucus model was used widely by the LGP, but was not the only model employed. Many of the LGP required that there were women members of the council, and in ethnically diverse and divisive city of Kirkuk there was an effort to provide a balanced ethnic representation on the councils. Some LGP teams developed election procedures that were applied in the election of local councils.46 In many instances of council formation the LGP worked with councils that had already been formed by the multi￾national forces. However by the late fall of 2003, councils began to lose credibility and legitimacy as council members realized their lack of authority. This led to political in-fighting and accusations that members were collaborating with the Americans. By January 2005 the election process overtook the caucus selection when the Baghdad Provincial Council was elected (concurrent with the election of members to the Interim National Assembly). This newly elected Provincial Council (PC) dissolved the Baghdad City Council. The City Council challenged the PC’s ruling in Court. The initial result is that the PC’s dismissal of the City Council was overturned. Currently the PC has appealed this Court ruling, but there has been no ruling on the appeal. One result of the PC election has been to replace a system with broad geographic council representation from the city, with one where members are elected at large. As a result, 28 of the new 51 member PC are from Sadr City. In LGP 2, the Baghdad local governance team continues to work with the Amanat departments to improve service delivery, while at the same time supporting the newly elected PC. The initial work of the Baghdad team in developing a Council Procedure Manual, as well as Charters for the City Council, the Provincial Council and the Regional Council has been carried over to their work with the PC. In LGP 2, activities with the PC Legal Committee include the use of LGP 2 advisors implementing Activity 2 of the contract agreement to ‘support clarification of the roles and responsibilities of different levels of government.’ The Baghdad PC has now drawn up a draft ‘provincial law’ that it is promoting as a possible basis or starting point for an enabling law for local government authority. The Baghdad PC has become a catalyst for the other governorate PCs in championing the presentation of this agenda to the new government. Such efforts need to be fully supported. Council members were a focus of training during LGP 1. LGP 1 reported that it ‘trained and mentored’ more than 20,000 councils members. Simple mathematics suggests that this may be incorrect. It implies that on average 26 members received training from each council. It is more likely that a fewer number of council members were trained on more than one occasion.47 Training modules for council members currently include: meeting procedures, codes of conduct, budgeting, council-executive duties, citizen participation and transparency. With the January 2005 elections most incumbent Provincial Council members trained by RTI were not elected to the new Councils. In many cases their re-election would have been impossible. Those incumbents on provincial councils who were not members of political parties the order for its removal. The initial court ruling has been in favor of the City Council based on LGPO 71 and the TAL. The PC has 30 days to appeal. 46 E.g., Dhi-Qar 47 Later reports revised this number downward. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 81 were unlikely to appear in the newly elected councils.48 This marks a transition where council formarion moved from the ‘caucus’ period to the ‘election’ period. Once the electoral process of election by political party slate was understood (the second half of 2004), the LGP 1 teams might have considered adjusting the kinds of training being offered to council members, or to the development of political parties. The evaluators found no evidence to support or reject this notion. Lessons Learned 1. The ‘caucus’ method for local advisory council formation promoted by the CPA provided for proxy ward-based citizen representation. This was effectively replaced by the party￾list provincial elections of January 2005. Party list elections tend to obscure accountability and encourage urban bias, as most of those elected tend to come from the urban core (e.g., in the Baghdad provincial council elections 28 of the 51 provincial council members come from Sadr City). USAID needs to be alert to the impact of the electoral system on local government and decentralization. 2. Better information about prewar institutions that existed would have been helpful in explaining to the public differences between the old and the new councils which would have given added legitimacy and credibility to the new councils. 3. In this case the short term objectives of restoring stability and control should have been balanced with the longer term strategies for building new legal frameworks, new institutions and policies that must be designed to promote democratic governance. 4. Using expatriates with an in-depth knowledge about a country and its regional setting can be useful in dampening popular suspicion about coalition partner motivations , particularly in a country like Iraq where oil rich resources are widely believed to have been the basis for the war. 5. To the extent possible, resources used for reconstruction should be channeled through newly developed democratic governance mechanisms to provide employment opportunities and add legitimacy to activities that are implemented. IV. DEVELOPMENT OF A LEGAL BASIS FOR IRAQI LOCAL GOVERNMENT ORGANIZATIONS, POWERS AND RESPONSIBILITIES When the Research Triangle Institute (RTI) arrived in Iraq in April of 2003, it began to experience the legal obstacles that it would confront in undertaking to improve the legal basis for local governance structures and powers at the provincial, city and municipal levels of 48 In some cases the original councils formed through caucus would have excluded members of political parties favoring members of the Iraqi middle class and professionals. Whereas the January 2005 elections, conducted in conjunction with the Transitional National Assembly, was by party list, meaning that the voters elected the party they supported, and the party leadership chose which candidates they wanted placed on the council. (ref: Provincial Support Team, US Embassy Baghdad Iraq, DAN BISBEE CAPT., AR, JOHN KELLER SGT, USAR, ALI AL-SHAMARY IQATF, 29 October 2005). United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 82 government.49 It was known that under the Saddam Hussein regime Iraq had a command and control form of central government. Less was known about the powers of local government and their complex relationships to the central government resulting from the dictatorial control exercised by Saddam Hussein and the influence exercised by the Baath Party over all government appointments. As documented by its May 2003 Monthly Report, the Senior Vice President of RTI met in Baghdad with officials of the Baghdad government, the highest level officials of the U.S. military and the Administrator of the Coalitional Provisional Authority, L. Paul Bremer, to develop the terms of a working relationship. It is believed that a document was prepared by the military setting forth the basis of cooperation; however this document has not been provided to the Evaluation Team. It is not known what constraints may have been placed on the rebuilding activities by the UN Security Council by such UN resolutions as 1483 and 1511. During this period the CPA Administrator directed that the RTI program (now called the LGP) should develop a city government for Baghdad and that this should be accomplished by July 30, 2003. The work was accomplished by the deadline. It is not known what, if anything, that the Administrator may have directed to have been done either formally or informally in order to establish local government structures throughout the remaining local governments of Iraq. Work on a Local Government Enabling Strategy in formulating provisions for powers for local governance and/or shared intergovernmental powers between the central and local governments were begun by the RTI Policy Research Team in the period of August to September 2003. As work progressed, several priority areas emerged that were pursued during the remainder of 2003 and into 2004. These included: 1. Work with the CPA to establish favorable provisions benefiting local government under the Transition Administrative Law (TAL) that was under preparation. 2. Develop a working relationship with a champion or champions at the central government level to support the cause of local government. Working relationships were established with Minister of Municipalities and Public Works (MMPW) and the Minister of State and Provincial Affairs (MoSPA). 3. Development of the framework laws and local government codes to give legitimacy to local government activities. 4. Create a lobbying association for Iraqi local governments based on recommendations of the International Union of Local Governments (IULA) and the examples of other countries. 5. Create a successful example of how powers should be shared between the local government and central government through an experimental project that became known as the Decentralization Project. 49 The principal administrative subunit of Iraq is the province or governorate. These terms are used interchangeably throughout this paper. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 83 6. Mobilize support for these undertakings from all provinces through a series of national, regional and provincial workshops facilitated by RTI in cooperation with the MMPW, the MoSPA and individual governorates. Work by the RTI policy research team continued through 2005 until the present time. The developments described below are in chronological order and based on information found in the weekly and monthly reports prepared by RTI. The policy research team initially began by using part time experts that were mobilized on an as needed basis to work on CPA related activities. The first activity reflected in the RTI documents indicates that LGP contributed comments on various drafts of the CPA ‘Local Government Strategy and Local government Plan, Post-15 November’ (later referred to as the 15th November Accord). This document enhanced local government powers regarding consultation on the hiring/firing of Directors General and other senior officials of the central government, and also approval of ministerial investment plans. In related work, the Muthanna Governorate LGP office established municipal working groups and began to define the powers of the provinces, including the concepts of partnerships between the provinces and central ministries. It is noted that the 15th November Accord, because of the timetable for CPA to hand over power to the Iraqis by June 2004 ‘has changes in the embryonic local governments over personnel decisions of central ministries, but more complete devolution and the power to raise revenues have been dropped.’ In November the RTI Local Governance Teams (LGTs) began preparing discussion papers with Governors and Provincial Councils (PCs) on principles of democracy and shared powers. In December 2003 the policy research team met with the Minister of Municipalities and Public Works at a joint meeting with the Governor of Kirkuk, who requested a list of authorities they would like to have delegated regarding issues of reconstruction and land use. The Minister and Governor agreed to form a joint committee to examine the issue of disputed lands and encroachment as well as the Kirkuk General Plan. They further agreed to hold a one-day workshop to negotiate a comprehensive urban plan, including water and sewer systems supporting the plan. The Evaluation Team sent an Iraqi Field Monitor to interview key Kirkuk officials concerning this nascent attempt (see Memorandum, 12/7/03, RTI Action Notes) and document later events in Kirkuk to clarify the role of local governments in Iraq. Carrying forward the work begun in Muthanna Governorate, the LGP conducted a Civil Dialog Program (CDP/DDA) with a number of workshops and presentations to consider changes in the TAL and to provide information on selecting local councils based on the paper “Structure of Local Government.” This paper described the suggested roles and responsibilities of the different levels of government. In March 2004, the LGP was extended for a second year, which allowed the program to focus on one of three new objectives, including “a policy framework that establishes and supports autonomous, strong sub-national governments”. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 84 During the period March to April 2004, LGP sponsored important conferences relating to “Fiscal and Administrative Decentralization” in Basra that stemmed from a February 12 national conference in Baghdad involving international experts who spoke on different aspects of decentralized governance. In April, LGP headquarters established a Policy Analysis Unit, and placed a senior policy analyst from this unit with the MMPW to review the TAL and Local Government Order No. 71. The analyst was tasked to work on enabling instructions, regulations, rules and procedures needed for the implementation of local government projects, and to clarify the roles and responsibilities of MMPW, the MOF and the Ministry of Planning in this regard. In July 2004 the Civic Dialog Program was also underway discussing the TAL and CPA Order No. 71 which ultimately reached 750,000 Iraqi citizens and helped create an informed electorate to vote in later elections on the proposed Constitution, and elections of members of the Legislative Body and Provincial Councils. Preliminary steps were also undertaken to create and implement an association of local governments. Also in 2004, the MMPW Minister agreed to establish a decentralized local government pilot project to determine the powers that the MMPW would need to devolve in order to enable a provincial government to create effective service delivery systems in three municipal service areas--garbage collection, street cleaning and curb maintenance. A task force drafted an inventory of existing legal, financial, administrative, technical and general authorities from all levels of government related to these three areas of municipal service delivery. Later it was recommended, however, that the Ministry adopt the TAL and CPA Order No. 10 as the legal basis for decentralization to support the pilot program. The task force also drafted an agreement between the Ministry and the Wassit Governorate Council where two pilot sites were chosen for testing the project. The projects were to prepare plans for needed capacity development programs, minimum service standards, operating and capital budgets required, strategic planning, and mechanisms for monitoring and evaluating the pilot projects. The Minister subsequently sent a letter to the Cabinet of Ministers requesting approval to delegate authorities and responsibilities to conduct the pilot programs. In return the Minister received a letter of disapproval because the Council would not accept the TAL and CPA Order No. 71 as justification for the pilot program. The cabinet objected to devolution of authorities to the Provincial Councils as they were not subordinate to MMPW, and further that “the devolution of authority is not allowed under current Iraqi law”. MMPW subsequently changed the program concept to a monitoring and evaluation activity together with a standards program. This decentralization project is now being held in a temporary status, but it is being followed up by additional pilot programs that are to be implemented under LGP 2. The TAL, formally known as the Law of Administration for the State of Iraq for the Transition Period, dated March 8, 2004, was effective from June 8, 2004 until the formation of an elected Iraqi government pursuant to a permanent constitution. The TAL sets forth several sections beneficial to local government, such as Article 3, concerning abridgement of the powers of the United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 85 regions or governorates and Article 4, which specifies that the system of government in Iraq shall be republican, federal, democratic with shared powers between the federal government and the regional governments, governorates, municipalities and local administrations. The main local government powers are set forth in Chapter 8 – Regions, Governorates and Municipalities, Articles 52-58. A synopsis of these articles is a follows: Article 52 designs of a federal system to prevent concentration of power in the federal government; Articles 53 – 54 relate to powers of the Kurdistan Regional Government as the official government in areas where they currently have administrative authority. Article 55 gives each governorate the right to form a governorate council, name a governor, and form municipal and local councils; Article 56 relates to the powers of local government is quoted in full below because it sets forth the limitations and constraints on the exercise of those powers: (A) The Governorate Councils shall assist the federal government in the coordination of federal ministry operations within the governorate, including the review of annual ministry plans and budgets with regard to activities in the governorate. Governorate Councils shall be funded from the general budget of the State, and these Councils shall also have the authority to increase their revenues independently by imposing taxes and fees; to organize the operations of the Governorate administration; to initiate and implement province-level projects alone or in partnership with international, and non-governmental organizations; and to conduct other activities insofar as is consistent with federal laws. (B) The Qada’ and Nahiya councils and other relevant councils shall assist in the performance of federal responsibilities and the delivery of public services by reviewing local ministry plans in the afore-mentioned places; ensuring that they respond properly to local needs and interests; identifying local budgetary requirements through the national budgeting procedures; and collecting and retaining local revenues, taxes, and fees; organizing the operations of the local administration; initiating and implementing local projects alone or in conjunction with international, and non￾governmental organizations; and conducting other activities consistent with applicable law. (C) Where practicable, the federal government shall take measures to devolve additional functions to local, governorate, and regional administrations, in a methodical way. Regional units and governorate administrations, including the Kurdistan Regional Government, shall be organized on the basis of the principle of de-centralization and the devolution of authorities to municipal and local governments. Article 57 expands the limitations of Article 56 in the following way: (A) All authorities not exclusively reserved to the Iraqi Transitional Government may be exercised by the regional governments and governorates as soon as possible following the establishment of appropriate governmental institutions. (B) Elections for governorate councils throughout Iraq and for the Kurdistan National Assembly shall be held at the same time as the elections for the National Assembly, no later than 31 January 2005. Article 58- relates to the concepts to be pursued by the Iraqi Property Claims Commission. Subsection (B) sets forth the procedures for resolving differences concerning boundaries, such as Kirkuk. Apparently throughout the process of drafting the TAL it was known that a further CPA Order would be adopted. CPA Order 71, “Local Government Powers” was signed by CPA United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 86 Administrator Bremer on April 6, 2004. The purpose of Order 71 is to describe the “authorities and responsibilities of the governorate, municipal and local levels of government”. While this Order does provide for separate funding of Governorate Councils, it does not provide for decentralization of central government powers to the local level. (The rationale for not accomplishing greater decentralization of powers at that time is not known.) The Order’s salient points relative to this evaluation are summarized below: Section 2 (2) provides for setting governorate priorities, commenting on Ministry projects, monitoring and recommending improvements in service delivery, representing constituents, collecting taxes and revenues, and organizing provincial administrations. It also provides that amendments to specific ministry projects “may not interfere with the efficient and uniform execution of national objectives as implemented by specific programs”. It provides for initiating provincial projects, but leaves it to the provinces to obtain their own funding for such projects. Section 2 (3) gives the Governorate Councils the authority “to perform their responsibilities independently from the control or supervision of any ministry; and limited powers to approve appointments of Directors General and senior position appointments. Section 2 (4) gives the Governorate Councils powers to develop criteria for selection of its members as agreed to by the CPA. Section 2 (5) gives the Governorate Councils the power to select and appoint Governors and Deputy Governors. Section 3 “Governors and Deputy Governors”, Paragraphs (1), (2), (3), (4) and (5) concern the powers of the Governor to appoint senior personnel and requires that the Governors regularly coordinate with one another on matters of common interest. This section also provides that the Governor is accountable to the Governorate Councils. Section 4 “Local Councils”, Paragraph (1) establishes procedures for forming local councils that will be accountable to “achieve greater efficiencies and economies of scale in coordinating the provision of public services, and to facilitate unified and coordinated administration of cities.” Paragraph (2) provides for selecting membership of local councils in accordance with regulations approved by majority vote of the Governorate Council. Paragraph (3) gives the local councils power by majority vote to elect Mayors and Deputy Mayors. Sections 2 and 4 both provide that members of the Provincial and Local Councils holding office prior to CPA Order No. 71 shall continue to hold office. Section 5 relates to the powers and appointments of Mayors and Deputy Mayors. Sections 3 and 5 provide that Governors and Mayors may attend as non voting members meetings of the Councils to which they relate. Section 6, “Chiefs of Police”, Subsection (1) provides that there shall be a Chief of Police and Subsection (2) provides for a sharing of responsibility in the appointment process with the Ministry of Interior providing lists of qualified applicants to the Governorate Council, who then selects the Chief of Police for a 3 year term. Paragraph (3) provides for removal of a Chief of Police either by the Minister of Interior or by a two￾thirds vote of the Governorate Council, for cause. Paragraph (4) requires the Chief of Police to prepare “an annual policing plan that will take into account local and national priorities”, but leaves the authority with local governments to monitor implementation of the plan. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 87 Paragraph (5) provides for sharing between the Minister of Interior and the Governorate Council in the recruitment and appointment of local police chiefs in the same manner as the appointment of the Chief of Police as set forth in Paragraph 2 above. Paragraph (6) provides for removal of local police chiefs, for cause. Section 7, “Grounds for Removal”, indicates the areas where there is “cause” for removal, the sanctions imposed by cause and the appeal rights for redressing wrongful removal actions. Section 8 “Effect of Other Laws”, Paragraph (1) sets aside all Iraqi laws inconsistent with Order 71, specifically identifying Law 159 of 1969, but continues the power of provincial and local councils to assess, collect, administer or expend taxes and similar revenues under existing Iraq law, including the Income of Municipalities Law No. 130 of 1963 as amended, and farther that “notwithstanding existing Iraqi law, Governorates are authorized to impose althma’am of no greater than 5 percent of relevant taxes.” Following the TAL and CPA Order No. 71, as provided for in the TAL, an Interim Iraq Government became the sovereign government in Iraq in June 2004. Elections were held in January 2005 for an elected government and a provisional National Assembly which was charged with writing a new Iraq Constitution. A vote on adoption of the proposed Constitution was held in October 2005 with a 70% turnout that may have happened in part owing to the preparatory work RTI had done through the Civic Dialog Program in educating 750,000 Iraqi citizens about the features of democracy. The Constitution was ratified pursuant to the terms of the TAL and became effective on March 16, 2006, again pursuant to the provisions set forth in the TAL. The provisions of the Iraq Constitution are organized in the following sections: Section One – Fundamental Principles, Articles 1 to 13 Section Two – Rights and Liberties, Articles 14 to 44 Section Three – Federal Powers, Articles 45 to 105 Chapter One – The Legislative Power, Articles 45 to 62 Note Article 62 concerning the establishment of the Federation Council to include representation by the regions and governorates. The importance of this Council is indicated by the potential governing powers that this Council could have as indicated in Chapter One, Article 46 which provides that “The federal legislative power shall consist of the Council of Representatives and the Federation Council.” Chapter Two – The Executive Power, Articles 63 to 83 Chapter Three – The Judicial Authority, Articles 84 to 98 Note Article 90 (8)(A), Settlement of disputes by the Federal Supreme Court in cases of competency disputes between the Federal Judiciary and the judicial institutions of the regions and governorates, and 8 (B) settlements of competency disputes between the regions and the governorates Chapter Four – Independent Commissions, Articles 99 to 105 Note Article 102 establishing a Public Commission with membership of the federal government, and of the regions and governorates for “fair participation in managing the various state federal institutions, missions, fellowships, delegations and regional and international conferences.” United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 88 Section Four – Powers of the Federal Government, Articles 106 to 111 Note Article 108, ownership of oil and gas belong to all people in all regions and governorates Note Article 110 joint management of federal government with producing governorates and regional governments in management of oil and gas; and distribution of revenues in a fair manner, which will be regulated by law Note Article 111 as quoted in full below: “All powers not stipulated in the exclusive authorities of the federal government shall be the powers of the regions and governorates that are not organized in a region. The priority goes to regional law in case of conflict between other powers shared between the federal government and regional governments “ Section Five – Powers of the Regions, Articles 112 to 121 Chapter One – Regions, Articles 112 to 117 Chapter Two – Governorates that are not incorporated in a region, Articles 118 to 119 Chapter Three – The Capital, Article 120 Chapter Four – The Local Administrations, Article 121 Section Six – Final and Transitional Provisions, Articles 122 to 139 Chapter One – Final Provisions, Articles 122 to 139 Note Article 122 has five provisions relating to amending the Constitution Note Article 126 - Existing laws shall remain in force, unless annulled or amended in accordance with the provisions of this Constitution. Chapter Two – Transitional Provisions, Articles 128 to 139 Note Article 133 providing for postponement of the Federation Council provisions until decision by the Council of Representatives “by a two-thirds majority vote in its second electoral term that is held after this Constitution comes into force”. Note Article 134 (Fifth) that provides for the approval and issuance of legislation and decisions by the Council of Representatives except the stipulations of Articles (114) and (115) that pertain to the formation of the regions”. Note Article 136, First providing for the “necessary steps to complete the implementation of the requirements of all subparagraphs of Article 58 of the Transitional Administrative Law”. Note Article 136, Second which requires the executive branch to comply with Article 58 of the TAL to complete a census and referendum regarding Kirkuk and other disputed territories in a timeframe not to exceed December 31, 2007. An analysis of the implications of the provisions of the Iraq Constitution is set forth in several documents prepared by Dr. Christine Fletcher and Dr. Talib Hamdoni of RTI’s policy research team, in particular “The Constitution of Iraq: Impact for Local Government,” of October 31, 2005.” In March, 2006 the Baghdad Provincial Council exercising a leadership role brought together 15 of the Provincial Councils (not including the three Provincial Councils of Kurdistan which have provisions in the Constitution regarding their powers) in a National Conference to consider development of a framework law for local government (i.e. a Local Government Code) and also to consider forming a Local Government Association, which would be a lobbying organization United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 89 for local government as well as providing information and services on common problems and issues. Following this National Conference, four regional conferences are planned. These will be followed by another national conference which will adopt the work on writing a Local Government Code. This Code will then be given to the Council of Representatives and the Iraqi government to implement the provisions of Section Five of the Constitution “Powers of the Regions”. At the national conference the Provincial Councils are also expected to ratify a concept for a Local Government Association. V. Conclusion and Recommendations for the Development of a Legal Basis for Iraqi Local Government Organizations, Powers and Responsibilities. After the war ended in April 2003, RTI developed a number of programs intended to benefit the development of local government in Iraq. The scorecard on achievements is mixed. The Rapid Response Grants Program, which helped to improve the delivery of basic services destroyed in the war, is considered an unqualified success. The training of Provincial Councils had little utility as most of the members trained were swept from office during subsequent elections. The TAL and Order No. 71 provided marginal benefits in establishing local government powers, as did RTI’s work with the Ministry of Municipalities which attempted an experimental decentralization project. The Iraq Constitution now provides strong powers for the local governments and appears to have established their central role as the mechanism for local service delivery. The projects underway by the provinces to develop a Local Government Code and to develop an Association of Local Governments should be pressed to completion as soon as possible to maintain the momentum that the regions and governorates now have. There are several areas in the Constitution that deserve immediate attention. For example, Article 116 provides that regions shall adopt a constitution, whereas Chapter Two, relating to governorates, speaks in terms of laws to implement its provisions. A constitution and a law may have very different concepts in their governance provisions. Similarly the powers that should be given to the Federation Council should be considered when the constitutions for the regions and the governance laws for the governorates are written. The same holds true for the Public Commissions provided for in Article 102. Provisions and concepts for revenue sharing should also be considered in the sections where they appear in the Constitution. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 90 Lessons learned during the implementation of LGP I and II appear to have been incorporated in the planning for the new Provincial Reconstruction Team program. In particular, the concepts for new training programs reflect RTI’s experience in this key area.50 . A. Compliance with Contract Deliverables The contract specified a wide range of deliverables that are shown in full in Annex 8 Those requirements are summarized below. These are followed by the specific findings of the evaluators on the Contractors compliance with the deliverables. Overall, the nature of the LGP project itself was one of ‘learning by doing.’ This is expressed in the need for initial assessments and a requirement for a quarterly update of the work plan based on assessments. The LGP 1 contract strongly expresses the need for performance indicators that will help ‘fine tune’ the program. There was considerable uncertainty about the operating environment that would be found. Requirements at Start-up: • no later than three weeks after arrival in permissive areas, the contractor shall have prepared a report documenting their assessment of local governance and civil society issues in those geographical areas where security concerns have or will soon allow for engagement. The assessment shall inform a quarterly work plan which will be provided within three weeks after entering a permissive area. • Contractor shall identify appropriate functional and “legitimate” Iraqi leadership • Within a month of in-country arrival the Contractor shall coordinate with local authorities for the delivery of … reconstruction assistance and coordinate such assistance in permissive areas. General details of this coordination will be included in reports to USAID, as agreed between the contractor and the CTO. Within 90 of arrival in permissive areas, the Contractor shall prepare a report which provides a framework, based on collaborative development, for decentralization of authority to local governance structures and shall update the framework with each quarterly work plan submission and as ongoing research is undertaken. Reporting Requirements: • Weekly Consultations: • Monthly Reports: The monthly report should also include the Contractor’s accrued monthly expenditures. • Quarterly Work Plans: The first quarterly work plans shall be submitted within three weeks of entering a permissive environment. • Final Report: Performance Monitoring/Indicators: • Proposing Indicators: At program start-up, and subject to USAID approval, the recipient will develop and propose indicators for monitoring performance. • Performance Monitoring: Within 30 days from the signing of the award, the recipient shall submit for USAID approval a Performance Monitoring Plan. The plan will necessitate the conduct of a baseline study to develop the required baseline data for the measurement of progress throughout the program. The PMP shall provide for periodic evaluation by the recipient of the impact of the various program components and operate as a “program feedback loop”, i.e., 50 3-Day PRT Training, The Way Ahead: PROVINCIAL AND REGIONAL RECONSTRUCTION TEAMS (PRT/RRT) United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 91 impact evaluation/feedback/fine-tuning of component implementation/new activity cycle. • The recipient will submit a semi-annual Performance Monitoring Report that details benchmarks toward achievement of performance indicators and results, the data collected and the method of collection. The first Performance Monitoring Report will be submitted no later than 30 days following the end of FY 2003 U.S. Government fiscal year, with reports submitted semi-annually thereafter. The requirement to immediately deploy teams was met. The LGP team grew from nil to 3000 expatriate and Iraqi staff within 10 months. Teams (sometimes individuals51) were deployed to 17 of the 18 governorates. The management challenge for LGP 1 and USAID was noteworthy. Few, if any, USAID contracts had been issued on this scale.52 The scale both in monetary terms and in personnel was historic. With rapid deployment there was little scope for team building and organizational development. Initial work plans and program implementation were based on assumptions with limited supporting baseline field work.53 While the deployment itself was commendable it lead almost inevitably to staff reaching the field without a clear understanding of what was expected of them. LGP governorate teams dispatched to the field could have had only a vague understanding of their specific activities.54 Fortunately, team leaders typically had advanced degrees coupled with long experience and, once in the field, put together work plans that engaged their staff in council building, improving local service delivery, identifying focal points for introducing gender issues, and seeking out civil society organizations. However, the activities of the LGP teams were largely independent of one another; there was no standardized method for developing councils; and no tried and true suggestion of how to improve the service delivery of specific departments at the local level (such as water, sewer, or solid waste). Instead there were many separate initiatives (often worthwhile). Monthly team leader meetings convened to present new materials and to share lessons learned, however, as security conditions worsened, the holding of these meetings presented significant logistical challenges. Contractually the LGP was to coordinate with the regional military commands in their areas. Initial monthly and weekly reporting demonstrates that coordination took place. It is uncertain whether this coordination continued. In some cases, CPA appeared in the field after the LGP had been deployed, and they would have been the coordinating body. When CPA left and sovereignty began, the framework for coordination changed and it appears that coordination became more haphazard. It is also true that the rotation of military commands meant that the LGP teams needed to brief new arrivals and ‘restart’ the process. The evaluators found through interviews that some military units found LGP's level of coordination and communication lacking.55 51 Sulimaniyah had one LGP representative. 52 A possible exception would be the USAID program in Viet Nam. 53 SOW in the contract agreement; and the PMP of May-July; The LGP contract envisioned that work plan changes would occur and required quarterly work plan update reports – these do not seem to have been prepared. 54 An LGP ‘Toolkit’ was mentioned, but we were unable to locate it. 55 As the numbers of agencies in the field increased the time spent communicating and coordinating became unproductive drawing scarce resources from core tasks. An analogy in software engineering United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 92 B. Reporting Requirements Overall 10 reporting requirements are identified or implied in the contract. The evaluators attempted to validate each of these contract report requirements: 1. A report documenting their assessment of local governance and civil society issues: the evaluators did not find evidence that this was completed no later than 3- weeks after arrival in permissive areas. The implication is that a report would be necessary each time the LGP moved into a newly permissive area. It was probably deemed not practical to do this. The requirement extended to providing continual updates (but no time frame was suggested in the contract). The intent of the assessment appears to have been to inform the rollout of the project to new areas expressed through the quarterly work plan. No evidence was found that an initial report assessing local governance and civil society issues had been prepared (lack of evidence is not sufficient to say that LGP did not prepare such a report). 2. A quarterly work plan was to have been provided within three weeks after entering a permissive area: as above, no evidence was found that this report was prepared. 3. Identify appropriate functional and ‘legitimate’ Iraqi leadership: the requirement implies that a report for each permissive area would be prepared within one-month of in￾country arrival. Alternatively it may have only intended that such people are identified as local focal points for partnership. Each LGP team would have initially identified working partners. The evaluators believe that LGP complied with this requirement. 4. Report on the efforts of the LGP to coordinate with local authorities regarding the delivery of assistance: the requirements for this report were to have been agreed between LGP and the CTO. The evaluation was not able to determine whether the CTO required this report. 5. Within 90 days of arrival in permissive areas, the Contractor shall prepare a report which provides a framework, based on collaborative development, for decentralization of authority to local governance structures and shall update the framework with each quarterly work plan submission and as ongoing research is undertaken. The decentralization framework shall articulate political and administrative functions as well as intergovernmental finance recommendations as appropriate: an initial report on the framework for decentralization was not found. However, the importance of this activity to LGP is evidenced through their growing concern for clarity in the law that enabled decentralization. Given its importance, it should have been reported quarterly in conjunction with a quarterly work plan (also apparently not done). However, the monthly reports included information on the framework as reported individually by the LGP teams concerned with issues of authority for councils vis-à-vis local departments/directorates. Concern grew and was recognized by LGP management in the formation of the Policy Advisory Unit (PAU). The continuing speaks of the ‘mythical man-month’ noting that as the number of people deployed on a complex task increases, the time spent in communication and clarification with them reduces productivity. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 93 importance of the framework resulted in the formation of the Policy Reform Team, the Decentralization Pilot Program, the formation of Local Government Associations in Iraq, and an elevation of this activity in LGP 2. The framework continues to be under review and is reported on regularly. The evaluators found that LGP was in full compliance with the substance of the requirement. 6. Weekly Consultations: intended initially as weekly informal meetings that would support the content of the Monthly Reports and to help shape the Quarterly Work Plans, these instead became weekly structured reports. The reasons for this are not clear, but perhaps they are related to limited USAID staff and restrictions on travel that made meetings difficult. Weekly informal meetings (with an implication that time was available) were apparently not realistic. LGP did prepare regular weekly reports with substantial input from the field. The documents were available to the evaluators, and found to be a valuable resource in determining project activities. Evaluators held that LGP was in full compliance. 7. Monthly Reports: LGP prepared monthly reports of activities as described in the contract and were in full compliance. The monthly reports were the main venue for reporting the activities and progress of the LGP program. Annexes in the Monthly Report showed the status of the Grants program.56 A possible exception is LGPs reporting on accrued monthly expenditures that were to have been included in the Monthly Reports. However, reporting on monthly accrued expenditures appear until June 2003 in the monthly reports, but then there is a hiatus until such reporting reappears in November 2003. No explanation was provided for the hiatus. 8. Quarterly Work Plans: these reports were to have been submitted within 7-days before the start of a new quarter. The purpose of these reports was alleged to be to update the work plan and the framework for decentralization. The report was to include monthly fuel requirements for up to one-year of program implementation. USAID was to respond to the work plan within 5 days. No record was found for Quarterly Work Plan updates. 9. Final Report: a Final Report was prepared by the LGP.57 The format of the report was to have matched “accomplishments to the specific paragraphs of the Scope.” The report followed a different format, but appears comprehensive in its coverage. Evaluators judged the report to be in full compliance. 10. Proposing Indicators: at program startup, LGP was to develop and propose indicators for monitoring performance. This was done in full compliance when LGP issued its first PMP for the period May to July 2003.58 11. Performance Monitoring: within 30 days from signing the award, LGP was to have submitted its PMP (this was accomplished above under item 10). A second PMP59 56 After March 200 57 USAID/Iraq, Local Governance Program, Final Report, June 2005, prepared by RTI, Research Triangle Park, North Carolina, USA 58 PERFORMANCE MEASUREMENT PLAN, May-July 2003 59 Iraq Local Institutions Support and Development Program (Local Governance Project), Performance Measurement Plan, Baghdad, Iraq, September 2003 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 94 specified that the performance data for Strategic Objective 4.1 are to be gathered through opinion survey administered quarterly. These are the quality of life surveys (QOL) that were conducted on approximately a quarterly basis beginning in October 2003. This activity provided the baseline data for the measurement of progress throughout the program. However, these data were not sufficient to inform the “program feedback loop” envisioned in the contract. Further development of the Results Framework took place, but this resulted in a far too complex program structure with indicators that were near impossible to measure. A further requirement was for a Performance Monitoring Report that was to detail the benchmarks towards achievement of ‘performance indicators and results’. This was to be a semi-annual report. The first, and as far as could be determined the only, attempt to do this began in November 2003. At that time performance indicators were in doubt so the collection of data to inform the report used key informant interviews. A structured interview document was prepared (see Annex 9 Midterm Review Key Informant Interview Form), and the LGP monitoring and evaluation team visited most of the LGP teams in the field to complete these interviews. That team attempted to validate the extent of council development, and the training provided. Similarly it sought to understand what measures were being taken to improve service delivery. The data collected were primarily qualitative data. The IBTCI evaluation team was unable to locate the Midterm Performance Monitoring Report, but were assured that at least one semi￾annual report had been completed. The IBTCI evaluators believe that there was at least partial compliance on this deliverable. Daily reports were not required in the contract agreement; however LGP was urged to provide input for USAID’s own daily report. Preparation of these reports were time consuming and necessitated the use of report writers at HQ. The daily, weekly, monthly, and annual reports provided volumes of material. However, simple summary reporting was absent except for RRG, the CDP/DDA and council formations. Little evidence was found to demonstrate that a systematic attempt to measure performance was made. This may be explained by events occurring in the context of the LGP. The slow development of the Results Framework and infrequent updates of the PMP after the initial year meant that performance indicators were not put in place until late in the project cycle; too late for them to be measured. The Statement of Work of LGP-2 (Section C) sets forth the following Program Objectives: “The objective of this contract is to obtain the technical assistance to support the development of democratic local governance in Iraq. The LGP 2 will emphasize increases in the “supply” of democratic governance by working primarily on the enabling environment for local governance and building the capacities of national and sub-national governments.” “The LGP 2 will involve modest efforts to increase the “demand” for democratic governance, while relying on CAP and other complementary efforts … to increase demand.” The contractor is directed to focus on the following activities: 1. Promote Policy Reform in support of Local Governance; 2. Support clarification of the roles and responsibilities of different levels of government; 3. Promote increased efficiency of local service delivery; United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 95 4. Assist in the development of regularized mechanisms of citizen participation in governmental decision-making processes; and 5. Capture learning through systematic study and reflection. Activities that were nascent in LGP I are refined in LGP II. These include: work on local government code (charters); the pilot decentralization programs, development of Local Government Associations, simplified work plans, CD ROM training modules; and toolkits. LGP 1 work with local councils and the ministries has put in place strong advocacy for decentralization. National and regional conferences have made decentralization a focal point. This has reached an advanced level in Baghdad with the PC legal committee and adopted charter. The way forward for decentralization should be clear before the Provincial Reconstruction Teams are in place. Without this we are likely to have a repeat of the frustrations experienced by the LGP teams. A special section on the Legal Basis for Decentralization has been added to this evaluation. This was done to highlight the main recommendation of this evaluation which is to ensure that the laws enabling decentralization are husbanded through the legislative process before the Provincial Reconstruction Teams start their engagement with the councils. LGP 2 is well positioned to achieve this. C. Security Although victory in the war by the Coalition Partners against the Saddam Hussein regime in Iraq has been achieved, serious security concerns have pervaded the work programs of RTI from the outset of LGP-1 to the present time. The limitations imposed on the work programs of all USAID contractors in Iraq are documented in the assessment report prepared by Management Systems International ,”USAID Assistance to Iraq: A program-Wide Evaluation, Second Draft, First Year of Operations, June ,2004.” The broad perspective of the security situation in Iraq and how it affects programs are outlined in Annex 10. However, there were specific events for the LGP that caused reversals and setbacks in the governorate level programs. Some of these are stated in the text box below. In addition, there was a continuing threat in Dhi-Qar, Maysan and Basra from different militias wishing to demonstrate their power, or simply wanting to occupy the LGP office space. Kirkuk, November 2003, The Kirkuk LGP team experienced interruptions in its work for several days due to a suicide bomber who exploded a vehicle into the Patriotic Union of Kurdistan (PUK) party headquarters, 100 meters from the government building where the team conducts business and meets with the community. November 2003, A bombing in Nasiriyah killed 27 people and occurred next door to the LGP offices. The office was severely damaged and occupants narrowly escaped fatal injury. For two days the team was confined to its compound. Improved movement has been possible in the days following the Eid. Karbala, Jan 2004 “The activities of LGP/ Karbala were severely hampered by the attack made on the Governorate Building, where its offices were located, by a suicide car bomber on December 27, 2003. The offices were rendered unusable, and a decision was made— based on a security assessment that the Governorate Building would continue to be vulnerable to terrorist attacks—to re-locate the offices rather than renovate those that had been occupied.” Baghdad, 2004, LGP team was fired upon in an apparent ambush in Baghdad. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 96 Al Anbar, March 2004, During this reporting period, the Al-Anbar LGT was confined in the coalition forces military base for security reasons. However, the LGT did hire four local staff for the LGP office in Ramadi while also hiring and training 34 civic dialogue facilitators and section leaders. LGP began furnishing the team’s offices at the Governorate Building and deployed two expatriate staff members to the Army Base in Fallujah. LGP has withdrawn the Ramadi team from Ramadi and Fallujah for at least 60 days. The LGT has trained its Iraqi staff to conduct specific LGP activities and is in close contact with them by email. However, the LGT has been unable to get adequate reporting from its Iraqi staff Baquba, April 2004 “Security Situation”, The insurgency in the governorate remained very high after the first week of April. The CPA compound was locked down on April 7 to preclude any movement of international staff, and all local staff members were evacuated for one day from the CPA compound. LGP evacuated local Iraqi staff from both the villas except for the security guards. The local staff members were advised to be extra careful, to halt all work-related movement, and to reduce any personal movement outside their homes. LGP Iraqi staff have been receiving threats from the insurgents. These warnings affected the DDA team; the DDA team leader and some 4 facilitators resigned from the job because of these threats. The insurgents were handing out two types of leaflets praising the Ba’athists and motivating people to fight against the Coalition Forces. To reinforce the threats, insurgents threw one hand grenade into the LGP office in Villa-I. It caused no losses to the building, property or life. Threats to the property and local staff continue.” Baghdad, Feb 2005, Central, Baghdad, 2-6-05: Members of LGT’s Democratization of Local Government (DLG) Group met with the Baghdad City Council during their regular scheduled meeting. The principal order of business was to elect a new chairman and vice-chairman who will serve in their capacity for the rest of the period leading up to local elections. The new Provincial Council, to be formed upon vote count completion, will decide on the rules and timing for local elections. As noted last week, meetings with the various councils have become extremely difficult due to security concerns in the lead up to the national elections on January 30. On Saturday February 5, a member of the Baghdad Provincial Council, Abbas Hassan Waheed Al Kinani, was gunned down outside the offices of the Baghdad Governorate as he was exiting his car. Concluding remarks on the Importance of the LGP: “The US really begins to win when it can find ways to match the military, political, economic, and governance dimension. Creating a real partnership with allies means also respect; it doesn’t mean creating proxies or tools. It means recognizing that creating the conditions for effective governance and police are as important as the military. So is the creation of effective ministries. Iraq shows all too clearly that if you focus on the ministry of defense and ignore the ministry of the interior -- and even more difficult if you ignore the ministry of finance -- this just doesn’t work. In most places, the actual counterinsurgency battle is local and as dependent on police and effective governance as effective military forces. In hyper-urbanized areas, which represent many of the places where we fight, the city is the key, at least as much as the national government. And, incidentally, Iraq has already shown time after time that it is difficult to sustain any victory without a lasting presence by local police and government offices.”60 60 Center for Strategic and International Studies, 1800 K Street, N.W., Suite 400, Washington, DC 20006, “The Iraq War and Lessons for Counterinsurgency”, Anthony H. Cordesman, Arleigh A. Burke Chair in United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 97 Annex 1: Thi Qar Governorate Local Governance & Management Systems Saturday, January 10, 2004 I. Diagnostic Results A. Organizational Relationship Due to the war and transition period, there is no clear definition of relationships of one department with the others. For instance, the relationship between department and governorate office, between department and city council, the relationships between department and central ministry, between departments and CPA. Government agencies try to follow the pre-war situation and at the same time accommodate new stakeholders in power. It causes confusion up to a certain extent but the government agencies manage to adjust and float. Recommendation: ¾ There should be clearer definition of roles and functions of various government agencies in the future administrative and political structures, preferably in the form of local government code or administrative law. 1. Vertical Relationship with Central Ministries (Annexes 1 & 2) Departments in the provinces, in the absence of a clear written order, still try to maintain loyalty to the central ministries. This direct attachment has been existing even before the war. The provincial departments have to wait for the orders and instruction from their central ministries in Baghdad. As a consequence, horizontal relationships in terms of cooperation and decision among departments are difficult to nurture. For instance, the budget of each individual department goes to the central ministries in Baghdad. The total amount of budget requirement for the entire province is not known at the budget proposal stage. Treasury department gets to know them individual budgets upon approval by MoF. Then the Treasury Department records the budget of each department in their accounting books, to ensure that actual expenditures do not exceed the budgeted figures. Recommendation: ¾ The government structures and relationships need to be defined, particularly the relationships between or among departments. Strategy, With the Assistance of Eric M. Brewer and Sara Bjerg Moller, Working Draft, Revised: March 16, 2006 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 98 2. Other Sources of Revenues As of this date, the departments have not other major sources of revenues or operating budgets other than Baghdad and CPA. The tax department and its records have been burned down during the war. The staff and officers of the tax department no longer report to work (although still receiving salaries). And furthermore, except for a few, people no longer pay taxes and dues since the war. Thus, provincial government agencies have become totally dependent on Baghdad. The concept of financial self-sustainability of earning departments like water and electricity is no longer relevant. Recommendation: ¾ Special task force (composed of international and local staff) should be established to identify the possible sources of revenue and to promulgate the needed laws and regulations to institute them. 3. Relevant Management Skills (Annex 3) Except for a few, there is no medium term plan and carefully crafted capital projects are in place. Usually, departments provide ‘wish list’ of items needed, which are not based on certain assumptions or baseline data. Consequently, this makes it harder to allocate, more rationally, the limited financial resources. Also, because the capital projects are not pretty well defined, it becomes more difficult for departments to implement projects on time resulting in the slow budget implementation. Recommendation: ¾ Government personnel should be trained in project identification and planning, project prioritization and selection, budgeting, and project design and implementation. 4. Project Management System There is lack of standardized and forma project management and monitoring system in place. Most capital projects that have careful studies and monitoring are those funded and assisted by international organizations. Locally managed projects need some sort of physical monitoring system to ensure that financial and physical inputs are properly utilized in the projects. For instance, building control is a department in the group of Works and Municipalities that was newly established to monitor construction of buildings. Its technical capacity is basically hampered due to lack of staffing; only the manager is the civil engineer and the rest are either security or administrative staff. Recommendation: ¾ Project management system guidelines should be promulgated. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 99 5. Procurement Guidelines There seems to be no clear standardized guidelines in procurement, bidding, and evaluation of bids. Most of the bidding and evaluation committees follow different rules, coming from Baghdad, or have evolved through the years. As a consequence, the bidding and evaluation may not be that transparent. Recommendation: ¾ Procurement guidelines should be standardized and have the force of the law. 6. Financial Controls and Reporting The auditing tools and techniques are based on the 1990 law. And the financial systems are based on 1940 law. These have been consistently followed through decades. The last technical training was a long time ago, in the 1980s. The staff and officers may no longer be updated on the international standards of accounting and auditing. Furthermore, there is no certification exam in the government, to certify accountants as in other countries. The financial reports generated monthly are based solely on transactions for the month and the previous months. No balance sheet is generated as required by international accounting standards, making it difficult to monitor fixed assets (if there are any) and accrued transactions. Because records were burned during the war, the Treasury Department had to start its financial history a few months after the war. Recommendation: ¾ The accounting and audit personnel should be updated in internal accounting standards. This should also have an updated accounting law. OVERALL RECOMMENDATION: ¾ Develop the local governance structure, preferably in the form of a local governance code. ¾ Develop local governance, planning and budgeting framework (basic features). ¾ Determine which portion of the framework would require immediate technical assistance; ¾ Design short, simple training programs, to address those parts of the framework that require immediate assistance; United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 100 B. Work Plan 1. Assess continually local governance & management systems This will require further interviews and observations of the present status of structures, and management systems. The team will visit other remaining city councils, provincial councils, governorate office, and departments. 2. Develop the planning and budgeting framework (Annex 4) This will require developing a chart that will illustrate the desired planning and budgeting features of the management systems. 3. Develop basic parliamentary rules in conducting council meetings. This will require developing brief presentation materials for presentation during informal discussion with council members on how they could improve the conduct of their sessions. The presentation will be one council at a time, slowly building up on the guidelines as we further conduct each group of participants. 4. Develop basic planning and budgeting concepts and discussion during meeting with one or two departments. This will involve developing basic and simple planning and budgeting concepts and discuss them initially with one or two departments. This will require slow spreading of continual improvement of the guidelines as we further conduct each group of participants. Eventually, councils will also have this kind of introduction to planning and budgeting concepts. 5. Attend conferences on management and public finance. Before the start of their field work in December 2003, the RTI Team has recognized the need to have a standardized approach in public finance improvement opportunities. An attempt as made to communicate with other public finance experts, but only a handful replied. On in the late December 2003 and January 2004, that everyone has recognized that inherence limitation of the LGP that everybody might be going in many several directions, doing their own thing. Starting this middle of February, it has been suggested that team leaders and public finance experts and those involved, should meet in Baghdad to develop a common strategic management and finance strategic vision and approach. 6. Assess needs in other areas of financial management. This will involve in assessment the relevance of generally accepted accounting principles (GAAP), project accounting, best practices in procurement, and project management. 7. Determine whether each council and department has complied with listing of their capital item needs. LGP has specifically mentioned that there could have been lost or destroyed items during the war; or the capital items have never been there due to ‘neglect’ by the prior regime. As a starting point for the capital budgeting concept, each government agency should list their capital item needs, e.g. vehicles, office furniture and equipment. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 101 8. Update continually the planning and budgeting framework according to new developments. This will involve being on the lookout for new developments in the political, constitutional, and administrative aspects in the entire Iraq. 9. Conduct workshops (as in the case of Local Governance Fund Workshops for City Councils) to slowly introduce the best management practices. The LGT-Nassiriyah Team (specifically Dariusz Milka, Antonio Salindong, Haider Abd Al Hussain Shehab) conducted 6 (six) workshops for Fund Committee members and Local Program Managers, at both the district and provincial levels, either at the CPA or RTI office. Thi Qar Province is rich in history, dating back to 7,000 years back to the Zikurat in Ur, where Abraham lived. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 102 Annex 2: Training and Development Initiatives Option Year One I. Baghdad Local Governance Team Training and Development initiatives will comprise a core element in the capacity building activities of the LGT. These initiatives will also include study tours and short-term subject matter experts in a TDY capacity. 1. Ten Day Study Tour to Philadelphia and Washington, D C to visit several Training Centers for 5 Trainers/Management from the Amanat.$ 45,000. 2. Bring a TDY consultant to Baghdad for one month to offer leadership/management courses for Senior management at the Amanat. $25,000 3. Four one day training workshops for trainers in Baghdad. $ 4,000. 4. Public colloquia, youth and women leadership development and training programs in Civic Participation. $75,000 5. Study Tours to support introduction of a computerized property management system. $ 30,000 6. Computer software and training for Audit Department in accordance with international standards. $ 50,000. 7. Study Tour to study financial systems and the management and operations of “best practice” finance departments. $ 40,000 8. Conduct study tour to Greece to see examples of consolidated regional service operations for water and sewer. $45,000 9. A study tour for middle management staff from BWA & BSA to Alexandria, Egypt as a second step of the study tour conducted in March, 2004. $40,000 10. Conference/workshop for up to 50 people on Billing and Collection systems for Municipal Services. To be held in Baghdad at a time/venue to be selected. $10,000 11. Conference/workshop for up to 50 people on Decentralization/Privatization of Municipal Services. To be held in Baghdad at a time/venue to be selected. $10,000 12. Training for IT Department run, and maintain the financial management systems network. $10,000 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 103 Annex 3: Evaluation Example from Al Muthanna Please provide the following information in the table below regarding your work with government offices and neighborhood councils: 1. How many offices have actually prepared budget based on your assistance/training? At which administrative level (district or governorate)? 7 program performance budgets prepared for ministry departments within the governorate 2. How many have actually done an audit? At which administrative level (district or governorate)? We have performed 5 audits of governorate departments and banks through the Bureau of Supreme Auditors 3. How many Employment Service Offices (promoting merit-based hiring) are fully operational? At which administrative level (district or governorate)? Information on this is not available. 4. How many Complaints Offices (recording and responding to citizen concerns/complaints) are fully operational? At which administrative level (district or governorate)? Professional complaints office opened in the Water department, many other offices and levels of government claim to have complaint service but they do not exist in any real terms. 5. How many neighborhood councils are established? Number of Offices prepared budget Number of Offices done an audit Number of Employment services offices fully operational Number of Complaints offices fully operational Governorate 7 5 N/A 1 – Water department. City 0 0 N/A 0 District 4 0 N/A 0 Note: Curriculum developed for Governmental Accounting, Auditing and Financial Reporting for the Technical Insitute for ongoing Capacity Building; classes to be part of regular ongoing Technical Institue curriculum. Classes Start March 4, 2004-03-02 Curriculum Developed for Training Municipal Finance Officers for all governate, city and nahia within Al Muthanna; ongoing training to be initiated March 6,7, 2004-03-02 Governmental Budgeting Training provided for Four District Councils Governmental Capital Budgeting Training provided for 33 individuals representing all ministry financial departments within Al Muthanna United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 104 Annex 4: Questionnaire Administered in Kirkuk to Assess Training Provided The purpose of this questionnaire is to assess training that has been received by the DG and the staff of the following Kirkuk directorates or departments. • City Engineers Office (Aziza, City Engineer) • Government Construction and Irrigation/Water (Abdulkadir) • Municipalities Directorate (Salar) and (Jamal) • Water Resources (Shahab) • Roads and Bridges (Abdulkarim) The DGs names for the directorates where the training occurred are shown above in brackets. Try to contact all of them and ask to speak to them and their staff about the LGP training they received. It is known that they received management training from LGP. QUESTIONS FOR KIRKUK INTERVIEWS LEAD IN STATEMENT: I am working for a company that is doing an evaluation of the Local Governance Program. We are interested in understanding the results of assistance that may have been given to your department by the LGP. We are also interested in any additional work that we might consider to help you accomplish your work.. It is in the light of helping us to better understand how we might best assist you that we would like to ask you several questions about your contacts and experiences with the Local Governance Program. May I have your permission to proceed to ask you several questions? This should take about 20-30 minutes. Respondent Agreed Yes No If no, ask for reason and record it here:_________________________________ Are you aware of the Local Governance Program? Yes No Yes,___________________, if No probe to see if they are aware of RTI or USAID Tick here if they are aware of RTI_____or USAID_________ If no, continue with the interview and see whether they received training of any kind. Let me begin by asking you some background questions about you and your department/directorate. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 105 Name of the Department/Directorate ________________________________________________________________________ Geographic Area of responsibility for the services you provide (example: name of municipality, district, sub-district where service is provided) ____________________________________________________________________________ ____________________________________________________________________________ ________________________________________________________________ What is your present position in the Department/Directorate? ________________________________________________________________________ How long have you been in your present position? (number of years, if less than one write ‘0’ ________________________________________________________________________ How long have you been at your current pay grade/rank? _____________________________________________________________________ Does training that you might receive improve your chances for advancement? ______________________________________________________________________ How many professional staff do you oversee? How many administrative staff? Have you ever received training to improve the operations of your department/directorate or to improve your own skills? ___________________________________________; if no GO TO END If yes, did the LGP, RTI or USAID provide this training? __________________________________________________________________ If not LGP, RTI or USAID probe to see whether in fact it may have been them (discuss this) What other institution provided training? __________________________ How were you contacted about the training (tick all that apply)? Method of contact 1 Preliminary visit by LGP 2 Training was requested by my department/directorate a What training was requested? (explain) b How did the need for training arise? (explain) United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 106 Method of contact c Did they understand the reasons and background of your training Yes No request? 3 Directed to attend by superiors 4 Other_________, (explain) Was there a preliminary selection of persons who should attend the training? ___________________________________________________________________ If yes, what criteria were used to select candidates? ___________________________________________________________________ Type of training provided (tick all that apply): Type of Training Tick all that apply 1 Management 2 Strategic Planning 3 Billing Systems 4 Procurement 5 IT, Computer Training 6 Other, specify: Was the training in a formal classroom setting? (tick one) Yes No Was the training ‘learning by doing”? (tick one) Yes No Were you given materials for self instruction? Yes No Did you receive a certificate of completion at the end of the training? (tick one) Yes No What was the duration of the training that you received? __________ Turning now to the quality of the training you received, how would you rank the presentation of the training on a scale of 1 to 10, where 1 is the lowest and 10 the highest rank (circle one). Lowest 1 2 3 4 5 6 7 8 9 10 Highest United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 107 Turning now to the importance of the training you received, how would you rank how important the training has been to your current position? Rank on a scale of 1 to 10, where 1 is not important and 10 very important (circle one). Not important 1 2 3 4 5 6 7 8 9 10 Very important Would you advise colleagues in similar Directorate positions that they should recieve the same training? Rank from 1 to 10, where 1 is that you strongly disagree, and 10 is that you strongly agree (circle one). Disagree Strongly 1 2 3 4 5 6 7 8 9 10 Agree Strongly Was the training helpful to you in improving your job performance, and if so, in what way? ____________________________________________________________________________ ____________________________________________________________________ Was there a promise of further training for either you or your staff? (tick one) Yes No Are you currently still in contact with the LGP team? (tick one) Yes No What suggestions do you have for LGP so that the training assistance we provide can be improved? ____________________________________________________________________________ ____________________________________________________________________________ ________________________________________________________________ END. Thank you for your participation. We will be interviewing other officials here and in other places in the hope of improving the services LGP provides THANK YOU AGAIN! United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 108 Annex 5: Provincial Council Composition Achieved through Caucus in Maysan Districts & Sub￾districts (7) Women (3) Tribes (7) Minority Religious Groups (3) Political Parties (7) Interim Leadership (4) Municipal Councils (3) Educated People (2) Represented Groups Provincial Administrative Council (40) Nominees for Interim Governor People of Maysan Interim Governor Interim Police Chief Police Committee (15) (7) (1) (7) ?? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 109 Annex 6: Council Activity and Evaluation in Diwayniyah From: Andre Pecikiewicz [mailto:apecik01@lgp-iraq.org] Sent: Tuesday, March 09, 2004 11:05 AM To: lcrave01@lgp-iraq.org Subject: RE: M&E performance indicators - Assistance requested Lamar, I guess we are getting over-swamped with documents. Let me comment on the M&E performance. PART 2 Basically, we do not have the data to illustrate some responses to the questions, but there is some descriptive that may be useful. Sub-IR 4.1.2.1 Table A. Columns: 3 – we have councils in every of 15 biggest cities around the province, and in Diwaniya there is 37 neighborhood councils. Some of these councils have been established in the period of April – June 2003, right after commencement of military actions, some councils were established during summer 2003. Basically they act in a kind of an “undercover” manner, many people in cities other than Diwaniya simply do not know about the local council. There is some resentment to the manner the councils were established and some voices calling for additional elections can be heard. 4 – The Provincial Council is built of 40 members representing all major cities, women groups, tribes, some CSOs and NGOs. Its composition seems to be established and all 40 members are on board. City councils comprise of 20 members chaired by a president. In many cities (including Diwaniya) the councils comprise of less than 20 members, some have resigned or withdrawn during the last period. Attempts to organize additional elections have been made, but the results were none or not fully satisfactory 5 – Only the Provincial Council has four member-women representing women-CSOs. To my knowledge there are no women on local councils or neighborhood councils. 6 – within the province the women constitute less than 1% of total council members (neighborhood councils have not been taken under consideration). Tribes are represented on both provincial and the local levels. 7 – Having visited most of councils around the province, my estimate is that about 50% of them does not have adequate meeting facility – the council buildings have been looted, ruined and brought to an unusable condition. Meetings have been held in court rooms, classrooms or police stations. If we take the Diwaniya council meeting facility, none of local councils has any comparable ones. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 110 8 – basically the furniture and office equipment go parallel with meeting facilities. Usually it is only a table and some chairs. There are no computers, printers, fax machines, no internet. Some of councils do not have any telephone lines, so getting in touch with them is close to impossible. Many provincial and local council members do not have private telephone lines; keeping contact with them can be established by intermediary services of a friend, neighbor and other members of family who have telephones. Gender stage – 1 (in most cases no women or disenfranchised groups are represented in local government) OCAT stages of organizational development – in most cases I would put “nascent” In order to get them into “emerging” stage we shall deliver a number of training similar to training carried out for Provincial Council hoping to help establish structures, practices, proper use of financial resources and responsiveness to basic service delivery. OCAT rating scale – needs urgent attention Sub_IR 4.1.2.2 Table. The councils meet regularly, nevertheless, the frequency of meetings differs. The Diwaniya council meets once a week, councils in smaller cities have fortnightly meetings or even once a month. All depends on what business is seen as urgent and on the commitment of the council chair and its members. Some councils have some rules of conduct; many still operate as rather a social meeting than a professional council meeting. The training we are offering should put most of them on the business approach to council meetings. An identification of key citizens’ inputs regarding service delivery is difficult to prove. Nevertheless, the councils understand well their needs weaknesses and present them to all parties recognized as sources of assistance. We kept receiving letters in this respect; numerous needs have been identified during meetings in November and December (I sent you a report on the meetings by the end of December). Very recently, while setting training of local councils, I transmitted to all 15 local councils around the province a very basic questionnaire on their operations, problems, etc. The aim of the questionnaire was to learn more about problems in managing the city and to provide ideas for future training. The responses are expected in the second half of March. They will be analyzed to constitute a basic database on councils. Sub-IR 4.1.2.3 Cheers - Andre United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 111 Annex 7: Neighborhood/District/City/Governorate Councils Region Governorate District Neighborhood 83141 1.1. Name of the Concil/association: Al Maymouna Qada Council 1.2. Address: South 1.5. At what administrative level does it reperesent its constituencies District 2. Scoring Table Maysan Al Maymouna N/A 5 = Acceptable, needs maintaining Nieghborhood/Distric/City/ Governorate Councils 1. General Information 2 = Needs major improvement 3 = Needs improvement in limited aspects 4 = Acceptable, room for some improvement Please score the end states for sub IR indicated on worksheet "End state" of this workbook, using scales from 0 – 5. The scales are interpreted as follows: 0 = Not applicable 1 = Needs overall improvement 1.4. How many people does this council represent United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 112 Annex 8: LGP 1 Deliverables, Reports, Monitoring I. Deliverables LGP 1 The contractor shall be responsible for the following deliverables in addressing the functional areas above: • The Contractor shall be prepared to immediately deploy teams to the region, as instructed by USAID, and to further deploy into Iraq as directed and when a permissive environment has been established. • Upon arrival to the field, the Contractor shall establish coordination with appropriate military commands and other agencies to move forward relief and reconstruction assistance where security considerations permit. • As soon as is possible, but no later than three weeks after arrival in permissive areas, the contractor shall have prepared a report documenting their assessment of local governance and civil society issues in those geographical areas where security concerns have or will soon allow for engagement. This document will be continually updated as collaboration and field-work commence and the geographical scope of this document will presage the geographical engagement of the Contractor in the field. • The assessment shall inform a quarterly work plan which will be provided within three weeks after entering a permissive area. USAID shall respond to the submitted work plan within five days. • Within a month of in-country arrival the Contractor shall identify appropriate functional and “legitimate” Iraqi leadership with which to partner on relief and reconstruction programs in permissive areas. • Within a month of in-country arrival the Contractor shall coordinate with local authorities for the delivery of a range of education, health, power, small infrastructure, food distribution, and other reconstruction assistance and coordinate such assistance in permissive areas. General details of this coordination will be included in reports to USAID, as agreed between the contractor and the CTO. • Within 90 of arrival in permissive areas, the Contractor shall prepare a report which provides a framework, based on collaborative development, for decentralization of authority to local governance structures and shall update the framework with each quarterly work plan submission and as ongoing research is undertaken. The decentralization framework shall articulate political and administrative functions as well as intergovernmental finance recommendations as appropriate. II. Reporting Requirements In addition to providing the services and reports described in section 5 (Deliverables), the Contractor shall also submit the following reports and other deliverables: • Weekly Consultations: A continuing requirement of this contract will be close collaboration, reporting to and engagement with the appropriate officers of USAID. Consultations will occur on a weekly basis at a minimum to discuss the internal specifics and external contexts of the on-going work. It is anticipated that these informal but United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 113 regular consultations will support the Monthly Reports and help shape the Quarterly Work Plans. • Monthly Reports: The Awardee shall provide to the CTO, within 5 days after the end of each month, a report on the activities undertaken during the month. The monthly report should seek to be a brief, lucid, description of the activities, with emphasis on issues that have arisen, impacts made, constraints encountered, and suggestions for additional actions that might be taken. The monthly report should also include the Contractor’s accrued monthly expenditures. The CTO is responsible for transmitting this information to the USAID financial management office responsible for the contract. • Quarterly Work Plans: Quarterly Work Plans shall be submitted within seven calendar days before the start of the new quarter. The scope of the quarterly reports will depend upon the extent of permissive areas. The first quarterly work plans shall be submitted within three weeks of entering a permissive environment. The work plan should include the estimated monthly fuel requirements for up to one year of program implementation, in meeting all program objectives within the contract. USAID will respond to the work plan within five calendar days. • Final Report: The Contractor will prepare a final report that matches accomplishments to the specific paragraphs of the Scope. The final report will be drafted to allow for incremental improvements in the process, both generally within USAID and specifically with respect to this assistance mission. III. Performance Monitoring/Indicators • Proposing Indicators: At program start-up, and subject to USAID approval, the recipient will develop and propose indicators for monitoring performance. • Performance Monitoring: Within 30 days from the signing of the award, the recipient shall submit for USAID approval a Performance Monitoring Plan. The plan will necessitate the conduct of a baseline study to develop the required baseline data for the measurement of progress throughout the program. The plan will include methodology on how data will be collected, interim and final targets, and a timeline for collecting data. Data should be collected for all indicators developed by the recipient. The Performance Monitoring Plan shall provide for periodic evaluation by the recipient of the impact of the various program components and operate as a “program feedback loop”, i.e., impact evaluation/feedback/fine-tuning of component implementation/new activity cycle. The recipient will submit a semi-annual Performance Monitoring Report that details benchmarks toward achievement of performance indicators and results, the data collected and the method of collection. The first Performance Monitoring Report will be submitted no later than 30 days following the end of FY 2003 U.S. Government fiscal year, with reports submitted semi-annually thereafter. A specific format for the performance report will be proposed by the recipient to USAID for approval 60 days following program start-up. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 114 Annex 9: USAID – Iraq Local Governance Project Mid-Term Evaluation Questionnaire Governate:__Babil_________________ Date: Tuesday, November 11, 2003 OVERVIEW/CONTEXT: 1) Please provide the name of your governorate and its approximate population level (M&E team to provide). 2) When was the office opened (day, month)? Day 5 Month June 3) How many staff do you currently have, and what is your currently planned target staff level? Current: 14/60 Planned 17/75 MID-TERM EVALUATION OF ACTIVITIES, BY GROUP A. (4.1.1) LGP Assistance in Restoring and Increasing Access to Basic Services 1. # of residents accessing restored services (direct and indirect beneficiaries by location) Location Direct beneficiaries Indirect beneficiaries Governate Name of District: 1 2 3 4 5 6 7 8 9 10 2. cubic meter of water produced by location 3. cubic meter of water distributed by location 4. number of hours of availability per day United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 115 Location Water produced￾Cubic meters Water distributed￾Cubic meters Hours distributed per day Governate Name of District: 1 2 3 4 5 6 7 8 9 10 Supply of water and electricity better now than before the War 5. kilowatt of electricity generated by location 6. kilowatt of electricity distributed by location 7. number of hours of availability per day Location Kilowatts produced￾Cubic meters Kilowatts distributed￾Cubic meters Hours distributed per day Governate Name of District: 1 2 3 4 5 6 7 8 9 10 United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 116 8. metric tons of solid waste removed, etc. by location Location Metric tons removed Governate Name of District: 1 2 3 4 5 6 7 8 9 10 9. How many residents (i.e., direct and indirect beneficiaries) now have access to these restored services? Location Direct beneficiaries Indirect beneficiaries Governate Name of District: 1 2 3 4 5 6 7 8 9 10 10% of Hillah is covered by sewer connections, but just half of this is working. The 90% that is not covered by sewer connection use septic tanks (20%), while the remainder have inside toilets, but illegal connections to open sewers. The centralized treatment plant only has a capacity to cover 10% of Hillah- there is no excess capacity. Should sewer coverage be expanded the treatment plants would also need to be expanded. Bechtel has just begun to repair the sewer plants in Hillah and Najaf. A Health Sector Infrastructure assessment has just been completed. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 117 B. (4.1.2) LGP Support for Developing and Expanding Transparent and Participatory Local Governance 1. With which types of councils (and how many) exist in your governorate, and enter figures (number of councils) reflecting how many councils meet each criteria? i. Provincial/governorate councils __1___: a. with women and marginalized peoples adequately represented_NO__; • (if women are represented, how many on each council?) b. that are properly housed & equipped_YES_; c. that have instituted formal rules of conduct_YES; d. that have financial and administrative management systems in place_NO; e. that meet regularly (at least twice monthly)_WEEKLY. Council has 25 members. Formerly had one woman member who resigned. Tables, chairs, computers and air conditioner have been supplied. Formal rules of conduct have been provided. This is a shortened form of Roberts Rules of Order. 3 training sessions on rules of conduct. No financial management in place, but 9 sub-committees have been formed. They have no budget, only salaries. Grant funds go to departments. Selection caucus: 25 groups (associations) each sent 3 elected members to the caucus. Each of the 3 elected members presented themselves to the full caucus, explaining why they should be elected to the council. One of the 3 from each group was elected by the full caucus to become a member of the council. Council acts as the moral watchdog for the city (discuss issues regarding questionable western influence). Identify problems of service delivery, send to departments for action. 6 auditors were hired to audit government departments and present the results to the council. This has resulted in the council devoting considerable attention to problems identified, and to the council members asking the departments whether they have solved these problems. ii. How many cities are there in your governorate? 3 NOT INCLUDING HILLAH a. how many city councils are there in your governorate_3____; b. with women and marginalized peoples adequately represented_IN 1 ONLY_; • (if women are represented, how many on each council?) c. that are properly housed & equipped_ADEQUATE d. that have instituted formal rules of conduct_NO; e. that have financial and administrative management systems in place_NO_; f. that meet regularly (at least twice monthly)_YES iii. How many districts are there in your governorate_SAME AS CITIES___ a. how many District Councils are there in your governorate _____ b. with women and marginalized peoples adequately represented_____; • (if women are represented, how many on each council?) c. that are properly housed & equipped_____; d. that have instituted formal rules of conduct_____; e. that have financial and administrative management systems in place_____; f. that meet regularly (at least twice monthly)_____. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 118 iv. How many sub-districts are there in your governorate_12_; a. how many sub-district Councils are there in your governorate 6_; NAHIA b. with women and marginalized peoples adequately represented_NO_; • (if women are represented, how many on each council?) c. that are properly housed & equipped_NO_; d. that have instituted formal rules of conduct_NO_; e. that have financial and administrative management systems in place_NO_; f. that meet regularly (at least twice monthly)_NO_. Jim Mayfield has met with 2 of these councils and was not impressed. The original 6 councils were formed by the Marines when they were here. 100 delegates at the Nahia level were selected (what selection process?) and asked to determine whether the existing councils should remain or be changed (this is a way of validating the councils). Action is currently underway to add 2 more councils, with a further third soon after (don’t count these in the evaluation). v. How many Neighborhood Councils are there in your governorate _FOR LLOYD____; a. with women and marginalized peoples adequately represented_____; • (if women are represented, how many on each council?) b. that are properly housed & equipped_____; c. that have instituted formal rules of conduct_____; d. that have financial and administrative management systems in place_____; e. that meet regularly (at least twice monthly)_____. Jim Mayfield: neighbourhood councils are the most rewarding. The councils are aware of the traditional vs. the more modern approach to council membership. Selection is now to be based more on the ability of members rather than on family connection. Sees a real ‘hunger’ for participation, and sees the process as very positive. There are in total 62 neighbourhoods in the governorate – only urban areas have neighbourhoods (confirm this with Jim and Lloyd). Lloyd: Neighbourhoods are defined by the domain of the mukhtar (this may differ from the statistical office definition). The Prefect appoints the Mukhtar. The Prefect was formerly appointed by the Ministry of Municipalities, this mechanism no longer exists and it is unclear what, if anything will replace this mechanism. The Mukhtar is the distributor of government largesse, and the postion dates from the Ottoman era. He is an important link to the neighbourhood, and provides information and links to the higher levels of government. vi. Does your governorate have Tribal Councils YES; a. with women and marginalized peoples adequately represented_NO_; • (if women are represented, how many on each council?) b. that are properly housed & equipped_NO_; c. that have instituted formal rules of conduct_NO; d. that have financial and administrative management systems in place_NO; e. that meet regularly (at least twice monthly)_YES, TUESDAYS. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 119 Tribal councils are an elite group that are in ways similar to a Rotary Club. They are seen as a nascent farmer’s political party. Sheikhs are seen as moderate people who are pragmatic and hold pragmatic views about religion and governance (they are not radical). They represent about 60% of the region (however tribal groups do not represent a specific geographic region). They need to be taught how to win elections, how to bring members to the constitutional review commissions. Seen as solidly pro-American. vii. Does your governorate have any Women’s Advisory Councils NO _; a. that are properly housed & equipped_____; b. that have instituted formal rules of conduct_____; c. that have financial and administrative management systems in place_____; d. that meet regularly (at least twice monthly)_____. 2. Characterize the relationships (if any) between the different levels of councils that have been formed? I will read the following statements to you, please respond by indicating your level of agreement or disagreement. i. Councils at higher levels delegate responsibilities to those at the lower levels. Circle One Agree strongly, Agree, Disagree, Disagree strongly (there is no delegation), Don’t know Narrative explanation for your choice: Higher levels would like to control the lower ones. This is not seen as a positive outcome. The concept is delegation of autonomy from the higher levels to the lower ones, not as giving orders to the lower levels. ii. The councils at all levels are involved in the governance process by hearing citizen complaints and opinion, and making representations to higher levels of council governments? Circle One Agree strongly, Agree, Disagree, Disagree strongly (there is no involvement or representation), Don’t know Narrative explanation for your choice: Problems and complaints are collected, but not taken to the higher levels of government. Instead they are taken to laterally to the concerned departments for remedy. This is positive. Where budget requests are concerned it is seen as positive to make requests for budget allocations or provisions so that problems can be remedy at the local level. iii. Council members understand the relationship between the higher and lower levels of council representation? Circle One Agree strongly, Agree, Disagree, Disagree strongly (there is no understanding), Don’t know United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 120 Narrative explanation for your choice: ______________________________________________________________________ ______________________________________________________________________ ___________________________________________ 3. To date how many citizen-outreach events have taken place? Attempts to influence neighborhood elections Media campaign – media invited to meetings. Individual council members going out to visit neighborhood council meetings 4. Has the LGP provided technical assistance or training to develop and strengthen local government officials’ citizen outreach capacity and skills? Please provide: a. Number of training sessions conducted; Mentorship to Mayor/Governor b. Training themes; c. Number of participants per training; d. Number of women per training. 4 staff have divided up the 50 government departments in order to focus attention of developing the outreach and technical abilities 5. What technical assistance or training was provided by LGP to support the selection/election process? Please provide: a. Number of training sessions conducted; b. Training themes; c. Number of participants per training; d. Number of women per training. Was the selection process for the governorate council and the neighborhood council described by Jim due to LGP? There were three instances: for the governorate, the sub-districts (nahia) and the neighborhoods. 30 facilitators receive 1 hour of training everyday, then go out to the neighborhood. Part-time work, the 62 neighborhoods are their clients. There is one man and one woman on the facilitator teams. Women facilitators have been asked to organize neighborhood women’s committees. Will then ask these committees to join the 7-man neighborhood council. Workshop on how to be elected 6. How many selections has your governorate conducted to date (per city, district, sub￾district, neighborhood)_____? CHECK WITH LLOYD In general the selection process has been fair and transparent. Circle one response category. Agree strongly, Agree, Disagree, Disagree strongly (there selection process has not been fair and transparent), Don’t know United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 121 Narrative example informing your choice: Process was completely transparent, but the voter/participation remains low. Lloyd says that between 4-13% voter turnout was realized. Jim says 300-400 people show up for the neighborhood selections, when perhaps 5000 are eligible. 7. Has the LGP provided technical assistance or training to develop and strengthen local government officials’ leadership skills and effectiveness (i.e., governor, mayors, councils, executive department heads). Please provide: a. Number of training sessions conducted; NO b. Training themes; c. Number of participants per training; d. Number of women per training. 8. Has LGP provided or supported civic awareness training? YES Please provide: a. Number of training sessions conducted; b. Training themes; c. Number of participants per training; d. Number of women per training. Raja confirmed, and Sibel will report details based on earlier monthly report. Human rights, Gender issues, How to vote. Legal basis for the Women’s Group constitution. Will hold a ratification for the 800 member women’s organization. 9. How many citizens are aware of their governor? (from quality of life survey) 10. LGP’s work activities related to enhancing the participation of women and youth in local governance processes. a. What activities is LGP implementing to promote increased representation of women and youth in local councils? Jeevan:- Soccer leagues have been built around the neighbourhoods. Unfortunately, it was expected that the development of the league would be through grant funding, but the grant moneys expired before this could happen. Alternative sources were sought and found through the CPA. The CPA has a widescale program that covers 5 governates. Technical assistance has been provided. The assistance is seen as the entry point for lgp objectives regarding governance and building democracy. Raja: Assessment of CSO’s in Babil. Currently 11 of 16 will be brought into the training process. Outreach to women at the universities. Looking for Iraqi business women to form an association. Developing a womens student union, and perhaps radio stations? 11. Is the LGP providing gender-focused technical assistance and training? If yes, Please provide: a. Number of training sessions conducted; TAKE FROM RAJA b. Training themes; United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 122 c. Number of participants per training; d. Number of women per training. 12. Is the LGP providing youth-focused technical assistance and training? If yes, Please provide: a. Number of training sessions conducted; 1 b. Training themes; Planning and Budgetting c. Number of participants per training; 8 d. Number of women per training. 3 C. (4.1.3) LGP Support for Developing and Enhancing Effectiveness and Efficiency of Local Service Delivery 1. LGP work activities related to strengthening local government service delivery capacity. a) The executive departments are able to plan, budget, implement projects, manage resources and monitor performance. Circle your choice: Agree strongly, Agree, Disagree, Disagree strongly (there is no ability), Don’t know Narrative explanation for your choice: Workshops have been done and are well received. Dennis Moore (CPA) will be working with the Ministry of Finance and will start covering this area. Note that 78% of MoF funds that have been provided, have not been spent. This is a problem area. b) The local government is able to effectively plan and budget for the 2005 budget year. Circle your choice: Agree strongly, Agree, Disagree, Disagree strongly (there is no ability), Don’t know Narrative explanation for your choice: Jim: By March 2004, this should be the case c) How many executive departments have received training__5_? Please provide: i. Number of training sessions conducted ____6___; ii. Training themes____________________________________________; iii. Number of participants per training _avg. 6___total 30 ; iv. Number of women per training __4_______________. Check these responses. Dept of Sewerage, Dept of Water Supply, Dept of Agriculture, Dept. of Irrigation, Department of Education (2 weeks training). Engineers taught: operational maintenance of the department, technical analysis (rates estimation), bidding process, specifications, engineering drawing, design, technical supervision on site. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 123 2. LGP work activities related to promoting and expanding CSO participation in local service delivery. a) CSOs have access to local government executive data and information regarding service plans, budgets and performance. Circle your choice: Agree strongly, Agree, Disagree, Disagree strongly (there is no access), Don’t know Narrative explanation for your choice: Raja: This is a focus of their attention, trying to make this happen. Hillah is engaging a private contractor to do rubbish collection b) How many CSOs are partnering with local government to provide services locally? _ NONE middle of doing this. c) What basic public services are provided by CSOs and in what localities? NONE_______________________________________________________________ _____________________________________________________________________ _____________________________________________________________________ ________ 3. What technical assistance and training had the LGP provided to strengthen technical capacities of CSOs to deliver basic services effectively and efficiently? NONE, working on it. Training of Trainers. The Contractors Association of Hillah has been trained. 85 contractors were trained in how to submit a competitive bid. Please provide: i. Number of training sessions conducted; NONE ii. Training themes; iii. Number of participants per training; iv. Number of women per training. D. (4.1.3) LGP work activities related to enhancing local service delivery capacity by strengthening inter-governmental relations. a) Government liaison mechanisms exist for citizens and advisory councils to discuss service delivery, planning and development issues. Circle one: Agree strongly, Agree, Disagree, Disagree strongly (there are no liaison mechanisms), Don’t know Narrative explanation for your choice: United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 124 Neighborhood councils work with departments supported by facilitators who link councils to departments. b) What current inter-governmental relations generated difficulties are faced by local government executive departments? Financial distribution: 78%of funds allocated to the Ministry of Finance have not been disbursed. Ministry of Irrigation has $900K earmarked for cleaning the canals. There are unemployed willing to do this work. The Ministry in Baghdad has written to say that all cleaning work should stop because the water is too cold. d) What kinds of activities has LGP implemented to facilitate inter-governmental relations and local service delivery? Be specific if possible: numbers of events/trainings: ______ numbers of participants:__________. E. (4.1.4) LGP support for enhancing effective advocacy and participation of CSOs. 1) Participation of CSOs: a) How many CSOs are you aware of in your governorate? _____________ b) With how many CSOs is the LGP working with: all CSOs______, women CSOs__________, youth CSOs, __________, media/press-type CSOs _____? c) # of assisted-CSOs that are institutionally and financially viable: _______ d) # of CSOs registered with the local government:______ e) # of workshop/seminars conducted on a legal framework for the registration and operation of CSOs _______ Legal basis for registering CSO’s does not exist. Get Best Practices from Raja for the Constitution being developed for the Womans Group 2) LGP work activities related to enhancing CSO capacity to promote and expand civic awareness and political activism a) How many CSOs are conducting public awareness workshops, seminars and conference on democratic values and practices? _______ What training was provided? ______________________________________________________________________ _____How many participants? ______ How many women? __________ b) How many CSOs were trained on civic awareness training and civic education? ________ What training was provided? ________ United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 125 How many participants?____________ How many women? _______________ c) How many local leaders were trained on political activism? ________ What training was provided? ___________________________________________________ ______________________________________________________________________ _____ How many participants? ____________ How many women? ________________ d) How many CSOs were trained in local conflict management and mitigation? __________ What training was provided? ________________________________________________ ______________________________________________________________________ __ How many participants? ___________ How many women? _______________ e) How many CSO leaders are politically active? _________________________ f) What specific LGP work activities were implemented to enhance CSO civic awareness training capabilities and to enhance CSO political activism? ______________________ ______________________________________________________________________ _ What activities were implemented? __________________________________________ ______________________________________________________________________ _ How many participants? _________ How many women?_____________ 3) Assisted CSOs regularly assess the performance of local government institutions and to disseminate information to their members and the wider public. Circle one: Agree strongly, Agree, Disagree, Disagree strongly (there are no assessments by CSOs), Don’t know Narrative explanation for your choice: ______________________________________________________________________ ______________________________________________________________________ ___________________________________________ 4) The local media and the press provide balanced reporting and civic awareness campaigns. Circle one: Agree strongly, Agree, Disagree, Disagree strongly (there is no balanced reporting), Don’t know Narrative explanation for your choice: ______________________________________________________________________ ______________________________________________________________________ ___________________________________________ United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 126 5) The LGP has facilitated local level discussions on a national legal framework for the registration and operation of CSO’s. Circle one: Agree strongly, Agree, Disagree, Disagree strongly (there has been no facilitation) Don’t know Narrative explanation for your choice: ______________________________________________________________________ ______________________________________________________________________ ___________________________________________ F. Critical Assumptions 1) Security a) What was the security situation over the past several months in the governorate? b) How has the security situation affected or influenced performance of your program activities? 2) CPA Interim Policies a) How has CPA’s and central ministries’ policies affected the impact of LGP support to local government entities, and civil society organizations? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 127 Annex 10: Table of Events Time Frame Major attacks per month June 03 to Feb 05 Main Security Events Law and Democracy DG events May - 03 UN Security Council Resolution 1483 ; sanctions lifted, special UN representative to Iraq named; task to establish the Interim Iraqi Administration (IIA) LGP arrives in Basrah PMP May-July 2003 June 200 July 500 Governing council introduced by UN’s de Mello August 600 Bombing of UN compound, Sergio de Mello, UN special rep is killed. UN pulls out LGP I Implementation Plan September 750 PMP revised September 2003 October 100 Red Crescent HQ and 3 Baghdad police stations bombed same day by 4 VIED. Results Framework prepared November 1100 Nassiriyah hit my massive car bomb killing 19 Italians (and destroying the LGP office next door) ‘15 November Agreement’ paves the way for sovereignty; preparation of the TAL and Interim Government. Sovereignty specified for June 30, 2004. Al Sistani insists on elections for the Interim government. CPA favors caucus. Quality of Life (QOL) Survey 1 based on PMP released Field work for Mid-term review begins. December 750 January - 04 750 February 750 55 Iraqis are killed and approximately 65 are ‘Rush to Sovereignty Campaign’ (CDP/DDA); educate the population about democracy, the CDP/DDA starts in LGP; new increase in staffing United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 128 Time Frame Major attacks per month June 03 to Feb 05 Main Security Events Law and Democracy DG events wounded by car bomb outside a police station, crowd nearly riots while chanting anti-American slogans. role of the TAL, and prepare them for elections. March 900 Al Sadr uprising in the South begins. TAL completed and signed March 8 April 1900 Several suicide bombs explode in Basra. Crowds assault the Coalition forces trying to reach the wounded with stones. Bombs kill 68. Local Governmental Powers Order 71 signed 6 April Many LGP teams in South and South Central evacuated to Kuwait May 2100 Teams in Kuwait rewrite their work plans based on a regional model. June 1800 Sovereignty declared 30 June LGP Base Year Annual Report July 1800 Car bomb kills 70 Iraqis outside of a police recruiting center in Baquba. LGP Option Year One, Work Plan August 2900 QOL Survey 2 Report QOL 1 and 2 Trends Report September 2200 Suicide car bomb kills 19 and wounds 67 more in Kirkuk. October 2400 Three minibuses filled with 49 Iraqi recruits for the 16th Iraqi Army Battalion, 7th Army Brigade ambushed and killed by insurgents.. November 2700 The battle for Fallujah begins. Approximately 10,000 US troops and Report on Government Institutions United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 129 Time Frame Major attacks per month June 03 to Feb 05 Main Security Events Law and Democracy DG events 2,000 Iraqi forces confront 3,000 to 5,000 insurgents. December 1800 Suicide car bomber kills 18 Kurdish militiamen in Mosul. More than 16 are wounded. QOL Survey 3 January - 05 2700 January 30, 2005—The Iraqi elections take place. 44 people are killed in insurgent attacks, but the number of casualties and attacks is far below the expected number. Elections to Interim National Assembly and Provincial Councils February 1700 February 8, 2005—In Baquba, a car bomb explodes outside of the provincial police HQ, killing 15 recruits and wounding 17 more. In Mosul, a suicide bomber detonates his explosives inside the grounds of the Jumhouri Teaching Hospital. 12 Iraqi policemen die and four others are wounded. March LGP I ends, QOL Survey 4 April April 13, 2005— Insurgents in Kirkuk detonate a bomb close to 12 Iraqi security force members who are defusing a roadside bomb. All 12 are killed. LGP I 45 day extension May May 4, 2005—The Army of Ansar al-Sunna claims credit for a suicide bombing in Erbil that targets an office of the KDP political party. More LGP II awarded United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 130 Time Frame Major attacks per month June 03 to Feb 05 Main Security Events Law and Democracy DG events than 60 are killed and more than 150 injured by the blast. May 18, 2005—A US military official states that 126 car bombs had been detonated or disarmed since February 27, 2005. June The Interior Ministry releases figures that show that insurgents killed 20 Iraqi civilians a day on average over the past year and a half. Approximately 12,000 civilians were killed by insurgents over the time period. LGP I final report July July 24, 2005—Southeast of Baghdad in the Al￾Mashtel neighborhood, a truck filled 500 pounds of explosives detonates in front of the Al-Rashid police station, killing up to 40 people and wounding 25, August PRT initiative begins September October Referendum on draft constitution: passed November LGP II reorients to PRT prepares new LGP II Work Plan December Council of Representatives Elections December 15, 2005 January - 06 Council of Representatives elections validated LGP II Work Plan agreed United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 131 Appendix 5: USAID-Funded Economic Governance-II Capacity Building Programs March 10, 2006 Debrief Presentation United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 132 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 1 INTERNATIONAL BUSINESS & TECHNICAL CONSULTANTS, INC. Monitoring and Evaluation Performance Program – Phase II (MEPP II) EVALUATION OF PUBLIC SECTOR CAPACITY BUILDING PROGRAMS For USAID Mission in IRAQ MARCH 10, 2006 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 2 Background/Context • Evaluation team: • Hal Lippman (Team Leader), responsible for BearingPoint Ministries of Finance (FMIS, SSN, and CBI) and Planning activities • Jim Douglas, responsible for BearingPoint Ministry of Electricity activities • Frank Hersman and Harvey Herr, responsible for RTI LGP I and II activities • Evaluation environment and requirements have shaped the team’s methodology • Collaborative relationship with Mission DG, EG, and Program Office • Primary reliance on documents • Reliance on BearingPoint and RTI sources • Severely limited access to activity beneficiaries United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 133 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 3 Current Status • BP/Ministry of Electricity completed; preliminary draft report segment written • BP/Ministry of Finance activities: – FMIS – complete – Social Safety Net – 75% complete – Central Bank of Iraq – 25% complete • BP/Ministry of Planning – 75% complete • RTI LGP I and II activities – 50% complete • Draft report due March 24, 2006 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 4 INTERNATIONAL BUSINESS & TECHNICAL CONSULTANTS, INC. USAID-FUNDED ECONOMIC GOVERNANCE-II PROJECT CAPACITY BUILDING PROGRAMS MINISTRY OF ELECTRICITY United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 134 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 5 THE FUNDMENTAL GOALS (ELECTRICITY) BearingPoint Goal (Overview Report, 12/13/2004): • “Develop and help establish a restructuring plan and market model for the electricity sector that facilitates private investment…” Traditional Public Policy Goal For A Power Company: • Provide sustainable and reliable power at just (non-discriminatory) and reasonable (affordable) rates to customers, without government subsidies. Observations • Activities to achieve the first goal are directed at establishing a vision for the future; by definition those activities will not produce near term benefits. • Activities to achieve the second goal are directed at establishing improvements to the power system; those activities should produce benefits as completed. • During the course of e.g.-ii, BearingPoint pursued the first goal, and much later focused on the second goal. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 6 A TIME LINE REFERENCE POINT ƒ EG-I was initiated in October of 2003, ending in June 2004, producing: 1) A Comprehensive 15-year Master Plan; 2) A Draft Regulatory Law; and 3) Reports: Organizational Structure and Staffing Assessment (OSSA￾I), assessing critical issues facing the ministry and providing a consistent set of recommendations. ƒ EG-II was initiated in October Of 2004, Although some operational reviews were initiated, the main focus was on the following three core functional areas: - Open Market Development and Privatization Programs - Regulatory Reform – Establishment of a Regulatory Agency - Electricity Laws – to Provide the Basis for Privatization Programs United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 135 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 7 A TIME LINE REFERENCE POINT (continued) ƒ Most advisors left for Amman for several months (December 2003- March 2004), for security reasons; full project staff only re –established in April 2004. ƒ In June 2004 the Governing Authority was shifted from CPA to the Transitional Government of Iraq. ƒ In June 2005, in response to Ministry concerns, the focus shifted and staffing changed, from core functions with long term goals to core functions of immediate concern to the Ministry. ƒ January 2006, with the exception of financial management and limited metering activities, BearingPoint has no active assistance programs in electricity. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 8 CORE FUNCTIONS: INITIAL FOCUS OF EG-II Findings • The initial focus was on long term core functions, at the expense of immediate issues of concern to the Ministry. • Work products were a series of analyses and studies containing sound recommendations for the future; they were submitted to the Minister; no action has been taken or is anticipated to be taken, in the near future. • The documented recommendations contained in OSSA-I and II were essentially ignored. Conclusions • Because OSSA-I and II addressed very critical problems: ineffective organization and huge overstaffing, it was a major mistake not to have developed a technical assistance follow up program. • The major needs of the Ministry are to produce and distribute power, reduce operating costs, and increase revenues, and achieve a balanced budget. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 136 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 9 CORE FUNCTIONS: INITIAL FOCUS OF EG-II (continued) Recommendations • Use the recommendations and action programs delineated in OSSA-I And OSSA-II reports and establish a broad-based technical assistance program, with increased efficiency of the electricity sector and cost reduction as a major goal • Do not use resources and political capital to push for privatization, regulation, or new electric laws in the near future. As noted in the OSSA-I Report, there is a need for a more stable environment before such programs are relevant. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 10 CURRENT CORE FUNCTIONS (Areas Of Technical Assistance) • Tariffs and Cost of Service (Major Pending Issue) • Financial Management (Two Significant Programs: Establishing a Corporate Management Group and Implementing Three Computer￾Based Financial Models, from Egypt are Nearing Completion) • Metering Programs Under Development • Procurement (No Longer an Active Program) • Human Resources (HR)/ Capacity Building (Did Not Form Part of an Active Program) United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 137 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 11 TARIFFS AND THE COST-OF-SERVICE Findings • BearingPoint conducted extensive and well documented cost-of-service and related studies, recommended major increases in tariffs, and addressed concerns over price increases effectively. • Final tariff recommendations are under review and will soon be submitted to the Government. The Government must take action. Conclusions • This tariff issue is critical for the Ministry, the Government, and the Iraqi people , to address effectively, and to address immediately. • The tentative tariffs are based on cost-of-service of current expenses/ budget. Those cost figures have not been reviewed critically and probably contain unjustified costs. That should give some room to establish lower tariffs than indicated by the cost-of￾service study. Recommendation • The tariff issue is of vital importance to Iraq. IRMO, USAID, DOD, UN, and the World Bank, should quickly take a united position on tariff increases, which may involve a compromise tariff, and very firmly urge adoption. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 12 CORE FUNCTION: FINANCIAL MANAGEMENT Findings The Focus of Technical Assistance Is in Two Areas: – Organizational development of a “Corporate Financial Group” (CFG) to conduct financial analyses and assist senior management in decision-making. – Systems development to implement three computer-based financial models, available in Arabic from Egypt, as effective tools for financial management. ƒ Currently, Ministry staff are in Egypt undergoing technical training in the three financial models, consistent with a well defined systems implementation program developed by BearingPoint. ƒ There is a clearly defined development and implementation program for establishing CFG as a functioning organizational unit within the Ministry. Conclusions It is a reasonable expectation that CFG will be established and that the financial models will be implemented successfully. ƒ This will be a very important achievement within the Ministry, but it is yet to be accomplished. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 138 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 13 CORE FUNCTION: METERING Findings ƒ In process is a metering survey to assess specific requirements. ƒ Meter technology was reviewed and recommendations are being formulated ƒ A detailed meter strategy is under development. Conclusions ƒ A reasonable program has been established to define metering needs, the appropriate meters to purchase, and metering replacement strategy; the final report and recommendations are still in progress. ƒ An effective billing and collection system is critical to improve the revenue stream to the Ministry; meters are a key element to success. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 14 CORE FUNCTION: METERING (continued) Recommendations ƒ Realistic cost estimates of the metering program need to be identified now and an effort made to secure adequate financing. ƒ Recognize that metering is but one component of what should be an integrated billing and collection system, and seek funding for technical assistance to update the total system. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 139 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 15 CORE FUNCTION: PROCUREMENT Findings ƒ Of the total EG-II resources devoted to the Ministry, relatively little has been devoted to procurement issues. ƒ A series of one to four day training programs were provided to Ministry Staff; they were generic, i.e., outlining general principles of procurement, concepts of international bidding, policies governing issuing tenders for major projects. ƒ Currently there is no technical assistance to the procurement function. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 16 CORE FUNCTION: PROCUREMENT (continued) Conclusions • Too few resources were devoted to this technical area to have any significant or lasting impact. • No indication that purchasing and procurement programs within the Ministry were reviewed or assessed in any systematic manner or in any detail during EG-II. Recommendation • Considering that purchasing and procurement involves hundreds of millions of dollars, a technical assistance program should be established to conduct a detailed needs assessment, with particular attention to the internal controls needed to effectively limit graft and illegal procurement activities. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 140 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 17 CORE FUNCTION: HR/CAPACITY BUILDING Findings • OSSA-I and II identified a series staffing and related personnel issues and in general terms recommended an action program to address those issues. no follow up action was taken relative to those reports. • In EG-I, to provide interface, the Ministry formed an Executive Work Group, with an Executive Reengineering Group under it. During the time between EG-I and EG-II, these groups became inactive, attributable to the lack of any mentoring. In EG-II, the Ministry reestablished it as a Steering Committee, with Active Sub-Committees for each core function (technical areas) to which technical assistance was/is provided. Conclusion And Recommendation • This Steering Committee is a very practical mechanism to coordinate programs and introduce change. It should be staffed with ongoing technical assistance and its role expanded, if not through USAID, then through IRMO. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 18 FUTURE TECHNICAL ASSISTANCE NEEDS FINDINGS AND CONCLUSIONS Findings • Currently only one staff is assigned to provide technical assistance (financial management area) to the Ministry, and one part-time staff to assist in the metering survey, and he is located in England. • In practical terms, BearingPoint is no longer providing any technical assistance to the Ministry. Conclusions • Given the critical needs of the Electric Sector there should be major technical assistance (capacity building) programs in effect that does not appear to be happening. • Given the enormous complex problems faced by the ministry, that assistance must be sustainable over a long period of time, with relevant staff resources and adequate funding—there is no quick fix. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 141 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 19 FUTURE TECHNICAL ASSISTANCE NEEDS RECOMMENDATIONS In its presentation to the State Department/IRMO, in December 2005, BearingPoint identified potential future technical assistance: ƒ FINANCE – Move to IAS Accounting Standards – Prepare Asset Register – Train in New Accounting and Finance Procedures ƒ BILLING AND COLLECTION – Develop New Billing System and Revised Collection Procedures ƒ WAREHOUSING AND INVENTORY CONTROL – Develop New Asset Management System – Develop New Systems for Warehousing, Logistics, Asset Management Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 20 FUTURE TECHNICAL ASSISTANCE NEEDS RECOMMENDATIONS (continued) ƒ PROCUREMENT – New Procurement and Tendering System – Implement Tenders for New Generating Capacity An “Organization and Staffing Component” should be added to this list. in developing and implementing improvements to billings and collection systems, warehousing and asset management, and procurement, a significant component should be the development of effective accounting, audit, and operating procedures to adequately protect the Ministry and its customers from graft and illegal activities that could be occurring. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 142 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 21 TECHNICAL ASSISTANCE PRIORITIES High Priorities, In Order: ƒ Finalize and document tariff recommendations, obtain a unified position (IRMO, USAID, UN) on recommendations, and secure agreement from the Ministry to raise rates. ƒ Develop and execute a detailed technical improvement plan (institutional development) to establish an efficient and effective billing and collection system, with integrated sub-systems: meters; customer accounting and billing; and collections. The model to follow is what is now being implemented for the CFG. ƒ Use OSSA-I and II Reports (EG-I) to develop and execute improvement programs for (1) organizational development and (2) staffing and personnel related issues. ƒ Initiate systematic evaluations of warehousing and inventory, and the related function of purchasing and procurement; prepare a detailed systematic improvement program for each, defining budget requirements and establishing priorities. NOTE: The two highest priorities relate to increasing the revenue stream. The other two priorities relate to developing effective cost control and cost reduction programs. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 22 LESSONS LEARNED ƒ Establish clear and realistic goals, the initial goals of the EG-II for electricity were neither realistic nor clear. ƒ Devote adequate resources (staff and funding) to the task (technical assistance) at hand, or limit tasks to the resources available. Examples: Sufficient Resources: Tariff Study and Financial Management. Insufficient Resources: Procurement and Metering. ƒ The magnitude of electricity needs, requires a broad-based technical program of assistance, reflecting a unified and coordinated approach by IRMO, DOD, USAID, UN, World Bank. It would not be unreasonable for IRMO to have the leadership role in developing and implementing an integrated program. ƒ Given the magnitude of the problems facing the ministry, and their complexity, the efforts of individual agencies will remain largely ineffective in introducing significant change. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 143 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 23 INTERNATIONAL BUSINESS & TECHNICAL CONSULTANTS, INC. USAID-FUNDED LOCAL GOVERNANCE PROJECT CAPACITY BUILDING PROGRAMS Local Service Delivery Local Administrators Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 24 Question 1 • What MMPW/local government functions or problems were addressed with capacity building activities? • What capacity building tools such as on-the-job training, ad hoc advice and formal training were used to induce improvements? • Which indicators have been used to measure improvements? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 144 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 25 What MMPW/local government functions or problems were addressed with capacity building activities? • Preliminary Finding (problems addressed): LGP access to MMPW and other ministries is crucial to the development of local government code and structure. – Several examples emerged where uncertain local department/directorate authority hindered achievement of program IR (4.1.3 service delivery) • At the outset LGP I was denied access at the ministerial level to MMPW • By October/November 2003 LGP was expressing concern about the realities of local governance enabling law, and began providing decentralization recommendations to a reluctant CPA. A small Policy Reform Team (PRT) supported by short term advisors provided these recommendations. • Some ministerial level response to request for clarification of local authority in Kirkuk made by LGT who had convened a meeting with the Minister, the Governor and others, December 2003. • By January 2004, LGP with USAID approval developed a long term PRT with access to MMPW Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 26 What capacity building tools such as on-the-job training, ad hoc advice and formal training were used to induce MMPW and MoSPA improvements? • Preliminary Findings (capacity building tools) – National conference on administrative and fiscal decentralization. – Regional conferences same topic in Basra and Erbil. – Local government assessment report on a framework for local government used to suggest a decentralization pilot program at MMPW – LGP advised the MMPW established steering committee – Activities carried over to LGP II • Lessons learned: – Early reluctance to permit LGP access to the MMPW may have resulted in a limited interpretation of decentralization in TAL admin Order 71. The capacity building tools have been used effectively to heighten ministerial/local government awareness of the need for legal clarity. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 145 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 27 Which indicators have been used to measure improvements in the MMPW activities? • No indicators were used Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 28 Question 2. • Which core functions were addressed with which capacity building tools such as on-the-job training, ad hoc advice and formal training. – i) strategic planning and policy development – ii) fiscal management – iii) personnel management – iv) leadership and communications – v) information technology – vi) administrative rules and procedures – vii) other critical technical skills for the ministries, organizations and local governments United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 146 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 29 Preliminary Findings Q2 • A matrix is being developed in response to this (core functions by tools used) • The matrix is to be informed by weekly and monthly reports, and possibly by quarterly reports if we are able to locate them. • Attempting to validate tool use and efficacy with one or two examples. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 30 Question 3. • Did the contractors meet the requirements established in their Scopes of Work? • Have they produced the required outputs on schedule? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 147 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 31 Preliminary Findings Q3 • Contract compliance is not easily documented. • Frequent revisions to work plans; redirection by CPA to accommodate “rush to sovereignty”; al Sadr uprising in the South; frequent changes in the CTO (5?), and program adjustments, all make this a difficult question to answer. – RF structures were complex at the outset with some 32 sub-IRs specified; this was simplified by OYO, but the original PMP was not updated. – Project work plans were revised an unusual number of times. • Volumes of reports, but a lack of systematic reporting on results to date; no systematic use of indicators in reporting; thus most evidence is anecdotal rather than quantitative and summarized. Exceptions are for Grants Management, council development and the CDP. – Indicators posed a significant problem to LGP I. – No indicators are mentioned in the LGP final report. • Quarterly reports not immediately available. Thus far have access to only one early report on the status of core activities by date and governorate. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 32 Question 4 • What were the main/most important capacity building outputs or achievements obtained from the USAID investment? • What are the impacts in terms of the effectiveness of local government capacities in delivering services to the citizens? • What are the main failures that the contractors experienced? • When possible, findings should be presented in terms of the core functions listed above, and in relation to relevant and applied indicators. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 148 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 33 Preliminary Findings Q4 • Capacity building outputs were not well measured or documented by LGP, nevertheless from specific examples that are being examined we expect a positive result. These are: – Solid waste in Baghdad – Decentralization policy development – Financial management – IT – Council training • Impact of local service delivery (under investigation) – LGP focused strongly on this issue, but results of activities were not systematically reported. Data on levels of service delivery were collected in the QOL surveys. These show an improvement in water, and a worsening of electricity. – Attribution to LGP was weak. • Main failures experienced by LGP – Early on LGT recognized that an understanding of lines of authority for local directorates/departments was missing. Individual governorates attempted to cobble together the TAL order, and historic law to convey to DGs how decentralization is supported. DG’s still knew that they reported to Ministries. – Many LGT devoted considerable effort to developing local councils below the province level. While this was a good effort in the development of participatory democracy, such councils had limited legitimacy in recognized government structure (TAL order 71; 1969 Governorates Law). – Significant council training occurred in all governorates, but with the exception of Kurdistan most provincial council members were replaced by the January 2005 elections. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 34 Question 5 • What additional indicators are being used or could be used to measure impacts and success of these programs, regardless of the timeframe in which they were developed? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 149 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 35 Preliminary Findings Q5 • The original PMP indicated that quarterly household surveys were to be used to monitor SO 4.1 (Iraqi Citizens Benefit From Improved Local Governance) • There was much debate regarding suitable performance indicators throughout LGP I – CTOs requested that more attention be paid to obtaining performance indicators. – Most activities were/are oriented to capacity building to improve LGC operations and outreach, or in local government departments to improve service delivery. USAID OCAT and examples from the ‘Handbook of Local Government Indicators’ were used to inform the development of indicators. However, by the time of LGP I OYO had the indicators in place, drastic LOE reductions were underway, and as far as is known there were no reports issued using the indicators e.g., EOY LGP I report. • By LGP II the RF was manageable with indicators suggested, and the PMP under development (but now overdue as it was a deliverable due 60 days from award date). • Lessons learned: – throughout the LGP, apart from the QOL Surveys, there has been little actual measurement of indicators. The PMP development process has not been applied, and the field staff not included in the measurement process. The monitoring process needs to devolve to include those from whom results are expected. In the LGP context this means the DGs, and the council members. Caution is needed to ensure that we do not measure an institution that is unauthorized to make the proposed changes that may improve it, or that does not have the funding to do so. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 36 Question 6 • Are there any data to support and demonstrate attribution to USAID’s investment for capacity building achievements in the sectors evaluated? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 150 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 37 Preliminary Findings Q6 • LGP conducted four nationwide Quality of Life surveys to collect performance data for SO 4.1 – Results from the first of these was made available in November 2003; the 4th and final report was issued in March 2005. – Trend data were provided for some indicators, such as council recognition, service levels, empowerment, and participation. – A ‘weak’ attribution to LGP activities is implied in some of the LGP reporting. – The reports were not widely distributed to the field. • Lessons Learned: – Data reporting on attribution is necessarily longitudinal. The QOL surveys were a good first tool for this assessment, but needed to be complemented by other tools capable of identifying ‘stronger’ attribution. Examples of these are local citizen satisfaction surveys, and levels of service delivery against agreed standards. Less frequent QOL surveys should be implemented to build on earlier findings. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 38 Question 7 • What specific or contextual reasons explain star performing activities on the one hand and non￾performers on the other? • What general lessons are evident from both star performers and non-performers? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 151 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 39 Preliminary Findings Q7 • Currently tracking and validating the following activities – Solid waste management in Baghdad has a complex environment with many players. It is proving difficult to assess the LGP role in isolation. – Work with the MMPW and MoSPA with decentralization and development of LGA. – Training activities with Baghdad councils. – Training activities with Kirkuk DGs. • Aspects of performance was not routinely measured with performance indicators. – Possible exception for the CDP/DDA activity where selected attendees were asked opinions before and after events. An internal LGP evaluation identified which thematic events had the highest likelihood of changing attendee’s opinions in a ‘positive’ way. • Non-performers (aside from those caused by external security factors) are expected to be caused by a lack of understanding of the underlying legal or administrative basis for the proposed intervention (to be validated, e.g., dismissal of the Amanat Council). Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 40 Question 8 • Out of the lessons learned, what are the pre￾requisites and/or the key assumptions for the different capacity building activities to be effective? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 152 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 41 Preliminary Findings Q8 • Apart from the most obvious security issues: – Critical assumptions were that a legal basis would be in place that supported decentralization, and council development. – Assumptions were made about the authority or willingness of DGs to support decentralization. There is some evidence suggesting that they do not. (USAID/Iraq Local Governance Project, Government Department/Directorate, Organizational Assessments Report: ‘The Public Utilities’, November 2004) • Lessons learned: – Include in the baseline studies a clear understanding of the legal and administrative environment of the institution where intervention is proposed. More emphasis has been given to this in LGP II and generally in the development of the PRT. Some of the institutions where interventions are proposed have been operational since the 1920’s (e.g., Amanat water and sewer). Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 42 Question 9 • Summarize lessons learned and suggest levels of USAID/Iraq involvement in managing projects. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 153 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 43 Preliminary Findings 9 • To follow completion of findings Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 44 Question 10 • Compare effectiveness of “up-front indicator development though explicit institutional analysis” with indicators developed during contractor operational activity. • Need clarification on the meaning of “up front indicator development through explicit institutional analysis.” United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 154 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 45 Preliminary Findings Q10 • Indicators changed significantly from the first PMP of September 2003 • An elaborate RF developed during September 2003 consisted of 7 PIRs (program IRs) and 32 Sub-IRs. Each Sub-IR had an indicator (s) associated with it. This was unmanageable. • The RF changed with the ‘Rush to Iraqi Sovereignty’ when the Civic Dialogue Program was added to the work program. LGP took this opportunity to change the RF and reduce the sub-IRs to 17. • The uprising in the South led to another revision of the RF shown in LGP I OYO work plan. The RF now had 13 sub-IRs, with indicators related to activities that had been developed from the bottom up. • By LGP I OYO it was clear that many indicators were not measuring results/outputs. This was especially so regarding capacity building activities. The organization capacity building literature, and other materials were researched in an attempt to achieve measurable capacity building indicators (example; using scales to measure institutional improvement in budgeting, planning or customer response activities that were the target of LGP training activities). These were not implemented. • By the end of OYO emphasis had moved to establishing service level standards for departments/directorates. • Lessons Learned: the measurement of performance indicators should be integrated with LGP activities such as ‘increased efficiency of local service delivery.’ The measurement of indicators by the Departments/Directorates as part of confirmation of increased efficiency should also provide for requirements under the PMP. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 46 Question 11 • Assess effectiveness of RTI’s assistance to improve local governments administrative capacity to carry out their administrative functions. • Are the skills acquired during the training applied in undertaking their responsibilities? • To the extent possible, determine where those trained, especially under LGP1, are now and what are they doing. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 155 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 47 Preliminary Findings Q11 • Assessing effectiveness of RTI’s assistance: it isn’t likely that we will locate sufficient information on this topic as effectiveness wasn’t typically monitored. – There may be some evidence from financial operations in Basra, or from the CDP. – Councils: we have some information on council members who were trained, and are attempting to interview them regarding the results of training received. – Directorates/Departments: sending field workers to Kirkuk to meet with DGs and staff known to have received LGP training. • Lessons Learned: – Pending. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 48 INTERNATIONAL BUSINESS & TECHNICAL CONSULTANTS, INC. USAID-FUNDED ECONOMIC GOVERNANCE-II PROJECT CAPACITY BUILDING PROGRAMS MINISTRY OF FINANCE MINISTRY OF PLANNING United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 156 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 49 Lack of Definitional Clarity • Finding: – BP uses two definitional elements interchangeably in describing their capacity building activities: • Training for individuals – ‘providing a skill or ability that wasn’t there before’ (the usual meaning of capacity building) and seeing to it that that new skill or ability is used in a sustained manner • Technical assistance provided by senior advisors to high￾level Iraqi government officials (ministers, deputy-ministers, and director generals) to help them do their jobs and carry out GoI and/or U.S. policy aims (e.g., meet IMF SBA requirements) Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 50 Lack of Definitional Clarity ( continued) • Conclusion: – The latter definition is not capacity building. An example is providing a high level official with a strategy to help accomplish an identified purpose. The interchangeable use of these two definitional elements causes confusion in identifying and reporting accomplishments and capturing the real effects of the intervention. • Recommendation: – Consider establishing a separate classification – e.g., functional TA – for technical assistance provided by advisors and other expat experts. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 157 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 51 Selection of Participants for Training • Finding – Significant problems have occurred in the selection of individuals to participate in FMIS training. • cultural attitudes • Personal advantage • Attractive per diem • Conclusions – abuse has means that additional training had to be done to get the appropriate people the necessary skills and abilities. • Recommendations – Consider establishing a certification process for proposed training participants. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 52 Underlying Cultural Behavior and Attitudes • Finding – Trained CBI officials failed to use 20 desktop computers they received from the CPA more than two years ago. • Conclusion – Concepts of productivity and efficiency are not being understood. • Recommendation – In its capacity building efforts USAID must develop and better incorporate activities that go beyond the specifics of training and address the fundamental issues of cultural transformation. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 158 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 53 Training Enhances Good Governance • Finding – FMIS capacity building efforts give rise to significant outcomes well beyond the purpose of the training. • Conclusion – Look at second and third order effects of training, e.g., networking and team building for better governance. • Recommendation – Monitor technical training to include aspects of institutional capacity building such as regular follow￾up meetings of technical groups. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 54 Monitoring and Evaluation • Finding – BP has no PMP. • Conclusion – BP has no systematic, across-the-board way of comparing what they say they’re going to do at the outset of an activity, what they are doing at any given point in time, and what they have accomplished. • Recommendation – review the level of effort in the contract to ensure that it includes a performance monitoring capability. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 159 Appendix 6: USAID-Funded Economic Governance Capacity Building Programs March 17, 2006 Debrief Presentation United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 160 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 1 INTERNATIONAL BUSINESS & TECHNICAL CONSULTANTS, INC. Monitoring and Evaluation Performance Program – Phase II (MEPP II) – Task A EVALUATION OF PUBLIC SECTOR CAPACITY BUILDING PROGRAMS For USAID Mission in IRAQ MARCH 17, 2006 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 2 INTERNATIONAL BUSINESS & TECHNICAL CONSULTANTS, INC. USAID-FUNDED LOCAL GOVERNANCE PROJECT Evaluation of RTI - (LGP I and LGP II) with respect to: 1. Promoting increased efficiency of local service delivery (Pilot Projects in Service Delivery – Ministry of Municipalities and Public Works) 2. Building the Capacities of Local Administrators United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 161 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 3 General Statement of Findings • LGP I was an effective proving ground for LGP II – There was general compliance with the contract provisions. – Initial contract obligations were met on schedule; the timeline for later outputs was obscure. • Capacity building tools developed in LGP I were found to be effective. • The concepts of decentralization have taken hold in Iraq largely through sustained efforts of LGP. – Key individuals made important contributions. • Initial assumptions about the state of enabling law supporting decentralization were problematic. • As Iraq moved to sovereignty through electoral processes some of the work of LGP was undone. • Project reporting was voluminous, but did not use indicators, did not tend to report on progress to date, and did not attempt to assess the result of project activities undertaken. – Numerous changes in CTO may have had an obfuscating effect. – An initial unwieldy Results Framework complicated the use of indicators. • LGP provided useful lessons learned. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 4 Evaluation Questions • SOW for the evaluation listed 11 questions to be answered. • The presentation is ordered according to these questions. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 162 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 5 Question 1 • What government functions or problems were addressed with capacity building activities? – Ministry Level – Local Government Level • What capacity building tools such as on-the-job training, ad hoc advice and formal training were used to induce improvements? • Which indicators have been used to measure improvements? Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 6 What Ministry functions or problems were addressed with capacity building activities? • Problem: lack of clarity in local government law – MMPW emerged as the lead ministry for decentralization issues and focal point for LGP I and II ministerial contact. • Need for ministry access unfolded based on lack of clarity in local government law that supported decentralization. – Later (January 05) MoSPA emerged without portfolio, but LGP encouraged them to support decentralization (Feb 05) and the LGA. • Lessons Learned: – Lack of clarity in enabling law caused uncertainty and confusion. – Article 56 of the TAL gives local councils the power only to review ministry plans and budgets sustaining the top-down view of government; this may have been different had there been high level access to the Ministry. • Recommendations: – Sustain LGP access to MMPW and other ministries as they continue to develop local government code and structure; build on Baghdad Charter development done in LGP I, and thus far in LGP II. – Sustain strong decentralization advocacy with DOS and US Mil to the National Assembly through LGAs and PC national and regional conferences. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 163 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 7 What capacity building tools such as learning by doing, ad hoc advice and formal training were used to induce MMPW and MoSPA improvements? • Preliminary Findings: – Capacity building tools for Ministerial activity • National conference on administrative and fiscal decentralization. • Regional conferences same topic in Basra and Erbil. • Learning by doing with respect to use of RTI’s framework and matrix for local government authority used to develop a decentralization pilot program at MMPW • LGP provided ad hoc advice to the MMPW internal steering committee on decentralization • Handbooks and toolkits for LGA development used in training and conferences (occasionally in conjunction with MoSPA). – The capacity building tools have been used effectively to heighten awareness of the need for legal clarity in decentralization, and to promote Iraqi Associations of Local Governments as advocates for decentralization. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 8 Which indicators have been used to measure improvements in the MMPW activities? • No indicators were used to assess the results of LGP ministerial level activities. • Recommendation: – Review institutional capacity building indicators as has been done for the PRT. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 164 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 9 What local government functions or problems were addressed with capacity building activities? • Problem: poor service delivery by local directorates/departments. – No customer complaint mechanism. – No apparent service level standards, or systematic data collection on services that were provided. – Limited provision for citizen input for strategic planning. – No regular training program to build technical and administrative capacity of the staff. – Archaic budgetary process based on centralized system. • Problem: nascent understanding of local council functions to support participatory democracy. – Historic recognition of local councils predates the LGPO 71 and the Law 159 of 1969, but there was limited or no implementation. – LGP assisted implementation of Mahafada, Qada, Nahiya, Mahala councils based on imperfectly understood enabling law. – Councils had limited knowledge of what was expected of them. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 10 What capacity building tools such as learning by doing, ad hoc advice and formal training were used to induce local government improvements? • Preliminary Findings: – LGP developed a full array of capacity building tools for application to councils and local directorates. • 6 Formalized CD ROM training modules for PCs emerged from some 47 training activities used in LGP 1. • Technical training aimed at specific departments (water, solid waste, etc.) • Generalized IT training, as well as specific training for technical departments. • Study tours for council members, and study tours for DGs (water and sewer, solid waste) [two of those on solid waste tour later assassinated] • Extensive ad hoc advice to councils throughout Iraq, but especially in the Baghdad Amanat and South Central region. • Learning by doing in the creation of a local charter (Baghdad) and in the election of councils (Al Muthana, Dhi Qar). – To the extent limited by the duration of the evaluation we found that the training provided had been well received. In specific instances LGP individuals were warmly remembered for the advice and commitment they provided. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 165 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 11 Which indicators have been used to measure improvements in the local government activities? • Indicators were proposed, but not implemented. • Field teams were reluctant to report on indicators voluntarily. • In the later days of LGP 1, simplified indicators were developed with the teams, but a reduction in staff resulted in them not being used. • Recommendation: – Ensure that there is a ‘buy in’ and inclusion of field teams in the indicator development. – Incorporate the beneficiary of the activity in the measurement. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 12 Question 2. • Which core functions were addressed with which capacity building tools such as on-the-job training, ad hoc advice and formal training. – i) strategic planning and policy development – ii) fiscal management – iii) personnel management – iv) leadership and communications – v) information technology – vi) administrative rules and procedures – vii) other critical technical skills for the ministries, organizations and local governments? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 166 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 13 Preliminary Findings Q2 CD-ROM Module Training Materials Handbooks Toolkits Learning by doing Ad Hoc Advice i) strategic planning and policy development, ii) fiscal management, iii) personnel management, iv) leadership and communications, v) information technology, vi) administrative rules and procedures, vii) other critical technical skills for the ministries, organizations and local governments? Capacity Building Tools Core Functions Formal Classroom Training Mentoring Self-Teaching Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 14 Question 3. • Compliance: – Did the contractors meet the requirements established in their Scopes of Work? – Have they produced the required outputs on schedule? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 167 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 15 LGP 1 Scope of Work • LGP SOW requirements (4 areas) – Provision of specified services – Deliverables – Report requirements – Performance Monitoring/Indicators Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 16 Q3 – Compliance on Services Provided • LGP 1 was to provide services in four functional areas: 1. Activity Research, Analysis, Design, Monitoring & Evaluation 2. Building the capacities of Sub-national Administration 3. Building the capacities of Civic Institutions to Participate in Political Processes; include being prepared, if directed, to provide Support for Sub-national Elections. 4. Provision/Administration of RRG and Technical Assistance • Findings (refer to above numbers) – 1. Partial compliance, through four Quality of Life Surveys, one Assessment of CSOs, one Assessment of Government Departments, Analysis of a Labor Force Survey, and the development of payroll lists. Rapid appraisals were called for, but either not done, or not reported. – 2. General compliance, but not uniform throughout Iraq – 3. Initial compliance with direct work with CSOs, and the development of neighborhood councils. Later the CDP/DDA events prevailed. – 4. RRG was fully functional in the provision of grants, but did not routinely assess the final outcomes of the grants. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 168 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 17 Q3 – Compliance on Deliverables • Deliverables specified: 1. Immediately deploy teams 2. On arrival coordinate with military commands… 3. Within 3-weeks document assessment of local governance and civil society issues 4. Within 3-weeks prepare a quarterly work plan 5. Within one month of in-country arrival identify legitimate Iraqi leadership with which to partner 6. Within one month of in-country arrival coordinate with local authorities for distribution of assistance 7. Within 90-days of arrival report on a framework, based on collaborative development, for decentralization of authority to local governance structures and shall update the framework with each quarterly work plan submission… • Findings (refer to numbers above) – 1. teams were deployed rapidly, the LGP grew from nil to 3000 in about 9-10 months – 2. initial coordination did take place, but this was not uniform – 3. not determined – 4. Yes, A work plan was prepared in August 2003 – 5. Could not validate, but likely compliance – 6. Could not validate, but likely compliance – 7. No report on a framework for the decentralization of authority to local governance was found that had been prepared within the 90 period. This happened much later. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 18 Q3 – Compliance on Reporting Requirements • LGP 1 Report Requirements – Weekly Consultations – Monthly Reports – Quarterly Work Plans – Final Report • Findings – LGP 1 provided ‘daily reports’, in addition to weekly, monthly and annual reports. Weekly consultations became formal reports. Daily reports were not required in the contract, but asked for by CPA/USAID. – Quarterly Work Plans were not available; but many work plans were prepared in response to changing events and guidance. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 169 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 19 Q3 – Compliance on Performance Monitoring/ Indicators • At program start-up propose indicators monitoring performance • Within 30-days of the award submit PMP – PMP ‘necessitates the conduct of a baseline study to develop baseline data for the measurement of progress throughout the program.’ – PMP to provide for ‘periodic evaluation by the recipient of the impact’ of the various program components and operate as a ‘program feedback loop’ • Semi-annual Performance Monitoring Report that details benchmarks toward achievement of performance indicators and results. • Findings: – Indicators provided in the initial PMP, but were not measured –‘program feedback loop’ not in operation – Semi-annual Performance Monitoring Reports were not available for review. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 20 Question 4 • What were the main/most important capacity building outputs or achievements obtained from the USAID investment? • What are the impacts in terms of the effectiveness of local government capacities in delivering services to the citizens? • What are the main failures that the contractors experienced? • When possible, findings should be presented in terms of the core functions listed above, and in relation to relevant and applied indicators. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 170 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 21 Preliminary Findings Q4 • Capacity building outputs were not well measured or documented by LGP, nevertheless from specific examples that are being examined we expect a positive result. These are: – Decentralization policy development with the MMPW – Development of training modules – Development of a Local Charter for Baghdad – Solid waste in Baghdad – Financial management in Basra – IT training • Impact of local service delivery (under investigation) – LGP focused strongly on this issue, but results of activities were not systematically reported. Data on levels of service delivery were collected in the QOL surveys. These show an improvement in water, and a worsening of electricity although attribution is weak. • Main failures experienced by LGP – Early on Local Governance Teams recognized that an understanding of lines of authority for local directorates/departments was missing. Individual governorates attempted to cobble together the TAL order, and historic law to convey to DGs how decentralization is supported. DG’s still knew that they reported to Ministries. – Many LGT devoted considerable effort to developing local councils below the province level. While this was a good effort in the development of participatory democracy, such councils had unclear authority in recognized government structure (TAL Order 71; 1969 Governorates Law). – Significant council training occurred in all governorates, but with the exception of Kurdistan most provincial council members were replaced by the January 2005 elections. In Baghdad, the Amanat council was dismissed. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 22 Question 5 • What additional indicators are being used or could be used to measure impacts and success of these programs, regardless of the timeframe in which they were developed? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 171 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 23 Preliminary Findings Q5 • The original PMP indicated that quarterly household surveys were to be used to monitor SO 4.1 (Iraqi Citizens Benefit From Improved Local Governance) • There was much debate regarding suitable performance indicators throughout LGP I – CTOs requested that more attention be paid to obtaining performance indicators. – Most activities were/are oriented to capacity building to improve LGC operations and outreach, or in local government departments to improve service delivery. USAID OCAT and examples from the ‘HANDBOOK OF DEMOCRACY AND GOVERNANCE PROGRAM INDICATORS’ were used to inform the development of indicators. However, by the time of LGP I OYO had the indicators in place, drastic LOE reductions were underway, and as far as is known there were no reports issued using the indicators e.g., EOY LGP I report. • By LGP II the RF was manageable with indicators suggested, with the PMP under development. • Lessons learned: – Throughout the LGP, apart from the QOL Surveys, there has been little actual measurement of indicators. The PMP development process has not been applied, and the field staff not included in the development of the measurement process. – The monitoring process needs to devolve to include those from whom results are expected. In the LGP context this means the DGs, and the council members. Caution is needed to ensure that we do not measure an institution that is unauthorized to make the proposed changes that may improve it, or that does not have the funding to do so. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 24 Question 6 • Are there any data to support and demonstrate attribution to USAID’s investment for capacity building achievements in the sectors evaluated? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 172 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 25 Preliminary Findings Q6 • LGP conducted 4 nationwide Quality of Life surveys to collect performance data for SO 4.1 – Results from the first of these was made available in November 2003; the 4th and final report was issued in March 2005. – Trend data were provided for some indicators, such as council recognition, service levels, empowerment, and participation. – A ‘weak’ attribution to LGP activities is implied in some of the LGP reporting. – The CDP/DDA program collected ‘beneficiary’ information from those attending its events. These were analyzed to determine the impact the events had on changing participants’ viewpoint. – The reports were not widely distributed to the field. • Lessons Learned: – LGP collected indicator related baseline information but it was not well disseminated or understood by the field. • Recommendations: – Data reporting on attribution is necessarily longitudinal. The QOL surveys were a good first tool for this assessment, but needed to be complemented by other tools capable of identifying ‘stronger’ attribution over time. Examples of these are local citizen satisfaction surveys, and levels of service delivery against agreed standards. Less frequent QOL surveys should be implemented to build on earlier findings. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 26 Question 7 • What specific or contextual reasons explain star performing activities on the one hand and non￾performers on the other? • What general lessons are evident from both star performers and non-performers? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 173 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 27 Preliminary Findings Q7 • Currently tracking and validating possible ‘performers’ – Solid waste management in Baghdad has a complex environment with many players. It is proving difficult to assess the unique role played by LGP. – Work with the MMPW and MoSPA with decentralization and development of pilot projects and the LGA. – Development of Local Charters with Baghdad councils. – Training activities with Kirkuk DGs from 5 directorates (underway). • Non-performers (aside from those caused by external security factors) are expected to be caused by a lack of understanding of the underlying legal or administrative basis for the proposed intervention (to be validated, e.g., dismissal of the Amanat Council). Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 28 Question 8 • Out of the lessons learned, what are the pre￾requisites and/or the key assumptions for the different capacity building activities to be effective? United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 174 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 29 Preliminary Findings Q8 • Apart from the most obvious security issues: – Critical assumptions were that a framework for decentralization of authority would be in place within 90 days. This did not happen until much later. – Assumptions were made about the authority or willingness of DGs to support decentralization. There is some evidence suggesting that they may not. (USAID/Iraq Local Governance Project, Government Department/Directorate, Organizational Assessments Report: ‘The Public Utilities’, November 2004) • Lessons learned: – Include in the baseline studies a clear understanding of the legal and administrative environment of the institution where intervention is proposed. More emphasis has been given to this in LGP II and generally in the development of the PRT. Some of the institutions where interventions are proposed have been operational since the 1920’s (e.g., Amanat water and sewer). Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 30 Question 9 • Summarize lessons learned and suggest levels of USAID/Iraq involvement in managing projects. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 175 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 31 Preliminary Findings 9 • LGP provided seven lessons learned; these will be supplemented by those that emerge with our final report. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 32 Question 10 • Compare effectiveness of “up-front indicator development through explicit institutional analysis” with indicators developed during contractor operational activity. • Need clarification on the meaning of “up front indicator development through explicit institutional analysis.” United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 176 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 33 Preliminary Findings Q10 • Indicators changed significantly from the first PMP of September 2003 • An elaborate RF developed during September 2003 consisted of 7 PIRs (program IRs) and 32 Sub-IRs. Each Sub-IR had an indicator(s) associated with it. This was unmanageable. • The RF changed with the ‘Rush to Iraqi Sovereignty’ when the Civic Dialogue Program was added to the work program. LGP took this opportunity to change the RF and reduce the sub-IRs to 17. • The uprising in the South led to another revision of the RF shown in LGP I OYO work plan. The RF now had 13 sub-IRs, with indicators related to activities that had been developed from the bottom up. • By LGP I OYO it was clear that many indicators were not measuring results/outputs. This was especially so regarding capacity building activities. The organization capacity building literature, and other materials were researched in an attempt to achieve measurable capacity building indicators (example; using scales to measure institutional improvement in budgeting, planning or customer response activities that were the target of LGP training activities). These were not implemented. • By the end of OYO emphasis had moved to establishing service level standards for departments/directorates. • Lessons Learned: the measurement of performance indicators should be integrated with LGP activities such as ‘increased efficiency of local service delivery.’ The measurement of indicators by the Departments/Directorates as part of confirmation of increased efficiency should also provide for requirements under the PMP. Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 34 Question 11 • Assess effectiveness of RTI’s assistance to improve local governments administrative capacity to carry out their administrative functions. • Are the skills acquired during the training applied in undertaking their responsibilities? • To the extent possible, determine where those trained, especially under LGP1, are now and what are they doing. United States Agency for International Development Final Report Appendices Public Sector Capacity Building Evaluation International Business & Technical Consultants, Inc. 177 Monitoring and Evaluation Performance Program – MEPP-II United States Agency for International Development (USAID), Iraq Mission – International Business and Technical Consultants, Inc. 35 Preliminary Findings Q11 • Assessing effectiveness of RTI’s assistance: it isn’t likely that we will locate sufficient information on this topic as effectiveness wasn’t typically monitored. – There may be some evidence from financial operations in Basra, or from the CDP. – Councils: Baghdad PC Council members have attended LGP classes; plus there are some former council members that are being contacted. • Probably will look to the preparation of a Baghdad Provincial Charter as demonstrating effectiveness. – Directorates/Departments: field worker is in Kirkuk meeting with DGs and staff known to have received LGP training and attempting to validate the experience. • Lessons Learned: – Pending.