Tatweer National Capacity Development Program Final Evaluation i TATWEER NATIONAL CAPACITY DEVELOPMENT PROGRAM FINAL EVALUATION Final Report This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by The QED Group, LLC, under contract number 267-M-00-09-00513-00. ii Tatweer National Capacity Development Program Final Evaluation TATWEER NATIONAL CAPACITY DEVELOPMENT PROGRAM FINAL EVALUATION Final Report April 2011 The QED GROUP, LLC 1250 Eye Street, NW Suite 1100 Washington, DC 20005 Phone: 202-521-1900 Fax: 202-521-1901 www.qedgroupllc.com DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Tatweer National Capacity Development Program Final Evaluation iii CONTENTS CONTENTS.................................................................................................................iii ACRONYMS.................................................................................................................iv ACKNOWLEDGEMENTS.......................................................................................... vi EXECUTIVE SUMMARY........................................................................................... vii INTRODUCTION ........................................................................................................1 BACKGROUND ...........................................................................................................2 EVALUATION DESCRIPTION..................................................................................4 OBSERVATIONS & FINDINGS .................................................................................6 ANALYSIS & CONCLUSIONS ................................................................................13 RECOMMENDATIONS.............................................................................................25 REFERENCES .............................................................................................................30 ANNEX 1: USAID/Tatweer Results Framework....................................................32 ANNEX 2: USAID/Tatweer Functional Organization Chart ................................33 ANNEX 3: Final Evaluation Tatweer, Iraq – Self-Compliance Monitor...............34 ANNEX 4: Scope of Work Final Evaluation Tatweer, Iraq ...................................35 ANNEX 5: Pre-Departure Workplan For Evaluation of USAID Tatweer Program.......................................................................................................................41 ANNEX 6: Interview Log...........................................................................................47 ANNEX 7: Methodology ............................................................................................48 ANNEX 8a: Data Base Review..................................................................................61 ANNEX 8b: Tatweer Database Review ...................................................................67 ANNEX 9: Tatweer Interview Responses................................................................73 ANNEX 10: Hilla Group Interview...........................................................................74 ANNEX 11a: Budget Execution................................................................................75 ANNEX 11b: Tatweer Document Review: Information Technology (IT) Expenditure.................................................................................................................83 ANNEX 11c: Tatweer Document Review: Cost-Sharing.......................................92 ANNEX 11d: Donors Partnership ............................................................................96 ANNEX 11e: Tatweer Document Review: Training Programs ..........................102 ANNEX 12: Budget Execution Tatweer Ministries ..............................................105 iv Tatweer National Capacity Development Program Final Evaluation ACRONYMS BOTI Basra Oil Training Institute CCA Contractor Classification Application CLIN Contract Line Item Number CMMS Computerized Maintenance and Management System COMFAR Computer Model for Feasibility Analysis and Reporting COMSEC Council of Ministers Secretariat COSIT Central Organization for Statistics and Information Technology COSQC Central Organization for Standardization and Quality COTR Contracting Officer’s Technical Representative CSC Civil Service Commission DFID United Kingdom Department for International Development DG Director General DPMO-EA Deputy Prime Minister’s Office for Economic Affairs DPMO-S Deputy Prime Minister’s Office for Services EU European Union GAO General Accounting Office GEGI USAID/Georgia Enterprise Growth Initiative Project GIS Geographic Information Systems GOI Government of Iraq HR Human Resources IDMS Iraq Development Management System IR Intermediate Result ISCM Integrated Supply Chain Management ISO International Standards Organization IT Information Technology L&C Leadership & Communications LGP Local Governance Program LOC Letter of Credit MA Master of Arts M&E Monitoring and Evaluation MMPW Ministry of Municipalities and Public Works MOA Ministry of Agriculture MOE Ministry of Electricity MOEd Ministry of Education MOF Ministry of Finance MOH Ministry of Health MOHR Ministry of Human Rights MOLSA Ministry of Labor and Social Affairs MOMD Ministry of Migration and Displaced MOO Ministry of Oil MOP Ministry of Planning and Development Cooperation MORC Midland Oil Refineries Company MOWR Ministry of Water Resources NCCMD National Center for Consultancy and Management Development NCD National Capacity Development Program NDP National Development Plan NEDB National Employee Data Base NIC National Investment Commission NPARP National Public Administration Reform Program Tatweer National Capacity Development Program Final Evaluation v OIG Office of Inspector General/Iraq O&M Operations & Maintenance OSTP Organizational Self-Assessment and Transformation Program OTJ On-the-Job PC Presidency Council PERFORM Performance Evaluation and Reporting for Results Management PMC Public Manager’s Cycle PMO Prime Minister’s Office PMP Performance Management Plan PMS Performance Management System PRT Provincial Reconstruction Team RTI Research Triangle Institute SCOP State Committee for Oil Projects SO Strategic Objective SP Strategic Planning SOP Standard Operating Procedures SOC South Oil Company SOW Scope of Work SSN Social Safety Net T&D Training & Development TMS Training Management System TIR Tatweer Interview Responses TOT Training of Trainers TBI Trade Bank of Iraq UN United Nations UNDP United Nations Development Program UNAMI United Nations Assistance Mission for Iraq USAID United States Agency for International Development USG United States Government WB The World Bank vi Tatweer National Capacity Development Program Final Evaluation ACKNOWLEDGEMENTS This report was prepared under the auspices of the United States Agency for International Development (USAID)/Iraq Performance Evaluation and Reporting for Results Management (PERFORM) Project in Iraq, implemented by The QED Group, LLC. It was written by Eve Andrews, Robert Krause, Robert Brandstetter, and an Iraqi team member. Sincere thanks go to the USAID/Iraq Mission for the collaboration, support, and warm welcome provided throughout the process of fieldwork for this evaluation. Mission management, team leaders, and implementing partner staff provided substantial information, resources, and insight into the USAID/Tatweer project. Much appreciation also goes to the PERFORM local staff members for their invaluable assistance. In addition, thanks are extended to Iraqi governmental officials who shared their experiences, expertise, and insights in interviews and group discussions. Tatweer National Capacity Development Program Final Evaluation vii EXECUTIVE SUMMARY Purpose: The objective of this Final Evaluation is to “assess the relevance, performance, and success of the Tatweer Program” and to “provide information to USAID on whether the underlying theories and assumptions used in the development and implementation of the program were valid, what worked or did not work, and why.” Methodology: Findings were developed through: 1) semi-structured interviews with Government of Iraq (GOI) entities and Tatweer managers; 2) group discussions with GOI civil servants; 3) GOI and Tatweer documentation and database research; and 4) email questionnaires. Respondents were selected in coordination with the Tatweer Office with the goal of meeting the largest possible number of key GOI entities assisted by Tatweer, given the security and time constraints. In total, 92 interviews were conducted with 158 individuals and 197 questionnaires were fulfilled. In addition to the Baghdad interviews, team members visited Basra and Hilla and also met with the USAID Provincial Reconstruction Team (PRT) representatives for those two regions. The team was not able to visit Kurdistan. The methodology applied the research questions derived from the Scope of Work (SOW) to the field interviews in a structured and disciplined manner. The evaluation process steps were: 1. Development of evaluation criteria and questionnaires from the SOW; 2. Semi-structured interviews with GOI entities and Tatweer managers; 3. Preparation of standard “Interview Summary” reports following each interview; 4. Analysis of each Interview Summary for evaluation criteria, capturing findings in each report; 5. Integration of findings into 11 Tatweer Interview Responses (TIRs), one for each criterion; 6. Uploading of Interview Summaries and updated TIRs each night to a shared documents site for review and revision (if applicable) by all team members; and 7. Application of all TIR findings to Tatweer final evaluation (Findings section and Analysis and Conclusions section) upon completion of field interviews. Research Evaluation Criteria: 1) Achieved Results; 2) Targeted Results Not Achieved; 3) Critical Lessons Learned, Challenges, and Performance Highlights; 4) Underlying Theories and Assumptions / Validity of Hypotheses of Results Framework; 5) Cross-Cutting Lessons Learned; 6) Impact; 7) Sustainability; 8) Client Satisfaction; 9) Cost Effectiveness; 10) Relevance. At the request of USAID, cost sharing, gender, and donor partnerships were also evaluated as criterion number 11. Achieved Results: Tatweer achieved major results in all three project Sub-Intermediate Results (IRs) of Strategic Objective 10. Despite significant security and other challenges, Tatweer did so by responding flexibly to client needs and by engaging with high quality Arabic speaking team leaders and local expert staff who were supported by competent core area experts. Tatweer produced its most important results when it delivered skills training, Training of Trainers (TOT) training, technical assistance, and embedded consultancy, sometimes supplemented by the provision of equipment. • Strengthening Public Administration Skills (Sub-IR 10.1.1) Tatweer delivered training to over 105,000 participants for 76,000 individual employees, far exceeding its original numerical target of 58,282 for training participants and representing nearly one-third of the 320,000 civil servants targeted for skills training in the GOI entities assisted by Tatweer. • Public Administration Training Capacity Expanded (Sub-IR 10.1.3) Tatweer’s training of more than 1,200 TOTs in all core areas (plus additional methodologies such as Standard Operating Procedures (SOP), Performance Management System (PMS), and TMS) represents another major achievement, which significantly increased Iraq’s infrastructure for training. The Tatweer TOT success was achieved primarily through effective leadership and talented individuals in the client organizations who were proactive and innovative as TOTs in the implementation of training. viii Tatweer National Capacity Development Program Final Evaluation • Ministries Establish More Effective Administrative Systems (Sub-IR 10.1.2) The official GOI data provided by Tatweer for the budget execution proxy indicator does show an increase from 71.4 percent in baseline year 2006 to 94.0 percent in 2009, the last year for which official data is available. Tatweer improved administrative systems significantly in a number of key GOI organizations, particularly when it combined training, TOT, technical assistance, and embedded consultants. A number of GOI entities are now implementing the Organizational Self-Assessment and Transformation Program (OSTP) for instituting organizational change throughout all departments; the Council of Ministers Secretariat (COMSEC), the Midland Oil Refineries Company (MORC), and South Oil Company (SOC) represent just a few examples. The National Development Plan (NDP) and Civil Service Laws are arguably Tatweer’s two greatest achievements, but it is too early to assess the full impact of these accomplishments. Results Not Achieved: The only major result not achieved as originally planned was the goal of turning the National Center for Consultancy and Management Development (NCCMD) into the lead training organization for the Iraqi Civil Service. The Tatweer arguments that the original design was flawed, that the goal was unrealistic, and that NCCMD lacked the capacity for high-volume skills training are compelling: In short, NCCMD was not ready for the technical assistance being offered. The Tatweer course correction, in consultation with USAID, to focus upon updating or creating training departments in GOI entities, to establish five regional training centers, and to emphasize TOT training was the right approach to overcome the problem and achieve the overall objective of expanding training capacity. From this perspective, the course correction represented an achieved result. The current pool of over 1,200 Tatweer-trained TOTs is well positioned to provide the new Civil Service Institute with a strong foundation in the future. Critical Lessons Learned: Most importantly, Tatweer achieved measurable results by increasing awareness and support of top-level managers. Lessons learned for improving training delivery include the need for serious follow-up, the need to conduct needs assessments prior to training delivery, more technical courses, higher-level courses, and more on-the-job (OTJ) training. Challenges: Security posed the greatest challenge for implementation, but there were other major challenges as well. Foremost among these were the low capacity of the civil servants, and senior officials’ old mentalities and lack of basic management skills. These challenges, of course, were the basis for the Tatweer intervention, but they constituted implementation challenges as well. Performance Highlights: In addition to the major results achieved, Tatweer’s performance highlights were marked by a highly effective Energy Group delivering major results the oil sector. Other highly effective Tatweer teams included those for the Ministry of Planning and Development Cooperation (MOP), the Ministry of Health (MOH), the Ministry of Municipalities and Public Works (MMPW), the Ministry of Agriculture (MOA), OSTP, and Civil Service. Validity of the Theories and Assumptions Underlying the Results Framework: The inferred hypotheses for design of the Tatweer project were: 1. If public administration skills training relevant to urgent and priority needs were provided to a sufficiently high number of Iraqi civil servants, then a critical mass of sufficiently skilled employees would be established to enable implementation of more effective administrative systems; 2. If training capacity were built to provide Iraqi ministries and executive offices with ongoing training, then the foundation for implementation of more effective administrative systems would be sustained; and 3. If training efforts were supplemented by technical assistance and focused expert team engagements, then departmental processes, policies, and procedures could be changed to Tatweer National Capacity Development Program Final Evaluation ix create more effective administrative systems, thereby enabling the trained critical mass of civil servants to apply their new skills. Together, these three hypotheses constitute the necessary and sufficient conditions for establishing and sustaining more effective public administration systems. The original program design based on these hypotheses was valid and remains valid. Impact: Tatweer achieved its intended results. Positive net impacts are both immediate and longer term. The immediate impacts are manifold. Tatweer project implementation also produced a number of unintended impacts. One of the unintended impacts was resolving the Letter of Credit (LOC) problem for the Ministry of Oil (MOO) with the Trade Bank of Iraq (TBI). Moreover, there were a number of intended impacts that were not captured by Tatweer performance indicators, including “soft impacts” such as organizational culture changes and the embrace of modern techniques and methods. Sustainability: Prospects for sustainability of Tatweer results are strong, but not assured for all GOI entities. The main constraint is that many senior level officials have not yet committed fully to administrative systems reform, nor have these officials recognized the urgency and importance of skills training. Prospects are strongest for MOO and its 16 operating companies, and the partial Tatweer success at the Ministry of Electricity (MOE) is at greatest risk. The scale was appropriate to ensure sustainability. Both the number of civil servants trained (76,000) and the 1,200 TOTs trained were sufficient. Further, Tatweer implemented systems reforms in most of the GOI entities it assisted. Also, the technical assistance resources for focused team engagements were sufficient, especially for MOO, all the executive offices, MOP NDP development, and civil service reform. Client Satisfaction: For the vast majority of Tatweer GOI clients, needs were met with high praise. Among these were important GOI entities, including MOO, MOP, MOE, the Ministry of Education (MOEd), MOA, MORC, MMPW, Presidency Council (PC), the Deputy Prime Minister’s Office for Economic Affairs (DPMO-EA), and the Deputy Prime Minister’s Office for Services (DPMO-S). Cost Effectiveness: The Tatweer program was implemented and managed cost effectively. Costs were estimated at $648 per GOI civil servant trained at $72 per day. While there are no donor benchmarks for cost per participant per training day, evaluation team member experiences with similar capacity building projects indicate that the net training day cost of Tatweer training is comparable: $50 for the United Nations Development Program (UNDP) and $75 for The World Bank (WB). Total net benefits included more than 76,000 trained employees, more than 1,200 TOTs, system improvements for the NDP, Civil Service Reform, MOO, MOP, and other GOI entities, and all other benefits noted in the Achieved Results section. The long-term value and multiplier effects of the TOTs and other assets Tatweer created (people, systems, and tools), coupled with the mindset changes Tatweer inculcated in the Iraqi Civil Service, will provide lasting benefits that go far beyond the immediate recipients of training and systems changes. Cost Sharing: The major examples of cost sharing achieved voluntarily in the Tatweer program were in-kind contributions. The most common examples were the GOI contributions of space and furnishings for the establishment of Information Technology (IT) and/or training centers, with Tatweer providing IT equipment and software. Notably, GOI reported that it contributed $32.2 million in 2009 to Tatweer systems improvements. USAID/Tatweer analysis indicates the total in￾kind cost sharing by GOI entities amounted to over $100 million during the course of the project, a sum that represents nearly $1 USD for every $3 USD invested. Donor Partnerships: Tatweer had strong partnerships with a number of donors, including the European Union (EU) and United Nations (UN) agencies. Tatweer worked closely with the United Nations Mission in Iraq (UNAMI) for NDP development and with UNDP for the Iraq Development x Tatweer National Capacity Development Program Final Evaluation Management System (IDMS) and the Computer Model for Feasibility Analysis and Reporting (COMFAR) III software implementations. Tatweer and the United Kingdom Department for International Development (DFID) also collaborated in their work at COMSEC. Another important partnership was with the World Bank for the Social Safety Net (SSN) system implementation at the Ministry of Labor and Social Affairs (MOLSA). Further, Tatweer closely coordinated its work with other USG/USAID programs such as Inma (agribusiness), Tijara (provincial economic growth), and the USAID/Research Triangle Institute (RTI) Local Governance Program (LGP). Gender: The principal conclusions from the Tatweer training data are: 1) the more developed the province, the higher the level of female participation; 2) the percentage of women participating in Tatweer training was virtually identical to the percentage of female GOI ratios and of TOTs; and 3) female civil servants sought and obtained additional training as frequently as male civil servants. Relevance: The Tatweer program is highly relevant to Iraqi development needs in the short, middle, and long term. Short term, the high number of civil servants trained in modern methodologies and the initial multiplier effect of TOT training creates immediate service improvement impact. Medium term, GOI should be in a much better position to establish the business-enabling environment that will not only attract increased foreign direct investment but also support the growth of the Iraqi private sector. Long term, GOI will be challenged with managing the rapid growth in its oil revenues in a way that provides real economic opportunities for the Iraqi population. That’s why the NDP focus on private sector development is so important. That’s also why the establishment of more effective administrative systems in MOO and its 16 operating companies is so important. Recommendations: The following recommendations could inform the National Public Administration Reform Program (NPARP) implementation modalities and/or USAID strategic planning, particularly for the introduction and implementation for new programs. 1. Ensure Engagement of GOI Entity Top Management Many of Tatweer’s potential systems improvement results were not fully realized in all the assisted GOI entities due to the lack of active engagement by top-level managers in certain GOI entities (e.g., MOE) to implement administrative reforms in procurement, human resources (HR), budget, and project management systems. In order to assure high success potential for full implementation of results, projects at the outset should develop and execute focused and innovative “top-level engagement” action plans customized to the particular conditions and likely barriers at each GOI entity. Given the difficult security situation in 2006/07, Tatweer could not have fully implemented this recommendation but it should be considered for future project implementation. 2. Improve Effectiveness of Technical Assistance Delivery Effective technical assistance delivery is a critical element for achieving targeted systems reforms. Effective delivery requires an action plan that targets the right technical assistance for each GOI entity and the recruitment of the right people to execute the action plan. The most important recommended action is to conduct a needs assessment for each GOI entity, taking into account all departmental needs, and to develop a capacity building strategy before the initiation of technical assistance. Further, GOI entities should be encouraged to continue Tatweer’s OSTP and TMS initiatives in order to better define needs. 3. Improve Effectiveness of Training Delivery Continued delivery of skills training to GOI civil servants is another critical element for achieving targeted systems reforms. Given the large number of civil servants yet to be trained and the need to provide higher level training to many who have already received competency level training, even greater emphasis must be placed on expanding TOT delivery. The Tatweer TOTs, Scholarship Tatweer National Capacity Development Program Final Evaluation xi Program returnees, and OSTP teams constitute a major asset for the NPARP implementation to accomplish this objective in conjunction with setting up the new Civil Service Institute. 4. Improve Coordination at All Levels Achieved results can always be improved through better coordination at all levels to leverage resources and to integrate more closely initiatives focused on administrative systems reforms. Several actions are recommended for project implementers, including: 1) meet regularly with key stakeholders at all levels to develop inputs required for coordinated action plans; and 2) implement new programs with closer coordination between the national GOI entities and the provincial entities as well as among provincial entities and district and even sub-district entities. 5. Increase Flexibility to Respond to Changing Demand Maximum achieved results can only be accomplished if a project can deliver the right technical assistance to the right places with the right people at the right time. New opportunities, obstacles, and conditions affecting demand cannot always be foreseen. Rather than looking to multiple contract modifications to respond to changing conditions, USAID might consider “demand driven” components of new programs that would give projects the flexibility to respond to changing conditions more rapidly. Developing Contract Line Item Numbers (CLINs) with “demand driven” components (e.g., 20 percent) is one alternative for increasing flexibility. With such CLINs, the contractor could propose appropriate interventions to USAID for approval. The USAID/Georgia Enterprise Growth Initiative Project (GEGI) used a “demand driven” CLIN in 2003-2005. Contingency funding mechanisms could also be considered. 6. Special Considerations for Industry-Specific Technical Assistance GOI entities have industry-specific technical assistance and training requirements in addition to commonly shared management and departmental needs. In order to implement GOI systems reforms that have real impact and actually result in the improved delivery of services to the Iraqi people, technical assistance and training must address the industry-specific needs as well as the commonly shared ones. Tatweer’s Energy Group and several other teams (e.g., MOP, MOH, MMPW, MOA, OSTP, and Civil Service) demonstrated the high level of results that can be achieved through focused team engagements led by highly qualified industry experts. Serious consideration should be given to replicating this model with all assisted GOI entities. Tatweer National Capacity Development Program Final Evaluation 1 INTRODUCTION In December 2010, QED PERFORM (Performance Evaluation and Reporting for Results Management) in Iraq were tasked by USAID/Iraq to undertake the Final Evaluation of the National Capacity Development Program). NCD, known by the Arabic word Tatweer (“development”) was a $340 million program that began in September 2006 and was due to be completed in 2010. Currently the project is in a six-month no cost extension phase until July 31, 2011. 2 Tatweer National Capacity Development Program Final Evaluation BACKGROUND The Iraq national elections in December 2005 led to the formation of the first permanent elected government since the fall of the Saddam regime. As a result of decades of war, neglect, and international sanctions, the new regime inherited a dysfunctional civil service administration with most ministries unable to fulfill the basic functions of government. During this entire period, the ministries, executive offices, and other government agencies had been isolated from modern international standards, systems, technology, and best practices. The government decision to move toward a more decentralized civil administration in the provinces contributed to the complexities of the situation, which continue to be difficult due to the persistently dangerous security environment. NCD was launched in 2006 to contribute to correcting the lack of governmental capacity in selected key GOI entities. Within USAID’s transition framework for assistance to Iraq, Tatweer was designed to fulfill Strategic Objective (SO) 10: Capacity of National Government Institutions Improved. Tatweer focused on Intermediate Result (IR) 10.1: Assisted ministries demonstrating sustainable performance improvements in core administrative functions. Three Sub-IRs form the Results Framework: 1) Strengthened Public Administration Skills; 2) Ministries Establish More Effective Administration Systems; and 3) Public Administration Training Capacity, and a total of 34 Sub IR Indicators were developed to ensure optimal tracking of the IRs. (See Annex 1: Tatweer Results Framework and Annex 8a: Database Review of PMP Indicators.) Since 2006, Tatweer has evolved in response to changing conditions including the government changes in late 2010. Phase I of the program was designed to train many of the 1.0 million Iraqi civil servants (excluding 1.6 million security and military personnel) across Iraq in modern public management concepts and methods. In the Government of Iraq (GOI) entities assisted by Tatweer, there were approximately 320,000 targeted government employees (Grade 8 and above). Phase II reflected the new “surge” program initiated by the United States Government (USG) in January 2008. These adaptations included a shift from direct training by Tatweer of GOI civil servants to implementation of a Training of Trainers (TOT) methodology and the rollout of these Iraqi trainers to ministries and their provincial directorates. At the same time, there was also a major increase in emphasis on improving and/or establishing GOI administration systems and processes. As part of the second surge, Tatweer increased its focused efforts by forming the Tatweer Energy Group to provide enhanced technical assistance to MOO, its 16 operating companies, and MOE. The provincial deployment was the last major modification. Special note should be given to the fact that the Tatweer evaluation team was not able to visit Kurdistan. Tatweer documents, however, provide solid findings of major achievements in all three project areas in Kurdistan. In fact, Tatweer trained more Ministry of Finance (MOF) civil servants in Kurdistan than Baghdad. The Kurdistan Ministry of Planning was particularly appreciative of Tatweer assistance. Tatweer installed a customized version of its Training Management System (TMS), its Inventory Management System (IMS), and its Contractor Classification Application (CCA) complete with screens and fields translated into Kurdish. It is now taking steps to implement international standard procurement regulations. Special note should also be given to the scale and complexity of the Tatweer project. There were 20 separate institutional teams, 18 cross-cutting functional teams, and five regional offices engaged. Tatweer developed 22 separate components of its annual work plans to manage its large number of tasks. The project employed 320 professionals, including an Iraqi staff of up to 230 experts and 30 local-hire senior advisers. Tatweer engaged over 35 expatriate Arabic speakers, and it worked with 10 ministries at the national and provincial levels, six executive offices, and several other GOI entities, including the National Investment Commission (NIC), three training entities, and MOO’s 16 Tatweer National Capacity Development Program Final Evaluation 3 oil operating companies and refineries. The Tatweer organizational chart (Annex 2) shows the matrix management structure employed for the successful implementation of such a large scale and complex project. 4 Tatweer National Capacity Development Program Final Evaluation EVALUATION DESCRIPTION This is a Final Evaluation of the Tatweer program and is part of the routine program monitoring process practiced by USAID. A team of three expatriates and one Iraqi expert, supported by the QED PERFORM Monitoring and Evaluation (M&E) Adviser and support staff, carried out the assessment over a 12-week period from mid-January 2011. Team members brought diverse experiences to the evaluation, including public administration reform expertise, USAID project experience, M&E knowledge, and Iraqi familiarity. Throughout the evaluation process, continuous consideration was given to the USAID Evaluation Policy of January 2011, directing evaluators to “self-monitor” to ensure the team maintained full compliance. (Annex 3) Purpose According to the Tatweer Final Evaluation SOW (Annex 4), the objective of this Final l Evaluation is to “assess the relevance, performance, and success of the Tatweer Program” and to “provide information to USAID on whether the underlying theories and assumptions used in the development and implementation of the program were valid, what worked or did not work, and why.” The aim is to “determine relevance, efficiency, effectiveness, impact, and sustainability.” Methodology During the first two weeks of the evaluation, the team undertook home-based desk research, studying Tatweer reports and related documents and preparing a Pre-deployment Work Plan. (Annex 5) Six weeks were then spent in Iraq conducting field interviews, reviewing Tatweer databases, and distributing distance questionnaires. A presentation of the draft evaluation report was given to USAID before the team returned home to finalize and deliver the final draft. Findings were developed through: 1) semi-structured interviews with GOI entities and Tatweer managers; 2) focus group discussions with GOI civil servants; 3) GOI and Tatweer documentation and database research to close gaps in evaluation criteria; and 4) email questionnaires. Respondents were selected in coordination with the Tatweer Office with the goal to meet the largest possible number of key GOI entities assisted by Tatweer, given the security and time constraints. In total, 92 interviews were conducted with 158 people, including 117 GOI civil servants with whom meetings could be arranged and 36 Tatweer staff who were available in Iraq when the team arrived. (Annex 6) Lastly, 197 questionnaires were completed by Tatweer trainees. The methodology applied the research questions derived from the SOW to the field interviews in a structured and disciplined manner. The evaluation process flow steps were: 1. Development of evaluation criteria and questionnaires from the SOW; 2. Semi-structured interviews with GOI entities and Tatweer managers; 3. Preparation of standard “Interview Summary” reports following each interview; 4. Analysis of each Interview Summary for evaluation criteria, capturing findings in each report; 5. Integration of findings into 11 Tatweer Interview Responses (TIRs), one for each criterion; 6. Uploading of Interview Summaries and updated TIRS each night to Google shared documents for review and revision (if applicable) by all team members; and 7. Application of all TIR findings to Tatweer final evaluation (Findings section and Analysis and Conclusions section) upon completion of field interviews. The team overcame a number of constraints during the evaluation process, most of which related to security and the inability to arrange all desired meetings at GOI entity locations. In Basra, the team adjusted by arranging a large number of interviews at a secure hotel. In Hilla, the team adjusted by arranging interviews and a focus group meeting at the Tatweer Regional Office and the Local Tatweer National Capacity Development Program Final Evaluation 5 Governance Program (LGP)/RTI Office. In Baghdad, the team was able to visit most GOI entities but also supplemented the number of interviews by inviting GOI civil servants to meet at the QED PERFORM office. Nonetheless, it is important to note the data retrieval limitations. First, the team was not able to meet with many of the Tatweer teams, as most teams had completed their assignments by the end of January. Second, the Tatweer Regional Office in Erbil had closed before the team’s arrival in Baghdad, thereby making a trip to Kurdistan unworkable for securing desired meetings. Third, the security lockdowns in Baghdad due to street demonstrations in early March limited interview responses from MOA and MOH. Fourth, as interviewees had limited knowledge about several evaluation criteria questions, document sources were required to fill gaps in interview findings for cost effectiveness, donor partnerships, gender, and cost sharing. Research Questions: Evaluation Criteria Evaluation Criteria from SOW Background and Purpose Sections Evaluation Criteria from SOW Research Questions Section 1) Achieved Results 6) Impact 2) Targeted Results Not Achieved 7) Sustainability 3) Critical Lessons Learned, Challenges, and Performance Highlights 8) Client Satisfaction 4) Underlying Theories and Assumptions / Validity of Hypotheses of Results Framework 9) Cost Effectiveness 5) Cross-Cutting Lessons Learned 10) Relevance Additionally, at the request of USAID, the evaluation team asked interviewees about gender, cost sharing, and donor partnerships. (The methodology, interview research questions, and evaluation criterion are presented in more detail in Annex 7). 6 Tatweer National Capacity Development Program Final Evaluation OBSERVATIONS & FINDINGS The findings were developed primarily from the interviews and focus groups with GOI entities and Tatweer managers. The team also reviewed GOI and Tatweer documents and Tatweer databases (Database Review: Annex 8a) to augment and triangulate interview findings. Distance questionnaires were used to validate training findings from the interviewees. The eleven TIRs contain all interview and focus group findings. (Annex 9) The following is a summary of major findings from all sources. Achieved Results Training Delivery: According to Tatweer documents, the project trained over 76,000 individuals in Baghdad and the 17 provinces, which include the three Kurdistan provinces. Training was provided to over 105,500 participants. Initially, expatriate Arabic-speaking trainers delivered most training. After 2008, training was delivered by a cadre of Iraqis using the TOT methodology, which markedly increased the number of GOI trainees. Tatweer’s five core training areas were Leadership and Communications (L&C), HR, IT, Fiscal and Project Management (PM). Tatweer added courses to meet client demands for Strategic Planning (SP), SOP, PMS, Training Management System (TMS), Training Evaluation and many others. (Annex 8b) Simultaneously, Tatweer increased the GOI training infrastructure by establishing five regional Training & Development (T&D) centers and centers within ministries and executive offices. Tatweer trained GOI entities to use the international standard TMS to increase T&D staff capacity and to promote sustainability of GOI self-provided training. Technical Assistance: Tatweer delivered technical assistance, mentoring, and coaching, and focused team engagements to their assisted ministries and executive offices. The technical assistance was provided by embedded teams drawn from an Iraqi staff of up to 230 experts, 30 local hire senior advisers, and over 35 expatriate Arabic speakers over the life of the project. The GOI entities interviewed reported that the technical assistance and tools were highly effective and valued. Examples of highly valued tools include the Integrated Supply Chain Management (ISCM), OSTP, and IDMS. National Development Plan: The former MOP Minister stated in the acknowledgments section of the NDP that Tatweer provided “invaluable extensive technical and logistics support” in assisting MOP to develop the NDP; approved by the Iraqi Council of Ministers in April 2010. A Civil Service Framework: Tatweer senior managers reported that Tatweer’s Civil Law Reform Team Leader was instrumental in the writing and approval of the Civil Service Commission (CSC) Law. He also drafted the proposed Civil Service Law, and he is serving as the only non-Iraqi member of the CSC. He noted that further contributions to the establishment of a professional Civil Service Framework include increasing the capacity of human resources units in GOI entities, full implementation of the National Employee Data Base (NEDB), Job descriptions, and performance management systems. Council of Ministers Secretariat (COMSEC) interviewees reported that standard operating procedures (SOPs) are also being adopted by many ministerial and executive office departments. More Effective Administration Systems: MOP advised that it has mandated the use of the IDMS for all capital investments planning, and management. Tatweer IT specialists helped develop IDMS and implemented COMFAR III software for feasibility studies and Primavera project management software to enable ministry and provincial linkage to IDMS. In addition, Tatweer helped implement Geographic Information Systems (GIS) at MOH and MOA. Tatweer is now working with MOLSA to digitize the SSN system. Central Organization for Standardization and Quality (COSQC) Tatweer National Capacity Development Program Final Evaluation 7 made its first outsourcing contract when it contracted quality control of imports to two foreign inspection firms. Capital Investment Budget Execution: According to MOP data, the 10 ministries assisted by Tatweer increased their capital budget execution from 71.4 percent in 2006 to 94.0 percent in 2009, the last year for which official final data is available. (Annex 11a & Annex 12) Results Not Achieved NCCMD as Primary Mechanism for Training Capacity (Sub IR Indicators 10.1.3.1 to 3.5): NCCMD was originally designated as the focal point for building training capacity. Tatweer reported, however, that as the demand for Civil Service training exceeded the capacity of NCCMD in terms of management, regional outreach, and number of trainers, Tatweer shifted its emphasis to a TOT methodology by which it established five regional training centers and other training centers within GOI entities. Improving Effectiveness of Administrative Systems (Sub IR10.1.1.1 to 1.9): GOI interviewee responses indicate that the full training impact was not consistently realized due to the lack of senior level support. For example, MOE has not incorporated the Public Manager’s Cycle (PMC) course into its training program, whereas MOO has embedded it. MMPW has established an HR Directorate, but all other ministries the team visited have just re-named their previous personnel departments. GOI interviewees also indicated that project management systems have not been established in all ministries. According to COMSEC and DPMO-EA, the PC remains the only executive office to have established a public policy office. Public Administration Skills Training (Sub IR 10.1.2.1 to 2.16): Suggestions for improving the quality of training delivery were expressed during the GOI interviews. These issues are presented in the Learned Lessons section. Cross-cutting Lessons Learned: Anti-Corruption A number of effective anti-corruption initiatives have been implemented with Tatweer assistance: • The Tatweer Basra Regional Office reported that it had established anti-corruption offices in each of the four provincial councils in its region. • The Tatweer Babil Regional Office reported that it had conducted three anti-corruption workshops, assessed recommendations made during these workshops, developed an anti￾corruption strategy based on this assessment, and submitted the strategy to the Integrity Committee in Baghdad under the Office of the Prime Minister for national coordination. • The Tatweer Babil Office also reported its success in streamlining the number of signatures required for procurement contract payments, thereby removing many corruption opportunities. • At the national level, MMPW reported that it had established anti-corruption complaint teams with contact numbers and email addresses advertised on posters and on their websites with the purpose of raising awareness and supporting the combating of corruption. The MOE website also has the same system. • Several Tatweer managers indicated that full implementation of the NEDB would result in the removal of thousands of “ghost workers” from the rolls and also expose endemic nepotism. Similarly, the Tatweer managers indicated that the SSN implementation will substantially reduce the number of shadow and duplicate beneficiaries, thereby reducing payment leakages substantially. • According to project documents, the MOP is now promoting the adoption of a cross￾ministry Contractors Classification Application (CCA) developed by Tatweer as another anti-corruption measure. The CCA provides GOI entities with an online listing of certified and blacklisted contractors and it makes contract tenders transparent. 8 Tatweer National Capacity Development Program Final Evaluation • MOP interviewees reported that the full implementation of IDMS will also have a significant impact on corruption control by putting the provincial project planning system online and also making the process of capital investment more public and transparent. Critical Lessons Learned, Challenges, and Performance Highlights Critical Lessons Learned: • Interviewees indicated that the personality, credibility, and length of service of team leaders heavily influenced the degree of results in individual GOI entities. Interviewees also spoke positively of the difference that credible Arabic specialists can make. • Provincial staff were found to be very cooperative, eager to learn, and capable of influencing central government improvements, e.g., development of the standard bidding document. • Among the 19 participants in the Hilla group interview, 42 percent of the participants had recently received HR training, but only 16 percent were able to apply it, as basic HR systems are missing in their directorates, including job descriptions and HR policies and procedures. It should be noted, however, that Tatweer Post-Training Surveys of over 10,000 trainees in all areas showed that 60 percent of respondents indicated that they “frequently” used their skills in the workplaces and that another 35 percent indicated that they “sometimes” used their new skills at work. • The Hilla and MMPW Group interviewees noted that training could be improved in a number of areas: needs assessments conducted at the outset, longer courses, more follow￾up, more practical and less theoretical courses, better translations, OTJ training, higher level and more advanced content, more technical and customized programs, better selection processes for trainees and trainers, more training conducted onsite, and more effort in increasing awareness at higher levels of management regarding the value of training. Challenges: The most frequently cited challenges were security issues related to the inability of international consultants to travel in Iraq and the geographic spread of the project. The low management capacity of the civil servants was mentioned in all interviews with T&D staff. Many mid￾managerial interviewees noted the “bureaucratic mindsets” of senior managers and their lack of basic management skills, adding that the senor mangers often did not understand the importance and urgency of public administration reforms. Performance Highlights: Clear examples of administrative improvements were reported by MOP, MOH, MOA, MMPW, MOEd, MOO, MORC, SOC, Basra Oil Training Institute (BOTI), and the Provincial Directorates. Interviewees most often cited the 105,000 enrolled trainees as the quintessential performance highlight. The Tatweer Energy Group was often cited as a stellar example of combined training, TOT, and focused team engagements. Other reported performance highlights include the establishment of five regional training centers and the training centers in GOI entities; effective implementation of the TOT model; the Civil Service Commission Law and the pending Civil Service Law; the NDP; and the MOP cross-ministerial mandate to implement IDMS. Validity of Hypotheses and Assumptions Underlying the Results Framework The key document findings were that the three Sub-IRs were not changed during the project and that all the results were achieved within the original results framework. Tatweer managers reported that course corrections were necessitated by changed conditions. These corrections included the shift away from NCCMD, the shift away from ministries blocking or not fully supporting Tatweer initiatives, the addition of new courses to meet GOI entity training demands, and the adoption of matrix management in mid-project to provide a closer alignment of Tatweer training and technical assistance with GOI entity priority needs. There were no other findings directly linked to this criterion. The Analysis and Conclusion section will consider all findings in addressing the validity of hypotheses and assumptions questions. Tatweer National Capacity Development Program Final Evaluation 9 Impact • Interviewees reported the beginning of a fundamental change in the mindset of senior managers and the organizational culture of GOI entities. They indicated that civil service cadres in Tatweer assisted GOI entities have gained a sense of pride and ownership in their work and services. In a meeting with senior managers of MORC, one manager stated that the management of his company “was the best in the oil sector.” • All interviewed GOI entities noted that the IT capacity developed with Tatweer assistance has enabled them to adopt modern and professional management systems heretofore unavailable to government civil servants. MOP, with the Tatweer technical assistance, developed Iraq’s first-ever integrated National Development Plan. According to MOP, the plan was developed through a broad-based approach in which all stakeholders contributed actively. According to DPMO-EA, implementation will depend upon careful follow-up by senior officials. • Tatweer interviewees advised that the current formation of the National Civil Service Commission and the approval of the pending Civil Service Law would together re-write the relationship between the government and civil service employees. They noted that the implementation of this legislation would fundamentally reform and modernize the Iraqi civil service, a precondition for sustained economic growth. • GOI entities also enthusiastically emphasized the current impact of many improvements in various government procurement systems. They maintain that the reformed procurement and project management practices will have a significant and lasting impact. Sustainability The impression gained from the interviews was overwhelmingly positive: the skills learned from the Tatweer training programs are sustainable in many cases, creating the potential for continuance of the new administrative systems and for the expansion of training capacity via the TOT mechanism. At the same time interviewees cautioned that sustainability is likely to be inconsistent due to different levels of buy-in from different entities. Examples from interviewees included: • PC interviewees cited significant staff performance improvements, project management, and strategic planning as indicators of sustainability. • MOO interviewees outlined their capacity development strategies, which include the institutionalization of the QMC program, TMS, ISCM, IDMS, and OSTP. • MOP reported that all national ministries and provincial directorates can now access IDMS and update project information for planning and budget execution. MOP also indicated that it is using Tatweer assistance for M&E follow-up for NDP implementation and that it will develop annual reports to show how the NDP is improving the lives of Iraqi citizens. • MORC reported that its annual training plan involves 200 trainees attending courses each day in six training halls. • SOC in Basra reported having an annual training budget of $12-13 million. • DPMO-EA cited the importance of the IT systems and capacity. • COMSEC cited the active role of its TOTs in implementing SOPs in all departments. • The Hilla Municipalities Directorate reported a plan to conduct 70 training courses in 2011, with the Tatweer TOT graduates being the main trainers. • The MMPW reported that its Training Plan shows that TOTs will provide over 500 courses across Iraq in 2011. • The USAID PRT representative in Hilla confirmed that MOA TOTs actively continue to train now that Tatweer has finished. Client Satisfaction A large majority of interviewees expressed a very high level of satisfaction with Tatweer assistance and the 197 questionnaire respondents averaged a satisfaction rate of 4.6 out of 5 with Tatweer Training. This high level of satisfaction was found at all levels of Tatweer beneficiaries, from administrative trainees to deputy ministers. MOP Deputy Minister cited Tatweer’s “instrumental 10 Tatweer National Capacity Development Program Final Evaluation assistance in the development of the NDP.” Also, The MOP IT Manager noted that “Tatweer was critical to the implementation of IDMS.” Similarly, MOO Training and Development Director stated that “Tatweer provided very expert specialists; we are very pleased with all of the assistance and advice received.” He specifically noted the modern concepts that Tatweer delivered, including project management, TMS, TOT, and job descriptions. The Head of Protocol at Deputy Prime Minister’s Office for Services (DPMO-S) offered that “I could talk for three days about Tatweer‘s assistance to the DPMO.” In Basra, the Electricity Directorate and SOC representatives reported that Tatweer activities were essential, especially the training evaluation. The PRT representative in Basra and Maysan stated that during his work in these provinces, he had never heard a single negative comment on Tatweer activities. Feedback from the Hilla and the MMPW focus group meetings (Annex 10) also indicated high client satisfaction. Cost Effectiveness Interviewees had little or no information about cost effectiveness. But notably, none offered any reports of perceived waste or financial mismanagement. One GOI interviewee in Basra did express concerns about abuse and possible corruption associated with the $13 per day training stipend, indicating that the stipend provided an incentive for favored and “connected” civil servants to attend multiple training courses, irrespective of suitability. Tatweer’s COSQC Team Leader also noted the COSQC study tour on measurement and testing was funded by the U.S. Commerce Department. Findings for Tatweer budget allocations were derived from the limited information available in Tatweer documents. Estimates for those budget allocations are presented below. The assumption for the security allocation was originally based on general experiences in Iraq, although Tatweer managers provided the final figure. The IT equipment estimate was based on a Tatweer Document Review. (Annex 11b) The estimated budget allocation finding will be used in the Analysis and Conclusions section for cost effectiveness. • Total Tatweer Budget: $339.5 million • Reported Expenditures – LOE expenditure: $121.1 million (36 percent) – Direct Costs and Grants: $218.4 million (64 percent) • Assumptions – $119 million: Security – $50 million: IT Equipment / TC Renovations / Learning Materials Relevance: Short, Medium, and Long Term Tatweer managers emphasized the importance of oil revenues to Iraq’s short, medium, and long￾term development, particularly Iraq’s long-term economic growth. They also pointed to the long￾term importance of the introduction of automated systems to replace manual systems, MOEd’s decentralization of procurement, and the establishment of five regional training centers. Tatweer managers also noted the short-term importance of the establishment of websites by MOEd and other ministries. MORC interviewees stressed the long-term importance of the training and computer labs established with Tatweer assistance. DPMO-S noted that Tatweer assistance was highly relevant to developing a long-term culture of public service. MOEd reported the major long￾term influence of tendering and procurement modernization. The Hilla LGP cited anti-corruption and strategic planning as highly relevant to developmental needs of Iraq in the short, medium, and long term. The Hilla USAID PRT representative cited the relevance of increased transparency from HR training in the short and medium terms. Lastly, in the Hilla Evaluation Focus Group Meeting, Babil Region civil servants noted that Tatweer training would benefit Iraq in the short and medium term. Tatweer National Capacity Development Program Final Evaluation 11 Cost Sharing, Donor Partnerships, and Gender Cost Sharing: Each interviewee was asked about cost sharing, some examples of which are outlined below. In addition, a Tatweer Document Review was undertaken to identify reported cost sharing activities. (Annex 11c) • PC reported sharing costs on 50/50 basis for establishing its language and IT training labs. • MOE noted its provision of space and furniture for its IT lab, with Tatweer supplying all IT equipment and software. • MORC reported the establishment of its IT lab on a similar basis. • BOTI noted its contribution of providing training facilities to Tatweer for training sessions there. • MMPW cited its contributions of transport and MI&E for participant training. • According to Tatweer documents, the MOP 2008/09 budget allocated $10 million for an operating budget for capacity building, and matched Tatweer’s $450,000 expenditure for IT equipment and training. • Similarly, COMSEC allocated $1 million for the infrastructure development and IT equipment for a new COMSEC training center. • DPMO-S also invested in renovating facilities, which included an IT center with a disaster backup capability. Tatweer collaborated with DPMO-S and a private firm to create a higher degree of operational effectiveness and security. • In 2010, MOH shared costs with Tatweer for the purchase and implementation of a state-of￾the-art GIS system that was established in three local health facilities in Maysan Province. • A number of ministries have been establishing their own T&D Units, staffing them with Tatweer-trained TOTs. • The GOI has reported that it had contributed some $32.2 million in 2009 to Tatweer system improvement initiatives. Donor Partnerships: Interviewees were asked about any knowledge of donor partnerships and provided some examples outlined below. In addition, a Tatweer Document Review was undertaken to identify reported cost sharing activities. (Annex 11d) • Tatweer managers reported that the SSN computerization of beneficiaries was divided between WB for hardware ($27 million) and Tatweer for software, training, and technical assistance. • PC indicated that the Tatweer and UNDP coordination was “great” for IDMS implementation: Tatweer improved IDMS with its capital investment program and also merged the MOP donor database with the MOP capital investment database, which the PC considered very helpful. The cooperation was among Tatweer, UNDP, and Synergy, a private company that had been engaged by UNDP before Tatweer. • According to Tatweer documents, one of the most important partnerships was the joint effort of UNAMI and Tatweer to assist the MOP in developing and publishing the NDP. • Tatweer and UNDP also collaborated on the implementation of UNDP’s COMFAR III feasibility study software, with Tatweer providing hardware, training, and technical assistance to ensure widespread implementation of the software. Gender: Findings derived from the NDP and Tatweer documents show that although gender is an important part of the NDP,”there is still a noticeable deficiency in attitudes towards women.” (NDP) In fact within the GOI itself, the gender ratio is 30 percent female to 70 percent male while the gender ratio in the population of the whole of Iraq is 51 percent male to 49 percent female. • The team interviewed many high-level, well-educated women, including the MOP Director General (DG) of International Cooperation, MOEd DG of Finance, MMPW DG of HR, DPMO-EA Acting Chief of Staff, and a Hilla Provincial Counselor. 12 Tatweer National Capacity Development Program Final Evaluation • The Tatweer COSQC Team Leader called COSQC the flagship of gender diversity, with women comprising 70 to 80 percent of its workforce. The study tour group to the US included five women and two men. • MOP noted that gender is an important part of the NDP. • MOE cited the high ratio of women in its T&D Directorate. • MORC noted that one-third of its trainees are women, that many trainers are women, and that women are fully integrated into MORC’s professional staff. • Four Basra interviewees reported that women are offered full and equal opportunities in their organizations. • The Hilla USAID PRT Representative stated that 13 of 32 (41 percent) enrolled in the 10- day training course on the Standard Bidding Document were women. • Tatweer’s training courses in less developed provinces such as Salah ad-Din and Anbar had far fewer female participants (17 percent and 7 percent respectively) than in more developed provinces such as Baghdad and Sulaymaniyah (46 percent each). Tatweer National Capacity Development Program Final Evaluation 13 ANALYSIS & CONCLUSIONS Achieved Results Tatweer achieved major results in all three project Sub-IRs of Strategic Objective 10: Capacity of National Government Institutions Improved. Despite significant security and other challenges, it did so by responding flexibly to client needs, by recruiting up to 230 Iraqi staff professionals and senior advisors, and by engaging over 35 high quality expatriate, Arabic speaking team leaders who were supported by competent core area experts. Tatweer produced its most important results when it delivered its full suite of skills training, TOT training, technical assistance, and embedded consultancy, sometimes supplemented by the provision of equipment. It should also be noted that according to a recent World Bank survey, most public administration reform projects worldwide have not achieved their intended results. Tatweer did succeed, and its accomplishments were not limited to large numbers of trained civil servants. Its major achievement in building more effective administrative systems was marked by its focus on people, systems, and tools. The Performance Management Plan (PMP) indicators supported Tatweer’s achieved results. Strengthening Public Administration Skills (Sub-IR 10.1.1) Tatweer delivered training to over 105,000 participants for 76,000 individual employees, far exceeding its original numerical target of 58,282 for training participants and representing nearly one-third of the 320,000 government employees targeted for skills training in the GOI entities assisted by Tatweer. In response to mounting pressure associated with “the surge” to rapidly increase the number of trainees, Tatweer adjusted quickly by creating a training machine that combined direct and rollout training through TOTs. Further, Tatweer’s clients expressed a high degree of satisfaction with the training curriculum and delivery. Data from project PMP indicators support this interview information. Over the four-year span of the project, 95 percent of the training graduates reported that they frequently or sometimes make use of their new training skills. Although different clients found certain courses more beneficial than others, there was universal appreciation for the value and importance of IT training in improving efficiency. In addition to the recruitment of many qualified Arabic speaking team leaders and a large number of Iraqi staff professionals, Tatweer achieved strong results by responding flexibly to training needs. Tatweer’s training delivery evolved from the general introduction of concepts to deeper penetrations of GOI entities via identified change agents. Recruitment and support of these champions was a critical success factor. To its six core training areas (L&C, HR, IT, Fiscal, PM, and SP), Tatweer added courses to meet client demands for SOP, TMS, PMS, and Training Evaluation. The latter complemented the effective work that Tatweer was also already doing in implementing job descriptions. Moreover, Tatweer introduced effective software applications that clients continue to use, including COMFAR III for feasibility studies, Primavera for project management, IDMS for planning and budgeting, and ISCM for supply chain management. Tatweer’s application of the OSTP methodology for organizational change also contributed to its success by getting clients to own their change process. The main factors enabling Tatweer success were high motivation on the part of most trainees to improve their skills, and the strong support of top managers in certain organizations (e.g., MOP, MOH, COMSEC, MOO, MORC, and SOC). Highly motivated individual champions within these organizations and effective managers of Tatweer Regional Offices were also important success factors. Public Administration Training Capacity Expanded (Sub-IR 10.1.3) Tatweer’s training of more than 1,200 TOTs in all core areas (plus additional methodologies such as SOP, PMS, and TMS) represents another major achievement. As several clients have indicated, TOT 14 Tatweer National Capacity Development Program Final Evaluation training may be Tatweer’s most significant accomplishment, fundamentally changing the training delivery methodology in Iraq and addressing both geographical and security constraints. These qualified TOTs are delivering training now but are also available to be integrated into the future development of the Civil Service Institute. Another critical factor for expanding training capacity was Tatweer’s establishment of five regional training centers (Anwar, Babil, Basra, Kirkuk, and Erbil) and training centers or departments in each assisted ministry and executive office. PMP indicators show that 455 people were trained as trainers, and 55,000 people received management training in these regional centers. Tatweer’s work with MOH is particularly noteworthy. MOH is a pioneer in obtaining TMS/International Standards Organization (ISO) 10015 certification for its training center. In order to take full advantage of this potential, however, measures should be taken to address the limitations of Tatweer TOT training associated with the need for more technical and customized training, higher level training, and more emphasis upon action / OTJ training. Tatweer’s Scholarship Program should also be recognized for its contribution to building training capacity. More than 100 civil servants from 15 provinces and representing 23 ministries have received Master of Arts (MA) degrees in public administration and public policy from universities abroad (e.g., Egypt, Jordan, and Lebanon) under the Tatweer-funded program. The Tatweer TOT success was achieved primarily through effective leadership and talented individuals in the client organizations, who were proactive and imaginative as TOTs in the implementation of training. TOTs are effective not only in reaching a large number of trainees, but in other areas as well: MORC succeeded in resolving its LOC problem with TBI. COMSEC succeeded in implementing SOP in all COMSEC departments and is now extending SOP to many other GOI entities. The partners succeeded in providing effective PM training to the Ministry of Water Resources (MOWR) and L&C training to the Hilla Provincial Council. The enabling factors for Tatweer’s TOT success included supportive top managers and the overwhelming demand for training in the Iraqi organizations. Without supportive top managers, however, qualified TOTs will not always receive approvals to conduct training sessions outside their own departments or Ministries. Ministries Establish More Effective Administrative Systems (Sub-IR 10.1.2) There are no direct quantitative indicators for the improvement of administrative systems. Nonetheless, for the 10 ministries with which Tatweer worked, the official GOI data provided by Tatweer for Budget Execution (Sub Indicator IR10.1: Percentage of Approved GOI Annual Budget Spent by Assisted Ministries) shows an increase from 71.4 percent execution in baseline year 2006 to 94.0 percent in 2009, the last year for which official data is available. Budget execution improvements were realized in all 10 assisted ministries partly due to Tatweer’s addition of project management to its original SOW-defined core training programs in October 2006, when Tatweer realized in consultations with MOWR that budget execution was more a project management capacity issue than a budget issue. Budget execution improvements were particularly impressive at MOEd and MOO. By itself, however, the budget execution indicator does not reveal the full extent of Tatweer’s positive results in establishing more effective administrative systems. Tatweer improved administrative systems significantly in a number of key GOI organizations, particularly when it combined training, TOT, technical assistance, and embedded consultants. As all GOI entities started with no IT systems and weak or non-functioning administrative systems, evidence of the high degree of significant improvements can be found in the following examples from interviews: • MOP reformed systems in all departments and also mandated IDMS for budget execution and standard bidding documents; Tatweer expert economists provided the methodology and trained 50 persons for NDP development; • The Central Organization for Statistics and Information Technology (COSIT/MOP) built the first phase of the NEDB in 2010 with the help of Tatweer experts; • COSQC/MOP improved inspection systems by changing mindsets and implementing its first outsourcing contract; Patent Office is now processing applications at a rate of several hundred per year; Tatweer National Capacity Development Program Final Evaluation 15 • MOO implemented a procurement system meeting international standards, developed complete job description manuals, and implemented PMC, ISCM, OSTP, and the Computerized Maintenance & Management Systems (CMMS); • MOH developed the National Health Plan with Tatweer assistance, and it is one of Tatweer’s major accomplishments; MOH is also obtaining TMS/ISO 10015 certification for its training center; • MOEd’s HR department is circulating questionnaires for job description development; directorates in Babil, Mosul, and Anbar are renovated and in full use; Tatweer IT developed the MOEd website; standardization of reporting in MinEd with use of ACCESS; three MinEd training halls werbuilt and furnished; more than 2,500 employees were trained. • MOA implemented a GIS system to improve the type of services that MOA delivers to farmers and livestock owners; MOA established a new GIS department and appointed a new DG for this department; Tatweer provided IT services, GIS software, training, and TA for implementation of three GIS centers in Iraq; • PC reformed all administrative systems, established its Public Policy Office, and also established its Arbitration Court with Tatweer assistance, the latter being one of its most important innovations; • COMSEC established its SOP Committee and is now working with each department, and it will hold three more SOP workshops in April; job descriptions are part of the SOP process; it conducted four workshops on tendering and contracts; it is using standard tender documents; it is also using Primavera project management software; • DPMO-EA is applying all courses to all departments; started the implementation of a system for management of document classifications, and at least 35 of 40 staff received competency level IT training; Tatweer IT also provided a fiber optic line; • DPMO-S is applying core public administration skills to all administrative systems; capacity has been expanded and improved in all areas; • The State Committee for Oil Projects (SCOP) implemented a decision to hire outside consultants with Tatweer assistance; it had previously resisted outsourcing; • MORC has conducted five project management workshops already in 2011; it is implementing TMS and ICMS; its IT lab is functioning on daily basis; it is extending LOC workshops; and • SOC revised procurement to standard bidding documents; it is now implementing training evaluation training systems; it is just about to complete first year of OSTP for organizational change with Tatweer assistance and will complete the second year on its own. Procurement and HR systems improvements stand out among the Tatweer successes. GOI’s implementation of standard tender documents and clearly defined job descriptions are noteworthy systems improvements, as are the outsourcing successes Tatweer achieved at COSQC and SCOP. Tatweer’s success in its assistance to MORC and MOO in resolving LOC problems for all operating oil companies also merits commendation. Additionally, Tatweer’s technical assistance helped improve many other GOI systems: the GIS implementation at MOA; international standard tender systems at North Oil and North Gas; the Arbitration Court at the PC; the Patent Office at COSQC; and the SSN at MOLSA. Tatweer’s technical assistance to MOH was also critical to the development of the National Health Plan. Moreover, a number of GOI entities are now implementing OSTP for instituting organizational change throughout all departments. These GOI entities are MOA (plus four state companies), MOO (plus seven state companies), MOWR, MMPW, MOP, MOE, COMSEC, PC, the Ministry of Human Rights (MOHR), and the Ministry of Migration and Displaced (MOMD), Among the many Tatweer accomplishments, the NDP and Civil Service Laws are arguably its two greatest, but it is too early to assess their full impact. The NDP is now in its first implementation year and continues to receive Tatweer assistance for monitoring and evaluation. In recognition of the depth of Tatweer technical assistance, the Tatweer Civil Service Reform Team Leader is the only non-Iraqi member of the High Commission of the Civil Service Commission currently being formed. 16 Tatweer National Capacity Development Program Final Evaluation The Civil Service Law itself is now in the final approval stage with the Shura Council and is expected to become law in mid-2011. Tatweer PMP indicators support the interview conclusions. Project Indicator 2.1 shows that the number of units / departments implementing Tatweer-recommended changes in administrative systems or procedures increased from 145 in FY2009 to 174 in FY2010. Similarly, Project Indicator 2.16 shows that the number of ministries issuing administrative orders to restructure HR increased from only two in FY2009 to 26 in FY2010 to 35 in Q1 FY2011. The Tatweer results were achieved by a variety of means, all grounded in a professional approach: 1. First, highly qualified Arabic speaking team leaders supported by Iraqi expert staffs were largely responsible for many of these achievements. In all cases, selective staff recruitment and effective training provided by Tatweer team leaders to their Iraqi staff through coaching and mentoring were critical success factors. 2. Second, technical assistance was generally required to supplement training, as illustrated by Tatweer’s expert assistance in drafting the proposed Civil Service Law. 3. Third, Tatweer identified and supported proactive champions within GOI entities to help achieve results. 4. Fourth, Tatweer’s embedded consultants helped focus attention and execute results￾oriented action plans, as in the case of the NDP development at MOP. 5. Fifth, Tatweer recognized the importance of changing mindsets in order to produce results and, therefore, dealt directly with this need as part of its action plans. COSQC agreement to outsource inspection services reflects this type of mindset change. 6. Sixth, the cultivation of good relationships throughout the client organization (not limited to the training department) was also important to Tatweer successes, particularly in the GIS systems implementation at MOA. Notably, Tatweer’s NDP achievement was only realized after proving itself by responding effectively to numerous specific assistance requests, including M&E. Tatweer’s success in gaining GOI client buy-in was often achieved by overcoming initial suspicions and building trust over time. 7. Seventh, the Tatweer “what can I do for you?” approach helped engage clients in a manner by which both Tatweer and the client could set plans for systems improvements and work jointly toward their accomplishment. Tatweer’s success at the PC is an excellent example. 8. Lastly, the close and effective support of the USAID Contracting Officer’s Technical Representative (COTR) was extremely important to Tatweer’s success in all three projects areas, especially in responding to changed conditions. The enabling factors were very similar to those for effective TOT implementation: high-priority needs and strong support from top officials at GOI entities. In assessing the degree of success Tatweer achieved in establishing more effective administrative systems, there are two important considerations. First, without the active engagement of top officials, Tatweer could only have limited success. Trainees could not implement changes in their own departments, as the established policies and procedures could only be reformed with the active engagement of senior managers. Thus, for example, Tatweer’s Energy Group was far more successful with MOO in implementing systems improvements than with MOE, which was a latecomer in terms of actively seeking Tatweer assistance. Second, even when top officials of GOI entities actively supported Tatweer assistance, changes were likely to be implemented in a step-by￾step manner in keeping with organizational culture and judgments by the senior managers about the pace of change to which its employees could adjust while continuing to complete their everyday tasks. As the COMSEC Training Manager pointed out, “we know our strengths and weaknesses, and we know the [pace and sequence] by which we can implement change.” Thus, Tatweer’s overall results can be viewed as a solid start. In short, not all assisted ministries have implemented significant systems improvements, and even in those that have, much work remains to be done in order to establish effective administrative systems. Tatweer National Capacity Development Program Final Evaluation 17 Note that Tatweer’s budget execution proxy indicator was approved by USAID. In its response to the Office of Inspector General/Iraq (OIG) audit, USAID stated in November 2008 that “the performance target of improved service delivery, given the level of resources, limits in time and limits in scope for this activity, is now recognized by the Mission as being outside the manageable interests of the Tatweer Program. The Tatweer PMP will be revised to add an indicator on budget execution. This will respond not only to the OIG report findings, but will also be consistent with several prior General Accounting Office (GAO) reports on capacity development that recommend measurement of budget execution as a proxy indicator for capacity development at the national level.” Results Not Achieved The only major result not achieved as originally planned was the goal of turning NCCMD into the lead training organization for the Iraqi Civil Service. The Tatweer arguments that the original design was flawed, that the goal was unrealistic, and that NCCMD lacked the capacity for high-volume skills training are compelling. NCCMD was not ready for the technical assistance being offered. The Tatweer course correction in consultation with USAID to focus upon training departments in GOI entities, to create training centers in those GOI entities, to establish five regional training centers, and to emphasize TOT training was the right decision. From this perspective, the course correction represented an achieved result. The current pool of more than 1,200 Tatweer-trained TOTs is well positioned to provide the new Civil Service Institute with a strong foundation. NCCMD can continue to provide certification training with respect to practices required by Iraqi law. With Tatweer assistance, NCCMD is now certifying TOTs and master trainers and endorsing training materials. It was also an early adopter of OSTP. Although it remains to be seen how NCCMD will relate to the new Civil Service Institute, it is clear that the Tatweer-trained TOTs will become critical factors for its success. Tatweer-trained TOTs can deliver across Iraq and are better suited to continuing skills training, especially when they are armed with more advanced and technical courseware. Other results were achieved but not with full impact, generally due to shortcomings in individual GOI entities. For example, the MOE’s inability to implement TMS and its decision to discontinue its Public Managers Program are directly related to its late adoption of Tatweer assistance. It should also be noted that Tatweer, in close coordination with USAID, only started its TMS implementations in 2010 in selected GOI entities. The fact that a number of Ministries have not yet fully established their HR departments is also related to political issues and delays associated with decisions to wait for new GOI legislation. The decision by the PC to halt implementation of new job descriptions is primarily related to internal factors. Similarly, the fact that the PC is the only executive office yet to establish a Public Policy Office can also be attributed to internal GOI political differences rather than inadequate Tatweer assistance. Other findings are more related to room for improvement than results not achieved. The findings reported from Basra, Hilla, and the MMPW concerning less than full realization of Tatweer’s training impact are directly related to areas in which training delivery could have been improved. These learned lessons include more standard application of needs assessments prior to training delivery, better trainee and trainer selection, better follow-up, better translations, better coordination at all levels, longer courses, more advanced courses, and more action / OTJ training. Also, training implementation results could have been improved by stronger efforts to secure the support of top officials, and by providing more technical assistance to the organizational departments in order to help implement departmental policies and procedures in consonance with the training concepts and principles. Note that special USG efforts should be continued to help the Tatweer Scholarship Students obtain Ministry of Higher Education certification of their advanced degrees, especially as scholarship interviewees were under the misimpression that Tatweer itself had not coordinated effectively with the Ministry of Higher Education. 18 Tatweer National Capacity Development Program Final Evaluation Cross-Cutting Lessons Learned By far, the most important cross-cutting lessons learned relate to combating corruption. Four major anti-corruption areas stand out: 1) systems reform and applications; 2) awareness; 3) strategies; and 4) organizational establishment. Two cross-ministerial systems applications driven by MOP are already having a positive effect. When it is fully implemented, the NEDB will help to remove fictional employees from the civil service salary rolls and expose nepotism (the NEDB is currently less than 75 percent populated). The CCA must also be fully implemented to help combat corruption in procurement by increasing the transparency of contract tenders. In Babil, Tatweer-initiated efforts are underway to streamline the payments approval process for procurements, as the large number of required signatures opens the door to corruption opportunities. Similarly, corruption awareness measures, including but not limited to workshops and conferences, should be expanded and applied within all GOI entities, which would also benefit from more anti-corruption training. The ability to recognize corruption and a deeper understanding of its deleterious effects are the first steps toward combating it. Further, the work done by the Tatweer Babil Office in helping its four provinces to develop anti-corruption strategies through feedback from workshop recommendations should be replicated throughout Iraq. All Provincial Councils need such strategies, and they should be integrated into a national strategy developed by the Integrity Committee under the Prime Minister’s Office. Organizational establishment at the regional level can only help the national Integrity Committee accomplish its goals in combating corruption. The work that the Tatweer Basra Office did in establishing ant-corruption offices in each of its four Provincial Councils should be replicated as well. Note that similar Tatweer anti-corruption initiatives took place in all provinces. A final learned lesson concerns the power of money to corrupt. Ministerial processes themselves can spawn corruption, as can special-interest legislation. For all ministries, administrative systems must be assessed from the standpoint of openings to corruption, particularly for MOF and others controlling large amounts of money. Legislation must also be reviewed and assessed from the same perspective, especially as it affects large cash flows. Critical Lessons Learned, Challenges, and Performance Highlights Critical Lessons Learned: All of the learned lessons deserve serious consideration for the implementation of any future public administration reform project in Iraq. Perhaps the most important is the need not only to increase awareness and support of top managers but also to win their active engagement in order to achieve optimal sustainable results. In all assisted GOI entities, Tatweer gained the support of top managers to enable delivery of its training programs and technical assistance. Tatweer assistance was most successful when it was able to provide its full complement of direct training, TOT training, and focused team engagements with embedded consultants. The level of assistance that led to systems changes required the active engagement by the GOI entity’s top management. Tatweer’s “center out” approach, by which Tatweer trained mid-level and upper mid-level officials, convinced top managers of the value of systems reforms in many cases but not all. The matrix management that Tatweer applied in 2008 also made good sense and improved the effectiveness of the “center out” approach. Requiring Tatweer core training area managers to “sell” their training services to GOI entities forced the service providers to tailor offerings to client needs, and it forced clients to evaluate more carefully the priorities of their training needs. Adjusting training implementation and technical assistance to the varying needs of different provinces is also important for the success of any project implementation. Finally, all of the many learned lessons for improving training delivery should be considered seriously. The most important of these findings are the expressed desires for serious training follow-up, needs assessments conducted prior to training delivery, more technical courses, higher-level courses, and more OTJ action training. At the same time, it should be noted that technical courses were never in the Tatweer scope and given the security situation, Tatweer embedded local experts pushed the OTJ boundaries as far as practicable under the circumstances. Challenges: Tatweer encountered a multitude of challenges, as indicated by the findings. Security posed the greatest challenge for implementation. Tatweer overcame the security challenge by Tatweer National Capacity Development Program Final Evaluation 19 embedding local staff in assisted GOI entities, the ubiquitous use of after-hours mobile communications between Tatweer advisers and GOI entity counterparts, and the expanded use of the Mansour compound as a meeting place. There were other major challenges as well. Foremost among these were the low capacity of the civil servants and old mentalities. These challenges, of course, were the basis for the Tatweer intervention, but they constituted implementation challenges as well. Training was required to start at very low levels, especially for IT. Tatweer overcame these challenges by recognizing that the problems were temporary and caused by recent historical events. By so doing, Tatweer was able to gear its initial training and mentoring to the correct level and then raise the levels quickly as the largely well-educated trainees embraced the new methods. Another challenge was presented by senior officials’ lack of basic management skills. This challenge always threatened to diminish the impact of training delivery, as senior officials would not approve administrative process changes that they themselves did not understand. Further, senior officials would not join training courses with more junior officials. Thus, it was always a major challenge for Tatweer to gain high-level support. Similarly, gaining the confidence and trust of GOI entities was another major challenge, as many GOI entities had become accustomed to “unkept promises” by donors. GOI entities were often skeptical of the ability of many donors to maintain a course of action. Tatweer overcame these challenges when it could apply innovative approaches to gain senior level support, including but not limited to developing upper mid-level champions who could influence the decision-making of top officials. MOO, MOP, COMSEC, and the PC are all good examples of this approach. With the support of top-level officials, Tatweer was better (though not always) able to overcome the challenge of employees being “too busy” for training. Once senior managers recognized the urgent need and benefits to be derived from training, trainees could more easily obtain approvals from their department heads to participate. Performance Highlights: In addition to the major results achieved, Tatweer’s performance highlights were marked by a highly effective Energy Group helping to improve administrative systems at MOO and several of its operating companies. The Tatweer Energy Group, comprised of 18 technical experts from the oil and electric power industries, achieved significant accomplishments in establishing more effective systems at MOO, MORC, SOC, South Gas Company, BOTI, North Oil Company, and North Gas Company. Notably, MORC established an IT Center and Center of Excellence with Tatweer Energy Group assistance and pioneered its New Project Management Program to mix older and younger engineers for intensive interactive training. The Tatweer Energy Group also introduced and implemented the ISCM) system in MOO and then helped in the extension of ISCM to a formal course taught at the University of Baghdad. In the words of the USAID PRT Representative for the Basra Region, “they know what they’re doing, they know oil, they know Iraq, and they know oil in Iraq.” Budget execution for MOO increased from 54.4 percent in 2006 to 99.8 percent in 2009 with the help of the Tatweer Energy Group. The mandate by MOP for all ministries to use IDMS was another performance highlight. Validity of the Theories and Assumptions Underlying the Results Framework The inferred hypotheses for design of the Tatweer project were: 1. If public administration skills training relevant to urgent and priority needs was provided to a sufficiently high number of Iraqi civil servants, then a critical mass of sufficiently skilled employees would be established to enable implementation of more effective administrative systems; 2. If training capacity were built to provide Iraqi ministries and executive offices with ongoing training, then the foundation for implementation of more effective administrative systems would be sustained; 3. If training efforts were supplemented by technical assistance and focused expert team engagements with selected GOI entities, then departmental processes, policies, and procedures could be changed to create more effective administrative systems, thereby enabling the trained critical mass of civil servants to apply their new skills. 20 Tatweer National Capacity Development Program Final Evaluation Together, these three hypotheses constitute the necessary and sufficient conditions for establishing and sustaining more effective public administration systems. The original program design based on these hypotheses was valid and remains valid. The skills training worked very well, subject to the areas for improvement addressed in Learned Lessons. The training capacity building did not work well initially under the assumption that NCCMD could provide relevant ongoing skills training to the very large number of Iraqi civil servants. It did not work well primarily because NCCMD simply did not have the capacity for that scale of training or the core competency for the types of skills training required. After Tatweer made the correct adjustment to TOT training and focus upon regional training centers and training departments and/or centers within GOI entities, the training capacity building did work well. The technical assistance and focused team engagements worked well, but only when GOI entities strongly supported such assistance (e.g., MOP, MOO, MOH, COMSEC, PC). The framework parameters have not changed. The challenge is how best to apply the framework parameters to changing conditions. Overall execution of the program design worked well largely because USAID adjusted to changing conditions, obstacles, and opportunities. It adjusted its curriculum to meet client needs. The five core areas (Leadership & Communications, Fiscal, HR, IT, Project Management) were expanded several times: Leadership & Communications led to Strategic Planning; Fiscal led to Procurement and Contract Management; HR added SOPs, which are now being applied to all departments in certain GOI entities; and Project Management led to feasibility studies and the PMC. Public Policy Management was added in response to needs identified in the executive offices. OSTP was added to address organizational change needs. The Tatweer decision to apply matrix management in 2008 was a logical response to the need for matching training offerings to client needs. USAID also adjusted its implementation strategy by shifting its resources to the GOI entities that openly welcomed Tatweer assistance and scaling back assistance (especially technical assistance and focused team engagements) to ministries that erected blocks. Any future project will confront similar challenges for adjusting implementation mechanisms to client needs and changing conditions. Impact Tatweer achieved its intended results, and they are explained in the Results Achieved section. Positive net impacts are both immediate and longer term. The immediate impacts are manifold. About 76,000 Tatweer-trained civil servants are now attempting to reform administrative processes within their own departments by bringing pressure upon senior managers to change, sometimes successfully and sometimes not. Most of the 1,200 Tatweer-trained TOTs are continuing to extend training throughout their organizations (and sometimes others as well). MOO and its 16 operating companies are now using international standard tender documents. MOP is gaining ground in the cross-ministerial application of standard procurement documents, IDMS for budgeting, and CCA for combating corruption and improved budget execution. SOP implementations are reforming all departments in COMSEC and spreading to other GOI entities. All 10 Tatweer-assisted ministries have established training centers and/or strengthened training departments. MOE has established an M&E department, and the PC has established its Public Policy Office. MOEd has reported a significant improvement in job performance. These improved organizations and skills illustrate the immediate net impact of the Tatweer assistance. Tatweer project implementation also produced a number of unintended impacts. One of the unintended impacts was opportunistic: resolving the LOC problem for MOO and its 16 operating companies by arranging workshops for a Tatweer TOT specialist to provide training to all credit managers on proper L/C document preparation. Another unintended impact was the development of an Operations & Maintenance (O&M) manual and delivery of training by the Tatweer Babil Office in direct response to provincial client needs for specific water and sewage treatment projects. Moreover, there were a number of intended impacts that were not captured by the performance indicators. Many of these were “soft impacts” and included Iraqi civil service reconnection with the Tatweer National Capacity Development Program Final Evaluation 21 professional world, mindset changes to embrace modern techniques and methods, organizational culture changes, increased ability of civil servants to cope with change, reductions in everyday mistakes, better decision-making, significant improvement in cooperation between national entities and provincial governorates, reduction in political and religious frictions through regional cross￾training, and the narrowing of the generational divide through such programs as the MORC New Project Management module, which combines older and younger engineers. Longer term, Tatweer set the foundation for deeper Public Administration Reform (PAR). The premise of the Tatweer Team Leader is on the mark: civil service reform is, indeed, a necessary condition for effective PAR, which is a necessary pre-condition for overall economic reform. By its achievements in civil service reform, Tatweer merits credit for longer-term impact. A similar argument can be made for the NDP. Nonetheless, the caveats of the DPMO-EA Acting Chief of Staff are germane. Much more work remains to be done, not only at the ministries that did not receive Tatweer assistance but also at those that did. Moreover, the remaining work requires further training initiatives and technical assistance at all levels, including senior officials and top managers. PMP indicators support the impact conclusions drawn from the interview findings. Per Section 6.1.3, the budget execution proxy indicator shows an increase from 71.4 percent in baseline year 2006 to 94.0 percent in 2009. Also per Section 6.1.3, Tatweer Project Indicator 2.1 shows that the number of units / departments implementing Tatweer recommended changes in administrative systems or procedures increased from 145 in FY2009 to 174 in FY2010. Similarly, Project Indicator 2.16 shows that the number of ministries issuing administrative orders to restructure HR increased from only two in FY2009 to 26 in FY2010 to 35 in Q1 FY2011. Measuring increases in the delivery of government services to the population proved to be impossible during the course of the Tatweer evaluation. Sustainability Prospects for sustainability of Tatweer results are strong yet not assured for all GOI entities. Prospects are strongest for MOO and its 16 operating companies, as they have the requisite commitment, foundations, and budget. MOO’s 125 TOTs are already providing ongoing training. It has a capacity development strategy in place, and it is supportive of reforms and innovations. It has taken over the Quality Management Cycle management training program introduced by Tatweer and has just started training the third round of 23 MOO managers with its TOTs. MORC is currently training 200 employees each day in its six training halls. MORC will also maintain the IT Center established by Tatweer. SOC has a $12-13 million annual budget for training, and it will not only continue training but also implement year two of OSTP on its own. SOC will also continue cooperative training activities with BOTI. Prospects are also strong for many other GOI entities due to implementations already underway and Iraqi legislation. MOLSA is supporting the SSN system implementation. COSQC is processing several hundred patent applications each year. MOP has engaged Tatweer for M&E assistance for NDP implementation and plans annual reviews. Provision for the Civil Service Institute is now in legislation. COMSEC’s TOTs are now developing new Council in all departments. MOEd has also embraced its commitment to skills development and systems improvements. The 1,200 Tatweer-trained TOTs provide the most important foundation for sustainability in these and other GOI entities. In Babil, the TOTs are also active. Even though the agricultural part of Tatweer has closed, agricultural TOTs in Hilla are actively training today, and the MMPW TOTs in Hilla are planning 70 training courses for delivery this year. The proven commitments of TOTs and civil servants to skills improvement along with the institutionalization of software tools such as IDMS provide a strong base for sustainability. At the same time, sustainability is not yet assured for certain GOI entities. The main constraint is that many senior level officials have not yet committed fully to administrative systems reform, nor have these officials recognized the urgency and importance of skills training. Unless and until the vast majority of GOI’s top managers embrace public administration reform and further skills training, sustainability will be at risk. This concern is foremost on the mind of the DPMO-EA Acting Chief of Staff. Her concern is legitimate, and it provides a strong argument for more technical assistance. For 22 Tatweer National Capacity Development Program Final Evaluation these reasons, the partial Tatweer success at MOE is at greatest risk. The discontinuation of the Public Manager’s Program and the stalled implementation of TMS are ominous signs. In Basra, MOA’s Science & Animal Production Department indicated a drying up of demand for core area TOT training. The issue in this case is not the absence of top-level support, but the unmet demand for more highly technical training. Budget availability for follow-on skills training is another risk factor, especially for Basra and other provincial departments dependent upon budget allocations from Baghdad. The scale was appropriate to ensure sustainability. Both the number of civil servants trained (76,000) and the number of TOTs trained (approximately 1,200) were sufficient. Further, Tatweer implemented systems reforms in most of the GOI entities it assisted. Also, the technical assistance resources for focused team engagements were sufficient, especially for MOO, all the executive offices, MOP NDP development, and civil service reform. Resource availability could have been increased, however, for higher level and more technical training to close some demand gaps. Had these demand gaps been closed with more TOTs trained in higher level and more technical courses, prospects for sustainability would have been increased even further. Client Satisfaction For the vast majority of Tatweer GOI clients, needs were met with high praise. Among these were important GOI entities, including MOO, MOP, MOEd, MOA, MORC, PC, DPMO-EA, and DPMO-S. The USAID PRT representatives in Basra and Babil also gave Tatweer high marks. Even MOE, which was a latecomer and turned down in its request for more assistance in late 2010, indicated that it thought that all aspects of Tatweer assistance were “good” and that it wanted more. Some of the feedback was absolutely effusive, including that from MOO (“the best consultants from any donor”) and DPMO-S (“I could talk for three days about Tatweer and USAID assistance to DPMO”). The only serious negative feedback came from NCCMD, which was understandably displeased with the Tatweer training relationship, given the change of course that Tatweer correctly made in 2007 away from NCCMD. As presented in the Learned Lessons section and discussed further in the Sustainability section, however, there were areas identified by GOI entities for training delivery improvement and unmet needs, particularly for higher level, more customized, more practical, and more technical training. Although Tatweer did make adjustments to its core curricula to add SOP, PPM, strategic planning, feasibility studies, procurement, contract management, and even O&M in the Babil region, it did not have the resources available to meet all client training demands, especially for highly technical training in agriculture (e.g., embryo transfer) or oil refining (e.g., reduced fluid carbon cracking). Cost Effectiveness Cost effectiveness is often expressed as an incremental cost effectiveness ratio: the ratio of change in costs to the change in effects. However, the SOW research question asked if the program was “implemented and managed cost effectively.” Thus, the evaluation field research focused on the SOW question. GOI entities could provide little or no feedback on Tatweer cost effectiveness. Notably, however, not one GOI interviewee registered any complaint whatsoever about perceived inefficient use of Tatweer resources or unnecessary expenditures. Tatweer interviewees could only provide anecdotal evidence of steps taken to reduce costs. One example was the two-week COSQC “measurement & testing” USA study tour for seven employees, funded by the U.S. Department of Commerce with no costs to Tatweer. Similarly, in Babil province, training operational costs were shared with USAID/LGP. The most frequently cited Tatweer cost effectiveness indicator was the training cost reduction realized through the TOT mechanism. From data provided by Tatweer, the Evaluation Team constructed the following cost effectiveness estimated calculation based upon the number of training participants, the number of trained civil servants, and the average number of training days per individual GOI employee. The Tatweer project Tatweer National Capacity Development Program Final Evaluation 23 team reviewed the cost data presented below on security, LOE, direct costs and grants, and estimates for IT equipment, training center renovations, and learning materials: Cost Effectiveness Estimated Calculation • Total Tatweer budget = $339.5 million • Gross cost per training day = $495 (using an average nine training days per trainee) • Gross cost of training a GOI individual civil servant = $4,452 • Reported LOE expenditures $121.1 million: 36 percent • Reported direct costs and grants $218.4 million: 64 percent • $119 million security • Assumptions: $50 million IT equipment, training center renovations & learning materials • Net cost per training day = $72 • Net cost for training a GOI individual civil servant = $648 Total Net Benefits (Impact) for that $648 per GOI Civil Servant • 76,000+ trained employees • Approximately 1,200 TOTs • The NDP, Civil Service Reform, MOO, MOP, and other GOI entity systems improvements, and all other benefits noted in the Achieved Results section While there are no donor benchmarks for cost per participant per training day, evaluation team member experiences with similar capacity building projects indicate that the net training day cost of Tatweer training is comparable: $50 (UNDP) and $75 (WB). Although the evaluation team did not investigate incremental cost effectiveness questions, one measurable incremental ratio is that the Tatweer assisted ministries achieved an average budget execution rate of 94.0 percent in 2009 versus 83.6 percent for all line ministries. See Annex 12. The overall conclusion is that the Tatweer program was implemented and managed cost effectively. Most importantly, the long-term value and multiplier effects of the TOTs and other assets Tatweer created (people, systems, and tools), coupled with the mindset changes Tatweer inculcated in the Iraqi Civil Service, will provide lasting benefits that go far beyond the immediate recipients of training and systems changes. Relevance The Tatweer program is highly relevant to Iraqi development needs in the short, middle, and long term. Short term, the high number of civil servants trained in modern methodologies and the initial multiplier effect of TOT training creates immediate service improvement impact in terms of better decision-making, and also builds the critical mass required for more fundamental and lasting public administration reform. The early-stage improvement of administrative systems (as presented in the Achieved Results section) also provides immediate service delivery impacts and sets expectations for further reforms, augmented by the first implementations of the NDP and civil service reform. Tatweer-provided IT systems and training serve to accelerate the change process. Medium term, as GOI administrative systems become more effective and more widespread, and as civil service reform and the NDP take hold, GOI should be in a much better position to establish the business-enabling environment that will not only attract increased foreign direct investment but also support the growth of the Iraqi private sector. Civil service reform and the establishment of an effectively functioning public administration are critical to the development of the Iraqi private sector and its associated economic growth. Only then will significant job growth be generated by a healthy private sector operating in free markets. Long term, GOI will be challenged with managing the rapid growth in its oil revenues in a way that provides real economic opportunities for the Iraqi population, not simply increased expectations for government jobs or stipends. This is why the NDP focus on private sector development, the establishment of more effective administrative systems in MOO and its 16 operating companies, and initial steps to combat corruption are so important. Tatweer has 24 Tatweer National Capacity Development Program Final Evaluation been an active player on all these fronts, and its early work will be highly relevant for many years to come. Cost Sharing, Donor Partnerships, and Gender Cost Sharing: It should be noted that cost sharing was never a part of the Tatweer agreement. The major examples of cost sharing achieved voluntarily in the Tatweer program were in-kind contributions, and the trend line is positive. The most common examples were the GOI contributions of space and furnishings for the establishment of IT and/or training centers, with Tatweer providing IT equipment and software. Other examples were GOI entities paying the per diem and transportation costs for civil servants to attend training and/or conferences. Of those interviewed by the evaluation team, only the PC reported a straight monetary 50/50 cost split for establishing its language lab and training center. Notably, GOI reported that it contributed $32.2 million in 2009 to Tatweer systems improvements. Most of this sum represented valuations of the space and furnishings contributions plus overheads. USAID/Tatweer analysis indicates the total in￾kind cost sharing by GOI entities amounted to over $100 million during the course of the project, a sum that represents nearly $1 USD for every $3 USD invested. Effective in-kind cost sharing was a Tatweer achievement that could be replicated in other projects. For 50/50 or better monetary cost sharing splits, MOO and its 16 operating companies would be likely candidates to target for future project implementations, given the greater financial resources available to them. Donor Partnerships: Tatweer had strong partnerships with a number of donors, including the EU and UN agencies. Tatweer worked closely with UNAMI for NDP development and with UNDP for IDMS and COMFAR III software implementations. Tatweer and DFID also collaborated in their work at COMSEC. Another important partnership was with the World Bank for the SSN system implementation at MOLSA. Further, Tatweer closely coordinated its work with other USG/USAID programs such as Inma (agribusiness), Tijara (provincial economic growth), and the RTI LGP. The WB SSN example, in which one donor provides equipment and the other technical assistance and software, is certainly worth consideration for future project implementations. The UNAMI and UNDP technical assistance collaborations also merit replication in future projects. See Annex 11d for a detailed description of Tatweer’s donor partnerships. Gender: It is striking that of the many male and female GOI civil servants interviewed by the Tatweer Evaluation Team, not one registered a concern about equal opportunities for women. Time and again, interviewees responded with “no issues” or “no problems” when asked about gender issues and equal opportunities. Moreover, findings were the same in Baghdad, Basra, and Hilla. It is also striking that the evaluation team encountered women leaders at high levels (e.g., MOP, MOE, MMPW, and DPMO-EA) and that several of the most effective TOTs interviewed were women (e.g., MOO, MORC, Basra MOWR, Basra Tatweer Office, Hilla MOWR, and Hilla MOO). Also, it is noteworthy that women comprise nearly 80 percent of all civil servants at COSQC, a technical organization. Yet the Social Development section of the NDP recognizes and attempts to address “noticeable deficiencies” in gender equality. Tatweer data also show a much higher percentage of female training participants in the more developed provinces. The fact that interviewees did not express concerns about gender issues is best explained by the fact that the evaluation team conducted interviews in provinces that had relatively high percentages of female participation in Tatweer training. The principal conclusions from the Tatweer data are: 1) the more developed the province, the higher the level of female participation; 2) the percentage of women participating in Tatweer training (31 percent) was virtually identical to the percentage of female TOTs (30 percent), which corresponds to the civil service gender split; and 3) female civil servants sought and obtained additional training as frequently as males. Tatweer National Capacity Development Program Final Evaluation 25 RECOMMENDATIONS As requested in the SOW, recommendations have been developed that could inform the NPARP “implementation modalities” and/or “other USG programming and strategic planning, especially as [they relate] to the introduction and implementation for new programs.” These recommendations are not intended for Tatweer implementation. Ensure Engagement of GOI Entity Top Management Many of Tatweer’s potential systems improvement results were not fully realized in all the assisted GOI entities due to the lack of active engagement by top-level managers in certain GOI entities (e.g., MOE) to implement administrative reforms in procurement, HR, budget, and project management systems. Consequently, trainees were frequently unable to succeed in the “center out” approach to convince top managers to implement the concepts and principles they had learned in their training. Arguments are compelling that the Tatweer “center out” approach was the most effective one available in Iraq during the life of the project. COMSEC has noted the initial suspicions of espionage among the obstacles to implementing a combined “top down / bottom up” approach in 2006. With the solid groundwork established by Tatweer during the past four years, however, there are now strong reasons for NPARP to consider multiple approaches to gaining the active engagement of top managers. In order to assure high success potential for full implementation of results, therefore, projects at the outset should develop and execute focused and innovative “top-level engagement” action plans customized to the particular conditions and likely barriers at each GOI entity. The objective would be to achieve systems reforms that will continue under any management change. Securing the engagement of top-level managers at the outset would be a means toward that end. Top managers are apt to support systems reforms only if they receive technical assistance and/or executive-level training that instill the priority systems reform concepts and principles in their own mindsets. The objective of the top-level engagement should be to get the top-level managers to “own” the systems reform agenda. “Center out” may still be the right approach in certain GOI entities, but others might now be ready for approaches that include top management from the outset. Thus, the following actions are recommended for future project implementations: 1) view the top￾level action plan as an “executive sales” initiative that anticipates all expected sales barriers and includes specific strategies and tasks to overcome them; 2) prepare and present a one-page program brief for each entity that describes the program, the outcomes and benefits proposed for each entity, and the specific commitments required by the entity; 3) determine if an approach including the active engagement of top management from the outset would be more likely to achieve results than a “center out” approach; 4) make initial presentation by the most senior level contractor manager available and perhaps accompanied by a senior USAID official; 5) consider the drafting and execution of a Memorandum of Cooperation (MOC) before initiating technical assistance and/or training; 6) as top managers are unlikely to join any courses for lower ranking civil servants, craft specialized courses for higher officials, including those delivered only to each GOI entity and those designed for cross-ministry and/or cross-executive office delivery. Top management targeted should include all managers at the Deputy DG level and above; 7) consider selective study tours for senior managers to witness firsthand best practices implementations in their specific areas of responsibility; 8) take into account that while many systems reforms can be implemented by the GOI entities themselves, others may require legislative changes, particularly in financial and procurement systems; and 9) be prepared, therefore, to provide law drafting and appropriate advocacy technical assistance. For the reasons noted above, the MOC objective should be not only to secure support for technical assistance to be delivered to the entity’s departments and personnel but also the direct engagement 26 Tatweer National Capacity Development Program Final Evaluation of top GOI entity management in the training and/or technical assistance. For the deepening of procurement systems reforms, it is of paramount importance to secure the engagement of the top management of MOF, which controls a critical part of the procurement life cycle through its budgeting and funding allocation authority. The efforts to secure top-level engagement should be supplemented by initiatives to continue to build the policy development capacity at all executive offices and to continue to reach out to the provinces with improved skills through “administrative decentralization.” Expanded public outreach efforts to reach new leaders and to demonstrate the benefits of new methods and systems to those GOI entities not yet embracing reforms should also be considered. A large number of mid-level managers support this recommendation, as do top-level managers at most executive offices, particularly COMSEC for training top managers at all levels above Deputy DG, and DPMO-EA for selective study tours for senior managers. Please note that given the difficult security situation in 2006/07, Tatweer could not have implemented this recommendation, but it should be considered for future project implementation. Improve Effectiveness of Technical Assistance Delivery Effective technical assistance delivery is a critical element for achieving targeted systems reforms. Effective delivery requires an action plan that targets the right technical assistance for each GOI entity and the recruitment of the right people to execute the action plan. The following actions are recommended: 1) once top management engagement is secured (or once support is obtained if employing a “center out” approach), conduct a needs assessment for each GOI entity taking into account all departmental needs and develop a capacity building strategy before the initiation of technical assistance; 2) consider opportunities for embedding mentors who could identify and develop expert “clones” within key GOI entity departments through a “learning by doing” action training approach; 3) expand technical assistance in governance, as good governance sets the foundation for effective implementation of all technical assistance; 4) as GOI entities are now seeking higher level and more advanced technical assistance and training, consider means by which more qualified Arabic speaking international experts might be engaged to meet specific needs defined in the capacity building strategies of each GOI entity; 5) give careful attention to the recruitment and training of qualified Iraqi staff to support the international experts, including their industry-specific technical qualifications; and 6) work more closely with GOI entity top-level managers to provide them with the information they need to define more clearly the TORs for international consultants. Please note that NPARP would not likely address all the identified needs, but the GOI entity would benefit by having a comprehensive needs analysis completed, especially if seeking additional assistance from other donors. The NPARP contractor would also benefit by understanding more precisely the specific gaps it planned to close. Further, GOI entities should be encouraged to continue Tatweer’s TMS and OSTP initiatives in order to better define needs. Several GOI entities have expressed support for these recommendations, but better TOR definitions are most important to MOP. Improve Effectiveness of Training Delivery Continued delivery of skills training to GOI civil servants is another critical element for achieving targeted systems reforms. Given the large number of civil servants yet to be trained and the need to provide higher training to many who have already received competency level training, even greater emphasis must be placed on expanding TOT delivery. The Tatweer TOTs, Scholarship Program returnees, OSTP teams, and master trainers constitute a major asset for the NPARP implementation to accomplish this objective in conjunction with standing up the new Civil Service Institute. Specific actions to consider include: 1) follow up the training they have already received with special care and attention to improving their skills further, especially in higher levels and more technical areas; 2) consider special conferences at which they can share experiences and develop new ideas for more effective implementation; 3) continue and deepen implementation of TMS training; 4) focus less on total numbers trained and more upon priority target needs such as procurement; and 5) review and consider all the training delivery suggestions presented in the Learned Lessons section, especially those related to needs analyses, higher level training, more technical training, and OTJ training. Tatweer National Capacity Development Program Final Evaluation 27 Improve Coordination at All Levels Achieved results can always be improved through better coordination at all levels to leverage resources and to more closely integrate initiatives focused on administrative systems reforms. For better coordination to be more than lip service, the following actions are recommended for project implementers: 1) meet regularly with key stakeholders at all levels to develop inputs required for coordinated action plans; 2) implement new programs with closer coordination between the national GOI entities and the provincial entities as well as between the provincial entities and the district and even sub-district entities; 3) develop mechanisms for improving understanding at all levels about the Iraqi laws that define legal authorities for each, including their limitations; and 4) by so doing, develop integrated strategies for effective implementation at all levels that can be supported at each level. The Tatweer Babil Regional Office strongly supports this recommendation. On a separate matter related to coordination between a USAID contractor and a GOI entity, increase transparency regarding the resources actually available for technical assistance. No GOI entity can develop an effective strategic plan for capacity building without knowing the resources available to execute the plan. Further, many GOI entities are challenged to optimize outcomes with resources from several donors, and that challenge requires a clear understanding of resources available from each. The need for more transparency regarding resource availability was voiced most strongly by MOP. Increase Flexibility to Respond to Changing Demand Maximum achieved results can only be accomplished if a project can deliver the right technical assistance to the right places with the right people at the right time. Changed conditions can block the timely delivery of technical assistance if flexibility is not built into the delivery mechanism. New opportunities, obstacles, and conditions affecting demand cannot always be foreseen. Due to changing conditions in Iraq, Tatweer negotiated 21 contract modifications. Rather than looking to contract modifications to respond to changing conditions, USAID might consider “demand driven” components of new programs that would give projects the flexibility to respond to changing conditions more rapidly. Developing CLINs with “demand driven” components (e.g., 20 percent) is one alternative for increasing flexibility. With such CLINs, the contractor could propose appropriate interventions to USAID for approval. The USAID/Georgia Enterprise Growth Initiative Project (GEGI) used a “demand driven” CLIN in 2003-2005. Contingency funding mechanisms might also be considered to help a project respond to changing demand conditions at the time it is most needed. Several GOI entities expressed support for this recommendation. Special Considerations for Industry-Specific Technical Assistance GOI entities have industry-specific technical assistance and training requirements in addition to commonly shared management and departmental needs. In order to implement GOI systems reforms that have real impact and result in the improved delivery of services to the Iraqi people, technical assistance and training must address the industry-specific needs as well as the commonly shared ones. Tatweer’s Energy Group demonstrated the high level of results that can be achieved through focused team engagements led by highly qualified industry experts. Serious consideration should be given to replicating this model with all assisted GOI entities. Needs assessments would be critical, as each would have distinct and varying priority needs, furthermore, each would have industry-specific needs for higher-level technical training. MOO and its 16 operating companies are good examples. They have special advanced technical assistance needs that are not common with other GOI entities, including oil processing, oil extraction, OTJ training delivered at refineries abroad, and technical training delivered onsite in the oil fields themselves. MOE, MOA, MOWR, and other ministries have their own specific needs for advanced technical assistance. New programs that replicate the Tatweer Energy Groups model in all GOI entities would have substantive impact. Given the extraordinarily high importance of oil revenues to Iraqi economic growth, serious consideration should also be given to a Phase II implementation of the Tatweer Energy Group, with focus on more advanced technical assistance. Also, while in-kind cost sharing opportunities should be pursued in all sectors, the oil sector would likely be the most receptive GOI entity to monetary cost sharing due to its stronger financial position. MOO supports this recommendation. 28 Tatweer National Capacity Development Program Final Evaluation Outline Summary of All Recommendations Secure Engagement of GOI Entity Top Management 1. Develop and execute focused and innovative “top-level engagement” action plans customized to the particular conditions and likely barriers at each GOI entity. 2. Prepare and present a one-page program brief for each entity that describes the program, the outcomes and benefits proposed for each entity, and the specific commitments required by the entity. 3. Determine if an approach including the active engagement of top management from the outset would be more likely to achieve results than a “center out” approach. 4. Craft specialized courses for higher officials, including those delivered only to each GOI entity and those designed for cross-ministry and/or cross-executive office delivery; top management targeted should include all managers at the Deputy DG level and above. 5. Be prepared to provide law drafting and appropriate advocacy technical assistance. 6. Continue to build the policy development capacity at all executive offices. 7. Continue to reach out to the provinces with improved skills through “administrative decentralization.” 8. Expand public outreach efforts to reach new leaders and to demonstrate the benefits of new methods and systems to those GOI entities not yet embracing reforms. Improve Effectiveness of Technical Assistance Delivery 1. Conduct a needs assessment for each GOI entity, taking into account all departmental needs, and develop a capacity building strategy before the initiation of technical assistance. Determine USAID project role within that overall strategy. 2. Embed mentors who could identify and develop expert “clones” within key GOI entity departments through a “learning by doing” action training approach. 3. Work more closely with GOI entity top-level managers to provide them with the information they need to define more clearly the TORs for international consultants. 4. Encourage GOI entities to continue Tatweer’s TMS and OSTP initiatives in order to better define organizational needs. Improve Effectiveness of Training Delivery 1. Follow up the Tatweer TOT training with special care and attention to improving their skills further, especially in higher levels and more technical areas. 2. Organize special conferences at which TOTs can share experiences and develop new ideas for more effective implementation. Improve Coordination at All Levels 1. Meet regularly with key stakeholders at all levels to develop inputs required for coordinated action plans. 2. Implement new programs with closer coordination between the national GOI entities and the provincial entities as well as between the provincial entities and the district and even sub-district entities. Increase Flexibility to Respond to Changing Demand 1. Consider “demand driven” components of new programs that would give projects the flexibility to respond to changing conditions more rapidly. A “demand driven” CLIN is one alternative for increasing flexibility, under which the contractor could propose appropriate interventions to USAID for approval. 2. Contingency funding mechanisms might also be considered to help a project respond to changing demand conditions at the time it is most needed. Tatweer National Capacity Development Program Final Evaluation 29 Special Considerations for Industry-Specific Technical Assistance 1. Replicate the Tatweer Energy Group model with all assisted GOI entities. 2. Implement a Phase II program of the Tatweer Energy Group, with focus on more advanced technical assistance. 3. Pursue in-kind cost sharing opportunities in all sectors and monetary cost sharing with MOO and its 16 operating companies. 30 Tatweer National Capacity Development Program Final Evaluation REFERENCES Annual Reports: Tatweer Project: Developing National Capacity in Public Management. Annual Report Year I (September 2006-September 2007). Management Systems International, December 19, 2007. Tatweer Project: Developing National Capacity in Public Management. Annual Report Year 2 (October 2007-September 2008). Management Systems International, October 30, 2008. Tatweer Project: Developing National Capacity in Public Management. Annual Report Year 3 (October 2008-September 2009). Management Systems International, October 30, 2009 Tatweer Project: Developing National Capacity in Public Management. Annual Report Year 4 (October 2009-September 2010). Management Systems International, October 30, 2010. Quarterly Reports: Tatweer Project: Developing National Capacity in Public Management. Quarterly Progress Report -1 (August-October 2006). 30 November 2006. Tatweer Project: Developing National Capacity in Public Management. Quarterly Progress Report -2 (November-December 2006). 30 January 2007. Tatweer Project: Developing National Capacity in Public Management. Quarterly Progress Report -3 (January-March 2007). 30 April 2007. Tatweer Project: Developing National Capacity in Public Management. Quarterly Progress Report -4 (April-June 2007). 30 July 2007. Tatweer Project: Developing National Capacity in Public Management. Quarterly Performance Data (Jul-Sep 2007). Management Systems International, 6 Nov 2007 Tatweer Project: Developing National Capacity in Public Management. Quarterly Progress Report -6 (October-December 2007). 31 January 2008. Tatweer Project: Developing National Capacity in Public Management. Quarterly Progress Report -7 (January-March 2008). 30 April 2008. Tatweer Project: Developing National Capacity in Public Management. Quarterly Progress Report -8 (April-June 2008). 31 July 2008. Tatweer Project. Quarterly Report -9 (July-September 2008). Submitted as Annex A to the Annual Report, Year Two. 30 October 2008. Tatweer Project: Developing National Capacity in Public Management. Quarterly Progress Report -10 (October-December 2008). 31 January 2009. USAID/Tatweer: Developing National Capacity in Public Management. Quarterly Progress Report -11. 30 April 2009. USAID/Tatweer Program. Quarterly Progress Report -12 (April-June 2009). 31 July 2009. USAID/Tatweer Program. Quarterly Progress Report -13 (July-September 2009). Submitted as Annex A to the Annual Report, Year 3. 30 October 2009. USAID/Tatweer Program. Quarterly Progress Report -14 (October-December 2009). 30 January 2010. USAID/Tatweer Program. Quarterly Progress Report -15 (January-March 2010). 30 April 2010. USAID/Tatweer Program. Quarterly Progress Report -16 (April-June 2010). 30 July 2010. USAID/Tatweer Program. Quarterly Progress Report -17 (July-September 2010). Submitted as Annex A to the Annual Report, Year 4. 30 October 2010. USAID/Tatweer. Quarterly Progress Report 18 (October-December 2010), 30 January 2011. Tatweer National Capacity Development Program Final Evaluation 31 Contract Amendments: National Capacity Development Program. Modification No.6. Contract No. DFD-I-00-05-00221-0. USAID: September 2007. National Capacity Development Program. Modification No.11. Contract No. DFD-I-00-05-00221-0. USAID: August 2006. National Capacity Development Program. Modification No.12. Contract No. DFD-I-00-05-00221-0. USAID: August 2008 National Capacity Development Program. Modification No.19. Contract No. DFD-I-00-05-00221-0. USAID/Iraq: December 2010. Documents: Iraq Institutional Capacity Building (ICB) Assessment. Submitted to USAID/Iraq, May 2010. (Prepared by Randall Thompson) Office of Inspector General. Audit of USAID/Iraqs’ National Capacity Development Program. (Audit Report No. E-267—09-001-P). November 25, 2008. United Nations/World Bank Joint Iraq Needs Assessment. October 2003. (http://siteresources. worldbank.org/INTIRAQ/Overview/20147568/Joint%20 Needs%20Assessment.pdf) Republic of Iraq. Ministry of Planning. National Development Plan: Years 2010-2014. Ministry of Plan: Baghdad, 2010. Task Order #1: National Capacity Development Program, Contract No. DFD-I-00-05-00221-0. USAID: August 2006. Tatweer Project. Ministry Baseline Report. Management Systems International (MSI), 15 September 2008. USAID/Tatweer Program. Performance Management Plan FY 2006-FY 2011. July 2010. USAID/Iraq. Tatweer, Curriculum Development Directorate. Training Analyst Course. 2 March 2011. (CD) 32 Tatweer National Capacity Development Program Final Evaluation ANNEX 1: USAID/TATWEER RESULTS FRAMEWORK Tatweer National Capacity Development Program Final Evaluation 33 ANNEX 2: USAID/TATWEER FUNCTIONAL ORGANIZATION CHART 34 Tatweer National Capacity Development Program Final Evaluation ANNEX 3: FINAL EVALUATION TATWEER, IRAQ – SELF-COMPLIANCE MONITOR Final Evaluation Tatweer, Iraq – self-compliance monitor. Source: USAID Evaluation Policy – January 2011 Criteria Comment The evaluation report should represent a thoughtful, well-researched and well-organized effort to objectively evaluate what worked in the project, what did not and why. The evaluation team followed a disciplined and structured process; Evaluation Criteria identification - Recommendations Evaluation reports shall address all evaluation questions included in the scope of work. Ten questions in SOW + additional upon request by USAID The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline need to be agreed upon in writing by the technical officer. Annex 3: SOW No modifications Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an Annex in the final report. Annex 5: Methodology and Evaluation Criteria Evaluation findings will assess outcomes and impact on males and females. Section 11c of Findings and of Conclusions: Gender analysis of Training and TOT attendees Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). Methodology: Constraints of security and also email questionnaire Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions. Findings should be specific, concise, and supported by strong quantitative or qualitative evidence. Triangulation of information was used to develop conclusions and recommendations; Interviews, Observation, Documentation. Sources of information need to be properly identified and listed in an annex. Annex: Interview Log plus References Section of Report. Recommendations need to be supported by a specific set of findings. Findings Section of Report Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. Presented to USAID Mission and to Tatweer Managers for review of actions recommended. Tatweer National Capacity Development Program Final Evaluation 35 ANNEX 4: SCOPE OF WORK FINAL EVALUATION TATWEER, IRAQ I. Project Description Since 2006, USAID/Iraq’s Tatweer National Capacity Development program has sought to build capacity in key Iraqi ministries by increasing the effectiveness of targeted ministries through reforming internal operational systems and instituting best practices and international standards. The program is designed to develop public management skills, improve operating systems, and institutionalize training activities at national and provincial levels. Core public administration areas include fiscal management, including contract and procurement management; human resource management; project management; leadership and communication; and strategic planning and information technology. The program works with the National Center for Consultancy and Management Development and nine ministries: Health, Education, Agriculture, Water Resources, Municipalities and Public Works, Human Rights, Migration, Oil, Electricity, and Planning. In addition, USAID/Tatweer works with GOI Executive Offices, including the Presidency Council, the Prime Minister’s Office, the Deputy Prime Minister’s Office, the Council of Ministers Secretariat and the Prime Minister’s Advisory Committee. Recently added were the National Investment Commission and the Social Safety Net program of the Ministry of Labor and Social Affairs. The Tatweer program works with counterparts through ministry engagement teams, a mix of expatriate and local staff, often embedded at partner ministries. The teams focus on: • Improving the public management skills of ministerial civil servants; • Permanent and sustainable improvements to management systems and standard operating procedures in partner ministries at both the central and provincial level. These include systems related to budget formulation and execution, human resource management systems, project management, and investment projects; • Expanding and institutionalizing the GOI’s training and consultancy activities at the ministries and the National Center for Consultancy and Management Development; and • Instituting civil service institutional and legislative reforms. Key content focal areas are civil service reform, provincial training and coaching, training of trainers, and best practices tools and governance methods. Key GOI and other implementing partners include: • Ministries at the national and provincial level: Health; Education; Agriculture; Water; Municipalities & Public Works; Human Rights; Migration; Oil; Electricity; Planning; Labor and Social Affairs (only as it relates to the Social Safety Net). • Executive Offices: Presidency Council; Prime Minister’s Office; Deputy Prime Minister’s office/Economic Affairs; Deputy Prime Minister’s office/Services; Council of Ministers’ Secretariat; Prime Minister’s Advisory Committee. • Other Entities: National Investment Commission; Oil Operating Companies; Oil Refineries. • Training Entities: National Center for Consultancy and Management Development; Al Quds School for Computers, Kurdistan Oil Training Institute, Basra Oil Training Institute. 36 Tatweer National Capacity Development Program Final Evaluation For more detailed information, see the Tatweer Fact Sheet attached to this SOW as Annex 1. II. Background: There are a number of challenges when facing the objective of creating a functional, representative, and effective government structure in Iraq. The former regime did not encourage the development of an effective civil service or efficient forms of governance. However, since 2003, the governance structures of Iraq have been rapidly changing, moving towards a more decentralized structure. The current Government of Iraq is the subject of frequent criticism which relates to a lack of effectiveness that can be traced to many causes. To name but a few: the country has been isolated from the global system for over 50 years; international standards, modern technology, and current best practices are lacking; and security still poses a major issue, with direct implications for assistance. The current Tatweer project was initiated in 2006, and as such was based upon a design that reflected the issues then current in Iraq. Much has changed in the governance landscape since that time, although many issues remain the same. This final evaluation is being commissioned to provide the Agency with documentation on Tatweer’s results achieved. The study will also provide insight as to how and why these results were achieved, or why results were not achieved as originally planned. This evaluation is part of the routine program monitoring process as practiced by USAID. The results of the evaluation will inform Mission management of critical lessons learned, challenges, and performance highlights of one of the most visible USG programs in Iraq. Additionally, USAID/Iraq will soon implement a successor program to Tatweer. The new National and Provincial Administrative Reform Program (NPARP) will build upon the successes achieved by the Tatweer program, and will introduce and anchor critical ministry systems, specifically those related to program management and budget formulation and implementation. It will also serve to support development and improvement of the policy framework for the various sectors, and will provide technical assistance to build civil servants’ capacity to identify and draft their own roadmap for reforms. This evaluation will not be used for the new program’s design, which has already been completed. However, it may be significant in terms of informing implementation modalities for NPARP. Also, due to the crosscutting nature of the scope and scale of the Tatweer program, the results of this evaluation will provide valuable information for other USG programming and strategic planning, especially as it relates to the introduction and implementation of new programs. III. Purpose: The objective of this final evaluation is to assess the relevance, performance, and success of the Tatweer program. This evaluation is being undertaken to provide information to USAID on whether the underlying theories and assumptions used in the development and implementation of the program were valid, what worked or did not work, and why. The aim is to determine relevance, Tatweer National Capacity Development Program Final Evaluation 37 efficiency, effectiveness, impact, and sustainability. The evaluation will also serve as a vehicle for extracting cross-cutting lessons from the Mission’s experiences and determining the need for country strategy modification. IV. Research Questions: Issues to be addressed during this evaluation mission should include the following elements: Impact Explain results and net impacts of activities and identify any unintended impacts. If results were not met, identify why not and provide recommendations for strengthening development strategy so they can be or will be met. To the extent possible, measure the increase in delivery of government services to the population. Sustainability What are the prospects for sustainability of the end results produced by the program? What identified results appear less sustainable and why? Was the scale appropriate to ensure sustainability? Client Satisfaction Determine if the customer (GOI) needs were met, and if not met what wasn’t met and why. Cost Effectiveness Was this program implemented and managed cost effectively? Relevance How relevant is this program to the short, middle and long term development needs of Iraq? Validity of hypotheses and assumptions underlying results framework Is the original program design framework still valid? Have framework parameters changed, and why? V. Suggested Methodology and Study Tasks: Desk-top review of key documents: Upon approval of the team composition, USAID/Iraq will provide the team with the Tatweer Scope of Work, copies of the Task Order and subsequent modifications, all reports and Performance Monitoring Plans submitted under this program, as well as any other supplemental documentation or reporting information required. Given the inherent logistical challenges of operating in Iraq, the team 38 Tatweer National Capacity Development Program Final Evaluation may not be able to move around as freely as may be preferred. The team will need to review these documents in depth to ensure that their limited time spent on the ground is based on an informed understanding of the programs. Field research: The field research will be comprised primarily of meetings and informational interviews with various individuals in Iraq. The majority of these interviews will take place in Baghdad. However, because the Tatweer program is operational in 14 provinces the team should plan to travel to one of the five regional MSI compounds and at least one visit to a PRT. Other individuals on the target interview list should include: ministry officials, Governors, USAID PRT representatives, leadership of Tatweer, Chief of Party, and other current MSI staff deemed appropriate. The research should also include meetings with the MSI home office. It is expected that the team will conduct informational interviews with existing USAID/Iraq staff. The team should also develop questionnaires or tools to be used for informational interviews and that can be emailed to individuals who are not available for in-person interviews. Development of draft report: The team is encouraged to engage in clarification meetings as a necessary part of developing the draft report. These meetings will include USAID staff who are involved in the oversight and management of the program. The team should plan to submit the draft report to USAID/Iraq by email, no less than three days before departure from Baghdad. USAID/Iraq will review the document and provide comments back to the team one week after receipt. At that point, the team will have an additional six working days to incorporate USAID/Iraq’s comments and submit the final report for approval. VI. Team Composition The technical team should consist of at least four individuals, each of whom should have prior experience in the area of capacity building and governmental administrative strengthening and/or program evaluation. USAID government capacity building, economic growth, agriculture and experience in post-conflict development contexts would be particularly useful. At least one member of the team should speak and read fluent Arabic; and one member should have substantive experience in the Iraq context. The following are the proposed positions/composition of the team: Team Leader – Possesses at least 10-12 years of experience in the area of government capacity building, preferably with a USAID-implementing partner, a USAID project, or with a similar international organization. The team leader must possess familiarity with USAID programming methodologies and USAID internal project management tools including PMPs. This person will have overall responsibility for designing and managing the research work plan. Deputy Team Leader – 7-10 years of experience in government capacity building assistance area, preferably with a USAID-implementing partner or international organization. This person will work closely with the Team Leader and Project Manager to develop the draft and final reports as well as the presentations. Tatweer National Capacity Development Program Final Evaluation 39 Team member – 5-7 years of experience in program design and evaluation, preferably with some experience in government capacity building. This person will guide the logistics of the evaluation as well as contribute to the production of deliverables. Team member – Substantive experience in the governmental capacity building sector, preferably an Iraqi national. VII. Schedule and Timeline The USAID/Iraq mission requests that evaluation team arrive in Baghdad no later than Jan 30, 2011. The following is the proposed schedule for the evaluation: Pre-deployment/background research: 1 week – LOE 5 days (+8 days for program reading material and evaluation design/plan before coming in-country. The team will conference call and draft the plan prior to deployment. Total LOE =13 days) Travel to Iraq: LOE 3 days Field research: 4 weeks (authorized 6-day work week) – LOE 24 days (+8 days for translation, testing and piloting of data collection tools, and USAID input. Total LOE = 32 days) Travel from Iraq: LOE 1 day Development of draft final report after leaving Iraq: 3 days – LOE 3 days (+5 days to develop first draft report and for copy editing. Total LOE = 8 days) Mission review of draft report and submittal of comments to evaluation team: 1 week Submittal of final report to USAID/Iraq: 1 week – LOE 6 days Total LOE: 63 days Total time period/commitment from evaluation team: 12 weeks (including USAID/Iraq’s 1 week review of draft report) Estimated completion date of the evaluation: April 15, 2011 VIII. Deliverables: 1. Questionnaires or tools to be used for informational interviews. 2. Oral presentation with PowerPoint to USAID on final day of deployment in Iraq. This oral report will summarize key findings and analysis to include results of field data collection and desk analysis. Initial draft report will also be submitted in hard and soft copy at this time. 40 Tatweer National Capacity Development Program Final Evaluation 3. The Final Report shall be no longer than 25 pages, excluding annexes, and shall include: A. A 3-4 page Summary Report of key findings, conclusions, and recommendations. B. Executive Summary C. Introduction D. Background and purpose E. Project Description F. Research Questions G. Observations and Findings H. Analysis and Conclusions I. Recommendations J. Lessons Learned K. Annexes Tatweer National Capacity Development Program Final Evaluation 41 ANNEX 5: PRE-DEPARTURE WORKPLAN FOR EVALUATION OF USAID TATWEER PROGRAM Final Evaluation of USAID/Tatweer Program Purpose, Work plan and Methodology 31st January 2011 1. Purpose and Approach The purpose of the final evaluation of the Tatweer program (2006-2010) is to assess the achievements of the program in terms of relevance, performance and success. In this respect the study is a routine final or summative evaluation. In terms of evaluative criteria, the evaluation methodology will be based on evaluation criteria noted in the SOW (Part IV) ‘Research Questions’ where there is reference to assessment of: I. Impact: Assessment of results and net impacts of activities and identification of any unintended impacts. If results were not met, identification of why not and recommendations for strengthening development strategy so they can be or will be met. Success stories will feature here. II. Sustainability: Assessment of the prospects for sustainability of the end results produced by the program. What identified results appear less sustainable and why? Was the scale appropriate to ensure sustainability? III. Client satisfaction: Were the customer (GOI) needs met, if not - what wasn’t met and why? IV. Cost Effectiveness: Was this program implemented and managed cost effectively? V. Relevance: How relevant is this program to the short, middle and long term development needs of Iraq? VI. The validity of hypotheses and assumptions underlying the results framework: Is the original program design framework still valid? Have framework parameters changed, and why? The SOW states that in addition to the above, the study can serve to inform implementation modalities for NPARP (the new National and Provincial Administrative Reform Program), the USAID/Iraq successor program to Tatweer. In terms of approach the team approach will be to focus on the criteria as they relate to the expected results shown below. The Tatweer program is highly visible and expansive, covering most of the GOI ministries and executive offices, which means not every participating ministry or site can be visited due partly to time and geographic constraints but primarily due to the ongoing security situation in Iraq. 42 Tatweer National Capacity Development Program Final Evaluation The Tatweer goals are broad and challenging: Tatweer is expected to improve the delivery of core public services to the Iraqi people. The three expected results of the program are: 1) Strengthen core public administration skills and fill national gaps in delivery of core public services; 2) Assist Ministries to establish more effective administration and process systems; 3) Expand public administration training capacity. The approach taken by the four person team will therefore be to select Ministries and GOI Institutions to visit where all three expected results are in progress or planned. In light of the general insecurity in Iraq and in particular in the ministries and other government bodies, the team will use appropriate assessment tools for informational interviews of individuals and / or small groups. In addition the team will work closely with the Tatweer team in facilitating security and access to the various ministries and other sites. This approach is expected to increase efficiency of the evaluation exercise. 2. Methodology The methodology to be used by the team will include three methodologies: a. Comparative Study of Tatweer Performance in Different Geographic Locations: Field visits to different geographic locations that have implemented Tatweer activities related to all three of the expected results will be carried out. By interviewing key players and comparing between the sites, the team will be able to understand the constraints and facilitating factors contributing to the degree of effectiveness. This will also support implementation modalities for the new project, showing alternative approaches. Comparative field visits will be made to sites where the program is visible in terms of completed activities related to the three expected results: training, systems reform, and training capacity development. From documentation study to date, the team has identified geographical areas where all three results can be studied: in Baghdad, Basra, and Hilla. The Team will develop a Performance Rating Tool based on each of the six evaluation criteria for the project as a whole. This tool will grade the Tatweer program based on four standard ratings; Highly Satisfactory, Satisfactory, Less than Satisfactory, and Highly Unsatisfactory. b. Use of Three Types of Tools: 1. Semi-structured Interview Guides –- Open Questions: -questions: which will be used for interviewing the various target audiences such as: Tatweer Project Managers, Chief of Party, Governors, USAID PRT representatives, GOI ministry officials, International Trainers, International Advisors, Iraqi Trainers (from the Training of Trainers programs), Trainees, and scholarship beneficiaries. These interviews will provide progress reviews of Tatweer in respect to the three expected results, status versus actual achievements, and other evaluation criteria.. The team will study and document these interviews at the end of every day. 2. Posted Questionnaires – Closed Questions: to determine beneficiary satisfaction the team will develop a tool similar to the one used in the midterm evaluation shown in the PMP’s Annex 6 for comparison. Tatweer will provide a list of individuals to whom the questionnaire can be posted. 3. Success story interviews: These will be conducted with individuals encountered by the team during the evaluation period to indentify and document additional stories to those on Tatweer National Capacity Development Program Final Evaluation 43 the Tatweer Success Story website. c. Documentation Review: In terms of documentation review the team will study a wide range of documentation, including the Performance Monitoring Plans (PMPs), as means of understanding and comparing both expected and actual performance. Many other documents will be studied, such as the Annual and Quarterly reports, midterm review, and Organizational Self Assessment tools. Others include the SIGIR (Special Inspector General for Iraq Reconstruction) Audit, Iraq Institution Capacity Building (ICB) Assessment, and the Implementing Partners PMP Review completed December 2010. In addition the team will study the various databases in the Tatweer M&E Department (Survey, Work-plan, Achievements and Training) to further evaluate the project in terms of the stated criteria and the three expected results. 3. Deliverables 1. Questionnaires or tools to be used for informational interviews; See sample Appendix 1 Interview Guide / Questionnaire. 2. An oral presentation with PowerPoint slides will be made to USAID four days before the team’s final day of deployment in Iraq. This oral report will summarize key findings and analysis to include results of field data collection and desk analysis. An initial draft report will also be submitted in hard and soft copy at this time. 3. The Final Report shall be no longer than 25 pages, excluding annexes, and shall include: a. A 3-4 page Summary Report of key findings, conclusions, and recommendations. b. Executive Summary c. Introduction d. Background and purpose Project Description e. Research Questions f. Observations and Findings g. Analysis and Conclusions h. Recommendations i. Lessons Learned j. Annexes 44 Tatweer National Capacity Development Program Final Evaluation APPENDIX 1: Sample Interview Questionnaire FINAL EVAUATION TATWEER Iraq INTERVIEW QUESTIONNAIRE 3: GOVERNMENT OFFICIALS AND CONTRACTORS A. INTRODUCTION USAID has commissioned this final evaluation to provide the Agency with documentation on Tatweer’s results achieved. The study will also provide insight as to how and why these results were achieved, or why results were not achieved as originally planned. The results of the evaluation will inform Mission management of critical lessons learned, challenges, and performance highlights of one of the most visible USG programs in Iraq. This evaluation is being undertaken to provide information to USAID on whether the underlying theories and assumptions used in the development and implementation of the program were valid, what worked or did not work, and why. The aim is to determine relevance, efficiency, effectiveness, impact, and sustainability. The evaluation will also serve as a vehicle for extracting cross-cutting lessons from the Mission’s experiences and determining the need for country strategy modification. The Tatweer project approach has been to identify and focus upon groups of specific priority systems changes for each ministry in terms of making impact on improvements in the ministries. Six key administrative function areas have been targeted for improvement: human resources, fiscal (including contract and procurement), IT, project management, leadership and communication, and strategic planning. Program results focus on the following: 1) strengthen core public administration skills; 2) ministries establish more effective administrative systems; and 3) expand public administration training capacity. The Final Evaluation Team for the Tatweer Iraq project would appreciate your candid insights and views about the questions overleaf. For your replies and comments, please note the program results focus (skills, more effective administrative systems, or training capacity), the specific key administrative function area(s), and the specific ministries and/or agencies. Interviewee Name: Province: Job Title: Institute: Interview Date: 45 B. CORE QUESTIONS 1. Achieved Results • What were the most important results achieved? • How were these results achieved? What factors enabled the successful achievement of these results? • In addition to Performance Indicators, were clear action plans developed to define target results with specific tasks, start and end dates, responsible task managers, and milestones? • Were actual results targeted or were they proxy results? • How were these results measured? 2. Targeted Results Not Achieved • What were the most important targeted results not achieved? • What specific obstacles prevented the achievement of results? • What are your recommendations for strengthening the development strategy so targeted results can be or will be met? 3. Critical Lessons Learned, Challenges, and Performance Highlights • What were the most important lessons learned from both the achievement of results and the failure to do so? • What major challenges are posed for future project implementations? • What were the most important performance highlights? Such highlights may or may not include key processes and/or implementation mechanisms directly relating to achieved results. 4. Underlying Theories and Assumptions • What were the most important development hypotheses for the Tatweer Project? • How do you rate the validity of each (high, moderate, low)? • What worked in implementing these development hypotheses, what did not work, and why? 5. Cross-Cutting Lessons Learned • What were the most important lessons learned in implementation of the Tatweer Project that would also apply to other USG projects? • What worked and what didn’t work in anti-corruption reforms (and why)? 46 Tatweer National Capacity Development Program Final Evaluation C. SUPPORTING QUESTIONS Impact What were the net impacts of key activities? Were there any unintended impacts? Did the delivery of government services to the population increase? If so, how can that increase best be measured? Sustainability What are the prospects for sustainability of the achieved results identified above? What achieved results appear less sustainable and why? Was the scale appropriate to ensure sustainability for these targeted results? Client Satisfaction Were GOI needs met? If not met, what wasn’t met and why? Cost Effectiveness Was the Tatweer project implemented and managed cost effectively? Relevance How relevant is the Tatweer project to the short, middle, and long term development needs of Iraq? Validity of hypotheses and assumptions underlying results framework Is the original Tatweer project design framework still valid? Have framework parameters changed, and why? 47 ANNEX 6: INTERVIEW LOG (Removed – For Internal Use Only) 48 Tatweer National Capacity Development Program Final Evaluation ANNEX 7: METHODOLOGY The Evaluation Process The evaluation team conducted the assessment over a 12-week period from January 10 2011. A six week period was spent ‘on the ground’ in Iraq supported by the QED PERFORM Monitoring and Evaluation Advisor and support staff in Baghdad. The Evaluation Process started with analysis of the SOW and finalized in the recommendation section of the final report as described in the steps and the flow chart below: • Analysis of the SOW • Question Development from Evaluation Criteria in the SOW • Design of the interview frameworks and the Trainee Questionnaire • Grouping of key document review information into Tatweer Document Reviews (TDRs) • Grouping of all interview notes into Tatweer Interview Responses (TIRs) • Categorizing the observations, findings and lessons learned • Disaggregation of the analysis and conclusions • Formulating the recommendations 49 1. Research Questions The evaluation criteria were set by USAID within the SOW in the form of research questions. They were extracted from an analysis of the SOW at the beginning of the project and became the basis for the entire evaluation process. A. Core Questions: The five core questions came from the body of the SOW. For example, where the SOW stated ‘This final evaluation is being commissioned to provide the Agency with documentation on Tatweer’s results achieved,’ the team used the evaluation criteria ‘Results Achieved’ to design questions to elicit responses from interviewees that would describe and verify any achieved results. 1. Results Achieved? • What were the most important results achieved? • How were these results achieved? What factors enabled their successful achievement? • In addition to Performance Indicators, were clear action plans developed to define target results with specific tasks, start and end dates, responsible task managers, and milestones? • Were actual results targeted or were they proxy results? • How were these results measured? 2. Targeted Results Not Achieved? • What were the most important targeted results not achieved? • What specific obstacles prevented the achievement of results? • What are your recommendations for strengthening the development strategy so targeted results can be or will be met? 3. Critical Lessons Learned Challenges, and Performance Highlights? • What were the most important lessons learned from both the achievement of results and the failure to do so? • What major challenges are posed for future project implementations? • What were the most important performance highlights? Such highlights may or may not include key processes and/or implementation mechanisms directly relating to achieved results. • How well was your team efforts coordinated with other Tatweer team efforts? 4. Underlying Theories and Assumptions? a. What were the most important development hypotheses for the Tatweer Project? b. How do you rate the validity of each (high, moderate, low)? c. What worked in implementing these development hypotheses, what did not work, and why? 5. Cross-Cutting Lessons Learned? • What were the most important lessons learned in implementation of the Tatweer Project that would also apply to other USG projects? • What worked and what didn’t work in anti-corruption reforms (and why)? 50 Tatweer National Capacity Development Program Final Evaluation B. Supporting Questions: These questions came directly from the Research Question section of the SOW. Evaluation Criteria Impact : • What were the net impacts of key activities? • Were there any unintended impacts? • Did the delivery of government services to the population increase? • If so, how can that increase best be measured? Sustainability : • What are the prospects for sustainability of the achieved results identified above? • What achieved results appear less sustainable and why? • Was the scale appropriate to ensure sustainability for these targeted results? Client Satisfaction: • Were GOI needs met? • If not met, what wasn’t met and why? Cost Effectiveness: • Was the Tatweer project implemented and managed cost effectively? Relevance: • How relevant is the Tatweer project to the short, middle, and long term development needs of Iraq? C. Additional Questions: These questions were identified in the initial Briefing Meeting with USAID on February 2 as being issues that should be considered during the evaluation process. • Cost Sharing • Donor Partnerships • Gender 51 2. Approaches and Methodologies Throughout the evaluation process a mixture of quantitative and qualitative data was employed. 2.1 Data Research: a. Quantitative research involves systematic investigation and analysis of information capable of being measured or expressed in numerical terms, e.g. percentage split of trainees by age, geographic location, or gender. The objective of quantitative research is to develop theories or hypotheses about the information being analyzed. b. Qualitative research produces information only on the particular cases studied, e.g. project activity examples discussed in interviews, and general conclusions are only hypotheses. Quantitative approaches can be used to verify which of these hypotheses are valid. 2.2 Methods: Three different research methods were undertaken: a. Core Methods (documentation review, semi-structured interviews) b. Sampling Related Methods (study of databases, email questionnaires) c. Discussion Methods (group work) 2.2.I Core Methods: a. Desk-Based Document Review: The team reviewed and analyzed a wide range of Tatweer￾related documentation, including quarterly reports and annual reports, as a means of understanding and comparing both expected and actual performance of the project. Additionally, UN reports, and other project evaluations such as the SIGIR (Special Inspector General for Iraq Reconstruction) Audit, the Iraq Institution Capacity Building (ICB) Assessment and the Implementing Partners PMP Review were also examined. In order to understand the ‘bigger picture’ of the project’s impact on the development of the Iraqi Civil Service, the team also reviewed the recent Iraqi National Development Plan (NDP), details regarding the February 2009 Law establishing a Federal Civil Service Commission (FCSC), and the Draft Civil Service Law which is currently with the Ministry of Justice. b. Semi-Structured Interviews: A semi-structured interview is a research tool frequently used because of the optimum flexibility, i.e. new questions can be brought up during the interview as a result of what the interviewee says. The alternative is a structured interview that has formalized, set questions. The interviewer in a semi-structured interview has a framework of questions built around the themes to be explored that are an informal grouping of topics that the interviewer can ask in different ways for different interviewees. The evaluation team designed Interview Guides (see example1) to focus the interviews on the topics without constraining them to a particular format. This helped the team conduct interviews with questions tailored to the interview and also to the context of the situation, as well as to the people they were interviewing. The interview guides changed slightly depending on the ‘target groups’: Tatweer Project Managers, Tatweer Team Leaders, GOI staff (clients), Iraqi Trainers, Trainees, and Scholarship Graduates. Despite the inherent security constraints, the team was able to undertake some limited travel and conduct interviews in Baghdad, Basra, and Hilla. In all, the team conducted some 92 52 Tatweer National Capacity Development Program Final Evaluation interviews and meetings, and interviewed a total of 158 individuals; 117 GOI officials, 36 Tatweer staff and 5 PRT/USAID Representatives. The team met with central and provincial government representatives from: the Office of the Presidency Diwan, the Council of Ministers (COMSEC), the Deputy Prime Minister’s Offices (DPMO), Economic and Services Offices, Provincial Councils, the Midland Oil Company in Baghdad, the South Oil Company in Basra, the National Centre for Consultancy and Management Development (NCCMD), the Central Organizations for Standardization and Quality Control (COSQC), the Ministries of Planning, Oil, Health, Education, Electricity, Agriculture, Water Resources, Labor, and Social Affairs, and Municipalities and Public Works. Lastly, meetings were held with USAID/Iraq and representatives of Provincial Reconstruction Teams (PRT) in Basra and Hilla. The purpose of the interviews was to answer the research questions and seek examples or evidence of the evaluation criteria related to the Tatweer Results Framework. Any Public Administration Reform initiative needs to take place at multiple levels within target government institutions in order to have positive impact on the overall performance improvements – these levels can be categorized as Strategic, Operational, and Administrative. Establishment of a Civil Service Framework • Strategic Intervention: Advice and assistance compiling the Civil Service Commission Law and the Civil Service Law • Operational Intervention: Establishment of Human Resources Units in GOI Institutions, the design of JDs, and PMS • Administrative Intervention at Central and Provincial levels: HR Training including principles, SOPs, JDs and PMS Strategic Admin: Central GOI Operational Admin: Provinces 53 2.2.2 Sampling Related Methods Sampling methodology was used to extend the reach of the team in order to deal with the complexity, numbers, and geographic spread of the project. A sample is expected to mirror the ‘population’ from which it comes, however, it is recognised that there is no guarantee that any sample will be precisely representative as sampling errors or sampling bias can occur. Hence properties were studied of a finite part or sample of the statistical populations to gain information about the whole but not used alone to state findings or observations. a. Review of Databases: The team reviewed samples of the established databases of the Tatweer Monitoring and Evaluation (M&E) Department to assess the information‘s relationship to the documentation review and the semi-structured interviews and to aid assessment of Time Based Patterns of Change. The four Tatweer M&E databases and examples of types of sampled reports are: Database Sampling Reviews Training Monitor 1. Training Participants split by year / month 2. Training Participants split by Province 3. Training Participants split by Institution 4. Training Participants split by Age 5. Training Participants split by Gender 6. Training Participants split by Employee Level 7. Training Participants split by Module 8. Participants views to compare with questionnaire responses Post-Training Surveys Annual Work Plans (Tatweer Team Leaders) Work Plans and Improvements reports from Ministry’s / Tatweer projects were reviewed prior to visiting each institution to provide the team with more in-depth details to enhance their understanding gained from the documentation reviews. And to enable the interviews to focus on specifics of the particular institution so as to seek verification of reports and to discuss examples of embedded improvements / achievements. Priority Systems Improvements Tracking 54 Tatweer National Capacity Development Program Final Evaluation b. Distance Questionnaire: A questionnaire was designed (Example 2) specifically to gather information from Tatweer Training Program Participants – both those trained directly by Tatweer International staff (Direct) and by Iraqi TOT Graduates (Rollout). This questionnaire, in Arabic, was distributed by email in order to seek responses from a random cross section of the approximately 76,000 individual trainees. The email addresses were obtained from Tatweer M&E databases. Email questionnaires are the least reliable of any sampling methods and in Iraq further issues must be recognized: • In the provinces, it is uncommon for people to have Internet access in their homes. Even for those who do, some trainees declined to provide this information and so were not included in the Tatweer database. • Most ministries do not have private email addresses for their employees. In many cases, a single email address serving the whole department was provided, which was not suitable for use this exercise. • Hence, the use of email addresses for the sampling exercise will tend to favour those that live in the provincial capitals and who either have senior positions in the government or a higher income, and so could be biased against participants from the more rural areas. However, the USAID Evaluation Policy states that each evaluation must ensure ‘application and use to the maximum extent possible of social science methods and tools that reduce the need for evaluator-specific judgments.’ Hence it would not be a policy-compliant evaluation without at least attempting to seek comments and views of the GOI Trainees who comprise the largest Tatweer ‘target audience’ and unfortunately emailing is the only available tool given the Iraqi specifics. Of the emailed questionnaires we received a response a response rate of 5.1 percent, which almost exactly matches international response rates to email questionnaires (5%). To combat this anticipated low response rate these questionnaires were also handed out to Tatweer trainees in GOI entities the team visited. Overall the questionnaire responses totalled 197. Thus 0.26 percent of the total universe of approximately 76,000 people trained by Tatweer is represented in this sampling exercise. Statistically responses from this percentage of the total number in the target universe provide a less than 3% sampling error, i.e. reduces prevalence to be positive or negative. In addition to providing numerical data, the responses contributed training related information from ‘real’ people: • 197 people provided information relating to their age, gender, job roles, training programs participated in and training programs they see as needed for the future. • 158 trainees commented on the question: ‘In your opinion does the current level of Training ensure that your organization can achieve its planned goals?’ • 122 commented on: ‘Please add any further comments or opinions that would help us to compile an effective Final Evaluation of the Tatweer program.’ 55 3. Discussion Methods (Focus Group Discussions) On some occasions, when the evaluation team was able to gather a number of people in one venue, the opportunity was used to seek opinions through group discussions. A focus group is a form of qualitative research in which a group of people are asked about their perceptions, opinions, beliefs, and attitudes towards a product, service, concept, or idea. Semi-structured questions are asked in an interactive group setting where participants are free to talk with other group members as well as the interviewees or group moderator. The size of the groups ranged from three to five people in individual ministries to groups of 19 inter-ministerial provincial employees in Hilla and 16 in the MMPW. 56 Tatweer National Capacity Development Program Final Evaluation • Example 1: Interview Guide • FINAL EVALUATION TATWEER Iraq Interview Guide: TATWEER TEAM LEADERS A. INTRODUCTION USAID has commissioned this final evaluation to provide the Agency with documentation on Tatweer’s results achieved. The study will also provide insight to how and why these results were achieved, or why results were not achieved as originally planned. The results of the evaluation will inform Mission management of critical lessons learned, challenges, and performance highlights of one of the most visible USG programs in Iraq. This evaluation is being undertaken to provide information to USAID on whether the underlying theories and assumptions used in the development and implementation of the program were valid, what worked or did not work, and why. The aim is to determine relevance, efficiency, effectiveness, impact, and sustainability. The Tatweer project approach has been to identify and focus upon groups of specific priority systems changes for each ministry in terms of making impact on improvements in the ministries. Six key administrative function areas have been targeted for improvement: human resources, fiscal (including contract and procurement), IT, project management, leadership and communication, and strategic planning. Program results focus on the following: 1) strengthen core public administration skills; 2) ministries establish more effective administrative systems; and 3) expand public administration training capacity. The Tatweer project evaluation team would appreciate your candid insights and views about the questions overleaf. 1a. Institutional Information: Please identify the ‘point person’ in the GOI institute in which you have been leading the implementation of the Tatweer program (s): Ministry / Institute ‘Point Person’ Directorate ‘Point Person’ Job Title ‘Point Person’ email address / contact details ‘Point Person’ Cell Phone # 57 B. CORE QUESTIONS 1. Achieved Results • What were the most important results achieved? • How were these results achieved? What factors enabled the successful achievement of these results? • In addition to Performance Indicators, were clear action plans developed to define target results with specific tasks, start and end dates, responsible task managers, and milestones? • Were actual results targeted or were they proxy results? • How were these results measured? 2. Targeted Results Not Achieved • What were the most important targeted results not achieved? • What specific obstacles prevented the achievement of results? • What are your recommendations for strengthening the development strategy so targeted results can be or will be met? 3. Critical Lessons Learned, Challenges, and Performance Highlights • What were the most important lessons learned from both the achievement of results and the failure to do so? • What major challenges are posed for future project implementations? • What were the most important performance highlights? Such highlights may or may not include key processes and/or implementation mechanisms directly relating to achieved results. • How well were your team efforts coordinated with other Tatweer team efforts? 4. Underlying Theories and Assumptions • What were the most important development hypotheses for the Tatweer project? • How do you rate the validity of each (high, moderate, low)? • What worked in implementing these development hypotheses, what did not work, and why? 5. Cross-Cutting Lessons Learned • What were the most important lessons learned in implementation of the Tatweer project that would also apply to other USG projects? • What worked and what didn’t work in anti-corruption reforms (and why)? 58 Tatweer National Capacity Development Program Final Evaluation C. SUPPORTING QUESTIONS Impact What were the net impacts of key activities? Were there any unintended impacts? Did the delivery of government services to the population increase? If so, how can that increase best be measured? Sustainability What are the prospects for sustainability of the achieved results identified above? What achieved results appear less sustainable and why? Was the scale appropriate to ensure sustainability for these targeted results? Client Satisfaction Were GOI needs met? If not met, what wasn’t met and why? Cost Effectiveness Was the Tatweer project implemented and managed cost effectively? Relevance How relevant is the Tatweer project to the short, middle, and long term development needs of Iraq? Validity of hypotheses and assumptions underlying results framework Is the original Tatweer project design framework still valid? Have framework parameters changed, and why? 59 • Example 2: Trainee Questionnaire (emailed) • (in Arabic) FINAL EVAUATION TATWEER Iraq INTERVIEW QUESTIONNAIRE: TRAINEES FROM TATWEER PROGRAMS B. Introduction The USAID/Tatweer program is collecting data as part of a Final Evaluation exercise to assess how well the project objectives have been achieved. To help us with this assessment we need your opinions – as one of the people who have been ‘clients’ of the Tatweer Training programs. Thank you for agreeing to spend a short time completing this questionnaire; we greatly appreciate your willingness to provide us with information. C. Demographic Information Ministry / Institution Name Job Title Gender (please circle or place an X) Female Male Province (please circle or place an X) Central Anbar Babil Baghdad Basra Dahuk Dhi Qar Diwaniyah Diyala Erbil Karbala Kirkuk Maysan Muthanna Najaf Ninawa Salah ad Din Sulaymaniyah Wasit Length of Service in your Ministry / Institution (please circle or place an X) Less than 1 year 1 to 3 years 4 to 6 years 7 to 10 years 11 to 14 years 15 years + Your Age (please state) D. Tatweer Project Related Information Q1 How long ago did you participate in Tatweer Training? 1 month or less 2 to 3 months 4 to 5 months 6 months or more Q2 What area(s) of Training did you Human Resources Fiscal Management Project Management Leadership / Communication 60 Tatweer National Capacity Development Program Final Evaluation participate in? IT Strategic Planning Other: please state Q3 How long was the Training program (s) (please state) Q4 On a scale of 1 to 5, where 1 is lowest and 5 is highest, please rate (by circling or placing an X): a) The Training Content including exercises etc 1 2 3 4 5 b) The Trainers Delivery / Presentations 1 2 3 4 5 c) The Training Materials (Handouts etc) 1 2 3 4 5 d) Your overall satisfaction level with the Training 1 2 3 4 5 Q5 What additional Training do you think will help you to improve your job performance? Human Resources Fiscal Management Project Management Leadership / Communication IT Strategic Planning Other: please state Q6 On a scale of 1 to 5, where 1 is lowest and 5 is highest, please rate (by circling or placing an X): a) The relevance of the Tatweer Training to your job role 1 2 3 4 5 b) The relevance to the performance of your Ministry / Institution 1 2 3 4 5 Q7 Please rate the level of positive impact that the Tatweer Training has had on your Ministry / Institution 1 2 3 4 5 Q8 In your opinion does the current level of Training ensure that your organization can achieve its planned goals?? Q9 Please add any further comments or opinions that would help us to compile an effective Final Evaluation of the Tatweer program. 61 ANNEX 8A: DATA BASE REVIEW Data Base Review: USAID/TATWEER PERFORMANCE INDICATORS SUMMARY & EXAMPLES PROGRAM RESULT 1: STRENGTHENED CORE PUBLIC ADMINISTRATION SKILLS Program Indicator Documented Results (By Year) References 1.1: Proportion of training graduates reporting that they frequently or sometimes make use of new training skills F = frequent / S = sometimes 2006 2007 2008 2009 2010 2011 PMP, Quarterly Report 17 and 18. F = 0 33% 54% 56% 52% N/A S = 0 60% 42% 38% 44% N/A 1.2: Proportion of training graduates reporting significant improvements or some improvements in the situation regarding their operational or training area in their unit/ministry F = 0 15% 23% 28% 26% N/A PMP, Quarterly Report 17 and 18. S = 0 58% 56% 53% 51% N/A 1.3: Number of trained trainers 0 188 416 317 272 106 PMP, Quarterly report 17, 18 and Tatweer Training Database. 1.4: Number of civil servants trained 0 3,079 26,532 52,133 20,663 3,256 PMP, Quarterly report 17, 18 and Tatweer Training Database. 62 Tatweer National Capacity Development Program Final Evaluation 1.5: Number of anti-corruption agencies staff trained - 38 1,257 967 779 41 PMP, Quarterly report 17, 18 and Tatweer Training Database. 1.6: Number of civil servants (central and provincial) trained by their ministry trainers in public administration areas - 376 15,753 31,034 6,6861 749 PMP, Quarterly report 17, 18 and Tatweer Training Database. 1.7: Number of post-graduate scholarship recipients enrolled and studying in post-graduate university programs - - 81 120 58 108 PMP, Quarterly Report 17 and 18. 1.8: Number of post-graduate scholarship recipients that complete courses of study - - 1 24 50 101 PMP, Quarterly Report 17 and 18. 1.9: Number of SSN staff trained in new information technology business process SSN=0 IT=0 SSN=0 IT=0 SSN=0 IT=4 SSN=0 IT=5 SSN=6 IT=81 SSN=0 IT=0 Training Database. PROGRAM RESULT 2: MINISTRIES ESTABLISH MORE EFFECTIVE ADMINISTRATIVE SYSTEMS Program Indicator Documented Results (By Year) References 2.1: Number of units/departments implementing USAID/Tatweer recommended changes in administrative systems or procedures 2006 2007 2008 2009 2010 2011 PMP, Quarterly Report 17 and 18. - 7 45 145 174 17 63 2.2: Number of anti-corruption agency offices implementing USAID/Tatweer recommended changes in administrative systems or procedures - - - 3 16 0 PMP, Quarterly Report 17 and 18. 2.3: Proportion of priority systems improvements in human resource management implemented by Ministries (central and provincial) - 0 44% 60% 50% T Performance Management Plan. 2.4: Proportion of priority systems improvements in fiscal management implemented by Ministries (central and provincial) - 0 37% 55% 50% T Performance Management Plan. 2.5: Proportion of priority systems improvements in information technology management implemented by Ministries (central and provincial) - 0 40% 60% 50% T Performance Management Plan. 2.6: Proportion of priority systems improvements in project management implemented by Ministries (central and provincial) - 0 39% 65% 50% T Performance Management Plan. 2.7: Proportion of priority systems improvements in leadership and communication implemented by Ministries (central and provincial) - 0 38% 65% 50% T Performance Management Plan. 64 Tatweer National Capacity Development Program Final Evaluation 2.8: Number of MOLSA sites with installed and functioning SSN applications (central and provincial) - - - 0 2 4 PMP, Quarterly Report 17 and 18. 2.9: Number of units/departments in ministries and executive offices with institutionalized self￾assessment process 0 2 7 13 8 N/A PMP, Quarterly Report 17 and 18. 2.10: Value of GOI financial contributions to USAID/Tatweer systems improvement initiatives - - - $32,231, 640 TBD PMP, Quarterly Report 17 and 18. 2.11: Number of workshops and focus groups that address planning and implementation of improved systems and procedures - - 210 921 1,139 (569 FG, 570 WS) 237 (122 FG, 115 WS) PMP, Quarterly Report 17 and 18. 2.12: Federal Civil Service Commission law enacted - - Bench mark 1 achieved Bench mark 2 achieved Bench mark 3 achieved Performance Management Plan. 2.13: Federal Civil Service Commission institutionalized - - 0 Bench mark 1 not achieved Performance Management Plan. 65 2.14: New Civil Service Law enacted - - - Bench mark 1 achieved Performance Management Plan. 2.15: Number of beneficiaries supported through SSN applications (central and provincial) - - - 0 850,000 40,000 PMP, Quarterly Report1 17, 18 and Annual Report 2010. 2.16: Number of ministries issuing administrative orders to restructure human resource function - - - 2 26 35 PMP, Quarterly Report 17 and 18. PROGRAM RESULT 3: PUBLIC ADMINISTRATION TRAINING CAPACITY EXPANDED Program Indicator Documented Results (By Year) References 3.1: Training and other databases developed at NCCMD 2006 2007 2008 2009 2010 2011 PMP, Quarterly Report 17 and 18. - - - 2 26 35 3.2: Number of trainees enrolled at NCCMD 1000 1,239 1,400 547 20 0 PMP, Quarterly report 17, 18 and Tatweer Training Data Base. 3.3: Number of trained trainers at NCCMD - 21 61 182 0 N/A PMP, Quarterly Report 17 and 18. 3.4: Number of NCCMD and Al Quds school staff certified as master trainers in different public administration fields by international accredited - - 0 0 0 Performance Management Plan. 66 Tatweer National Capacity Development Program Final Evaluation organizations 3.5: NCCMD curricula (in the 5 key areas) validated by NAPA as meeting international service standards - 0 0 0 T 10 Performance Management Plan. 3.6: Number of trainees enrolled at regional training centers - 1,616 15,259 28,71 0 10,299 1,293 PMP, Quarterly report 17, 18 and Tatweer Training Data Base. 3.7: Number of trained trainers at regional training centers - 92 256 135 64 0 PMP, Quarterly report 17, 18 and Tatweer Training Data Base. 3.8: Number of ministries that have established a public administration training program - 3 5 4 0 N/A PMP, Quarterly Report 17 and 18. 3.9: Number of anti-corruption agency staff trained by ministry training centers - - - 138 610 23 PMP, Quarterly Report 17 and 18. 67 ANNEX 8B: TATWEER DATABASE REVIEW Tatweer Database Review: Training Modules Training Module Training Module Training Module 1 Cash Management 2 Financial Management and Reporting 3 Governmental Internal Controls and External Audit 4 FMIS Introduction, Budgeting – Preparation 5 Execution, Controlling and Reporting 6 Government Procurement 7 Fundamentals of Project Management 8 Communications 1 9 Communications 2 10 Project Management Case Studies 11 Business English 12 Correspondence Writing 13 Business English: report writing 14 Tatweer Organizational Self￾Assessment and Transformation Program 15 Strategic Planning, Job Analysis 16 Job Descriptions 17 Specifications and Standards 18 Recruiting, Job Search, and Selection and Appraising Performance 19 TOEFL 20 Self Assessment 21 Anti-Corruption 22 Practicum in Capital Budget Forms 23 ISO 9000 24 Invest and Portfolio Management 25 Anti-Corruption Inspector General Workshop 26 Governorate Communication 27 Change Management 28 ICDL 29 Human Resources Management 30 VISIO, Contract and Project 68 Tatweer National Capacity Development Program Final Evaluation Software Management 31 Vender Supply 32 Rights and Duties of employees and The attachments laws 33 A+ 34 Business English -Special Purposes (ESP) 35 Cisco CCNA 36 Crimes in violation of public employment duties 37 Statistical Management 38 Awareness Raising of Inspector General Offices 39 The Manners and Means of Management 40 Governorate Planning Units 41 Epi Info 42 Primavera 43 Comp TIA CTT+ 44 Feasibility Studies and Evaluation of Project 45 M&E 46 Application Development 47 Participatory Learning And Action 48 Project Management Workshop 49 Integrated Supply Management 50 Comp bell Weather Stations 51 Project Design 52 Governmental Accounting 53 Applied Statistics For Evidence Based 54 GIS 55 General, International Donor Assistance 56 Team Building 57 Organizational Design 58 Capital Investment Project 59 Primavera Awareness 60 Health Information 61 Inventory control System 62 Safety In Power generation and Supply 63 IT Basic Training 64 System Analysis and Design 65 Mechanism for Monitoring Human Rights violations 66 Health Planning Project Management 69 67 Project Execution 68 COMFAR III 69 SharePoint Fundamentals 70 Auto CAD 71 Budget Preparation 72 Training Management System 73 How to use Contractor Classification Application 74 Evaluation Process 75 Process Analyses 76 Taji Irrigation Canal 42 Project 77 Oracle Database 78 Accounting and Auditing 79 Drilling Service Technology 80 Preparing the economic plans 81 Standard Operating Procedures Instructions 82 Chemical Treatment of Oil 83 HSEMS 84 Civil Service Awareness Raising 85 Visual Basic .NET 86 Circulating of Human Rights Principles 87 Web Site Fast-Track Training 88 IT for Public Cycle Managers 89 Performance Evaluation 90 ISCM, Auditing and Internal (External Control) 91 Administrative Anti-Corruption 92 Financial Accounting 93 Methods of Administration and its relationship with administer sociology 94 Database Introductory 95 E-Learning Workshop 96 Vouchers cycle and financial reporting and Position 97 Health Economic 98 Proactive Safety Process 99 Quality Assurance, Accounting 100 Safety Level II for ME 101 Computerized Maintenance Management 102 Network Design and Management 103 Management and Maintenance the 104 CISCO Network Fundamentals 105 MS Access Training 70 Tatweer National Capacity Development Program Final Evaluation Environment of Windows Server 2003 106 Cost Estimating 107 Developing Employee Data Bank 108 Unified Accounting System 109 Contracts/ Letters of Credit 110 Interpersonal Communication 111 IT Strategic Planning 112 Project Management Practical Workshop 113 Inventory control System 114 Businesses Process Analysis Workshop 115 Job Description and Performance Management 116 Software Development and Management Principles 117 Ethics and Transparency 118 Contracts Management/ Administration 119 Accounting and Auditing 120 Microsoft project for software development 121 Health Strategic Planning 122 GPR 123 Training management system, Auditing Control 124 SQL Server 125 Knowledge Base on governmental Investments program 126 Principles of Fiscal Management 127 Media and Communication 128 Civil Service Act 129 IT Mentorship with IT center at HG 130 Coding and Performance Indicators 131 Governmental Financial Statistics 132 Protocol and Donor Communication 133 Civil Service law workshop 134 HRM process design for health 135 Project Technical Assistance 136 Infrastructure Assessment 137 Training Analyses 138 Contracts for the procurement of goods and services 139 New Project Manager 140 Cost Analysis and Cost Center 141 TOT Safety Training 142 SQL Database Design and 143 CMMS and KKS 144 Health care and GPS data modeling 71 Implementation Part 1 workshop 145 Knowledge base Methodology for HP 146 COMFAR Awareness 147 proposal writing using logical frame work 148 Inspection Training for MOO and ME 149 SSN Application Deploying Administration 150 Understanding HTML and CSS Part2 151 IT Management 152 Technical Report Writing 153 Potential DG Training Policy 154 Windows Server 2003 network infrastructure implementation 155 International Communication and Protocol 156 IT Management Competency 157 Implementing 158 Managing and Maintaining SSN work 159 Training and Evaluation course 160 Fourth Data Entry Training Course 161 Media and Monitoring 162 Programming Windows using C# (Academy R1) 163 GIS Database Concepts 164 Budget Basic 165 Planning and Implementing Win 2003 Active directory infrastructure 166 Government Financial Statistic 167 Special budget mentoring workshop 168 Fundamentals of Computer 169 Information Security Training 170 Employees performance management 171 Human Resources- Appraisal 172 Arc GIS advanced analysis 173 GAD Application 174 Project management mentoring workshop 175 Construction Management Workshop 176 Manpower planning 177 Management Analytic 72 Tatweer National Capacity Development Program Final Evaluation 178 Budget follow up 179 SQL Server Fundamentals (Academy R2) 180 COMFAR III TOT Coaching Workshop 181 Process Analysis and Procedure Development 182 Problem Solving and Decision Making 183 Capital Planning 184 Joint Venture Accounting 185 Performance Appraisal 186 Fundamentals of Remote Sensing 187 Fundamental word processing 188 Website Master and ICDL 189 Information and Communication Technology Alignment 190 Conden 191 Strategic Orientation for the procurement Process 192 CCA Workshop 193 Budget and Fiscal Management for PMC 194 Public Manager Cycle 195 Modern Engineering Workshop 196 Technical Assistance with the provinces 197 Project Follow up, Curriculum Development 198 HR Management for PMC 199 Feasibility Studies for health projects 200 Communication (Monitoring) 201 Project Management for Rebuilding 202 Fuel Supply Monitoring Training 203 Knowledge Awareness Workshop 204 Budget monitoring Process 205 Agricultural Policy 206 Database Installation 207 Master Trainer 73 ANNEX 9: TATWEER INTERVIEW RESPONSES (Removed – For Internal Use Only) 74 Tatweer National Capacity Development Program Final Evaluation ANNEX 10: HILLA GROUP INTERVIEW (Removed – For Internal Use Only) 75 ANNEX 11A: BUDGET EXECUTION The Tatweer Results Framework (see Annex 1) is based upon results and indicators that support the USAID Strategic Objective 10: Capacity of National Government Institutions Improved and the Intermediate Result 10.1: National Capacity Development – assisted ministries demonstrating suitable performance improvements in core administrative functions. The Tatweer PMP indicator for IR10.1. is: ‘Percentage of approved GOI annual budget spent by NCD assisted ministries.’ Following the PMP Review of 2010 an additional sub-indicator was added to the existing sub-indicators to ‘respond not only to the OIG report findings, but also to be consistent with several prior General Accounting Office (GAO) reports on capacity development that recommend measurement of budget execution as a proxy indicator for capacity development at the national level. The new sub-indicator is ‘Percentage change of capital budgets executed by selected Ministries in targeted provinces.’ Information relating to any budget expenditure is not forthcoming for public consumption by Iraqi ministries nor easily evidenced during meetings and interviews. Hence the table below documents the FETI team’s ‘evidence’ extracted from Tatweer reports and interviews with Tatweer personnel in order to attempt to assess Tatweer’s Results Achieved or Not Achieved in terms of Capital Budget Execution. The ratio of Iraq’s Capital Budget as a percentage of the national budget is shown in the table below (Source: Special Inspector General for Iraq Reconstruction, Quarterly and Semi Annual Report to the US Congress – April 5th 2010.) The actual Budget execution percentages of the Capital Investment Budget are uncorroborated as the figures are not made public in Iraq but information from the MOP (December 2010) shows that Capital Investment Budget execution increased from 71.4 percent in 2006 to 94.0 percent in 2009, the most recent year for which figures are available available. .Year Iraq Capital Investment Budget as a Percentage of the National Budget 2006 18.2% 2007 24.5% 2008 34.7% 2009 23.5% 2010 28.0% 76 Tatweer National Capacity Development Program Final Evaluation Ministry Baseline Survey 2007 2008 2009 PMP Review 2010 Government of Iraq Budget reporting and preparation starts at directorate level in each province, is approved by governorate councils and forwarded to Ministry HQs for approval / inclusion in the final budget to be submitted to MOP. Some procedural steps have been proposed to reduce time for approval. Policy changes and modifications have to be introduced to reduce time for approval and to prepare project selection criteria The key area of concern since 2006 has been budget execution - the lack of GOI capability to spend its budget, especially its capital investment budget. The first concern is to increase the percentage of the capital investments budget as part of the total budget. This percentage has increased gradually from 18.2 percent (2006), to 24.5 percent (2007), to 34.7 percent (2008). There was a drop in 2009 to 23.5 percent due to the fall in world oil prices. This has been adjusted upwards in the 2010 submitted budget to 32 percent. The IMF recommends that Iraq increase its capital investment budget to 40 percent of the total budget - the GOI is determined to move toward that benchmark. Budget execution increased significantly since the rate of 28.2 percent in 2006. The rate reached 68 percent in 2008 and, as of August 31, has already reached 52.9 percent in 2009, with one-third of the year remaining. But key spending ministries such as Oil, Electricity, and Water Resources still had execution rates below 30 percent in 2008. Tatweer is working to help them improve this in 2009. (Annual Report 2009) Iraq’s ability to spend its overall budget has improved each year but capital expenditures remain under-budgeted and under-spent. With Tatweer assistance, the GOI has been able to spend more of its capital budget, but these vital investments still represent a small percentage of the total. Tatweer has helped build the foundation for excellent budget execution through increasing GOI staff capacity and enhancing requisition systems. These activities will continue in 2010 in order to reach the goal of a GOI budget execution process that is productive as well as sustainable. (Annual Report 2009) Priority Improvements contained in 2008 Work Plans (Year 2): •Strengthen Linkages Between Budget Execution Rates and Program Completion Rates. •Develop a Guide to all Budget Execution Rates Reporting Utilized by MOP, MOF, GOI, USG, World Bank, and others in an effort to harmonize the Reporting Language and Jargon and Increase Understanding of Actual Spending and Associated Completion of Capital and During Tatweer’s partnership with the MOP and other line ministries, the GOI budget execution rose from 43 percent to 86 percent. From 2007 to 2009, Tatweer-affiliated ministries averaged approx. 27 percent improvement in budget execution rate, while the figure for other ministries overall during the same period is approximately 17 percent. (Annual Report 2009 to 2010) The adoption of the new CIP for 2010 will help implement system￾wide reforms in how the GOI funds projects - from its current year- 77 Operational Budgets (PMP Page#7) to-year funding to a multi-year cycle. These reforms will change current business practices for all ministries and provincial councils. Tatweer provides assistance to the ministries to strengthen periodic performance reports, particularly with regard to budget execution, consistency, and adherence to set timelines (Annual Report 2009) Deputy Prime Minister’s Office GOI has no Training center as such and no organized training in Fiscal Management. Very cumbersome budget execution workflows. Tatweer established an active role with the DPMO through regularly scheduled meetings with the Chief of Staff, providing assistance, reviewing DPM draft policy and coordination papers, preparing a draft Handbook on Government Procurement Procedures, attending the GOI conference on Budget Execution, and carrying out IT assessments of the DPM in preparation for their move to a new facility. ( 3rd Quarterly report 2007) As part of the provincial rollout, Tatweer team leaders went to their counterparts’ provinces to understand the challenges that provincial DG’s face with budget execution. In Babil Province the Governor and Tatweer jointly sponsored a conference to form a Planning Commission in the province to be responsible for coordinating the capital budget execution of the Ministries operating in Babil. Tatweer is assisting the Governor’s office in its efforts to follow best practices in coordinating ministerial efforts. (Annual Report 2009) 78 Tatweer National Capacity Development Program Final Evaluation Ministry of Planning The project led 23 senior officials of COMSEC, MOP and several key economic ministries to a week-long Executive Strategic Planning course in Dubai. Course feedback indicated high impact on the management approach of these key leaders. Tatweer provided impact meetings and quick training courses to address bottlenecks in the process of capital project approval and the progress review. These were delaying the release of funds, and so were slowing down budget preparation and budget execution across the government. The Tatweer MOP advisor drafted new guidelines for capital goods procurement used to accelerate large steel pipe procurement for the Ministry of Oil. ( Tatweer Annual report 2006-2007) The Tatweer IT Services Group provides internal IT support to the project and to the important IT components of the ministry engagement program. This will be important in light of the upgrading of ministry IT resources and the introduction of business process mapping in order to facilitate the development of procurement and project/contract management systems which will aid the ministry in improving budget executions. (Quarterly Report Oct￾Dec 2007) Tatweer: - Provided technical support to the MOPDC to adopt a five-year program for the Government’s Capital Investment Program (CIP). - Assisted the Ministry in updating its Feasibility Studies instruction manual. - Recommended the introduction of a recently adopted law that mandates the incorporation of international standards when submitting feasibility studies for all strategic projects using the COMFAR III software. - Provides assistance to the ministries to strengthen periodic performance reports, particularly with regard to budget execution, consistency, and adherence to set timelines. (Annual Report 2008-2009) In 2009, Tatweer conducted extensive training to help the GOI move forward with substantive capital investment projects. Our support for the COMFAR III approach provides senior Iraqi project managers and project finance decision makers with critical tools to conceive, award and implement key projects. Tatweer also revised the GOI’s project proposal process, provided cross-ministry training programs and a series of ministry specific presentations on appropriate methods to make use of the GOI’s By 2008, the regional planning capacity of the MOP had severely diminished due to sparse communications and a shortage of experienced staff. Tatweer engaged with the PPUs to bolster planning capacity and coordination with the MOP HQ. The approach included training such as project management and feasibility studies, and technical assistance and support. E.g. dissemination of information on the regulations of Iraq’s capital investment process to each PPU, provided IT equipment to PPU offices to strengthen activities and communications with MOP. Impact: contributing to improved provincial budget execution, accelerated centralization of basic services, a more consultative planning process between provincial councils and the MOP. In Wasit province the governor mandated that all Capital Investment activities be undertaken only with PPU consultation. (Annual Report 2009 to 2010) The Iraq Development Management System (IDMS) is a high-impact system to enhance Iraq’s budget execution. The IDMS software developed by Tatweer, the UNDP, and the Ministry of Planning to automate the capital investment project cycle and manage Iraq’s 79 project proposal forms. The GOI’s use of an annual plan for capital spending created complications when attempting to implement multi -year projects. Within the CIP, Tatweer helped the MOPDC introduce a mandatory approach to capital investment project preparation by mandating and automating international standards in project feasibility analysis. There are already indicators that the CIP is being embraced by the ministries. E.g. the MOH Project Management Working Group discussed the ramifications of the MOPDC’s Capital Investment Budget Guidelines and subsequently ordered feasibility studies for the Rasafa and Karkh Departments of Health, and the review and update of the studies prior to submission to the MOPDC. (Annual Report 2008 -2009) The Tatweer team and MOPDC have made progress in reforming the MOPDC capital investment program and expediting budget execution. After the success of the pilot project implementing single copies of COMFAR III software at seven ministries, 200 additional copies were ordered for distribution. The MOWR started using the software and reported that they had rejected two feasibility studies prepared by an outside consultant after they compared the consultant reports with the COMFAR III analysis. Tatweer and MOPDC have finalized debugging and testing of the Contractors Classification Program capital investment portfolio. Fully functional in Arabic, as well as English, the IDMS is installed on the MOP HQ’s servers. The IDMS was mandated by the MOP as the software to be used in managing capital investment projects in the 2010 -2014 National Development Plan. (Annual report 2009 -2010) Tatweer Advisor to assist MOP in developing a framework to implement NDP capital investment project portfolio (Quarterly report 18: Dec 2010) 80 Tatweer National Capacity Development Program Final Evaluation which includes registered and black-listed contractors. Tatweer trained MOPDC staff to use the program, which will be an important anti-corruption tool as well as a major performance improvement. The program is designed to register contractors and give them the facility to submit applications online while allowing ministries to verify authenticity of contractor certificates online. The program will have a positive effect on budget execution by reducing the length of time needed for contractor verification. (Quarterly report 10 Oct-Dec 2008) Energy Group: Ministry of Oil and Ministry of Electricity Tatweer rapidly staffed and is managing a group of experts focused on budget execution and operational efficiencies in these key ministries. The program is delivering assistance in the critical areas of project management, technology and procurement for the Iraqi electricity, petroleum, and gas industries. These experts assist their Iraqi counterparts to develop the skill sets and tools needed to rebuild the Iraqi energy sector, from best practice in tendering for plants and services to the project management required to build multi-billion dollar energy infrastructure like refineries and power plants. Priority Improvements contained in Tatweer 2008 Work Plans (Year 2): Institutionalize changes fitting Budget Execution Module (BEM) (PMP page 9) 81 (Quarterly Report 11 Jan-March 2009) Ministry of Health The MOH created a new accounting unit within the engineering section of the Medical City (part of the HQ in Baghdad) as part of its system reform assisted by Tatweer through assistance to the Fiscal Management Working Group. This process strengthens the link between budget execution rates and project completion in the capital budget process. One of the problems of budget execution is the late start of the process. From early 2007, Tatweer budget training emphasized the importance of preparing budgets within a medium term cycle of three to five years, part of a longer term strategic planning process which takes into consideration sector growth. In recent workshops Tatweer discussed budget preparation for 2009 simultaneously with 2010, within the five￾year plan, part of a strategic framework of multiple five-year cycles. The Minister of Health immediately supported the process and encouraged his leadership team to plan for an early beginning of the 2010 budget preparation cycle. (Annual Report 2007-2008) The first quarter saw the development of working groups and committees focusing on immediate returns, primarily in the area of capital budget execution. MOH employees and Tatweer advisors formed the Fiscal Management Working Group. The FMWG addresses the basic prerequisites required for sound capital budget preparation including improvements in budget preparation processes, execution, procurement, opening letters of credit and project execution. A procurement unit is being introduced to increase the focus on the procurement as a critical link to budget performance and service delivery. (2nd Quarterly Report 2008) Priority Improvements contained in Tatweer 2010 Work Plans (Year 4): Identifying budget execution hurdles (PMP page #24) Ministry of Municipalities and Public Works MMPW engineers and Tatweer advisors are working hand in hand, instituting end-to-end management systems for the full project cycle for major construction projects in seven provinces as the model for improving budget execution in this ministry, which leads the way in budget execution for the entire government. (Annual report 2007- 2008) MMPW is focusing its capacity development on its Provincial Project Management Teams in Baghdad, Babil, Najaf, Anbar, Kirkuk, and Basra, with Tatweer providing coaching and workshops on project management cycle. MMPW has adopted Primavera as its official project management tool, and its provincial units are Priority Improvements Contained in Tatweer 2009 Work Plans (Year 3): Assist MMPW to implement selected Projects in HQs, Baghdad, Babil, Najaf, Anbar, and Basra Provinces (budget execution and practical procurement exercises) (PMP Page #12) Priority Improvements Contained in Tatweer 82 Tatweer National Capacity Development Program Final Evaluation developing Standard Operating Procedures for every process in the project life cycle. Accelerated budget execution will lead to a more effective delivery of core services to Iraqis. (Annual Report 2009) 2009 Work Plans (Year 3): Conduct central￾province level business process mapping of communication methods, budget execution, procurement and reporting (PMP page #17) Ministry of Agriculture Budget execution bottlenecks and solutions were identified at meetings with the MOA and the MOPDC. Outcomes included identification of where Tatweer can provide targeted technical assistance, including workshops on capital budget procedures and feasibility studies. A chart outlining 2009 capital budget procedures, bottlenecks and solutions was created by Tatweer and is being utilized by the ministry. During the first quarter Tatweer embedded a fiscal management expert in the MOA to develop a new asset inventory system. The system will comply with the GOI’s Supreme Audit Board classification and will be rolled out with the help of Tatweer’s IT team. (2nd Quarterly Report 2008) Ministry of Water Resources Tatweer assisted MOWR’s Fiscal Management Working Group to reduce capital budget preparation time. This included facilitating a FMWG meeting with the DGs of Finance and Administration. Advisors and Working Group members explored internal steps to be taken to improve capital budget execution and identified external budget execution bottlenecks such as lack of inter-ministerial communication. (2nd Quarterly Report 2008) 83 ANNEX 11B: TATWEER DOCUMENT REVIEW: INFORMATION TECHNOLOGY (IT) EXPENDITURE Tatweer Document Review (TDR): Information Technology (IT) Expenditure Ministry/ Entity IT Improvements Descriptions 1 MOP MOP advanced decentralization by opening 15 provincial planning units. MOP shared the cost of providing them with IT equipment. (Annual Report 2007) Tatweer qualified several IT specialists from the Al Quds in information technology skill sets like Oracle. These new skills allow Al Quds to conduct courses, seminars, talks, and workshops on current technologies that will benefit the entire public sector. ( Annual Report 2008) Four computer applications for COSIT were redesigned using Oracle and statistical reports about crop harvesting, car accidents and parallel education were published for the first time in four years. Tatweer helped develop a well structured Master Statistical Plan that follows WB guidelines outlining all surveys and census for the next five years. ( Annual Report 2009) Tatweer qualified IT specialists from Al Quds in information technology. New skills allow Al Quds to conduct courses, seminars, talks, and workshops on current technologies. In terms of institutional strengthening Al Quds has completed a new operations manual including internal policies and procedures, job descriptions, and process maps. ( Annual report 2009) According to the WB, the (COSIT) this year improved its national capacity ranking from a score of 28 to 47 on a scale of 100. Tatweer helped develop COSIT’s capacity to generate timely and vital economic statistics through (1)providing a significant investment in IT equipment, (2) developing a software- training system in which COSIT staff learned to write current applications using Oracle, and (3) providing a six-week course in Oracle during which COSIT participants used their own computers and applications, allowing a seamless transfer of skills back to COSIT. ( Annual Report 2009) 84 Tatweer National Capacity Development Program Final Evaluation Tatweer’s COP, DCP, and members of the media were in attendance for the opening ceremony of newly-modernized Central Organization for Standardization and Quality Control (COSQC) buildings. The project focuses on the integration of new physical infrastructure and the systems to make best use of them, such as the IT modernization of COSQC, which paired a complete hardware and software outfitting with a revised IT department structure and set of SOPs. ( Annual Report 2009) The installation of a new archive system in the MOP is another Tatweer accomplishment. The system facilitates the electronic storage and retrieval of all documents entering and leaving the MOP, streamlining MOP operations and reducing bureaucratic delays. Additionally, the Tatweer and the MOP implemented a CCA System which is assisting the procurement of goods and services. The system has been steadily populated with contractor information and is an invaluable tool in the GOI’s contractor and project management ( Annual Report 2009) • Establish Assets Tracking Database 2008 • Improve website usage and documents/forms access for the Ministry and NCCMD 2008 • Establish online software library for NCCMD 2008 • Foster project management software licensure and usage • Assist ministry to develop and upgrade web site 2009 for the ministry and COSQ • Conduct competency and advanced level IT training courses 2009 • Conduct Phase 2 activities of Contractors' Classification Management Information System development 2009 • Conduct specialized IT training courses (ASP.Net; CCNA; Dreamweaver; ICDL; MCSI; GIS; Oracle; etc.) 2009 • Create and enable 300 email addresses for MOP staff • Develop a Training Information Management System (TIMS) • Develop e-government strategy (Phase 2) • Develop Government Assistance Database (GAD) • Donor and Government Assistance Projects Follow Up System ( PMP) By 2008, the regional planning capacity of the MOP had severely diminished due to sparse communications with the provincial offices and a shortage of experienced staff. Tatweer seized the opportunity to assist by engaging with the ministry’s Provincial Planning Units (PPU) through an integrated approach aimed at bolstering planning capacity and coordination with the MOP HQ. This approach included specialized training in fields such as project management and feasibility studies, which were in turn complemented by technical assistance and support. An example of the former is Tatweer’s dissemination of a comprehensive package of information regarding the regulations of 85 Iraq’s capital investment process to each PPU. In terms of support, Tatweer has provided an infusion of modern IT equipment to PPU offices to assist them in their activities as well as strengthen their communications linkages with the MOP in Baghdad. This was made crystal clear in Wasit province when the governor mandated that all capital investment activities be undertaken only with PPU consultation. Each of Tatweer’s activities has multiple outputs affecting both people and policy, and together these outputs form a foundation from which Iraq’s regional planning capacity will flourish. ( Annual Report 2010) 2 KRG New training center furnished by Tatweer support. The facility also has a video conferencing system with global connectivity.( Annual Report 2009) Tatweer has promoted a principal of standard practices across all ministries. For example, the adoption of the COMFAR III feasibility study software by key ministries represents a system improvement that facilitates ministerial uniformity in support of improving budget execution. This year, Tatweer received approval by several ministries and COSIT on the design for a National Employee Database, which integrates and merges the HR data of 21 out of 37 ministries. The completion of several major procurements, including the roll -out of (VTC equipment to 15 ministries in 5 cities, as well as the installation of IT infrastructure in the MOPDC/COSIT, highlight another dimension of improved integration among ministries. ( Annual Report 2009) Business continuity and disaster discovery plans (DRP) SSN; KRG-SSN Coordination of the activities of the internet services providers (ISP), the software vendor and local support partner SSN; KRG-SSN Finalization of the SQL application SSN; KRG-SSN Migration of legacy data into the new system KRG-SSN Method for auditing beneficiary applications and entering the records into Visual FoxPro/ MS Access or directly into SQL, SSN; KRG-SSN Medium-term plan for IT resources (hardware and software) necessary to support the SSN system SSN; KRG-SSN 2010 Inventory Management System (IMS) KRG-MOP (PMP) Despite the short time frame, Tatweer successfully implemented a number of activities and systems now in use at MOP-KRG, including: a comprehensive IT Assessment enhanced network design, the CCA, modified TMS, software, the Inventory Management System, and the IDMS. Tatweer has provided training in Erbil for MOP-KRG leadership and staff in the operation of these new systems and software. (Annual Report 2010) 3 MOLSA Despite some uncertainty regarding the law mandating the disassociation of the Social Affairs Directorates from MOLSA (which was later repealed), the SSN package was completed, approved by MOLSA leadership, and is now being deployed at the MOLSA headquarters and Al-Sadr SSN sites. Tatweer is working closely with MOLSA to ensure that data is being entered and migrated correctly into the SSN application, and is providing extensive training for over 30 MOLSA IT staff to prepare them to manage and maintain the SSN application. 86 Tatweer National Capacity Development Program Final Evaluation The USAID/Tatweer SSN team continued the development of the Kurdish language version of the SSN application. The MOLSA in the KRG successfully completed their initial data entry program and have 40,461 records entered into a Microsoft Visual FoxPro database ready to load into the SSN application once deployed. The Tatweer team has developed the capability of transferring the benefit claimant’s records from the Visual FoxPro database into the SSN’s SQL database. Deployment is planned to take place in the first half of 2011. ( Annual Report 2010) 4 MOH Helped MOH with wiring to improve connectivity and email functions. Assistance in construction and completion of the network in the Ministry video teleconference available but not working. Working to help the ministry activate the Video Conferencing system. (Baseline Survey) • Assist ministry to establish IT Training Centers in two sites • Assist ministry to restructure IT Department • Provide technical assistance for the development of an IT Strategic Plan2009 • Provide technical assistance towards improvement of MOH Web Site • Review Kimadia Inventory Management System (IMS) and propose recommendations for improvement • IT management training MOH ( PMP Report) At the request of Maysan’s DOH, Tatweer provided training, technical assistance, and GPS tools to upgrade its system and entered into a cost-sharing agreement to procure and install state-of-the-art GIS computer hardware and software at three key healthcare facilities: DOH HQ, and Al-Sadr and Al-Zahrawe Hospitals. This allows for the secure storage of health data collected through Maysan’s HVP ( Health Visitor Program) and at its 41 primary and 26 secondary health centers. Maysan’s HVP saw DOH information technology (IT) specialists pair GPS technology with related USAID/Tatweer assistance and training, mapping the governorate into small units containing no more than 1,000 homes. Consistent with HVP’s mission to provide home health care to Iraqis who would not otherwise receive medical attention, teams of 5-10 health care professionals and local volunteers were then assigned to these small units, provided with vehicles, and tasked with mapping the precise location of each house. In Maysan’s 14 marshland units, where it can take up to three days to see a doctor, HVP teams were provided with small wooden boats. Once installed, GIS servers will allow instant and direct communication between Maysan’s three main health care facilities and Baghdad’s MOH. Protected and transparent, the system will enable storage and analysis of critical health data that will augment health-related decision making and support critical disease remediation efforts by tracking vaccinations, infant birth and mortality rates, illnesses such as diabetes and cancers, and epidemic outbreaks such as avian flu, cholera, and tuberculosis. Resulting health reports can be transmitted instantaneously to the MOH central headquarters in Baghdad for further analysis and review. Future expansion of GIS will establish a modern data center by upgrading MOH network connections and 87 providing digital phone service and protected official government e-mail accounts. At present, government officials use personal cell phones and email addresses to communicate. Communication that once took more than a week will occur immediately. ( Annual Report 2010) 5 Presidency Diwan Award made and work begun to equip the Presidency Diwan Public Policy Unit with various office and IT equipment items( Annual Report 2010) Installation of a language lab for the Presidency Diwan ( Annual Report 2010) 6 MOHR IT staff involved in training. Over 20 percent of identified employees have been trained. ( Baseline survey) The Ministry of Human Rights (MOHR) opened the doors to its newly constructed IT Training Center in another well-publicized ceremony. ( Annual Report 2009) Improve IT staff computer skills MOHR Improvement of media department skills MOHR 2010 ( PMP) 7 Salahadin University￾KRG Opining Training Center at Salahadin University, along with the Center at MOP-KRG, is to become the basis of a Center of Excellence in Public Management which will produce high-caliber civil servants for decades to come. ( Annual Report 2009) 8 MODM Looking towards the long-term, USAID/Tatweer works hard on sustainability within the framework of accelerated systems improvement. The USAID/Tatweer IT team, in conjunction with their GOI counterparts, completed the design of an interactive system that will register professionals within the Iraqi diaspora who are planning to return home. Diaspora involvement in post-conflict reconstruction and development is truly essential to real Iraqi ownership of a development process that maintains any forward momentum. This system is one component of a revitalized website for the Ministry of Displaced and Migrants (MODM). ( Annual Report 2009) • Improve website usage and documents/forms access- 2008 • Asses Internally Displaced People (IDP) database system and recommend upgrade 2009 • Assist ministry to develop and upgrade website 2009 • Activating and uploading website and provide training, managerial skills, and technical support to website team for NCCMD 2009 • Conduct IT training needs assessment for web design and security, and implement registration system for Iraqis abroad 2009 (PMP) 88 Tatweer National Capacity Development Program Final Evaluation 9 MOWR In 2009, with newer tools and software, these freshly-trained inter-ministerial teams demonstrated how a Geographic Information System (GIS)-based inventory and conditional assessment of irrigation resources can be utilized in a water-distribution rehabilitation program. MOWR staff under USAID/Tatweer guidance used collected data to estimate costs to rehabilitate the canals, which the MOWR estimates at $6 million, including a pumping station that will operate 24 hours a day. ( Annual Report 2009) ESRI ArcGIS applications and geo-databases MOWR Data communication plans for linkages between the center and provinces MOWR; IT help desk support in the headquarters MOWR IT network infrastructure MOWR, IT management training MOH; MOWR • Establish Assets Tracking Database. Map and create guidelines for existing operations MIS • Assist ministry to develop IT strategic plan to guide future IT implementation 2009 • Assist ministry to establish and/or rehabilitate GIS Center 2009 10 Ninawa Province In Mosul, under a very restrictive security situation, the Tatweer Regional advisor has sent a steady stream of Ninewah civil servants to be trained in Erbil, arranged for several classes in IT to be delivered in Mosul itself. (Annual report 2007) 11 MOED MOED Ministry website , IT Technical Library IT Strategic Development Roadmap, Information and Communication Center Information Security; COMSEC ICDL Centers, Education Management Information System (EMIS), Basic computer use MOED ( PMP) Education Management Information System (EMIS): IT Staff trained. MoED and UNESCO taking over responsibility. ( Quarterly Report 18) 12 MOA • MOA has infrastructure for on line connectivity. • 67 participants have taken training in competency and advanced IT. • Tatweer provided a training center with 11 computers. IT department has used the IT center for an additional 100 trainees. (Baseline Survey) • Implement identified GIS centers 2008 ( PMP- system improvement indicator) Conduct GIS field Surveys in Provinces and present results for decision making 2009. Expansion of Animal Clinic database to include Animal Identification Data MoAGPS/GIS training in ministry and provinces MoA. • MoA GIS unit application of GPS/GIS tools in agricultural in Musayeb GIS project (Babil Province) MoA • Improvement of ICT services for key ministry directorates MoA • Anchor and transfer IT management courses to ministry ToTs MoA • 5-year Information and Communication Technology (ICT) Strategic Plan MoA ( PMP) Five-year Information and Communication Technology (ICT) Strategic Plan ( Annual Report 2010) 89 13 GOI Develop IT systems upgrade plan and procure needed software and hardware ( PMP- system improvement indicator) 14 COMSEC • Establish IT Administrative database systems 2008 • Establish online software library • Conduct IT assessment and design IT management plan 2009 • Conduct IT needs assessment and implement relevant recommendations 2009 • Procure hardware and software for administrative support to match needs for e-learning Intranet 2009 ( PMP- system improvement indicator) 15 MOJ Establish IT Administrative database systems ( PMP- system improvement indicator) AlQuds School Tatweer successfully qualified several IT specialists from the Al Quds in information technology skill sets, like Oracle. These new skills allow Al Quds to conduct courses, seminars, talks, and workshops on current technologies that will benefit the entire public sector. (Annual report 2007) • Training in CTT+, Primavera, Oracle g10 , GIS software, A++, ICDL, webpage design; Update training materials in Cisco, Oracle, MS Office Suite, CIW, MS Visio and SPSS, ( PMP- system improvement indicator) • Activating and upload website and provide training, managerial skills and technical support to website team. • Assist Al Quds in becoming a certified training center in CISCO, Oracle, Microsoft, CIW, Comp TIA, • Assist Al Quds in becoming an ICDL certified training center 2009 Networking and infrastructure improvements: • Improve Internet connectivity bandwidth • LAN improvements such as routers and firewalls • Servers, Exchange, File, Web, Database, Desktop Computers, Admin, Training • Projectors and other teaching aids • Video Teleconference equipment 2009 ( PMP) 16 NCCMD In order to further promote the NCCMD’s ability to assist GOI entities to identify and address reforms, Tatweer has assisted the NCCMD to establish Self-Assessment Teams. Tatweer has trained and coached 20 NCCMD staff divided into two teams to perform this function. These specialists are trained in five main areas: Finance and Budget; Human Resources; IT; Communications and 90 Tatweer National Capacity Development Program Final Evaluation Leadership; and Project Management. (Annual report 2007) • Improve website usage; and documents/forms access for the Ministry and NCCMD 2008 • Collaboration between the NCCMD and Tatweer to follow up on the databases developed during 2008. Technical assistance and necessary training will be provided to the users of the developed databases. Tatweer will continue designing new databases complementary and in harmony with the databases implemented by Tatweer in 2008 such as: 1- consultancy missions,2- civil service , 3- governmental performance, 4- complaints unit, 5- key performance indicators ( PMP- system improvement indicator) • E-Learning applications in training and development unit COMSEC; NCCMD • Documentation manuals for NCCMD IT applications NCCMD 2010 ( PMP) 17 MMPW • Implement identified GIS centers 2008 • Assist ministry to develop and upgrade Web Site 2009 • Provide technical assistance to IT staff in Human Resource Information System • Provide technical assistance to MMPW IT staff in data migration from Oracle 8i to 11g • Development of IT management TOTs MMPW • IT help desk support in the headquarters MMPW; • Data communication plans for linkages between the center and provinces MMPW ( PMP- system improvement indicator) 18 Tatweer Energy • Establish IT unit for service provision, database creation and usage and webpage design and maintenance (2009) • Remote sensing and diagnosis (TE) (2009) • Procure and establish Training Centers (17 in Round 1) (2009) • Design Procure and establish GIS Centers (number TBD) (2009) ( PMP- system improvement indicator) 19 PMO • Advise PMO in establishing an ERP solution for the GOI • Conduct IT needs assessment and implement relevant Recommendations 2009 • Develop an E-Gov Policy Paper/Strategy that can be adopted by all ministries • Develop an Enterprise Architecture Policy Paper as an institutionalized standard of MIS in the GOI 2009 • Introducing ISO for public sector 2009 ( PMP- system improvement indicator) 91 20 COSQC • Develop and implement inventory management system • Develop Patent Information Management System (PMIS) • Develop Standards Management Information System (SMIS) • Establish Local Area Network and Provide necessary hardware and software 2009 • Propose restructuring of the IT Unit • Scan Patents and Standards files • Inventory Management System (IMS) COSQC 2010 (PMP) 21 COSIT • Develop the Employees Data Bank • Develop the Heavy Industries Application • Develop the Parallel Education 2009 (PMP) 22 DPMO • Equip with IT hardware and software, train IT staff, and complete internal fiber-optic wiring • Operate 300 email addresses for COSQC staff (PMP) 23 SSN • Business continuity and disaster recovery plans (DRP) SSN; • Coordination of the activities of the internet services providers (ISP), the software vendor and local support partner SSN; Finalization of the SQL application SSN; • Method for auditing beneficiary applications and entering the records into Visual FoxPro/MS Access or directly into SQL • SSN Medium-term plan for IT resources (HW and SW) necessary to support the SSN system SSN, Maintenance agreement2010 • Conversion of legacy data into the new system SSN 24 Oil Ministry • IT Steering Committee guides the development of an IT system that is a fully compatible and systematic approach for the state oil companies • IT strategic planning process 2010 ( PMP) 92 Tatweer National Capacity Development Program Final Evaluation ANNEX 11C: TATWEER DOCUMENT REVIEW: COST￾SHARING Tatweer Document Review (TDR): Cost-Sharing Ministry/ Entity Cost-Sharing Description 1 MOP SUSTAINABILITY IN TATWEER’S OPERATIONS / GOI EXAMPLES OF COST-SHARING AND COST ASSUMPTION – TATWEER PROGRAM The GOI has begun cost-sharing and cost assumption initiatives for procurements and extending funding for Tatweer-originated concepts. This commitment shows that the GOI has adopted Tatweer’s emphasis on public administration and is prepared to invest its own resources in labor and expenditures to build its core capacity. The USG did not originally expect Tatweer to provide cost-sharing by the GOI as an indicator of success of the program. The project originally included providing grants and equipment purchases for ministries. As the relationship between the ministries and the Tatweer matured, oil revenues increased, and the security situation improved, the ministries’ sense of confidence increased. Ministries targeted to receive Tatweer assistance are now operating on a strong cost sharing basis, particularly for new capital investment. For example, most new IT projects require the client ministry to cover 100 percent of the capital and recurrent costs with Tatweer providing assistance with the development of technical specs and advice on the procurement and management systems. There are many examples of organizations taking Tatweer-initiated concepts and securing funding to sustain implementation of those concepts and activities within their organizations. The GOI plans to continue a range of Tatweer public administration contributions in terms of ministries’ assumption of training delivery, technical assistance, IT hardware and software procurement, and civil works investments. (Annual Report 2008) BUDGET ALLOCATIONS/CASH CONTRIBUTIONS MOP The MOP has allocated $10 million in its 2009 budget for capacity building. $6 million is going towards a scholarship program. This program, modeled on the Tatweer scholarship program, is for graduate studies. Tatweer working with the MOP on a 50/50 cost share to equip new MOP Planning Units in 15 provinces. An illustration of this approach is the joint MOP/Tatweer upgrade of the Ministry’s IT systems. Tatweer will provide $450,000 worth of IT equipment and software, and MOP will spend $450,000 for the related electronic archiving system. Part of MOP’s $30 million rehabilitation of its HQ, it is dedicating an entire floor for training programs. More indication of increasing MOP 93 ownership is its decision to fund all future national and international conferences. Tatweer had shared the costs for two major conferences. (Annual report 2008) The MOP has committed to allocating substantial funding in its 2010 operational budget for capacity building programs to develop the skills and knowledge of MOP staff. A portion of those funds has been earmarked for a specially tailored scholarship program that MOP has begun to implement. This program, modeled on the USAID/ Tatweer scholarships, is for post-graduate studies and is entirely GOI-funded. ( Annual Report 2009) 2 Council of Ministers COMSEC has allocated $1million to capacity development training in its 2008-09 budgets. This funding will be used to expand and internalize the approach taught in Tatweer courses. Tatweer training. COMSEC has approved the creation of a formal Executive Training Dept. to be conducted by Tatweer-trained COMSEC staff and has one floor of HQ dedicated to training. Technical costs of the training center are shared. COMSEC is furnishing hardware, and Tatweer has committed $40,000 for video teleconferencing and about $30,000 for other training equipment. COMSEC’s T&D Unit will be able to integrate the networks of the PM. ( Annual Report 2008) 3 DPMO COMSEC has allocated $1million to capacity development and training in its 2008-09 budgets. This funding will be used to expand and internalize the approach taught in Tatweer courses. Tatweer training. COMSEC has approved the creation of a formal Executive Training Department to be conducted by Tatweer-trained COMSEC staff and has one floor of HQ dedicated to training. Technical costs of the training center are shared. COMSEC is furnishing hardware, and Tatweer has committed $40,000 for video teleconferencing and about $30,000 for other training equipment. COMSEC’s T&D Unit will be able to integrate the networks of the PM. (Annual Report 2008) 4 VPO The VPO has agreed to fund its IT system, based upon on an IT assessment by Tatweer. Procurement of the system will be a collaborative cost-sharing effort, with the VPO contributing the funding (estimated at approximately $250,000) and Tatweer providing in-kind technical assistance services. This commitment to cost-sharing exhibits both the extent of VPO/Tatweer collaboration, and an increasing mutual confidence that technical assistance provided by Tatweer will be internalized and sustained. 5 NCCMD From the outset, Tatweer’s contribution of resources and personnel to the Tatweer/NCCMD training program has declined and NCCMD has assumed an increasing portion of costs. This has enabled Tatweer advisors to concentrate on the design and delivery of advanced level courses which are increasingly being taken over and delivered by NCCMD training staff. 94 Tatweer National Capacity Development Program Final Evaluation NCCMD has added $330,000 to its 2009 operating budget to assume the costs of enhancing the skills of its training staff. NCCMD has contributed over $170,000 to the technical communication and infrastructure costs of expanding its distance learning capabilities and the upgrading of its branches in Basra and Mosul. NCCMD supports the transfer of the training delivery to GOI trainers in the ministries who now deliver all basic level training. Over half of all training sponsored by the Project has now been delivered by GOI trainers who are graduates of the project’s TOT training. The GOI has assumed a responsibility which, had Tatweer utilized locally-hired private training staff, would have cost the USG some $600,000 to date. (Annual Report 2008) 6 MOA In recognition of Tatweer training’s impact on performance, the MOA’s upper leadership included the “Agricultural Strategic Planning Center for Training and Capacity Building” in the 2009 budget and allocated $5.8 million for initial construction costs and $5.1 million as an annual operating budget. Proposals for construction and annual expenditure were jointly developed by MOA and Tatweer teams. This center will become a premiere training facility and is a Tatweer legacy. The MOA committed to expanding on the successes of Tatweer’s OSTP program and is committed to building its internal OSTP capacity development planning process. The Minister has allocated $400,000 for this effort in the 2009 budget. ( Annual report 2008) The MOA’s upper leadership showed their commitment to sustainability by including the “Agricultural Strategic Planning Center for Training and Capacity Building” in the 2009 budget and allocating $5.8 million for initial construction costs and $5.1 million as an annual operating budget. Proposals for construction and annual expenditure were jointly developed by MA and USAID/Tatweer teams. This center will become a premiere GOI training facility and will leave a clear USAID /Tatweer legacy in Iraq. ( Annual Report 2009) 7 MOWR The MOWR and the MOA have been working closely on the South Rashidiya Irrigation Pilot Project. This inter-ministry, multi-organization project brings together different ministries to repair canals critical to agriculture in the South Rashidiya area near Baghdad. Tatweer has successfully worked with counterparts at the MOA and the MOWR to have the SRP included in their 2009 capital budget planning. The MOWR has promised to decide on the budget submission for this activity based on the submission of the project cost analysis presented by the MOWR Rashidiya team; the cost could be as high as $40 million. ( Annual Report 2008) After USAID/Tatweer and the MOWR developed an intensive project management training program for the Najaf, Dhi Qar, and Anbar Water Resources Directorates, each directorate created a team of 25 civil servants directly involved and affiliated with province-specific projects in their respective directorates. This combination of decentralization and embedding of a new capacity in the provinces is a long term improvement in project management. ( Annual Report 2009) 95 8 MOH The MOH, with Tatweer assistance, held a meeting on strategic planning during the conference on National Health Care Reform. The Ministry funded the bulk of the costs associated with this event. The MOH has begun funding travel and training in conjunction with Tatweer, including a recent feasibility studies training workshop in Jordan. The MOH has recently moved10 Tatweer-trained Master Trainers from their previous positions to become the core of an expanded training capacity in their Training Center. The installment of IT hardware is completed in the ministry HQ. The ministry and Tatweer contributions are roughly equal. MOH provided the computers and servers. The ministry now 25 computers (both new and existing) on each of the 10 floors (a total of 250 workstations), and is planning on adding another 250 workstations next year. (Annual Report 2008) The MOH has recently moved 10 USAID/Tatweer-trained trainers from their previous positions to become the permanent core of an expanded internal training capacity in their Training Development Center. ( Annual Report 2009) At the request of Maysan’s DOH, USAID/ Tatweer provided training, technical assistance, and GPS tools to upgrade its system and entered into a cost-sharing agreement to procure and install GIS hardware and software in three healthcare facilities: DOH HQ, Al-Sadr and Al￾Zahrawe Hospitals enabling the secure storage of health data collected through Maysan’s HVP and at its 41 primary and 26 secondary health centers. ( Annual Report 2010) 9 GOI Value of GOI contributions to USAID/ Tatweer systems: $32,231, 640 ( PMP) Value of GOI contributions to USAID/Tatweer systems improvement initiatives for the year 2009 was $32,231,640 96 Tatweer National Capacity Development Program Final Evaluation ANNEX 11D: DONORS PARTNERSHIP Donors Partnership: Year Description 1 PMP USAID Strategic Plan and Program Results The USAID/Tatweer Program is the major USAID contract in support of USAID’s Strategic Objective 10: Capacity of National Government Institutions Improved. In addition to USAID/Tatweer, the Economic Governance (EG) project, implemented under USAID’s SO 8, addresses critical national policy, legal, and regulatory environment issues relevant to the success of SO 10. USAID/Tatweer also coordinates with the Local Government Program (LGP) and the PRTs regarding provincial dimensions of ministry capacity development. Furthermore, there are important USG actors in addition to USAID, namely the Iraq Technical Assistance Organization (ITAO), the Department of Treasury, and the Gulf Regional Department of the U.S. Army Corps of Engineers (GRD), USDA, and DOD initiatives. Other donors, especially DFID, contribute to the accomplishment of SO 10. UNDP is also contributing a broad range of Capacity Development support. 2 2007 COMSEC has established a training unit and allocated a budget of $1 million, Tatweer-trained staff as trainers, assisted in development of training schedule, and other elements (DFID is also involved with COMSEC). In Mosul City, Iraqis Test their Foreign Language Skills The clock started. The Iraqi test takers opened their notebooks to begin the three-hour International Test Program’s TOEFL test. For many of the Iraqis involved, this was a first step towards a chance for a scholarship at a Middle Eastern or western university. For the Ninewa community, this was the first testing event of its kind. Despite the ongoing dangers in war-ravaged Mosul, 60 residents of the Ninewa province came to the test center that day, choosing opportunity over fear. The test offered those interested the chance to qualify for the language skills portion of the USAID-funded Tatweer scholarship. For those who pass the test and continued through the scholarship process, the next step is to compete against fellow Iraqi citizens for a full scholarship to study public administration at an accredited university. 97 “The feedback was very positive from all the students, government officials, local institutions, and NGO representatives,” said Tatweer advisor Atia Moor, who oversaw the test center planning. In order to offer the ITP TOEFL test in Mosul that day, preparations took the combined efforts of the local Ninewa Business Center, Provincial Reconstruction Team, and representatives from USAID, Tatweer, the U.S. military, and AMIDEAST. All worked in close partnership to ensure the safety of the applicants at the test center. Because of the demand for more testing by its residents after the first test event, the local Ninewa Business Center now seeks to administer the test itself. Being the first test of its kind in Mosul, news spread quickly throughout the province and the northern city of Tal Afar also requested to be a test center. (Annual report 2007) 3 2008 MINISTERIAL PARTNERSHIPS WORK TO KEEP WATER FLOWING IN SOUTH RASHIDIYA CANALS In the midst of Iraq’s struggles with crippling drought, the USAID-funded Tatweer program has facilitated a partnership between the MOWR and MOA to help secure a budget allotment to keep the water flowing through the South Rashidiya canals. The South Rashidiya Pilot Project, the first inter-ministry, multi-organization approach with the USAID-funded programs Tatweer and INMA and the local Provincial Reconstruction Team, could be a pilot for other development projects involving Iraqi Ministries and U.S. assistance. The problems with South Rashidiya irrigation systems are not new. Water used to flow freely in the area’s agricultural canals, serving 4,000 hectares. Deterioration of the irrigation infrastructure, combined with drought and shortage of water supply, has left many of the water channels empty. Extraordinary measures are needed to re-establish what was once considered a robust agricultural community of 700 farms. Natik Joudi, who was an engineer for the Ministry of Irrigation in the mid 70s and now works with the Provincial Reconstruction Team, explained that the longtime agricultural region was famous for its grains and orchards and populated by farmers who have handed down their land for generations. The project will preserve the region’s rich agriculture and traditional way of life. Tatweer advisors, in cooperation with their counterparts at the MOWR and the MOA, helped initiate a field survey and conditional assessment of all the irrigation systems serving the South Rashidiya area, which was conducted by 40 technical staff representatives from the MOWR and the MOA. Trained staff from the MOWR, with assistance from MOA technicians, will utilize an existing Geographic Information System provided under a previous U.S. donor project to process the data gathered by the field survey. The end result will be a report that will clearly identify the maintenance and capital reconstruction effort required to return irrigation water to the area and include capital rehabilitation requirements for South Rashidiya in the MOWR 2009 Budget. Although the river’s low water flow and the slow deterioration of the area’s irrigation systems are of primary concern, power 98 Tatweer National Capacity Development Program Final Evaluation shortages do have an impact. Currently, power shortages do not allow the canal pumps to keep a consistent enough flow to prevent stagnation in some of the canals, but Iraqi engineers proposed a solution for their ministerial counterparts. “Any way you look at it, there is a need for more electricity,” Robert Kirkman, advisor to the MOWR said. “I’m encouraged there’s a design for getting new pumps and more electrical supply. It seems to me we’re close to solving this!” The ME has now joined the partnership and will provide a “dedicated service” of uninterrupted electricity to keep the canal pumps working despite reduced water levels. The intent is for future surveys to be conducted by the MOWR and the MOA without help from Tatweer, INMA, or the Provincial Reconstruction Team, eventually leading to a complete survey of all the irrigation systems and agricultural land throughout Iraq. Once this has been achieved, the GOI will accurately be able to evaluate the supply and demand requirements for water resources. This information will carry forward at the national level, and will provide the GOI with a firm basis for negotiating utilization of water resources with its neighbors. In addition, the MOA will be able to accurately monitor and assess crop production vs. market demand, and introduce programs for pesticide distribution, crop rotation methods, and best irrigation practices. (Annual Report 2008) TE is applying its comprehensive problem solving approach to improve the MOO’s procurement systems with international suppliers and to upgrade inefficient financial and legal systems, which will boost the confidence of potential international investors and allow the GOI to move effectively into the world market. The MOO’s capacity to manage projects and strategically plan needs for the oil and electricity sectors is improving with the aid of Tatweer’s core skill building in integrated supply chain management and project management. Concurrently, TE is working in conjunction with the Energy Fusion Cell (EFC) and the Gulf Regional Division of the Army Corp of Engineers (GRD) on key projects to enhance oil and gas output which will allow the MOO to leverage in-country assets in a more effective manner. The Ministries of Oil and Electricity are now engaged in joint long-term planning discussions necessary for an efficient national power production system. This level of cooperation is a significant break from the past, where both ministries worked in extreme isolation. The first evidence of this cooperation is at the Mussaib power plant, where a budding partnership between both ministries will provide the technical help to integrate power plant operations with an on-site oil refinery. TE is also helping to facilitate broader collaboration by involving leaders from both ministries in capacity building training events such as contract procurement and management, and project management and control. TE helped organize training in which middle managers from both ministries met and discussed common problems and solutions for the first time. This cooperation is the foundation of an updated energy system which supplies the right amount and type of fuel to the right power plants to produce sufficient electricity for all Iraqis. TE collaborates with the EFC, the GRD, and the Iraq Transition Assistance Office (ITAO) programs to leverage strengths of the different contributors. Together, these programs combine near-term focus on specific projects and budget execution with long- 99 term capacity building at the ministry level and throughout the regions of Iraq. There is a TE group liaison officer at the EFC to facilitate coordination with the EFC, the U.S. Embassy, GRD, and ITAO. (Annual Report 2008) 4 2009 National Development Plan (NDP) USAID/Tatweer assisted the Ministry of Planning and Development Cooperation (MOPDC) to develop a five-year capital investment plan. With USAID/Tatweer assistance, 15 GOI committees developed sectoral investment plans based on inputs from ministries, provinces, and stakeholders. Toward the end of the year, this “investment” plan expanded in importance and is becoming the NDP. With the full cooperation of all ministries and provinces, and with the involvement of the United Nations and the World Bank, the NDP is becoming the main framework for national development, replacing the earlier, donor-driven International Compact for Iraq. (Annual Report 2009) USAID/Tatweer, the World Bank and Iraqi staff in the Ministry of Labor and Social Affairs (MOLSA) are conducting functional testing on the Social Safety Net (SSN) information system to make the SSN more efficient and egalitarian. The SSN will begin service in coming months, with 22 offices across the country. It will operate on a credit card-based system and will structure basic day-to-day welfare for hundreds of thousands of disaffected Iraqi citizens, including disabled persons, widows, and orphans. (Annual Report 2009) In 2009, USAID/Tatweer conducted extensive training to help the GOI move forward with substantive capital investment projects. Our support for the World Bank’s COMFAR III project management and feasibility studies approach provides senior Iraqi project managers and project finance decision-makers critical tools to conceive, award and implement key projects. USAID/Tatweer complimented these courses by revising the GOI’s project proposal process, providing cross-ministry training programs as well as a series of ministry specific presentations on appropriate methods to make use of the GOI’s project proposal forms. (Annual Report 2009) The Rashidiya survey was the first in a series of three surveys sponsored by USAID/Tatweer in cooperation with the USAID￾funded INMA project and the Taji Irrigation District. The project will train, equip, and institutionalize a program within the MOWR and MOA for the survey of all irrigation systems and farmland throughout the Iraq. It will also permanently document the results of these surveys within geographic information systems. “The Ministry of Water is there to provide water resources when and where needed,” said USAID/Tatweer Senior MOWR Advisor, Robert Kirkman, who has led the GIS-mapping and irrigation 100 Tatweer National Capacity Development Program Final Evaluation canal surveying efforts. “Ninety percent of water in Iraq is used for agriculture, so 90 percent of the reason for the existence of the Ministry of Water Resources is agriculture. Right now the Ministry is not sure of the cost-effectiveness of fixing deteriorated canals and pumping stations. They need the numbers to give key decision makers the knowledge to make strategic decisions.” (Annual Report 2009) The Rashidiya survey was the first in the series of three such assessments sponsored by USAID/ Tatweer in cooperation with the USAID-funded INMA project and the Taji Irrigation District. The project is training, equipping and institutionalizing a program within the MOWR and MOA for the survey of all irrigation systems and farmland throughout Iraq. It will also permanently document the results of these surveys with GIS. Most importantly, this rare example of ministerial cooperation has led to a desire by the GOI to expand this work to address irrigation problems nationally. (Annual Report 2009) NEW TOOLS DEMONSTRATED AT JULY 2009 WORKSHOP; U.S. ARMY – USAID/TATWEER-PARTNERSHIP In coordination with USAID/Tatweer, the US Army’s 56th Stryker Brigade Combat Team presented a week-long workshop about the newest GIS mapping tools in Baghdad’s International Zone to MOWR and MOA technicians. Twenty technicians learned data processing procedures that they will use to inventory the Iraqi farmland and irrigation infrastructure. The workshop showed participants how to compile data, gathered recently by field technicians in the Rashidiya pilot project, to measure canal layouts and amounts of acreage used for growing various crops. Ministries are now using the raw data to conduct cost analyses of improvement projects. Such projects are sorely needed, as many parts of the canal system in rural Iraq are over 30 years old and in various states of disrepair. (Annual Report 2009) 5 2010 Decades of poor funding, crippling sanctions, incoherent investment policies, and a lack of stakeholder input led to severe under￾performance across all Iraqi sectors. Given the responsibility of creating the NDP, the MOP recognized the need for a unified approach that incorporated modern concepts of planning and transparency and harmonized GOI priorities with those of the international investor and donor communities. With its most seasoned planners either in retirement or living outside of the country, the MOP requested assistance from USAID/Tatweer advisors to coordinate the divergent interests of stakeholders and provide the MOP with technical and logistical support. Working to ensure the NDP reflected local community needs, USAID/Tatweer simultaneously steered GOI coordination with the United Nations, World Bank, and other international organizations toward incorporating internationally accepted best business practices, thus contributing to donors’ widespread support for the NDP. (Annual Report 2010) 101 Ministry of Planning: Iraq Development Management System. The MOP approved the joint USAID/Tatweer–United Nations Development Programme (UNDP)-developed Iraq Development Management System (IDMS) as the official system to be used to request funding for capital investments. Data is being entered and migrated correctly into the SSN application, and is providing extensive training for over 30 MOLSA IT staff to prepare them to manage and maintain the SSN application. The USAID/Tatweer SSN team continued the development of the Kurdish language version of the SSN application. The Ministry of Labor and Social Affairs in the KRG successfully completed their initial data entry program and have 40,461 records entered into a Microsoft Visual FoxPro database ready to load into the SSN application once deployed. The USAID/Tatweer team has developed the capability of transferring the benefit claimant’s records from the Visual FoxPro database into the SSN’s SQL database. Deployment is planned to take place in the first half of 2011. (Annual Report 2010) The Iraq Development Management System (IDMS) is perhaps the most high-impact system introduced by USAID/Tatweer and will enhance Iraq’s budget execution for years to come. The IDMS is state-of-the-art software developed by USAID/Tatweer, the UNDP, and the Ministry of Planning to automate the capital investment project cycle and manage Iraq’s capital investment portfolio. The IDMS consists of several separate modules or databases, each of which facilitates the planning, coordinating, and monitoring of development endeavors throughout Iraq. These include a comprehensive library of all relevant investment laws, rules, and regulations, a budget execution roadmap, a project proposal form, and a mechanism that facilitates electronic requests for funding. The IDMS is also set to include a module for comprehensive projects and national-level monitoring, as well as a procurement status tracker. Fully functional in Arabic as well as English, the IDMS is installed on the MOP HQ’s servers. The IDMS was mandated by the MOP as the software to be used in managing capital investment projects in the 2010-2014 National Development Plan. (Annual Report 2010) 102 Tatweer National Capacity Development Program Final Evaluation ANNEX 11E: TATWEER DOCUMENT REVIEW: TRAINING PROGRAMS Tatweer Document Review (TDR); Training Programs: 1. Project Management 2. Fiscal Management 3. Human Resources Management 4. Information Technology Management 5. Leadership and Communications 6. Anti-Corruption 7. Training of Trainers General Program 8. Monitoring and Evaluation 9. Technical Assistance Training 10. Strategic Planning 11. Quality Management (Tatweer Database) PUBLIC MANAGEMENT SKILLS TRAINING PROGRAM Tatweer’s civil service training project designs and delivers training modules and workshops to enhance the capacity of Iraqi civil servants to deliver essential services to the population. Tatweer’s program has, in one year of operations, trained over 2,900 participants throughout the country while creating over 180 trainers who now deliver new skills and perspectives within their own ministries. Tatweer’s activities include the provision of skills training in core functions of public administration, including: Project Management: How to conceive, design, and implement projects from a legislative tracking system in the Council of Ministers Secretariat to large capital project procurements under the Ministry of Electricity. Fiscal Management & Procurement: Budget formulation and execution, current accounting principles, and a series of procurement courses based on Iraqi law and international best practices Leadership & Communications: Empowering staff to meet the needs of decision makers, clearly communicating goals and objectives to staff and managing resources, especially staff resources, effectively. Strategic Planning: Preparing ministry staff like the Ministry of Oil to undertake comprehensive strategic plans that account for its diversified structure. (Annual report 2007) 103 TATWEER’S ONGOING CIVIL SERVICE TRAINING PROGRAM Although Tatweer’s training efforts were challenged throughout the year by changes in upper-level management at some offices and ministries, concerns about adding civil servants from newly￾supported ministries, and security issues, more than 25,000 individuals were involved in Tatweer training. These enrollees completed direct training courses conducted by Tatweer core area advisors, and roll-out courses conducted by civil servants who had been certified as TOTs. Follow￾up surveys indicate that these individuals are putting their new skills to good use, and by the end of the year more than half of this training was provided by Iraqi trainers who had been through the Tatweer training process. Each of the 10 Tatweer-supported ministries have actively participated in training activities, selecting staff to participate in courses that cover the core areas of Project Management, Fiscal Management, Human Resources Management, IT Management, and Leadership and Communication. In the case of the small, newly-established Ministry of Human Rights, all staff members had participated in at least one Tatweer-designed course by the end of the reporting period. The implementation of rollout courses taught by TOTs was crucial to this effort, as it allowed the flexibility to insert a small number of MOHR staff into courses primarily delivered to another ministry. Increased Iraqi investment in training capacity is evidence of their government’s commitment to building internal ability across ministries and provinces. With Tatweer’s assistance, the government’s civil service training center, NCCMD, has substantially enhanced its capabilities; regional centers have been developed in Erbil, Basra, and Mosul, and many Ministries have established and funded public administration training programs within their Ministries. SPECIALIZED TRAINING In addition to the core competency courses, Tatweer responded to requests from various ministries, and to needs identified by Tatweer advisors, and designed a variety of specialty training programs. Specialized training consists of workshops, specially designed training sessions, and working meetings. Among the topics presented as part of Tatweer specialized training are procurement and contracts management, project management, capital budget development, and strategic planning. Some specific examples of specialized training are: Computerized Maintenance and Management Systems (CMMS): The ME requested assistance in implementing CMMS - computerized systems for parts inventory management, scheduling of preventive maintenance of equipment used in ME operations, and ancillary functions such as automatic production of work orders. TE conducted a workshop involving 14 representatives from ME facilities from across Iraq to identify priorities and develop a work plan. Project Management: The joint TE/ME Project Management workgroup held a four-day workshop for 11 senior project/cost managers from the ME to discuss earned-value based cost control, uniform project monthly reporting, and Primavera based scheduling. Two specific ME project activities were presented by participants in order to receive informal risk-assessment feedback. Participants discussed future corrective actions and TE mentoring of the managers during implementation of cost and schedule control programs on one or two major ME projects. Governmental Accounting: Forty-eight employees from the ME Economic and Finance Departments participated in three sessions of the governmental accounting course requested by the Director General (DG) of Economic and Finance Department and implemented by the Training Workgroup. This course was designed specifically to address the fundamentals of governmental accounting as outlined by the DG. It will be followed by two or three workshops to help in solving specific issues the Finance Department is currently facing in their work performance. Strategic Planning: North Oil Company and North Gas Company employees participated in a Strategic Planning coaching session led by Tatweer Senior advisors from the 104 Tatweer National Capacity Development Program Final Evaluation Leadership/Communications and strategic teams. This session has resulted in an improved leadership capability amongst the employees who attended. Risk Analysis: Top Iraqi engineers from the MOMPW, MOWR, and MOA have gained valuable hands-on problem solving experience using a rigorous risk analysis approach. Tatweer designed a specialized Advanced Contract and Project Management Course (C/PMS) to introduce a rigorous risk analysis approach and to provide mentoring using real-world projects as the engineers work through the course. C/PMS participants analyzed the design and construction of a $125 million glass wool factory as case study for the course. They were tasked to utilize course concepts and apply contracting strategy, organizational approach, and systems and procedures to the potential problems and risks inherent in building and managing this facility. (Annual Report 2008) 105 ANNEX 12: BUDGET EXECUTION TATWEER MINISTRIES Update Dec 6,2010 Million 2006 Operations Budgets (1) Investment Budgets (2) Total Budget (3) Actual Spent for Investment Budgets (4) % of Execution (4/2) Line Ministries $ 32,088.20 9,643.58 41,731.79 4,959.62 51.4 IQ 38,024,521 11,427,645 49,452,165.8 5,877,152 51.4 Provincial Councils $ 27.05 2,531.65 2,558.70 0.00 0.0 IQ 32,059.53 3,000,000.00 3,032,059.5 0 0.0 KRG $ 2,973.72 632.91 3,606.64 632.91 100.0 IQ 3,523,863.60 750,000.00 4,273,863.6 750,000.00 100.0 Supplemental budget $ IQ Agricultural Initiative $ IQ Total Budget $ 35,088.98 12,808.14 47,897.12 5,592.53 43.7 IQ 41,580,443.88 15,177,645.00 56,758,088.88 6,627,152.00 43.7 Total Federal Budget $ 47,897.12 IQ 56,758,088.88 % of Capital Investment Budget to Federal Budget 26.7 # of Projects 2006 Line Ministries 51.4% 106 Tatweer National Capacity Development Program Final Evaluation Update Dec 6,2010 Final Close Out This Year 2007 Millio n Operations Budgets (1) Investment Budgets (2) Total Budget (3) Actual Spent for Investment Budgets (4) % of Execution (4/2) Line Ministries $ 23,818.00 6,603.78 30,421.78 3,981.81 60.3 IQ 28,224,328 7,825,480.30 36,049,808.6 4,718,444 60.3 Provincial Councils $ 43.32 4,014.20 4,057.52 1,869.16 46.6 IQ 51,329.42 4,756,827.36 4,808,156.8 2,214,955.78 46.6 KRG $ 3,391.84 1,698.41 5,090.25 1,698.41 100.0 IQ 4,019,324.67 2,012,620.00 6,031,944.7 2,012,620.00 100.0 Supplemental budget $ IQ Agricultural Initiative $ IQ Total Budget $ 27,253.15 12,316.39 39,569.54 7,549.38 61.3 IQ 32,294,982.4 3 14,594,927.6 6 46,889,910.0 9 8,946,020.08 61.3 Total Federal Budget $ 39,569.54 IQ 46,889,910.09 % of Capital Investment Budget to Federal Budget 31.1 # of Projects 1412 2007 Line Ministries 60.30% 107 Update Dec 6,2010 Final Close Out This Year 2008 Million Operations Budgets (1) Investment Budgets (2) Total Budget (3) Actual Spent for Investment Budgets (4) % of Execution (4/2) Line Ministries $ 47,146.99 15,200.01 62,347.00 12,393.51 81.5 IQ 55,869,183.75 18,012,013.800 73,881,197.6 14,686,309.600 81.5 Provincial Councils $ 311.61 6,378.24 6,689.85 3,160.03 49.5 IQ 369,258.91 7,558,216.600 7,927,475.5 3,744,639.000 49.5 KRG $ 4,292.31 3,748.34 8,040.65 3,748.97 100.0 IQ 5,086,384.98 4,441,787.000 9,528,172.0 4,442,526.000 100.0 Supplemental budget $ 587.64 587.64 IQ 696,358.00 696,358.0 0.0 Agricultural Initiative $ IQ Total Budget $ 51,750.91 25,914.24 19,302.51 74.5 IQ 61,324,827.64 30,708,375.40 92,033,203.04 22,873,474.60 74.5 Total Federal Budget $ 77,665.15 IQ 92,033,203.04 % of Capital Investment Budget to Federal Budget 33.4 # of Projects 1939 2,008.0 Line Ministries 81.54% 108 Tatweer National Capacity Development Program Final Evaluation Update Dec 6,2010 Final Close out this year 2009 Million Operations Budgets (1) Investment Budgets (2) Total Budget (3) Actual Spent for Investment Budgets (4) Actual Spent for Investment Budgets without Advance (5) % of Execution (4/2) % of Execution (5/2) Line Ministries $ 40,400.96 8,543.99 48,944.94 7,142.56 4,284.40 83.6 50.1 IQ 47,875,133.36 10,124,624.00 57,999,757.4 8,463,930.80 5,077,010.00 83.6 50.1 Provincial Councils $ 813.96 2,167.36 2,981.32 1,849.05 1,849.05 85.3 85.3 IQ 964,547.72 2,568,319.00 3,532,866.7 2,191,120.40 2,191,120.40 85.3 85.3 KRG $ 5,046.27 1,943.75 6,990.02 1,927.69 1,927.69 99.2 99.2 IQ 5,979,833.61 2,303,338.40 8,283,172.0 2,284,308.00 2,284,308.00 99.2 99.2 Supplemental budget $ IQ Agricultural Initiative $ 73.27 73.27 0.00 0.00 0 0.0 IQ 86,830.50 86830.50 0.00 0.00 0 0.0 Total Budget $ 46,261.19 12,728.36 58,989.56 10,919.29 8,061.13 85.8 63.3 IQ 54,819,514.69 15,083,111.90 69,902,626.6 12,939,359.20 9,552,438.40 85.8 63.3 Total Federal Budget $ 58,989.56 IQ 69,902,626.59 % of Capital Investment Budget to Federal Budget 21.6 # of Projects 1922 2009 Line Ministries 83.60% 109 Update Dec 6,2010 2010 Million Operations Budgets (1) Investment Budgets (2) Total Budget (3) Actual Spent for Investment Budgets (4) % of Execution (4/2) Line Ministries $ 44,419.88 15,654.76 60,074.63 11,402.70 72.8 IQ 52,637,557.34 18,550,884.793 71,188,442.1 13,512,200 72.8 Provincial Councils $ 1,047.29 2,711.76 3,759.05 1,883.98 69.5 IQ 1,241,035.00 3,213,441.312 4,454,476.3 2,232,519 69.5 KRG $ 5,952.83 2,901.64 8,854.47 2,901.96 100.0 IQ 7,054,102.37 3,438,448.404 10,492,550.8 3,438,817.00 100.0 Supplemental budget $ IQ Agricultural Initiative $ 169.17 IQ 200,469.059 200,469.1 69977.00 Total Budget $ 51,420.00 21,437.34 72,688.16 16,188.64 75.5 IQ 60,932,694.71 25,403,243.57 86,335,938.3 19,253,513.00 75.8 Total Federal Budget $ 72,857.33 IQ 86,335,938.28 % of Capital Investment Budget to Federal Budget 29.4 # of Projects 2083 2010 Line Ministries 72.84% 110 Tatweer National Capacity Development Program Final Evaluation Update Dec 6,2010 2011 Million Operations Budgets (1) Investment Budgets (2) Total Budget (3) Actual Spent for Investment Budgets (4) % of Execution (4/2) Line Ministries $ 48,073,395.81 17,195,819.93 65,269,215.75 0.0 IQ 56,966,974,038.00 20,377,046,620.00 77,344,020,658.0 0.0 Provincial Councils $ 455,753.09 3,644,388.19 4,100,141.28 0.0 IQ 540,067,414.00 4,318,600,000.00 4,858,667,414.0 0.0 KRG $ 5,498,382.88 3,596,887.08 9,095,269.96 0.0 IQ 6,515,583,714.00 4,262,311,184.00 10,777,894,898.0 0.0 Supplemental budget $ IQ Agricultural Initiative $ IQ 0.0 Total Budget $ 54,027,531.79 24,437,095.19 78,464,626.98 0.0 IQ 64,022,625,166.00 28,957,957,804.00 92,980,582,970.0 0.0 Total Federal Budget $ 78,464,626.98 IQ 92,980,582,970.00 % of Capital Investment Budget to Federal Budget 31.1 # of Projects Line Ministries 111 All Amounts in Millions Final close out this year Ministries/Province 2006 Allocation Amounts (1) Actual Expenditure (2) Budget Execution Rate Dinar Dollar Dinar Dollar (2/1) Ministry of Oil 4,431,650.0 3,755.6 2,409,500.0 2,041.9 54.4 Ministry of Electricity 1,746,030.0 1,479.7 1,746,030.0 1,479.7 100.0 Ministry of Water Resources 299,704.0 254.0 299,704.0 254.0 100.0 Ministry of Agriculture 36,699.0 31.1 35,718.0 30.3 97.3 Ministry of Municipalities and Public Works 574,387.0 486.8 553,298.0 468.9 96.3 Ministry of Human Rights 3,000.0 2.5 0.0 0.0 0.0 Ministry of Education 35,190.0 29.8 26,990.0 22.9 76.7 Ministry of Planning 38,300.0 32.5 37,837.0 32.1 98.8 Ministry of Displacement and Migration 2,000.0 1.7 1,000.0 0.8 50.0 Ministry of Health 31,542.0 26.7 30,667.0 26.0 97.2 6,100.4 2006 4,356.6 Total 10 Ministries 71.41% 8 Ministries (no Electricity or Water) 4,366.8 60.06% 2,622.9 112 Tatweer National Capacity Development Program Final Evaluation All Amounts in Millions Final close out this year Ministries/Province 2007 Allocation Amounts (1) Actual Expenditure (2) Budget Execution Rate Dinar Dollar Dinar Dollar (2/1) Ministry of Oil 2,875,000.0 2,436.4 1,732,398.0 1,468.1 60.3 Ministry of Electricity 1,745,055.0 1,478.9 1,315,550.0 1,114.9 75.4 Ministry of Water Resources 338,613.0 287.0 279,075.0 236.5 82.4 Ministry of Agriculture 62,816.0 53.2 41,889.0 35.5 66.7 Ministry of Municipalities and Public Works 628,290.0 532.4 353,043.0 299.2 56.2 Ministry of Human Rights 2,000.0 1.7 88.0 0.1 4.4 Ministry of Education 361,330.0 30,147.0 25.5 8.3 Ministry of Planning 100,000.0 84.7 69,977.0 59.3 70.0 Ministry of Displacement and Migration 1,000.0 0.8 370.0 0.3 37.0 Ministry of Health 98,250.0 83.3 28,280.0 24.0 28.8 4,958.5 2007 3,263.4 Total 10 Ministries 65.81% 8 Ministries (no Electricity or Water) 3,192.7 59.89% 1,912.0 113 All Amounts in Millions Final close out this year Ministries/Province 2008 Allocation Amounts (1) Actual Expenditure (2) Budget Execution Rate Dinar Dollar Dinar Dollar (2/1) Ministry of Oil 3,854,602.0 3,266.6 3,647,967.0 3,091.5 94.6 Ministry of Electricity 4,709,129.0 3,990.8 4,451,985.0 3,772.9 94.5 Ministry of Water Resources 1,057,816.0 896.5 853,094.0 723.0 80.6 Ministry of Agriculture 179,400.0 152.0 135,109.0 114.5 75.3 Ministry of Municipalities and Public Works 1,448,014.0 1,227.1 1,600,928.0 1,356.7 110.6 Ministry of Human Rights 5,712.0 4.8 414.0 0.4 7.2 Ministry of Education 481,183.0 407.8 214,476.0 181.8 44.6 Ministry of Planning 132,318.0 112.1 31,631.0 26.8 23.9 Ministry of Displacement and Migration 1,630.0 1.4 842.0 0.7 51.7 Ministry of Health 142,263.0 120.6 79,533.0 67.4 55.9 10,179.7 2008 9,335.6 Total 10 Ministries 91.71% 8 Ministries (no Electricity or Water) 5,292.5 91.45% 4,839.7 114 Tatweer National Capacity Development Program Final Evaluation All Amounts in Millions Final close out this year Ministries/Province 2009 Allocation Amounts (1) Actual Expenditure (2) Budget Execution Rate Dinar Dollar Dinar Dollar (2/1) Ministry of Oil 2,603,550.0 2,206.4 2,599,255.0 2,202.8 99.8 Ministry of Electricity 1,375,383.0 1,165.6 1,276,972.7 1,082.2 92.8 Ministry of Water Resources 689,258.0 584.1 567,620.0 481.0 82.4 Ministry of Agriculture 181,900.0 154.2 131,387.0 111.3 72.2 Ministry of Municipalities and Public Works 610,483.0 517.4 610,483.0 517.4 100.0 Ministry of Human Rights 4,250.0 3.6 3,067.0 2.6 72.2 Ministry of Education 212,500.0 180.1 170,593.0 144.6 80.3 Ministry of Planning 51,000.0 43.2 39,439.4 33.4 77.3 Ministry of Displacement and Migration 6,800.0 5.8 3,606.0 3.1 53.0 Ministry of Health 531,500.0 450.4 488,114.0 413.7 91.8 5,310.7 2009 4,992.0 Total 10 Ministries 94.00% 8 Ministries (no Electricity or Water) 3,561.0 94.00% 3,428.8 115 All Amounts in Millions Update Feb 28-2011 Regarding First report for 2010 Ministries/Province 2010 Allocation Amounts (1) Actual Expenditure (2) Budget Execution Rate Dinar Dollar Dinar Dollar (2/1) Ministry of Oil 3,103,550.0 2,630.1 2,644,841.0 2,241.4 85.2 Ministry of Electricity 4,256,503.0 3,607.2 3,997,418.0 3,387.6 93.9 Ministry of Water Resources 1,145,682.6 970.9 738,158.0 625.6 64.4 Ministry of Agriculture 209,414.6 177.5 97,093.0 82.3 46.4 Ministry of Municipalities and Public Works 1,749,476.0 1,482.6 1,504,702.0 1,275.2 86.0 Ministry of Human Rights 7,150.0 6.1 2,458.0 2.1 34.4 Ministry of Education 270,975.8 229.6 65,480.0 55.5 24.2 Ministry of Planning 66,376.5 56.3 25,449.0 21.6 38.3 Ministry of Displacement and Migration 18,194.0 15.4 11,984.0 10.2 65.9 Ministry of Health 1,077,000.0 912.7 216,553.0 183.5 20.1 10,088.4 2010 7,884.9 Total 10 Ministries 78.16% 8 Ministries (no Electricity or Water) 5,510.3 70.26% 3,871.7