May 24, 2007 This publication was produced for review by the United States Agency for International Development. It was prepared by Kathleen Webb and Stark Biddle, with sample design and statistical analysis by Robert Torene and Harvey Herr, of International Business & Technical Consultants, Inc. Building on Transition Iraq Civil Society Program (ICSP) Final Evaluation Final Report Building on Transition Iraq Civil Society Program (ICSP) Evaluation Final Report International Business & Technical Consultants, Inc. 8614 Westwood Center Drive Suite 400 Vienna, VA 22182 USA Contracted under: Monitoring and Evaluation Performance Program, Phase II (MEPP II) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. TABLE OF CONTENTS PAGE ACRONYMS EXECUTIVE SUMMARY ............................................................................................................... i CHAPTER 1: BACKGROUND ......................................................................................................1 1.1. Country Context .....................................................................................................................1 1.2. The USAID/IRAQ Program ....................................................................................................2 CHAPTER 2: THE ICSP EVALUATION .......................................................................................5 2.1. Overview of the Evaluation ....................................................................................................8 2.2. Constraints faced during the Evaluation ................................................................................5 CHAPTER 3: Description of the ICSP Program ...........................................................................9 3.1. Purpose and Contractual Expectations..................................................................................9 3.2. Project Activities...................................................................................................................11 3.3. Project Organization ............................................................................................................11 CHAPTER 4: FINDINGS.............................................................................................................13 4.1. Findings - ICSP Management .............................................................................................13 4.2. Program Design and Relevance ..........................................................................................19 4.3. Program Effectiveness.........................................................................................................22 4.4. Impact of the ICSP...............................................................................................................34 4.5. Efficiency..............................................................................................................................37 Chapter 5: Sustainability.............................................................................................................41 5.1. Contractual Obligation – Sustainability ................................................................................41 5.2. Sustainability Plans..............................................................................................................42 5.3. Sustainability of Civil Society in Iraq ....................................................................................44 5.4. Role of CSRCs in Sustainability...........................................................................................45 5.5. Sustainability of the ICSP Program......................................................................................45 5.6. Sustainability of the CSRCs.................................................................................................46 5.7. Sustainability Planning.........................................................................................................47 Chapter 6: The ICSP Program: The Pros and Cons of Alternative Future Directions....................................................................................................................................48 6.1. Background considerations..................................................................................................48 6.2. The Identification of Alternative Strategies ..........................................................................48 6.3. Factors to be considered in evaluating the pros and cons of alternative approaches .........49 6.4. The Pros and Cons of Alternatives ......................................................................................50 CHAPTER 7: CONCLUSIONS ...................................................................................................53 7.1. Program Effectiveness.........................................................................................................53 7.2. Program Efficiency...............................................................................................................53 7.3. Management ........................................................................................................................54 7.4... Sustainability......................................................................................................................54 CHAPTER 8.0: Recommendations.............................................................................................56 8.1. Fundamental Strategy Recommendations...........................................................................56 8.2. Detailed Recommendations.................................................................................................57 8.3. Recommendations for the USAID Results Structure ...........................................................58 8.4. Recommendations for the USAID Results Framework........................................................58 LIST OF ACRONYMS ADF America’s Development Foundation ALNAP Active Network for Accountability and Performance in Humanitarian Action CAG Community Action Group COP Chief of Party CPA Coalition Provisional Authority CSO Civil Society Organization CTO Cognizant Technical Officer DAC District Advisory Council DCOP Deputy Chief of Party DG Democracy and Governance DOD Department of Defense DOS Department of State FY Fiscal Year GOI Government of Iraq HQ Headquarters HR Human Resources IIACSS Independent Institute for Administration and Civil Society Studies IBTCI International Business & Technical Consultants, Inc. ICAP Iraq Community Action Program ICSP Iraq Civil Society Program IFES International Federation of Election Systems INL International Narcotics Law IP Implementing Partner IR Intermediate Result IRD International Relief and Development IRI International Republican Institute IT Information Technology LG Local Government LGP Local Governance Program LOE Level of Effort LOP Life of Project or Program M&E Monitoring and Evaluation MIS Management Information System MOE Ministry of Education MOU Memorandum of Understanding N/A Not Applicable n.d. No Data NDI National Democratic Institute NGO Non-Governmental Organization OECD Organization for Cooperation and Development PMP Performance Management Plan PC Provincial Council PRS Project Reporting System PRT Provincial Reconstruction Team Q Quarter RF Results Framework RFA Request for Application SO Strategic Objective SOW Scope of Work TA Technical Assistance UN United Nations US United States USAID United States Agency for International Development USG United States Government Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) i EXECUTIVE SUMMARY This is a final evaluation of the 3-year, $60 million USAID funded program to build and strengthen civil society in the Republic of Iraq. The program, entitled the Iraq Civil Society Program (ICSP) is being implemented through a contract with the America’s Development Foundation (ADF). It began in August 2004 and will terminate on June 30, 2007. Purpose The purpose of this evaluation is to provide guidance and, if needed, recommendations for course corrections for the current program, determine if objectives are being achieved, and provide lessons learned in the event of a follow-on project. The evaluation focuses on the management of the program, the model developed by ADF, the quality and impact of training provided and on the effectiveness and sustainability of the four Regional Civil Society Resource Centers (CSRCs) that have been established in Iraq. The evaluation does not deal with the media program being implemented by ADF nor does it include a critique of the anti-corruption program. These elements were explicitly excluded from the Scope of Work. Context After decades of tyranny, Iraqi citizens are now starting to adopt some of the attributes of what has been called “civil society” - - - the capacity of individuals to come together voluntarily for the good of their community and to articulate and advocate for change that will improve their quality of life. At the same time it is abundantly clear that this phenomenon is in its infancy, that the ethic of community action is only beginning to take hold and that the institutional infrastructure that will need to be in place for civil society to flourish is at a very early stage of formation. The Project The USAID/ICSP initiative was intended to: promote an informed, sustainable and active indigenous Iraqi civil society that effectively and responsibly participates within a democratic system of governance” .This objective was to be achieved through the establishment of four CSRCs; the provision of training and technical assistance through these centers and the management of a small grants program to “reinforce” training and technical assistance. Special emphasis was to be placed on civic education, human rights, women’s advocacy and anti￾corruption. The contract heavily emphasized the importance of sustainability and local “ownership” of the CSRCs. Advisory Boards were to be established for each CSRC and to gradually assume a governance role. The ADF Model The ADF model has the following attributes: centralization and governance from the top; common format and templates; broad based approach; a diverse training curriculum; and the uses of conferences and workshops to focus on a particular issue. Factors that shaped the current model included: the necessity for speed; the weakness of the CSO sector; the desire to create sector-wide competence; the absence of a baseline survey or needs assessment; the use of a contract as opposed to a cooperative agreement; and difficulty of monitoring and oversight due to the security situation. strengths of the model include: speed, broad outreach, and the establishment of relatively uniform standards for organizational performance; uniformity of approach and the use of organizational assessment tools (OAT); the establishment of a rich data base; the importation of a training model from another country; potential access to donor funding; consistency with project objectives. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) ii Weaknesses of the model include: a “one size fits all” approach to training; centralization of grant making; tensions between the Egyptian training curriculum and the Iraqi context; inflexibility in responding creatively to opportunities; and the use of an inappropriate procurement vehicle for establishing a strong community constituency. Findings: Project Management The overall ICSP management structure is based on a traditional “hub and spokes” concept with authority and influence radiating out from a central point of direction. Ultimate policy and operational authority is located at headquarters in Baghdad and delegated out to the constituent units in degrees based on a judgment as to whether or not these units (the CSRCs) have the capacity to assume responsibility. Regional Directors have been given primary authority for managing the CSRCs. The initial rationale for establishing these positions was that the CSRCs units lacked the substantive and managerial expertise to function on their own. In practice, these positions have assumed line responsibility. The CSRCs do not have the authority to make policy nor are they routinely expected to offer policy alternatives. Virtually all of the CSRC Directors that were interviewed for this evaluation desired a greater degree of autonomy, flexibility and ability to adapt to the local situation in their given regions. The existence of multiple reporting channels makes it exceedingly difficult for the local Directors of the CSRCs to establish effective control. All of the CSRCs have made an attempt to establish local advisory boards. However, these entities were not started at the onset of the program, have not matured and are viewed as peripheral to the operation of the ICSP. None of the four CSRCs have developed a long range plan, a transition plan or an exit plan. The CSRCs have not made an effort to seek funds from other donors, individuals, local companies or multinational business. There has been no effort to seek funding from participating CSOs or to require fees for conference participation or workshops or to explore fee for service opportunities. The consultants were impressed by the caliber of local and expatriate staff. While additional training is required, these individuals manifest the competence to sustain the ICSP once ADF leaves. Findings: Program The ICSP has provided widespread training and technical assistance to approximately 1800 CSOs. The demand from Iraqi CSOs for training is high and growing and CSOs report high levels of satisfaction with the training that they have received. The CSRCs have sponsored numerous conferences and workshops that have been well received by the CSO community. However, there is growing demand for more specialized courses, for courses that reflect local issues and concerns and for courses tailored to the specific needs of the CSOs in that area. ADF has fully or partially complied with the following primary deliverables: • CSRCs have been established in the four regions and a full range of capacity building training has been installed. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) iii • Staff for all four centers have been hired and trained. They are motivated and competent although further training is warranted. • Twenty two training modules have been established and are given good marks by both CSOs and local CSRC staff. • An organizational assessment tool has been adapted to Iraq and administered to a large number of CSOs. While effective, there are a few minor administrative deficiencies that need to be corrected. • The assessment tool has yielded a very valuable data base of information upon which to build a second stage civil society project. • Training programs have been designed and delivered in the 3 emphasis areas: anti￾corruption, women’s advocacy and civic participation. Interview feed back indicates that these courses were of good quality. ADF has had difficulty complying with the following: • The establishing of CSRC satellite centers. Initial participants have not been able to sustain these programs. • The establishing of local advisory boards. The role of these boards will remain questionable until greater authority is delegated to the CSRCs. • The goals of institutional sustainability, discussed below. Findings: Impact In terms of wider effects of the ICSP (such as social, economic and technical effects), there is evidence of some impact based on the measurement of the critical assumptions of the ICSP. CSOs are more involved in the identification and mobilization of resources for advocacy campaigns, which in turn mean the CSOs can advocate for causes before local government elected or advisory councils. In many cases CSOs have advocated jointly with other public institutions for a common cause. These and other findings verify that the CSOs have had impact as measured by the indicators drawn from the critical assumptions of the program.1 Findings: USAID/ADF Communications Ambiguities in the USAID results framework and in the ADF contract, coupled with constant change in the country context have made communication between USAID and ADF difficult. Constraints on direct field observation, a complex bureaucratic structure and the intense pressure to achieve results have exacerbated this situation. The problem was made more difficult by the fact that the procurement instrument was a contract with the rigidity inherent in 1 The critical assumptions for the impact survey were that 1) the level of capacity building provided to the CSOs is adequately measured by the number or trainings attended, workshop participations, conference attendances, and technical assistance received; and 2) that the impact the CSOs are having is measured by the following: • The number of times they have facilitated a community forum or campaign; • The number of times they appeared before local government elected or advisory councils; • The number of times they appeared before local government departments; • The influence they believe they have had to change or influence policy decisions; • Whether the organization has mobilized resources from its members or from the community to carry out an advocacy campaign; • The number of times they have worked closely with other CSOs to advocate for a common cause; • How often in the past 12 months they have petitioned government officials or political leaders for improved services and the result of those petitions; and • The perception of empowerment that CSOs have in making the community a better place to live. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) iv that instrument rather than a cooperative agreement which would have encouraged thematic adjustment as USAID priorities evolved. In general, the extent and depth of detailed USAID oversight appears to have created a managerial burden that has forced and perhaps legitimized the establishment and maintenance of a powerful central office in Baghdad anathema to a philosophy of decentralization, local empowerment and local “ownership”. Findings: The ADF Contract The target dates for activity completion were wildly unrealistic. In particular, the plan to establish, staff, operate and be in a position to develop a long range sustainability plan for the four CSRCs within 60 (later 90) days of inception was impossibly ambitious and likely to force precipitous decision making. It is unfortunate that time constraints did not permit a civil society sector survey or a needs assessment that would have identified institutional deficiencies. The use of the contract format, coupled with an emphasis on time-specific deliverables appears in retrospect to be inconsistent with the type of capacity building work that was to be carried out under the contract. The extraordinary volatility of the Iraqi situation, the inherent difficulty of measuring impact and the consequent necessity of continual flexibility and adaptation suggest the preference for a cooperative agreement. Findings: Sustainability - the Civil Society sector The Iraqi context is deeply problematic. The CSO sector has not yet established the habits and practices of seeking financial support from individuals, business or foundations. While CSOs have a strong sense of social purpose, very few manifest an understanding of how to identify, build and nurture a constituent base of support. In general, CSOs appear to have a mindset of growing reliance on the international donor community. While a few CSOs have initiated income generating activities, the incremental funds from this source are limited and the time and energy needed to operate these initiatives may deter the group from work more directly connected to their social purpose mission. However, there is a deep humanitarian and philanthropic tradition in Islam that in the long run may provide the attitudinal structure necessary for charitable giving to flourish; there are roughly two million Diaspora Iraqis that could be cultivated for purposes of supporting Iraqi-based social purpose groups and when and if stability returns, multi-national firms can be expected to supply significant amounts of community based giving. Primary conclusions based on the findings A. Positive 1. With the exception of the area of sustainability, and in the context of extraordinarily difficult operating conditions, ADF has done a very good job in complying with the deliverables set forth in the initial and subsequently modified contract. 2. The overall ADF model was effective in the rapid delivery of training and technical assistance support to a large number of Iraqi CSOs. 3. As a result of the ADF program a nascent civil society sector appears to have been stimulated in Iraq. 4. A small but important number of highly competent and effective CSOs particularly in the area of human and civil rights appear to have emerged. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) v 5. The four CSRCs that have been established have the potential to make a significant contribution to the further development of civil society in Iraq if sustained and empowered to do so. 6. The local and expatriate staff of the ADF Project is competent, motivated and professional and CSRC staff appears capable of assuming the responsibilities of independence if empowered for this task. 7. There was no instance of report falsification which came to the attention of the Evaluation Team; issues of staff turnover relate to the centralization of authority that was necessary for implementation. ADF has taken remedial steps B. Negative 1. The targets and time lines set forth in the original and subsequently amended contract were excessively ambitious. 2. The use of a contract procurement mechanism was an error because it focused attention on whether or not deliverables were being produced instead of developing an adaptive program strategy. 3. The ADF management structure is too centralized and has not delegated authority commensurate with the growing competence of the CSRCs. ADF’s confusing internal communication system has created tensions that have undercut morale and hampered progress. 4. Effective Regional Advisory Boards have not been created in large part as a consequence of the top down, centralized nature of the project. 5. Lack of momentum toward the establishment of a sustainable structure at the national and local level constitutes a serious project failure when measured against the objectives set forth in the contract. Both USAID and ADF have been derelict in their disinclination to grapple with this deficiency, to define what is meant by sustainability, to recognize contradictions in program design and to make mid-course corrections. 6. Although the survey findings indicate increased advocacy activities including identification of resources for advocacy, these cannot be directly attributed to the training and grants received by the program and are likely related more to the CSOs histories. 7. The close out of the project will almost certainly mean that the CSRCs will collapse as they are not sustainable. Primary Recommendations 1. It is recommended that the ICSP Program transition from generic capacity building to a focused concentration on a core group of CSOs that are working in areas of high priority to USAID. This transition process would be phased over the next 12 to 18 months. 2. It is also recommended that the CSRCs be supported to evolve into locally owned and independent entities with local boards of directors, local staff and individualized programs that reflect the needs of the local communities. 3. The current head office organizational structure is recommended to be gradually compressed and ultimately consist of a small financial office, a monitoring and evaluation unit and a public relations office devoted to nation-wide advocacy for civil society. This entity would be transferred to the Iraqi Civil Society Institute (ICSI) and have the authority to receive grants from USAID and other donors and make sub-grants to the CSRCs. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) vi 4. USAID (through the IP) would work with the CSRCs to install financial control systems compliant with USAID regulations so that these entities can receive direct grant support from USAID if appropriate. 5. Administration of the OAT is recommended to be continued for approximately one year or until ICSP has conducted at least two assessments for each active CSO. 6. Training and the provision of technical assistance is recommended to be modified over the transition period to focus more sharply on tailoring assistance to the specific and individualized needs of the CSOs. Alternative Future Strategies Recommended There are four alternative future strategies recommended by the consultants for consideration: terminate the program as scheduled on June 30, 2007; continue the program with no fundamental change in approach; shift from generic capacity building to focused support for a limited number of CSOs or place primary focus on the establishment of strong CSRCs. • Factors to be considered to shape a choice of strategy include purpose, feasibility, sustainability and likelihood of complementarities. • An analysis of the strengths and weaknesses of each of these alternatives suggests the desirability of a Phase 2 Civil Society Program with the following characteristics: o Ultimate independence of the CSRCs under governance of local boards of directors. o Maintenance of a small central office in Baghdad to advocate for civil society and raise funds for distribution to the CSRCs. (The CSRCs would be affiliate members.) o The establishment of a new project to fund CSOs working in USAID priority areas. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 1 CHAPTER 1: BACKGROUND 1.1. Country Context In August 2004 USAID/Iraq launched the three year Iraq Civil Society Program (ICSP) in support of the United States Government (USG) efforts to foster participatory democratic governance in Iraq. The goal was to strengthen civil society’s role in the economic and political development of indigenous Civil Society Organizations (CSOs) in Iraq. A $43 million contract was awarded to the America’s Development Foundation (ADF) to accomplish this goal. This figure has now been modified to $60,880,157.002 . An understanding of the context in which ICSP was implemented provides insights into constraints and challenges of the program and has implications for its future and the future of other civil society programs. The Republic of Iraq is bordered from the north by Turkey, from the west by Syria and Jordan, from the south by Saudi Arabia, Kuwait and the Arab Gulf and from the east by Iran. Iraq has an area of 435,025 sq km in which 18 (eighteen) Governorates (five northern, nine central and four southern) comprise the key regions3 . Iraq has an ancient history but several more recent events have contributed to high vulnerability of the general population with respect to minimum living standards and life expectancy. This situation exists despite Iraq once being described by the United Nations (UN) as a high-middle-income country with a modern social infrastructure. For example, ten percent of the world’s oil reserve is found in Iraq; that represents the second largest oil reserve in the world after Saudi Arabia. The former Ba’athist regime ruthlessly discouraged the formation of community groups that might challenge the established order or give voice to alternative viewpoints. They repressed the formation and development of community based grass roots organizations that could advocate for change and reform. Centralization of power, the eradication of voices of opposition and the establishment of a State created and managed institutional structure neutralized attempts at voluntary association.4 Ongoing repressive rule by the Ba’athists under President Saddam Hussein was characterized by serious human rights abuses. This and the subsequent Gulf War of 1991 and overthrow of Saddam Hussein by the United States led coalition in 2003, worsened the situation. Continued unrest and fighting mean that social conditions and quality of life for the Iraqi people have deteriorated dramatically in the last five years. The result of this, up to 2003, when Saddam Hussein was overthrown, was a deadening of the habits and practices of what in the West is called “philanthropy”, and which the Iraqis refer to as Ál Jamiáat Al Khayria’, where local self help groups (often with religious or socio-cultural ties) come together to support their communities and liaise with local government leaders to make change. The fall of Saddam Hussein's regime opened a new chapter in Iraqi history and the opportunity emerged in the country to build a durable and open society with the attributes of community representation, local participation and a capacity to confront social, cultural and 2 Amendment/Modification 9 ADF Contract No. GEW-C-00-04-0001-00, page 3. 3 Four of these regions, namely Northern Region with its central city of Erbil; Southern Region with its central city of Basra; South Central with its central city of Hilla, and Central region with its central city of Baghdad, are the subject of this evaluation as the ICSP was implemented there. 4 A comprehensive analysis of the role, structure, strength and funding of civil society in Iraq is beyond the scope of this evaluation. Indeed, this is an immensely complex subject that would require a sensitive understanding of regional, tribal and religious distinctions. The following comments are based on interviews with civil society leaders and with the staff of the Civil Society Resource Centers. Some of these are based on anecdotal reporting and on the opinion of individuals whose views may be biased or based on incomplete information. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 2 economic issues unfettered by rigid centralized control. Indigenous civil society groups began to emerge/re-emerge and demand a say in the future of Iraq. The decline and slow emergence of these local groups or CSOs can be attributed to the almost complete disappearance of any vestige of free and voluntary association for purposes of social change. The new Government of Iraq (GOI) inherits a challenging governance apparatus characterized by weak and disorganized institutions lacking in transparency and accountability. There is poor inter-ministerial coordination and ineffective public outreach and communications infrastructure. The civil service is unmotivated and poorly managed. Service delivery and policymaking capabilities remain weak. Inadequate public information and discourse on proposed national governmental reforms result in a public perception that the GOI has no strategy and operates from crisis to crisis.5 Iraqi independent elections have been held in January and December 2005, with a Constitutional Referendum in October 2005. This process was supported by the USG and other governments through the International Republican Institute IFES (formerly the International Foundation for Election Systems) and National Democratic Institute (NDI). Many Iraqis worked with these organizations and benefited from technical assistance and training courses. There is growing evidence that after decades of tyranny, Iraqis are now starting to adopt some of the attributes of what has been called “civil society” - the capacity of individuals to come together voluntarily for the good of their community and to articulate and advocate for change that will improve their quality of life. At the same time it is abundantly clear that this phenomenon is in its infancy, that the ethic of community action is only beginning to take hold and that the institutional infrastructure needed to be in place for civil society to flourish is at a very early stage of formation. The ability of Iraqi CSOs to flourish has been negatively affected by the ongoing insecurity, compounded by an influx of insurgents from neighboring countries. Another factor limiting CSOs is the dispersion of native Iraqis to other countries. This began early in the regime of Saddam Hussein and is continuing through 2007. Many of those who left were the educated and experienced Iraqis who could have made a great contribution to the emerging CSOs. 1.2. The USAID/IRAQ Program The USAID/Iraq program (2006-2008) attempts to address the issue of state failure described above by dealing with the root causes of violence and by re-engaging a disenfranchised citizenry. USAID/Iraq has adopted a three year transitional strategy aimed at: 1) stabilizing Iraq through supporting economic and social stabilization efforts in key areas affected by insurgent violence; 2) supporting capacity building and governance of local and national government; and 3) increasing economic opportunity6 . The USAID program is designed to help build core functions within the government including strategic and policy planning, budgeting, feasibility analysis, human resource development, and management. 5 The Iraq national government is organized into 18 provinces (also known as governorates). Traditionally, each province has had a governor and a provincial advisory council. Under the Coalition Provisional Authority (CPA), enabled by the Transitional Administrative Law (TAL) and by relevant Administrative Orders, members to the Provincial Councils (PCs) were elected by popular vote. Interim provincial elections were held in January 2005 concurrently with the elections to the Transitional National Assembly (TNA). The Iraqi Constitution was validated in a referendum held on 15 October 2005 and the Council of Representatives was elected two months later on 15 December. The GOI is expected to hold a new round of provincial elections, but it is unclear whether or not provincial elections will be held in 2007. 6 Revised USAID/Iraq Performance Management Plan 2006-2008, p.1. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 3 By successfully improving the effectiveness and legitimacy of the national and local government and by improving service delivery, it is hoped that the USAID strategy will help the GOI establish a stable, democratic and prosperous Iraq. USAID technical support for future elections will also encourage citizen electoral participation, awareness of governmental issues, and involvement in the policy directions to be taken by the GOI. Recently a structure of Provincial Reconstruction Teams (PRT) has been established to better coordinate various USG programs and reflect unique local concerns. These teams include representatives from the Department of State, Department of Defense (DOD), USAID and the USAID Local Governance Project II (LGP II). It is hoped that this structure will facilitate dialogue with local government and identify synergies among different projects. USAID faces twin challenges of not only implementing extensive activities in an unstable and insecure environment but also monitoring and managing these activities for results, mostly by remote control. “Pressures to produce immediate accomplishments primarily at the output level can override actions geared toward meaningful long-term results. Extensive media exposure of the political and security situation in Iraq and a microscopic focus on all USG interventions is without precedent in recent history.”7 The USAID Results Framework (Annex I, Pages 211-212) is a detailed tracking of the evolution of USAID/Iraq’s performance structure. It appears that the primary documents that articulate what USAID is trying to achieve through its contract to ADF are the contract itself, the November 2005 Transition Strategic Plan and the 2006-2008 Performance Management Plan. These documents constitute a reasonably clear set of higher level concerns and objectives against which the ICSP could be assessed. In the 2005 Transition Strategic Plan, the development of civil society was a component part of the strategic objective designed to strengthen local democratic governance. A dual approach was designed to improve the institutional effectiveness of local government while simultaneously building the capacity of civil society to advocate for community concerns with a special emphasis on women’s issues and anti-corruption. An increase in the number of participating CSOs was employed to assess the effectiveness of the program. In the 2006-2008 Results Framework, the impact of the civil society program was recognized as assisting in the pursuit of three strategic objectives including strengthening of local government, building the capacity of national government and civic stabilization or reducing the incentives for violence.8 More specifically: • Under Strategic Objective (SO) 9: Strengthen Responsive and Effective Local Government, intermediate result (IR) 9.3 envisions greater involvement of citizens in decision making and employs the size of the CSO community as an indicator of progress. 7 USAID/Iraq Performance Management Plan 2006-2008 8 There appears to be a lack of clarity regarding the location of the ICSP Program within the USAID results framework. The Scope of Work for this Evaluation states that pursuant to both the Transition Plan and the Implementation Plan the ICSP program is located under Strategic Objective 9, Effective Local Government. While this is correct with respect to the Transition Plan, under the Implementation Plan, the ICSP program appears to fall primarily under Strategic Objective 10 with secondary implications under 7 and 9. Although both results frameworks are comparable, there are nuanced differences that have important implications with respect to delivery mechanisms. This observation illustrates the confusion that may arise when objectives are not clear. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 4 • Under SO 7: Focused Stabilization - Reduce the Incentives for Participation in Violent Conflict IR 7.2 envisions the mitigation of conflict by increasing citizen participation in community activities. Progress toward this result was to be measured by the number of joint activities undertaken by CSOs and local government. • Under SO 10: Improve the capacity of the National Government; IR 10.3 envisions a stronger capacity of civil society to advocate for citizen interests. With respect to a strengthened civil society, progress toward this result was to be gauged by two indicators: the number of policy changes influenced by CSOs and improvement in organizational capacity as measured by a standard questionnaire. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 5 CHAPTER 2: THE ICSP EVALUATION 2.1. Overview of the Evaluation This section provides an overview in terms of the background, timeline and scope of the evaluation. 2.1.1. Evaluation Team The evaluation conducted between March 5-April 18, 2007 was a final or summative one carried out by three external consultants with long-term experience in the subject matter. These were: a team leader with experience in monitoring and evaluation (M&E) in conflict and post conflict countries, a team member with experience in USAID funded and civil society programs and a statistician with long term experience in statistical analysis. The statistician provided support in statistical analysis from Washington, supported by IBTCI’s Long Term M&E Expert based in Iraq. The Team Leader and Team Member worked in both Washington and Iraq carrying out the main evaluation tasks shown in the Scope of Work (Annex A). The core team of consultants worked under International Business & Technical Consultants, Inc. (IBTCI), which is charged with the task of implementing the USAID/Iraq’s Monitoring and Evaluation Performance Program II Phase II (MEPP II) contract. In Iraq, the consultant team was supported by the IBTCI Chief of Party (COP), management staff and support staff who facilitated the evaluation exercise and the survey management. In addition, the IBTCI supervised survey and data collection carried out by an Iraqi consulting firm, the Independent Institute for Administration and Civil Society Studies (IIACSS). IACSS works under a sub￾contract with IBTCI. During the evaluation period, IACSS administered the survey to CSOs based in the urban and rural areas within the four operating regions of the ICSP. The IACSS expertise was included in the evaluation exercise due to their experience working under IBTCI and knowledge of the four regions where ICSP was implemented. It was also clear that the consultants could not visit all areas in the four sites. The survey was therefore designed to supplement the evaluation exercise. The Implementing Partner (IP), namely the America’s Development Foundation (ADF) assisted the evaluation team in the exercise throughout both in Washington and in Iraq by ensuring key staff in CSRCs and CSOs was available to be interviewed. A full list of the key staff interviewed is shown in Annex B. Names of the CSO members interviewed are not included for security reasons. 2.1.2. The Timeline of the Evaluation In terms of the timeline for the final evaluation of ICSP, pre-evaluation activities started in late February 2007, when the IIACSS survey team, working with IBTCI Baghdad, began the design of a sample frame and draft questionnaire for sampling 258 CSOs in the four regions. The formal activities began on March 5-7, 2007, when the three consultants met in IBTCI’s Northern Virginia office to begin planning for the evaluation exercise and the field exercise in particular. This meant planning the evaluation work plan (Annex C) for submission to USAID and the design of various open ended questionnaires based on the expectations in the SOW. March 5- 16, 2007 were spent on documentation study and meetings with USAID and ADF respectively. The team leader and team member traveled to Iraq on March 17-18, 2007 and conducted field activities in each of the four regions where ICSP is working, namely the Central, Northern, South and South Central regions. The field work was completed on April 5, 2007 and the first draft of the report was submitted or schedule by April 14, 2007. The two consultants returned to ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 6 their respective countries, namely Kenya and the United States, and finalized the report, after incorporating the survey results and addressing comments from USAID. The timetable for the fieldwork is shown in Annex D. 2.1.3. Scope of the Evaluation The purpose of this evaluation was to: • Provide guidance and, if needed, recommendations for course corrections for the current program. • Determine if objectives are being achieved in the components of the program under review. • Provide lessons learned in the event of a follow-on project. In particular, the evaluation was to focus on and make recommendations regarding: • The overall management model used by ICSP to determine its effectiveness in ensuring that strategic and programmatic objectives are being met in the most efficient and cost￾effective manner possible with particular attention paid to the regional structures that are currently in place. • The efficacy and sustainability of the CSRCs as tools to achieve the program objectives. More specifically, the evaluation was to provide: • An assessment of the training, curriculum and activities being offered through each CSRC. (The survey instrument provides information regarding all of the training programs offered - duration, location, subject - the numbers of participants attending each and a look at the curriculum.) • An assessment of the technical assistance offered at each CSRC including who is benefiting from this assistance and how this assistance is translated into tangible results. • An assessment of the grants being issued by each CSRC including a list of grants specifying purpose and amount. The assessment is expected to include feedback from the grantees regarding grant process, administration of grants by ADF, etc. • A review of each CSRC’s financial and programmatic sustainability action plan and the implementation status of the plans. • An assessment of whether the CSOs that have received assistance under the project have utilized and benefited from this assistance. It was intended that the evaluation be forward - looking, consider how the lessons learned might be applied to the ICSP activities and how they might better be integrated into the PRT system. This evaluation does not deal with the media program being implemented by ADF nor does it include a critique of the anti-corruption program. These elements were explicitly excluded from the Scope of Work. 2.1.4. Evaluation Methodology As the project is expected to be completed on June 30, 2007, the evaluation was considered to be a summative or final one in that it measures both the process and completion of the activities, including effectiveness, impact, efficiency, relevance and lessons learned. The team therefore used several methods for evaluation as shown below: ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 7 • Literature Review: An in-depth review of literature made available to the two evaluators in Vienna, VA and Iraq by IBTCI, USAID and ADF, was conducted. This data included contract deliverables and modifications, fact sheets, implementation plans, data bases on CSOs, training and grants files and reports. This basic project data was studied to enable the team familiarize themselves with the project, understand information collected and identify gaps. A full list of the literature made available to the consultants is shown in Annex J of this report. • Key Informant Interviews: The evaluators formulated a series of questionnaires designed to elicit information from the relevant people involved with the program. These were USAID, ADF, CSRC management staff and CSRC board members. The 115 questions formulated covered all aspects of evaluation such as relevance, effectiveness and impact. These questionnaires are shown in Annex E. • Focus Group Discussions: In order to explore specific issues in depth, check and verify activities and create new ideas through debate and dialogue, the evaluators conducted focus group discussions (FGD) with CSOs in each of the four regions. A total of 20 CSOs, or 100 persons, drawn from a wide range of CSOs were interviewed. • Survey: The evaluation team could not interview a sufficient number of CSOs to draw reliable conclusions regarding the impact of the training and technical assistance being provided under the ICSP program. To address this limitation, a survey instrument was designed and conducted and questionnaires administered to CSOs that had received support from ICSP. An analysis of the survey is included in Annex F, additional cross tabulations are provided in Annex G and the survey instrument itself is included at Annex H. The survey was intended to compare the experiences of program participants with those of non￾participants. Drawing from four separate databases, the “ICSP Activities Database”, “CSOs General Information Database”, “Grants Database” and the “Organizational Assessment Tool (OAT) Database”, the CSOs were sampled and linked to the activities they had participated in. In this case, only ”workshop” activities were identified, rolling up the total number of training days each CSO has received across the number of workshop activities it attended. Data were extracted from the ICSP Activities Database so that just those activities identified as ”workshop” were included. A relationship table from the CSO Information database was used to link the workshop activities to specific CSOs. The sampling frame for the survey is shown in Annex I. From the 1,847 CSOs in the CSO General Information database, 1,529 of them had attended at least one workshop. The difference between the 1,847 and 1,529, or 318 CSOs is the number of CSOs who had received no training (the non-participant group). The 1,529 CSOs with training were then divided into two groups based on the median number of days training received (this defined the two participant groups). Together they defined the three survey design groups. A random sample was drawn from each of the three groups. Based on a ”sample power” analysis, 86 CSOs were selected from each group making a total sample size of 258. The survey field work was completed by IIACSS on April 4, 2007 and data entry, analysis and presentation of results followed. From a sample list and the survey responses it was understood that 207 of the 258 questionnaires were successfully administered in each of the four regions, with the exception of two sites (Diyala and Al Anbal) in the Central region which were very insecure at the time. Of these 67.8%, of the questionnaires were completed. For 73 cases (28.3%), the enumerators were not able to find the CSOs. In 4 cases or 1.6% CSOs were not at home and in 6 cases, or 2.3%, the CSOs refused to be interviewed. These results are shown in the table below, which is extracted from Annex F. The results of the table show that there were cases of CSOs refusing to be interviewed, not at home, not found or not reached due to insecurity. By far the largest category was “CSO not found”, with cases spread evenly in the ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 8 three categories, including the non-participant group, i.e. those who did not receive assistance from the program. Table 1: Survey Participation by Strata Strata: CSOs Grouped by Workshop Training Days (strata) No Workshop Training Days 1 to 6 Workshop Training Days More that 6 Workshop Training Days Total Completed Count 56 56 55 167 Column N % 64.4% 65.1% 64.0% 64.5% Refused Count 1 0 3 4 Column N % 1.1% .0% 3.5% 1.5% Not at home Count 4 0 3 7 Column N % 4.6% .0% 3.5% 2.7% CSO not found Count 22 21 20 63 Column N % 25.3% 24.4% 23.3% 24.3% Security situation Count 3 9 5 17 prevented access Column N % 3.4% 10.5% 5.8% 6.6% Other Count 1 0 0 1 Column N % 1.1% .0% .0% .4% Count 87 86 86 259 Result of interview Total Column N % 100.0% 100.0% 100.0% 100.0% Source: Final Report ICSP Annexes, Annex F. The survey results were then incorporated in the evaluation findings where they serve to corroborate the other methodologies. 2.2. Constraints Faced During the Evaluation There were several constraints faced during the evaluation. First of all, security concerns made it difficult to meet all stakeholders and CSOs. For example, the team could not meet government officials due to insecurity. This meant the team could not administer the questionnaire for government officials shown in Annex E. It also meant the team met stakeholders (ADF, CSOs) in brief meetings kept short due to concern for the safety of Iraqi nationals and the consultants. Table 1 shows that the security situation prevented the survey team from reaching 6.6% of the CSOs. The lack of a baseline survey typically conducted at the start of a program like ICSP leaves many unanswered questions about the CSOs formation and history, which could have added to the evaluation. Finally, the consultants find that the strata selected for the survey left out the important strata of CSOs receiving grants. However cross – tabulations were developed by the US – based statistician to incorporate CSOs which received grants. (See Annex G). Finally the team determined that since the CSOs surveyed differed from those interviewed in the FGDs, some differences in results emerged as a result of the different methodology. Where there are differences, it is indicated in this report. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 9 CHAPTER 3: DESCRIPTION OF THE ICSP PROGRAM This section provides a brief description of the ICSP in terms of its purpose and contractual obligations, management and organization. 3.1. Purpose and Contractual Expectations The ICSP was started in August September 2004 when USAID entered into a $43 million contract with ADF entitled “Support for Civil Society” with a view to facilitating the growth of Iraqi CSOs and an independent media. The program which is due to end on June 30, 2007 is designed to facilitate the growth and development of Iraqi civil society organizations and an independent media9 . This work is being implemented throughout Iraq pursuant to the contract with (ADF) although in the recent past, there was also a subcontractor for the independent media component.10 There have been nine contract modifications during the performance period. Most of these involved administrative or budgetary adjustments and did not constitute a change in strategy or alteration in basic purpose. In May, 2005 modifications were made to increase projected staffing levels at the Center and at the CSRCs, attenuate several target dates particularly the deadline for submitting individual sustainability plans and delete language that required the contractor to ensure that the CSRCs were “…increasingly owned [and] has to be viewed by area CSOs as being created with their own cooperation and serving their interests.” As stated in the contract, the purpose of the ICSP initiative was to: promote an informed, sustainable and active indigenous Iraqi civil society that effectively and responsibly participates within a democratic system of governance”. This objective was to be achieved through: • The establishment (or “strategic management”) of five (later reduced to four) CSRCs. The deliverables related to this objective consisted of a series of organizational benchmarks designed to lead to and culminate in a structure of independent and sustainable CSOs. • The provision of technical assistance and training that would serve a broad range of CSOs while at the same time emphasizing assistance to CSOs working in the areas of civic education, advocacy for women and anti-corruption. The deliverables under this objective consisted of increased citizen participation in government, increased CSO capacity to educate citizens on women’s rights and measurable progress in fighting corruption. • The design and management of a small grants program to “reinforce” the training and technical assistance. • The development of a professional, independent media sector in Iraq. (Not included in this evaluation.) Important background and contextual factors that are pertinent to the purpose of the grant include: • The ICSP was to be part of USAID’s Democracy and Governance Program. 9 Note that the evaluation exercise is not directed at the media component 10 The media component of the program was sub-contracted to the International Research and Exchanges Board (IREX). ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 10 • The goal of the program was to contribute to the “development and institutionalization of a broad cadre of indigenous civil society organizations in Iraq.” • Priority was placed on the importance of establishing “… a sound management and financial plan that increases the stake of local partners in the future of the centers and their prospect for sustainability.” The CSRCs were to act as “…technical service organizations that offer a sustainable resource of ideas, organizational innovation and technical training on a broad range of issues…with increasing Iraqi ownership”. • The contract envisioned a gradual integration of local partners into the management structure of the CSRCs and increasing “ownership” of the centers by local CSOs in order to reflect community priorities. • The identification of synergies with other USAID funded activities such as the Community Action Program (CAP) and the Local Government Program (LGP) was stressed. • There was an explicit directive that “…self –sustainability be built into the program from the very beginning. Specifically, the contract envisioned that ADF “…has to quickly examine other potential resources [for the centers] including generating revenue through membership fees, developing a fee for service structure… [etc].” The Contract also included several important managerial directives that are relevant to this evaluation and to the findings that are set forth in Chapter 4. These include: • The mandatory establishment of a central Baghdad office. • A clear statement that the Directors of the CSRCs would be “…responsible for the functioning of all aspects of the CSRCs…including management and administration of the center’s personnel.” • Stress on the importance of coordination with USAID and the Embassy in order to avoid supporting CSOs with extreme or sectarian political positions. • A mandate that ADF manage the small grants program independent and separate from the management of the CSRCs. • A requirement that USAID approve the sustainability plan that would transfer final authority to the newly established centers and a requirement that USAID approve the grants manual. • Full mobilization of staff and development of an implementation plan within 30 days, establishment of performance indicators within 30 days, and the development of a “sustainability plan” for each center within 90 days. • Weekly USAID consultation, weekly activity reports, quarterly implementation plans and monthly activity reports. • Specification of hours of level of effort with consequent reporting linked to fee payment with the provision that once the level of effort has been approved, the contract is complete. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 11 3.2. Project Activities ICSP is expected to advance the democratic process in Iraq through substantial engagement of civil society in good governance activities (such as the recent constitutional and electoral processes) using CSOs as entry points. Five thematic areas or sections in the design of the program are advancement of anti-corruption; human rights; women’s rights and civic education agendas; establishment of mechanisms for an independent media; and the capacity building of ongoing civil society organizations through training and grants. Although there are no other projects in Iraq that provide the breadth, quality or type of support for civil society that is provided under the USAID ICSP, two other USG programs were providing support to good governance through programs at the time ICSP started. The LGPI program addressed good governance through district and provincial governments, and the ICAP I project worked with common interest groups on good governance and social welfare projects. Both these projects have now entered their second phases. Many of the staff working in the ICSP are originally from the LGPI project. The civil society component of ICSP focuses on building the capacity of Iraqi CSOs to be effective public actors and provides special targeted assistance to organizations working in civic education, women’s advocacy, anticorruption, and human rights. The three core activities of this component are: 1. Establishing four CSRCs staffed and managed by Iraqis, to serve as regional sites for the delivery of training and technical assistance to Iraqi CSOs. 2. Providing training and technical assistance to Iraqi CSOs in order to directly impact on Iraq’s emerging democratic processes and institutions. 3. Providing a small grants program to CSOs to reinforce training and technical assistance and support advocacy and public awareness projects and activities. Although not under evaluation, the media support component also gave training and support in three areas, namely professional media skills, media business development and media law advocacy. 3.3. Project Organization ICSP works in four regions, namely Northern, South Central, Central and Southern, where it has established four regional CSRCs (located in Erbil, Baghdad, Hillah, and Basrah) that are expected to serve CSOs (and independent media institutions until 2006 when the GOI took over the media) in all 18 governorates of Iraq. The Iraqi-staffed CSRCs are supposed to provide training, technical assistance, small grants, and consulting services to build the capacity of civil society organizations (CSOs) across the country. In addition to the institutional development support, the four sections provide specialized services to CSOs in: civic education; human rights; women’s advocacy; and anti-corruption. The ICSP activities are carried out through a team of expatriate and Iraqi personnel with skills in management, training and the subject matter.11 This includes a team in the central administrative office located in Baghdad and four CSRCs, one of which is located within the central Baghdad office complex. The program is supervised by ADF offices located in Alexandria, Virginia, USA. The central Baghdad office is comprised of approximately 35 people including a Director, Deputy Director and Finance and Administration Managers. The Central office also contains a 11 This organigram in Annex K, represents the management structure for one of the centers. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 12 monitoring and evaluation unit and six thematic units comprised of topical coordinators in such areas as anti-corruption and human rights and trainers who train in these areas. In addition to this core structure, the central Baghdad office includes four Regional Directors who oversee the activities of each of the CSRCs, which have approximately 150 personnel amongst them.12 PMPs were jointly set between ADF and USAID and the indicators presented were based on these results. The evaluators found that ADF made use of the PMPs to carry out activities and monitoring. The contract between ADF and USAID13 specifies recruitment of staff to fill three major categories, namely: key personnel; other long term personnel; and support staff for CSRCs. The three categories were expected to provide technical, and professional and administrative support to implement and monitor the three core activities of the ICSP. Therefore the structure was expected to allow for close coordination between the thematic areas (civic education, etc) and finance and administration and both top/down and bottom /up communication. There are essentially two structures: one for ADF overall management of the program, which is based on Baghdad and has overall authority in all areas, and four others in the four regions which take direction from the headquarters. 12 The role and function of the Regional Directors has been recently modified by consolidating responsibility from 4 to 2 positions with each Regional Director responsible for 2 regions. In addition, it appears that the Regional Directors will no longer be located at the CSRCs but rather have their offices in Baghdad. 13 Contract document Part V: Personnel. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 13 CHAPTER 4: FINDINGS This chapter is inclusive of all the findings of the evaluation starting with ICSP management, relevance, effectiveness, impact and efficiency. Each section in this chapter presents findings followed by recommendations. 4.1. Findings - ICSP Management 4.1.1. Management Structure The consultants note that in the first few years of the program, coordination and communication in the program structure allowed for smooth completion of the implementation activities, but over time this has changed. This is because of many factors: larger portfolio of CSOs; the introduction of the grants activities (and high demand for them); the growing demand by the CSRCs’ management for more autonomy and increasing insecurity. These factors have meant that where in the past the structure served ICSP well, this is not the case today. Efficient coordination of the program activities have therefore declined. The consultants note that there are essentially two structures: one for ADF overall management of the program, which is based on Baghdad and has overall authority in all areas; and four others in the four regions which take direction from the headquarters in Baghdad. There is a sense that with respect to strategy and policy formulation, responsibility is shared in differing degrees among USAID (as articulated in the contract and modified pursuant to changes in USAID program strategy), the ADF offices in Alexandria, Virginia, and the central office in Baghdad.14 Owing to the use of training modules replicated from the ADF Egypt office, there is also heavy reliance on the Egyptian ADF expertise with respect to training design and re￾design. The overall ICSP management structure (Annex K) is based on a traditional “hub and spokes” concept with authority and influence radiating out from a central point of direction. Ultimate policy and operational authority is located at the central office in Baghdad and delegated out to the constituent units in degrees based on a judgment as to whether or not these units (the CSRCs) have the capacity to assume responsibility. The central Baghdad office appears to exercise close operational control over the CSRCs. Some examples of centralization in the current structure are outlined below: • Work planning including course selection and scheduling is managed at the central office in Baghdad and implementation plans are disseminated to the field. Although there is interaction between the topical units in the field and in Baghdad and some feedback regarding changing areas of priority and the emergence of opportunities that only the field units are aware of, overall command of the program is centered in Baghdad. 14 The evaluation team was able to meet with the ADF President and staff in Alexandria, Virginia, although two telephone call interviews were conducted with the Vice President, and Director of Programs. The bulk of the analysis with respect to the ADF structure was done in the field. However, from field interviews it appeared that the Alexandria office required extensive reporting, close oversight and approval of new initiatives and changes in approach. Indeed, there was anecdotal indication that on occasion the Alexandria office had ignored field recommendations and substituted an alternative approach without consultation. There is insufficient information to determine whether or not these decisions were justified as there is little evidence of pre-testing of modules. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 14 • While grant applications are sent to the CSRCs and they can and do make recommendations, the final review and approval is done in Baghdad. With respect to grants, the CSRCs are not members of the selection panel.15 • Attendance at training sessions is closely monitored by headquarters with frequent input on the composition of the trainees’ groups. 4.1.2. Civil Society Resource Centre (CSRC) Management Regional Directors have been given primary authority for managing the CSRCs. The initial rationale for establishing these positions was that the CSRCs lacked the substantive and managerial expertise to function on their own. To redress this situation, the Regional Directors were to perform a staff function of mentoring, supporting and facilitating. In practice, these positions have assumed line responsibility and, until very recently, the Directors of the CSRCs have been under the managerial control of the Regional Directors. 16 While mentoring and support may indeed take place, and the skills of the Regional Directors are very apparent, the effective role of the Regional Directors is quite different than the role that was initially envisioned in the contract with ADF. Based on extensive interviews it does not appear that the CSRCs have the authority to make policy nor are they routinely expected to offer policy alternatives. Virtually all of the CSRC Directors that were interviewed for this evaluation desired a greater degree of autonomy, flexibility and ability to adapt to the local situation in their given regions. Two of the four expressed an explicit wish that the central technical offices be decentralized to the regional level. Although there are differences among CSRCs, there is a broad inclination among local staff to view the CSRC Directors as a component part of the Baghdad led ICSP program rather than staff of a local Regional entity. In fact several of the CSRC staff reported that they worked for ADF and in some interviews it appeared that staff members were not aware that the CSRCs constituted units that were eventually to become independent.17 CSRCs function primarily as training centers as opposed to CSO CSRCs. Some examples to illustrate this finding are shown below: • CSRCs have not yet been rooted in the community or structured in a way that would reflect local the needs and priorities of local CSOs. As noted below, the CSRCs are governed from headquarters. Local advisory bodies have either not been established or (with the exception of Baghdad) are not functional. • The present mechanism for tracking CSOs is valuable but it has not been used to any great extent as a mechanism to track the CSOs weaknesses and strengths or as a tool for management to change directions, institute quality control and other checks and balances, to name a few. • Using local ”experts” (drawn from advisory boards) for problem solving for CSOs due to their ability to reach grassroots level CSOs. This finding is illustrated in Figure 3 (Annex F) where advisory councils are not identified as an important source of help for CSOs having difficult problems. The major source of ”help” is Boards of Directors: 43.7% of 15 CSO applicants are quite conscious of the fact that their grant proposals are sent to and reviewed in Baghdad. Several respondents indicated unhappiness with this structure because the selection panel was not knowledgeable about local affairs. 16 Interview notes, March 21. 17 It is indicative and perhaps unavoidable that at least in Erbil, the ICSP vehicles carried “ADF/ICSP” license plates. For security reasons this was not the instance in other regions. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 15 CSOs seek them out when they have problems. FGDs conducted in the four regions also confirmed that advisory boards were mainly not in place. • Although they offer training, the CSRCs do not offer the diverse type of services normally associated with a support center. These include: a clearing house function to match the interests of donors and grant applicants; open office space with a full range of computers, faxes, copying machines, etc; a library of material on non-profit management and on the global NGO community; advisors and mechanisms to link Iraqi NGO with overseas NGOs; a sub-committee structure of local CSOs to work on such matters as codes of conduct, regulatory reform, and the passage of enabling legislation; a public affairs office and/or program to communicate the role and purpose of CSOs to the community at large; and most importantly a membership structure designed to reflect constituent interests. Table 8 (Annex G) of the survey asked CSOs stratified by training and/or grants “what is your most important source of help on management issues?” It was found that only 10.1% of CSOs sought help from the CSRCs with no differences where training and/or grants provided. In terms of type of support received from CSRCs, Table 10 (Annex G) shows that 46.7% of CSOs (77.7% where CSOs received training and grants) received support in the form of logistics, finances, organizational development and training. The survey results confirm that intended assistance was provided, but emerging needs (clearing house, open office space, etc.) are not met. • The CSRC Directors appear to have very limited operational authority to make changes in the program, suggest new initiatives, adapt to changed field conditions or directly oversee the activities and personnel of the Centers. • The existence of multiple reporting channels and the fact that staff at the CSRCs believe they report either to the Regional Directors or to one of the technical offices in Baghdad (or to both) makes it exceedingly difficult for the Regional Directors to effectively coordinate activities with the CSRC Directors.18 • Although the ostensible purpose of the Baghdad- based technical offices is to provide technical oversight, in practice the field units feel they are reporting to these central offices and under their direction.19 4.1.3. Communication Systems Communication between the CSRCs and the central office in Baghdad flows through three channels: between the finance units in Baghdad and the finance units in the field; between the technical coordinators in Baghdad and the technical coordinators in the field; and between the Regional Director in the field and the central Baghdad office. Much of this communication is ineffective due to double reporting, tendency of some communication links to bypass some staff and lack of reinforcement of communication means with more frequent staff meetings. Work planning and program scheduling is initiated at headquarters through the preparation of a three month work plan.20 This document is reviewed and approved by USAID and distributed to the CSRCs that is reviewed by the CSRC and approved by the ICSP central office In Baghdad. The consequent implementation plan lays out a training schedule by day and by topic, a technical assistance schedule by organization and a list of conferences, if these are planned. 18 To varying degrees, all of the Regional Directors desired greater authority, increased flexibility and an expanded capacity to respond to unique regional conditions and opportunities. 19 Field notes from Erbil, Basrah and Hillah. 20 The evaluation team was told in one instance that the three month work plan was developed on the basis of feedback from the CSRCs as a result of scores obtained from OATs. It was not possible to determine whether this practice was universal or to what extent feedback from the OAT influenced implementation planning. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 16 The CSRCs report back to the central office in Baghdad on the basis of this implementation plan. Internal planning meetings are held less often with respect to discussing staff issues, but frequent meetings are held to discuss the CSOs. 4.1.4. Advisory Boards All of the CSRCs have made an attempt to establish a local Advisory Board pursuant to the intent of the contract. The role, function and stage of development for individual Boards appear to differ from case to case. In Erbil, the initial attempt to establish an Advisory Board was not successful because of a perception that CSO representatives had a potential conflict of interest. In Hillah, an Advisory Board has been established and does meet on a periodic basis. In Basrah, there was an initial attempt to establish a community based Advisory Board but it is no longer actively functioning. In Baghdad, the Advisory Board appears to be relatively active. Regardless of their stage of development, the Advisory Boards do not appear to be viewed as essentially important to the overall operation of the ICSP. None of the Advisory Boards possess significant policy making responsibilities. In no case did a Regional Director indicate that an initiative had been undertaken in response to an Advisory Board recommendation and in no case did a CSRC put forward a staged plan for the development and maturation of these regional advisory structures. 4.1.5. Long Range Planning None of the four CSRCs has developed a long range plan, a transition plan or an exit plan. Although some of the staff at three of the CSRCs were aware that efforts were being made to register an Iraqi organization that could perpetuate the work of the ICSP, their knowledge of this initiative was limited and anecdotal. It did not appear to the evaluation team that there had been an organized and systematic effort to brief local staff on future directions or on the future role of the CSRCs. At this point in time, the four CSRCs appear not to have made an effort to seek funds from other sources. There has to date been no attempt to explore funding possibilities with other donors, individuals, local companies or multinational business. There has been no effort to seek funding from participating CSOs or to require fees for conference participation or workshops or to explore fee for service opportunities. It can be said that the structure has served the project reasonably well over the three year period (in terms of setting up the project especially standardizing with respect to training and grants administration) up until the last two quarters. It also allowed for Baghdad based directors and managers working under the Chief of Party to direct the four CSRCs during the implementation period. Today due to the changing context of Iraq and other factors already discussed, the structures no longer promote effective coordination and communication especially from the CSRCs to the headquarters and there is significant overlap of the ADF and the CSRCs to cause communication breakdowns. They also do not allow for regional differences related to history, socio-cultural issues to emerge or stimulate innovation to the level expected by the CSRCs. Another emerging issue is the growing insecurity, especially from 2006, limiting key personnel such as the directors and managers, from visiting the four CSRCs. Finally, the CSRCs increasingly want independence and the structure does not allow for this. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 17 4.1.6. Monitoring and Data base Systems The program has a central monitoring and evaluation (M&E) office and staff in each of the CSRCs. These gather data related to the activities of the CSOs, grants they receive, and training they have undergone. The data bases are quite impressive with respect to content which has provided the teams with much information for planning purposes and, to some degree, monitoring. This information is shared between the CSRCs and ADF Baghdad. However, the evaluators found that many of the M&E staff lack any type of training and need direction on how to analyze and use the data between departments within the CSRCs. M&E systems (mainly databases and based on PMPs) are set at the central level. Within the grants and thematic areas, officers work with M&E officers in each CSRC. This may mean field visits but in some cases, emails and documentation have sufficed due to insecurity. 4.1.7. Recommendations for ICSP Management The recommendations of the evaluators with respect to management are: • It is recommended that the ADF management structure in Baghdad be further decentralized to allow a stronger role for regional management through the centers. The ADF management structure is not well linked to the center structures to prevent bypassing of authority. For example, the thematic area staff (women’s empowerment, etc.) report directly to the regional directors or thematic area directors and often bypass the CSRC Director. Thematic managers in the centers have more authority than other centre managers such as grants managers and M & E specialists. • It is recommended that finance and administration be decentralized to allow for faster decision making. The ADF management structure has not decentralized essential functions such as finance and administration. This means there is much red tape to procure equipment and supplies, which leads to the disruption of activities. In the end, implementation plans are not always followed. It is recommended that finance and administration be decentralized to allow for faster decision making. • It is recommended that the grants sections in the regions take a stronger role and that decision making for grants (and other areas) be largely in the hands of CSRC Directors. Grants functions are situated in the less powerful finance and administration sections of the CSRCs’ structures. This means key decisions with respect to selection of grantees are in the hands of the thematic area staff who consult with ADF management in Baghdad. Although a selection committee has addressed this, grants issues beyond selection (quality control and accountability of funds) are not as well addressed. • It is recommended that ADF explore options in terms of structure (e.g. ADF Baghdad serves largely as a liaison office and technical directors be posted in regional offices), frequent bypassing of CSRCs by the ADF central office in Baghdad and by thematic section managers has caused a lot of confusion and contributed to communication breakdowns and in some cases resignations. In a few cases, this bypassing has meant inexperienced staff making management decisions which they should not make without consultation. Recently, ICSP has taken the initiative to design and put in place a long term structure that could potentially perpetuate the program once the ADF contract has run its course. While the design is in a formative stage, ADF appears to envision a central entity under the aegis of a registered Board of Directors to be comprised of members elected or appointed by the CSRCs which in turn would have their own local Boards of Directors. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 18 4.1.8. Staffing There is approximately 150 staff employed by ICSP including 14 expatriate staff. Staff can be grouped into three categories, excluding support staff. These are: Key Personnel which included the COP and Deputy Chief of Party (DCOP) amongst others; other long term personnel which include Regional Directors, Grants Manager and other managers (Training, Civic Education to name a few); and CSRC staff. The latter include the CSRC Directors, Program Coordinators, and Finance Administrators. The Evaluation Team had an opportunity to visit the key personnel and long term personnel in Baghdad, charged with managing the ICSP. They also visited the four CSRCs in Erbil, Baghdad, Hillah and Basrah and conducted background interviews with approximately 30 CSRC staff. Each of the four CSRCs has roughly the same organizational structure with a staff of approximately 45 people in each case. The CSRCs are under the management of a CSRC Director who reports to the Regional Director who in turn reports to the Director of the central office in Baghdad. The CSRCs are staffed by trainers and program coordinators grouped under six thematic areas including: women’s advocacy; human rights; anti-corruption; civic education; capacity building; and media. These clusters range in size from two to six individuals with a normal composition of three comprised of a technical director and two trainers. The largest units are normally in the area of capacity building. The thematic groups work closely with the thematic located in Baghdad and communicate directly with these groups. The Contract between ADF and USAID specifies that effort would be made to hire ”personnel with expertise working in the Middle East, preferably with Arabic and Kurdish language” From the start, the program experienced difficulties hiring Iraqis for any of the three key categories. This was explained to the consultants as being due to lack of Iraqi staff with skills as trainers and expertise in the thematic areas. Certainly, other than women’s empowerment, lack of Iraqi personnel could have been the case at the start of the program. However, the consultants noted that despite the mentoring process going on, few Iraqis made it to the top positions. There is of course, the additional factor of insecurity in light of threats on the lives of local staff who worked with USG programs. Several cases have occurred in the central and northern regions where staff has had to leave the program due to security threats. The consultants found that the staff hired for the project was of high caliber, in terms of technical skills and experience from many outside countries and several sectors. Local staff had a high level of dedication. Some local staff were former RTI staff that had benefited from civic education training, but had little experience. The majority of middle level and support ICSP staff had no background in the sector in which they were now working. Despite this, they had learned quickly by reading and discussing files and documents related to their sector. The expatriates from Egypt were particularly impressive as they had brought into the program a training curriculum which at the start was sound and largely acceptable to trainers and the CSOs. Although the majority of ICSP staff expressed job satisfaction, a few expatriate staff and some Iraqis in key and long term personnel positions expressed concern that ADF Baghdad consulted with ADF Egypt with respect to the design and redesigning of the training rather than with the staff in the ICSP. Increasingly, the CSRCs expressed the need to be consulted and to contribute to curriculum development. There were a few staff who expressed concern with regard to employment decisions, that while the CSRCs are responsible for advertising for vacant staff positions and for the construction of a “short list” of applicants, the final hiring decision is made in Baghdad.21 21 For reasons that are not entirely clear, knowledge of English is required. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 19 A second major concern of staff who were satisfied with their jobs and the few, who expressed dissatisfaction, was the need to stop mentoring and start staff development programs. They felt the latter was ineffective as it was done by infrequent visits from the Regional Director, by email or by telephone. There was also concern that some of the CSOs had better training skills than the section staff (i.e. the staff working in the five thematic areas), as they had benefited from capacity building and other training courses. 4.1.9. Staffing: Recommendations The consultants make the following specific recommendations with respect to staffing: • Staff development has to receive priority using methods other than mentoring, or if mentoring is used, more frequent mentoring in groups. Staff in-services or visits outside Iraq would also be part of the program. • More Iraqis need to be employed and undergo staff development whereby wherever possible there is greater balance between numbers of expatriates at senior levels as compared to Iraqis. • Conduct a training needs assessment for each of the regions. • Carry out workshops to revise curricula based on regional needs and suggestions. 4.2 Program Design and Relevance This section assesses the degree to which the activities in the ICSP program are in line with local needs and priorities; as well as priorities of the GOI. 4.2.1. Findings on Program Design and Relevance The consultants find that the program is highly relevant today as it was when it was first implemented since it meets the needs of Iraqis working for their communities through CSOs. Iraqis have a long history of forming self help and common interest groups. However, most of these were suppressed during the Saddam era. Upon his overthrow, and during the pre and post election periods, people developed an interest in civil society as a means to promote change and create a new way of thinking with respect to key issues such as good governance, democracy, and the empowerment of women. The consultants visited the four regions in which ICSP has been implemented and found that the program was highly acceptable to the local people. This finding is based on the positive interview results indicating appreciation for both the training and the grants as it enabled the CSOs to work in various sectors and effect public policy. This is shown in the table below extracted from the survey findings. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 20 Table 2: Policy And Decision Affected By Assisted CSOs 1. Modifying The Law Of Distributing Lands For Martyrs‘ Families 2. Women Rights 3. Elections Rights 4. Administrative Corruption 5. Security Block For Marshes Citizens 6. Job Opportunities In All Institutions 7. Assisting Orphans And Widows 8. Improving Services In The Governorate 9. Increasing Electricity Power 10. Freedom Of Journalism And Protecting Media Staff 11. Law Project Of Provincial Councils 12. Distributing Fuel 13. Improving Prisoners` Status Inside Jails 14. Granting Lands For Illegally Occupants Of Houses 15. Granting Lands For Organizations 16. Changing Water Supply Mechanism 17. Changing Amount Of Chemical Materials Given To Farmers 18. About The Decision Of Choosing Basic Committees In Reconstruction 19. The Issue Of The Security Of Female Employees` Salaries 20. Taxes 21. Giving Immigrants Their Rights 22. Leveling The Towers Of Communications Office 23. Work Style Of Trade Ministry Offices 24. Activating The Law Number 688 Which States Condemning Murdering Civilians Crimes Using Military Rules 25. Modifying The Law Of 1970 26. Modifying The Article Of Civil Society Law In The Constitution 27. Changing The Law Of Distributing Lands Regardless Of Birth Place 28. Forming A Consultant Engineering Council 29. Making Use Of Public Money 30. Improving Sewer System In The Sub District 31. Improving Salaries Of Teachers 32. About Reconstruction Projects 33. Starting A Campaign Against Expired Milk 34. To Deletion Of Paragraph 41 Of The Constitution 35. Reduce Fuel Prices 36. Resolution 137 Of The Personal Status 37. The Implementation Of The Demands For The Region 38. To Deletion Of Paragraph 41 Of The Constitution Survey results from the field survey and analysis of ICSP databases also confirm that there was a need for services for CSOs. The ICSP contributed to meeting this need. The 61% of the CSOs interviewed were established in the years 2004-2006 when the ICSP was operational (Annex F, Table 6, Figure 2). Approximately 12% of CSOs now registered on the ICSP database reported that they were established on or before 2002, another 28% were established in 2003, 27% in 2004, 26% in 2005, with the remaining 8% in 2006. Older CSOs such as the Iraq Red Crescent Society and various professional groups have been operating in Iraq since the 1950s and before. From ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 21 1991, CSOs emerged in the Kurdish North that was then in the no-fly zone, under UN development assistance, and governing itself. By 2003 there was a rapid growth of CSOs in the remainder of Iraq. This growth apparently tapered off by 2006 (based on data from the ICSP database), plus there has been some consolidation in the sector according to evidence from the CSO survey. Logically, we expect that older CSOs will have had time to accumulate more members, and Table 7 from Annex F confirms this showing that more than 50% of the older CSOs have 100 or more members. Overall, it is estimated that the number of CSOs grew from less than 200 before the invasion to as many as 2,000 by the year 2007. The survey found that number of CSOs grew from less than 200 before 2003 to a current estimated 2,000, about one-third of CSOs were established before 2004 (Annex F, Table 7). Despite the results of this table, results in Figure 1 indicate that 30.7% of CSOs learned about the concept of civil society from workshops held in their region. The survey did not verify if these workshops were exclusively funded by the ICSP, but obviously a considerable number would have been supported by the program. This means that the ICSP acted somewhat as a catalyst towards increasing awareness about civil society. The same figure shows 6% of CSOs heard about ”civil society” after visiting a CSRC; another category ”other22” comprised 28.9% of CSOs. Additional survey results related to relevance are shown below: • Weighted estimates of CSO membership suggest that there may be as many as 600,000 CSO members; • Weighted estimates of new CSO members joining in the past year exceeded 200,000 (however from a data quality assessment point of view this number has low reliability); and • A large percentage, but not a majority, of new members were women; the results also show that training had an impact on new membership. The total number of new members in the survey analysis, show that more men than women joined in the past year, but the number of new women members is encouraging. However, since the inception of the ICSP, many things have changed. First of all, insecurity is of serious concern both for expatriates and local Iraqis. Many Iraqis found to be working with Americans have been targeted, threatened and even killed. The ICSP has responded to this change by underplaying the name of the organization and its links to the USA. The field visits by key expatriate staff such as Regional Directors have also been reduced due to insecurity and the number of Regional Directors down to two from four. In many cases, workshops have been held outside the country to protect the participants. The consultants find that these changes to the program were appropriate and also in line with decentralization. The program has also evolved since its inception. The demand for CSO training and grants is very high in the four regions, far beyond what the program can support. Demands reflected in letters of request to the program indicate that there is demand for both training and grants. Tables 9 (Annex F) and 4 (Annex G) show the primary purpose of the CSO at the time of the survey. The results in Table 9 (Annex F) show the most common purpose was woman’s advocacy (23.8%) followed by human rights (22.3%). The third largest category (21.5%) where CSOs indicated purposes outside the six sectors designated by the ICSP- the largest percentage were those not assisted by ICSP (25%). This result was also analyzed by CSOs receiving training and/or grants in Table 10 and the results were similar. 22 The category ‘other’ was not explored further. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 22 There are also increasing demands by ICSP trainers on staff to redesign the curriculum and change the thematic areas to fit regional needs, sometimes stated as a need to shift into humanitarian aid. In many cases, ICSP staff have taken the initiative to re-design the curricula. There was no evidence to suggest that curriculum re-design was managed from Egypt ADF or any other Egyptian institution. CSOs are designing more and more innovative proposals to reach sectors of the society never before involved, such as secondary school children. Many young people now have an interest in forming CSOs. The evaluation team finds that ICSP has provided widespread training to approximately 1,800 CSOs. In many cases these CSOs have continued with activities, expanded and taken a stronger role in their governments. However, the vast majority have not. This suggests that the design of the program has to be changed to accommodate fewer CSOs that are capable of sustaining their activities and playing a strong role in their communities. 4.2.2. Program Design and Relevance: Recommendations. The consultants make the following recommendations to make the ICSP more relevant to the needs of Iraqis while at the same time retaining a civil society image: • A combined team of Iraqis and expatriates carries out curriculum re-design with more emphasis on emerging areas (i.e., a women’s unit to capture all thematic areas) reconciliation activities (Northern Region has already funded two CSOs in this area), and access to justice. • Identification of a linkage point (link pin) at which the ICAP II program common interest can be linked to the ICSP and CSOs and can be linked to the LGP II program. • Revival of the CSRC Boards of Directors so that they can play a strong role in ICSP which is more than advisory. The Boards can provide checks and balances to the program to ensure that there is fairness in training and provision of grants. • More focus on the staff development aspects of the ICSP so that the CSRC staff can better carry out their work with CSOs and also so they can ”market” their skills, e.g. training of trainers. This means a wider range of training opportunities for the local staff. • Management changes which are discussed in other sections of this report will also make the program more relevant. 4.3. Program Effectiveness This section of Chapter 4 deals with effectiveness at the level of the deliverables and the model used for the program. Deliverables in this case are the latest agreed upon between USAID and ADF in July 2006. As the USAID PMP was intended as a recurring point of orientation for planning23, the consultants find that they did not affect effectiveness in any way, but they did affect efficiency. Therefore the USAID PMPs are discussed under the section ”Efficiency”. 4.3.1. Degree to which Deliverables were Met The main objective of the ICSP is "to establish an informed sustainable and active indigenous Iraqi civil society that effectively and responsibly participates with a democratic system of governance’’. The rationale behind the ICSP is that by carrying out four major activities, establishing sustainable CSRCs, carrying out targeted and technical assistance, providing grants and developing an independent media, activities for good governance would follow. Indicators were set for the expected results or deliverables. These deliverables form the base for reporting to USAID in weekly and monthly reports. As the media component is not the 23 PMP 2006 page 9. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 23 subject of this evaluation, the consultants studied only the deliverables related to the first three activities; however the assumption exists that all four activities are mutually dependant on each other. Documentation for monthly and quarterly reporting is extensive and there seems to be some misunderstanding between ADF and USAID on the contract level and sub contract level of expected results or deliverables; hence variations on numbering of the deliverables. The table below shows reflects the summarized completed activities of the ICSP from July – December 2006, as submitted to USAID on April 10, 2007. Table 3: ICSP Completed Activities, July – December 2006 Activity Sector Staff Development Workshop Technical Assistance Forums Regional Conference National Conference CSO Activities CSO impacts Capacity Building 21 57 242 1 28 -- Civic Education 2 50 146 44 1 114 7 Women's Advocacy 42 206 10 180 2 Anticorruption 37 286 2 1 201 8 Human Rights 20 108 5 115 9 Independent Media 91 59 3 Total 23 297 1047 62 2 0 641 26 Source: PMP July 1- December 31, 2006. Contract Deliverables The overall contract deliverables set in the contract (GEW-C-00-04-00001-00) signed into effect in August 2004 were met by ADF. These included mobilizing and deploying key staff and long term staff within 30 days and 60 days respectively24. This was done but many staff had to be recruited from outside the country. ADF found that there were few qualified local staff at that time. The consultants concur with this view as in the current situation there is a shortage of qualified local staff. A format for performance monitoring and reporting systems for the project were set up and agreed upon with USAID as per the original Deliverables 2-7, which reports have generally been timely and accepted by USAID. Annex L shows the latest PMP and expected deliverables. The consultants refer to the latest agreed upon deliverables as these reflect agreed upon changes to the PMP. Below the consultants examine each deliverable and make findings based on the expectations of the contracts and the revised PMP: 1. Establishment of 4 Civil Society CSRCs (Indicator 1.1.): In terms of deliverables it was expected that CSRCs would be established by the program, staffed and managed by Iraqis to serve as regional hubs for the delivery of training and technical assistance resources to Iraqi CSOs. The design was expected to offer a full range of capacity building and other resource support services to the CSOs. CSRCs were also designed to become self sustaining 24 Deliverable #1 ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 24 institutions, providing services that enable the growth and development of Iraq’s civil society sector. • The CSRC’s have been established in the four regions, although the one in Baghdad is less visible as it is inside the same compound as the ADF/ICSP central office and therefore shares resources. The CSRCs are staffed by Iraqis but managed by the program which has mainly expatriate senior staff. The design offers a full range of capacity building training but less training was conducted in the thematic areas than for capacity training. Still, the CSOs found that capacity building training responded to their needs. Other services such as library and general consultation are not fully in place. Finally at this point the CSRCs are not self sustaining as the design allowed for the setting up of the CSRCs did not promote enough local ownership to allow for sustainability (active local boards, membership fees are some examples of what was not done. • According to the CSOs interviewed in FGDs, some existed before the ICSP started, but the program’s presence (in established CSRCs) and legitimacy (accepted by the GOI) stimulated the formation of many new CSOs. The survey results and analysis of ICSP databases confirm that the CSRCs were established as expected in that they deliver a full range of capacity building and other resource support services (grants) to the CSOs. Some important results are: o 46% of CSO survey respondents said that they had received some type of support from the ICSP/CSRC. o 74% of the CSOs said that they had visited the CSRC during the last year, but this was not statistically different across the control groups (strata). The analysis of visits by CSOs to CSRCs based on training and/or grants showed where CSOs had not yet received assistance (training or grants) they visited the CSRCs more than when they had received assistance – 42.4% of CSOs that had not yet received assistance had visited the CSRCs seven times in the last 12 months (Annex G, Table 11). o CSRC staff had visited the sampled CSOs an average of three times in the last 12 months with an average of 10 visits in a 12 month period where both grants and training were provided. This result confirms FGD findings where CSRC grants staff stated they supervise CSOs that receive grants. o When CSRC staff visited the CSOs there was a positive impact on the number of advocacy events undertaken. 2. Design of CSRC Staff development plans to develop a cadre of highly qualified and experienced Iraqi trainers and facilitators (Indicators 2.1. and 2.2). The plans were expected to include staff performance standards, evaluation mechanism and staff training programs that upgrade the professional skills of ADF staff in management, training, service delivery, administration, research, M&E, quality control, public relations, information technology and networking. • The consultants find that the staff in all four centers are highly experienced and dedicated to their jobs; most were not qualified when they entered into employment and many remain academically unqualified. This can be attributed to the challenge in finding qualified local Iraqi staff. ADF developed a cadre of highly qualified and experienced Iraqi trainers and facilitators, which were able to deliver a wide range of training courses. The staff performance standards were obviously high; however, many of the trainers were not from the CSRCs. In fact, CSRC trainers expressed concerns that some external trainers had received more training that CSRC staff especially in the area of ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 25 facilitation (TOF). As a result of the TOT and TOF training, opportunities for income generation have arisen and CSOs and government have gained income. The CSRC trainers remain concerned that the program should have focused more on CSRC staff. Secondly, the evaluators found that the staff in the other departments (Grants, M&E) received little training, and this was done on an informal basis with a few exceptions. For example, in every region M&E and grants staff had done self training by reviewing documentation as most of the 38 staff development activities were using mentoring. The consultants note that the ICSP training program refers to staff training on 32 technical areas to better meet the needs of the CSOs but not enough to meet requirements of ISRC staff such as M&E and grants. The table below extracted from ICSP PMP reports to USAID shows training workshops conducted by region during the project life. The results show how the IP paid attention to training in each of the regions, including provinces where insecurity is very high. Table 4: Training Received by CSOs at Workshops Training Events Males trained Females trained Total trained Percent that regions are Region Province of total Count Sum Sum Sum Percent Central Anbar 8 133 34 167 Baghdad 82 1,022 622 1,644 Diyala 6 52 104 156 Salah Al-Din 2 39 10 49 Sub-Total 98 1,246 770 2,016 18.7% North Dahuk 23 246 109 355 Erbil 31 499 179 678 Ninewa 23 314 140 454 Sulaymaniah 17 192 123 315 Tameem 27 308 163 471 Sub-Total 121 1,559 714 2,273 21.1% South Basrah 66 845 273 1,118 Missan 32 430 142 572 Muthanna 31 362 161 523 Thi-Qar 43 466 271 737 Sub-Total 172 2,103 847 2,950 27.4% South Central Babil 67 650 472 1,122 Karbala 36 322 342 664 Najaf 35 327 350 677 Qadissia 32 337 282 619 Wassit 24 267 192 459 Sub-Total 194 1,903 1,638 3,541 32.8% Total Total 585 6,811 3,969 10,780 100.0% Source: ADF Data base on training Results from the survey confirm that extensive training was conducted: • 22% of the CSOs said that they received most of their training from other institutions. • 47% of CSOs said they attended one or more conferences last year. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 26 • The combination of training events, workshops and conference attendance meant that on average CSOs attended 20 to 21 events last year; this may be taxing their ability to do advocacy. • A total of 1,529 CSOs attended workshops where they received training.25 • Approximately 24% of CSOs in the survey said they received most of their training from other institutions. These included RTI and other international NGOs. 3. CSRC training materials and resources are developed, adapted, validated and constantly upgraded (Indicators 2.2. and 2.3). ADF has region-specific approaches to CSO training delivery and tailors services to support democratic changes in Iraq. • The consultants found that capacity building workshops were held and that these workshops were used by the CSOs to improve their management skills. A total of 22 training modules were designed by ADF-Egypt and adapted for Iraq. This activity was mostly done by Egyptian expatriates and local Iraqi staff found them to be useful and acceptable to the CSOs. For the most part this deliverable was well met owing to the positive response of the CSOs and the quality of the training materials. Local Iraqi staff in ADF CSRCs are now in the process of re-designing the modules. There is some concern that a re-design plan should be set up to ensure the training programs meet evolving regional needs such as emerging gender concerns. • The survey findings suggest that the management training provided under the ICSP does not appear to be directly correlated with an increased level of advocacy work or with the perception by CSOs that they have greater influence on the outcome of public policy decisions.26 This is an important finding because it suggests that generic training is not sufficient to energize CSOs to engage in advocacy work nor is this type of training directly related to a perception of positive results. 27 It is important to emphasize that this does not mean that management training is devoid of value or that Iraqi CSOs have not benefited from this training. Indeed, CSOs that did receive training indicated a higher level of self confidence and empowerment than those that did not. Nor does this finding mean that specific and concentrated training in advocacy techniques is not effective since this course category was only a small part of the overall ICSP curriculum. (Less than a third of those CSOs interviewed for the Survey had taken a course devoted expressly to advocacy.) 4. The Organizational Assessment Tool (OAT) is developed and implemented in every CSRC (Indicators 3.1., 3.2. and 3.3.). The OAT is intended to be used as a management tool during capacity building training. • The consultants find that this deliverable was partly met as not all the CSOs trained and supported with grants received the OATs. They note that the tool is similar to other global models and has the potential to develop CSOs in the four key areas of internal governance, advocacy, general management and financial management. However, the consultants find that the tool was not administered to all CSOs who received capacity building training or to all grant recipients. The tool was also not used in a participatory 25 Based on an analysis of the ICSP Activities database from January 2007 linked to the ICSP CSO Information database 26 At this time demonstrations of impact are weak, but not entirely absent. The impact of workshop training seems limited, but technical assistance delivered through ICSP staff visits to the CSOs often shows significant impact. 27 A possible explanation for the absence of correlation may be that there has been insufficient time for the training to take hold. Another explanation may be that the situation within Iraq has deteriorated so that advocacy work is less likely to succeed than was the case three years ago. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 27 manner where CSOs get the chance to do self evaluation. For the OAT to be effective, CSOs need to know their results and be able to challenge them. Grant recipients may or may not have had an OAT done and so far the tool has only been administered once, whereas the recommended number is twice per year. It is important to note that OATs were intended to identify the capacity of CSOs in key areas related to civil society. ADF used this information to identify training needs and grants recipients. The tables below shows sample OAT scores for Advocacy, General Management and Financial Management in the four regions. Table 5: OAT Advocacy Scores by Region OAT Score Advocacy Region Mean Median Count Central 38 42 86 North 31 30 108 South Central 19 15 149 South 25 25 126 Total 27 26 469 Table 6: OAT General Management Scores by Region OAT Score General Management Region Mean Median Count Central 35 36 86 North 22 18 108 South Central 13 10 149 South 20 15 126 Total 21 17 469 Table 7: OAT Financial Management Scores by Region OAT Score Financial Management Region Mean Median Count Central 35 39 86 North 32 33 108 South Central 20 19 149 South 21 20 126 Total 26 24 469 Source: ADF data base on OATs 5. CSOs receive assessments (OATs) prior to receiving any ADF assistance and get a re￾assessment every six months before receiving any assistance (Indicator 3.4). The intention with this deliverable was for key CSOs from all the governorates in the four regions to receive training in a respective thematic area such as women’s advocacy, civic education, etc. The skills received would then be applied in workshops and public conferences where CSOs could meet with government and others. The deliverable implies that over time (not specified) the process of blanket training would be decentralized so a few CSOs would have sharpened skills and teach other CSOs. • The evaluators found that this deliverable was partly met in that CSRC staff visited CSOs (or vice versa) to conduct OATs. In some cases the OATs were followed by capacity building training, general training in one of the five thematic areas and grants. Generally, this was not always the case as some CSOs received OATs after their ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 28 training or not at all. At the time of the evaluation, CSOs in the database had received a single OAT, but additional OATs had only been carried out for a few CSOs. • The financial management component of the OAT paid limited attention to measures to address the low scores of most CSOs with respect to fundraising (which was part of the financial component of the OAT). This means data was collected in OATs but little was done with the results. The evaluators find that the OAT results should have been considered more when giving out grants (i.e. a CSO scoring low in fundraising would need special attention to this issue during capacity building training). • Where OATS were carried out, they were generally followed by capacity building or a given workshop or conference for the public. This was found to be appropriate with the exception above. Training was highly appreciated by the CSOs who were then able to apply what they learned in the training to community activities Some CSOs were paid for giving out training services and this income helped sustain their organizations. However, the decentralization process (where fewer CSOs would provide community training) is still not completed. The CSRC staff remains unconvinced that this is the best approach as they also want to market their training skills. • CSO staff members said they visited the CSRCs a total of 19,000 times in the past one year.28 • ICSP/CSRC staff paid some 6,000 visits to CSOs in the past one year.29 6. Foster and support the development of CSRC satellite centers owned and operated by qualified CSOs (Indicator 3.5.). • The consultants find that this deliverable was not met despite efforts in all four regions to carry it out. Four to five CSOs with training skills, acquired from ICSP were identified and agreed to be satellite centers for the community CSOs. For some months, this was successful. But for the most part, CSOs were not able to sustain the activity as they did not have office equipment or internet to communicate easily with each other. Other than their ”blessing” ICSP did not provide them with any support. Therefore, few satellite CSOs are able to support other CSOs. 7. CSRC business plans are in place (Indicator 4.1). The consultants were not provided with business plans. 8. CSRC advisory boards are in place to guide development of sustainability plans and actions (Indicator 4.2.). • The consultants find that this deliverable is only partially started. First of all, the Advisory Board is not fully defined and it is not clear how it will work with the CSRCs. This is difficult as the CSRCs are also not independent of ADF Baghdad. Secondly, the consultants note that there is a shift in thinking to have a central institute representative of all four regions. This is one area the CSRCs seem not yet to have agreed upon. Therefore the consultants find that the current sustainability plans have not involved Advisory Boards to the extent expected. 28 Based on weighted CSO survey results. This number is not verified through documentation. Data quality analysis is low. 29 As above. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 29 9. ADF training staff and key staff are trained to deliver quality services in the five thematic areas and gender and gender advocacy (Indicators 5.1, 5. 2, 5.3, 6.1, 6.3, 8.1.and 8.2.). • The evaluators find this deliverable has been met. Other than the ADF training staff wanting to take a stronger role to sustain the CSRCs, CSOs have received the training and applied it. 10. Key CSOs organize advocacy and awareness campaign on civic education targeting CSOs and the public specifying actions that public can take to be participatory (Indicator 5.4, 6.4. and 6.5). • The evaluators find this deliverable has been met as confirmed from interviews and the survey exercise. Only a quarter of the CSOs interviewed were involved in advocacy activities. Still, CSOs feel more empowered due to the training and support they have received. Many have held advocacy campaigns and are more involved with government bodies. • Women’s advocacy was identified by 23% of CSOs as the primary purpose of their organization. This was the largest single percentage among the six ICSP sectors. • The most important skill that CSOs sought to acquire was fund raising, followed by communication skills and advocacy. These are skills that can be taught by ICSP. • An analysis that predicted the number of advocacy events found that the ”number of times an ICSP/CSRC staff member visits a CSO” is the strongest positive factor in predicting increased advocacy events; while the number of conferences attended had a modest positive effect. Workshop attendance and visits by the CSO to the ICSP have no effect on the number of advocacy events.. • 44% of CSOs said they petitioned government officials or local leaders more than five times regarding the improvement of local services. The number of petitions were not differentiated across the control groups. More training apparently did not lead to more petitioning. • Successful petitioning was not found to relate to any of the capacity building activities collected in the survey. • The extent to which local government and local leaders took into account the concerns of CSOs was not related to the control groups; however there is a modest positive association with visits that ICSP staff make to the CSOs. • 55% of CSOs reported that they had no or limited influence to change policy or decision making with elected officials or advisory councils, and there was no significant difference across the control groups implying that workshop training did not improve this outcome. • CSOs saying that they did have influence provided good examples of policies that they have influenced; this is a very positive outcome. • Half of the CSOs said that they had been able to mobilize community resources to conduct campaigns; this was not related to the control groups. CSOs that were able to mobilize resources petition government more often were more successful at it. • CSOs have been good at partnering with other CSOs to work jointly for a common cause. Although there are not significant differences across the control groups, CSOs who visited the CSRC joined with other CSOs more often (perhaps as a result of networking). ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 30 • Table 25 (Annex F) illustrates that the average number of times CSOs facilitated a community forum or campaign in the last 6 months was 6.27 times. The average number of appearances before local government councils to advocate for a cause was 2.12 times, while the total number of appearances before local government departments was 1.77 times. • Table 29 (Annex F) shows that 29.1% of the CSOs perceived they had influence over routine community issues. This confirms the findings in the FGDs and key stakeholder interviews. Table 14 (Annex G) shows that 50% of CSOs perceived they had influence over important community issues. In Table 1 (Annex F) ”influence over important community issues” was compared for CSOs which had received training and/or grants. There was no difference in the results. • Table 32 (Annex F) and Table 16 (Annex G) examine the area of petitioning government officials and political leaders. The largest category of CSOs responding to this question (43.7%) stated that they had petitioned more than five times. There was no significant difference whether training or grants were provided or not, indicating likely influence may have been other factors such as the OAT exercise, which was found to be empowering, or attendance at conferences. • Table 17 (Annex G) shows similar positive findings. The largest category (33.1%) had occasional influence on government who took into account the concerns of CSOs when they made decisions. Table 35 (Annex F) shows there was less influence on local councils where the largest category (31.7%) stated they had no influence, while 25.2% had a limited influence. Table 18 (Annex G) confirms this result and shows no difference based on training and grants. • Tables 37 (Annex F) and 19 (Annex G) show that 49% of CSOs had mobilized resources for advocacy campaigns. • Table 20 (Annex G) shows that CSOs face obstacles to conduct campaigns, mostly due to lack of financial resources (58.8%). This finding confirms FGDs where CSOs expressed concern that funding for workshops was not enough as demand by the public to attend the workshops was often double the expected number. 11. Small grants are awarded to CSOs engaged in civic education activities gender advocacy and other thematic areas (Indicators 5.6. and 8.6). • The evaluators find that these deliverables have been met and that small grants were administered across the board to CSOs in the governorates of the regions. In some cases, security prevented the ICSP from giving out grants equally as some governorates could not be reached. There was a high demand for small grants and those who received them were able to benefit from several cycles. This enabled them to expand their activities and gain recognition in their communities. The evaluators find that the success of this deliverable is largely due to the grants system of selection and monitoring, which although not foolproof, is designed to protect the funds and ensure they go to CSOs who will ”deliver”. FGDs suggested that grants were instrumental in the CSOs conducting activities and reviving the CSOs where they had been inactive. 4.3.2. Effectiveness of the ICSP Model This section will discuss the current ISCP model with reference to its effectiveness in achieving the objectives set forth in the contract agreement with ADF and with regard to evolving USAID objectives. The first part of this discussion will examine the model in its entirety and the second part will focus on the CSRCs. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 31 The ADF approach is derived from a judgment that civil society in Iraq is in its infancy and that CSOs need immediate assistance in virtually all aspects of management and program planning and implementation. While the ADF approach does include areas of program priority, the predominant emphasis is on offering a large menu of training and technical assistance in order to build a broad based civil society sector.30 Important attributes of the current ICSP approach include the following: 1. Centralization and Governance from the Top • Management, policy setting, work planning and employment decisions are centralized at headquarters with implementation in the hands of the CSRCs. • The menu of ICSP capacity building programs is developed at the central office in Baghdad and is based on central judgments regarding the technical and managerial needs of CSOs. While individual CSRCs can make recommendations regarding the courses that they will offer and the courses that be designed and developed within a curriculum established and maintained at the central office. • The content of each training course has been pre-established and training materials are provided from the central office in Baghdad to the constituent regional units. Centralized quality control is further sustained through the establishment of central office - based coordinators who maintain the curriculum and provide training to their counterparts in the field. • The selection of functional areas of priority such as anti-corruption and women’s advocacy were specified in the contract and a roughly similar organizational structure is used in each CSRC to administer these areas of priority. Technical oversight and quality control are based at the Baghdad central office and significant changes cannot be made without their approval. • CSRC Directors have not yet been given the authority to manage and direct the CSRCs. Regional Directors serve (until very recently) essentially in a deputy role to the Regional Director. Reporting flows up to headquarters through at least three funnels and by￾passes the CSRC Director. 2. Common Format and Templates • ICSP applies a roughly similar approach to each of the four CSRCs. Systems, procedures, curriculum design, program content are designed at headquarters and are basically identical for each CSRC. • The approach to capacity building is highly structured and sequential. It is based on a model that has been used in Egypt and draws on training materials initially designed for Egypt and adapted to Iraq.31 The training strategy employs a baseline survey of organizational capacity and competence, the selection of training programs from a common menu to address deficiencies and a repeat of the baseline survey to measure growth in organizational capacity. This model is employed at all CSRCs and the evaluators confirmed that while in many cases the capacity building results were used to 30 Indeed, this emphasis is wholly consistent with the central purpose of the original contract which stated that: “The objective of this activity is to promote and informed, sustainable and active indigenous Iraqi civil society that effectively and responsibly participates within a democratic system of governance. This will be achieved in part through the strategic management of 5 civil society CSRCs that will offer basic training and technical assistance in leadership, management principles and the financial skills necessary to operate effective and sustainable CSOs”. 31 Adaption of the Egyptian model to Iraq included references to the gender and political context of Iraq and case studies specific to Iraq. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 32 identify ongoing training needs and grants provision, in many other cases grants were given without capacity building training being done. The evaluators find that this should not have been the case. 3. Broad Based Approach • The ICSP program is broad-based and inclusive and designed to reach the largest possible number of CSOs in the shortest possible time. Selection criteria for eligibility for training, technical assistance and grant assistance appear to be simple and easily met. Participation has included a wide range of organizations from individuals to groups with paid staff of 50 or more. 4. Diversity of Training Material • The range and diversity of training courses, particularly in the area of capacity building, is very extensive. The capacity building curriculum includes training in almost all core areas of management. 5. The use of conferences and topical workshops to focus on a particular issue • Although the ICSP approach is “top down” and the training is formatted and pre￾established, the use of periodic conferences focused on special issues does allow the program to reflect local interests and needs. Indeed, the selection of conference topics appears to be quite substantively rich. Although the evaluation team was not able to attend a conference, anecdotal remarks from staff and CSO leaders appeared to validate their relevance. FGDs confirm that attendance in the conferences and topical workshops increased advocacy where the primary purpose of the CSO was advocacy. This was not confirmed however, in the evaluation survey. 4.3.3. Factors that Shaped the Current Model The strengths and weaknesses of the current model are discussed below. In this regard it is essential to keep in mind the driving factors that shaped the approach and the strategic assessment that was made when the program began. • The necessity for speed. The goal of establishing five (then four) sustainable CSRCs and training enough CSOs to establish the outline of a legitimate civil society sector with expertise in several priority areas in war-torn Iraq was immensely ambitious.32 These optimistic goals necessitated a central command structure, a template approach and the prior existence of a workable model that could be quickly lifted and adapted to the Iraqi situation. • The weakness of the non-governmental sector. There is broad agreement that the non-governmental sector in Iraq was in a state of institutional infancy at the time the ICSP was initiated. Very few CSOs existed. There was virtually no tradition of western style charitable giving. The practice of grass roots activism that might have developed was stifled by an authoritarian government. While there may have been deep roots of charitable giving and a rich community life that are imbedded in Islam, the family structure and tribal traditions, these had not emerged as an independent political force or as an adjunct to the growth of a democratic system. This situation has an important implication. Because the demand for training, support and assistance was very limited a centralized or “top-down” approach was necessary in order to stimulate this demand. 32 One of the challenges faced by the project is the absence of a definition of “civil society” and what this Western term might mean in the Iraqi context. Without this modifying and clarifying consensus, there is an inevitable tendency to think of civil society through Western eyes and Western standards. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 33 • The desire to create sector-wide competence. The strategy that was chosen by USAID, and that is embodied in the contract, was to initiate a process that would stimulate a sector wide competence as opposed to strengthening the ability of a limited number of organizations. This approach appears to be predicated on the assumption that civil society would emerge spontaneously if given training, technical assistance and modest funding and that a sector wide movement was necessary in order to support and complement local government decentralization. • The absence of a baseline survey or needs assessment. While the absence of this analysis was not a causative factor, the lack of information about the sector may have limited the capacity of the project design team to incorporate elements that would have shaped the ICSP in alignment with Iraq’s unique situation. In view of the immense pressure for speed and the deteriorating security situation this is indeed a forgivable omission. • The use of a contract as opposed to a cooperative agreement. For reasons that are not entirely clear at this later stage, USAID chose to employ a contract rather than a cooperative agreement. This may have reflected the importance of speed, a desire to concentrate on a specific and very limited number of deliverables and a judgment that flexibility was less important than tangible and immediate results. Right or wrong, this approach tended to stimulate rigidity with respect to both management and implementation of the ICSP, necessitate central oversight, require a template approach with little room for adaptation, and a command structure with little delegation. As one staff member remarked, “the approach was driven by the oppression of the deliverable.” • Inherent weakness in oversight and monitoring and reporting due to the security situation. The inability of USAID to personally monitor at the field level and the risk of distorted data and information necessitated the design of a delivery system that would provide tight centralized control. 4.3.4. Strengths of the Current Model The advantages of the model employed by ICSP are straightforward and include: • Speed, broad outreach, and the establishment of relatively uniform standards for organizational performance. • The uniformity of approach and the use of OATs will eventually provide an extremely valuable data base for donors and a mechanism for measuring the maturation of civil society and by inference the development of voluntarism and citizen participation. • Although imported from another country, the Egyptian training modules had been proved effective and did not require re-design from the ground up. (Indeed the absence of a needs assessment or a baseline survey of civil society in Iraq would have made this difficult to do in any case.) • A centralized model is one that will facilitate access to additional donor funding whether from USAID or other sources. • Centralization of finance, administration and M&E facilitates effective oversight, accountability and organizational learning. A centralized system may be better able to capture lessons learned and convert these into changes in approach. • The approach used was fundamentally in line with the objectives set forth in the contract and consistent with a desire to jump start a civil society, if in fact this was feasible. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 34 4.3.5. Weaknesses of the Current Model The primary disadvantage of the ICSP model are: • A “one size fits all” approach to training may not reflect the individual needs of different organizations. • Centralization of grant making neglects an opportunity to engage community leaders in the identification and resolution of local issues. • In general, a structured and centralized approach is less adept at creative response to opportunities and a nuanced understanding of local issues and needs. • A centralized approach makes it more difficult to establish a community constituency for the CSRCs and to engage local CSO leaders in the governance of these Centers. 4.4. Impact of the ICSP The evaluators find that the ICSP has not had significant impact on the communities it serves in terms of effecting significant social change. This is attributed to the lack of measures taken towards sustainability. The program was expected to provide training to more than 1,000 CSOs followed by grants. This type of approach where many CSOs are assisted spreads benefits quickly, but these are not sustainable. The program would have had more impact if its approach had addressed sustainability and assisted a small number of CSOs. The critical assumptions for the survey was that impact would be measured by the following: • The number of times they have facilitated a community forum or campaign; • The number of times they appeared before local government elected or advisory councils; • The number of times they appeared before local government departments; • The influence they believe they have had to change or influence policy decisions; • Whether the organization has mobilized resources from its members or from the community to carry out an advocacy campaign; • The number of times they have worked closely with other CSOs to advocate for a common cause; • How often in the past 12 months they have petitioned government officials or political leaders for improved services and the result of those petitions; and • The perception of empowerment that CSOs have in making the community a better place to live. Based on these indicators, the program could be assessed as having positive impact. However, the analysis of data does not link the capacity building (training and grants) as directly responsible for the results. The lack of a baseline survey further complicates matters as there are no benchmarks on what a successful civil society program comprises. The survey results do suggest that the ICSP, as it is configured in the current social and political context, will have limited impact on the ability of CSOs to advocate effectively for policy change unless advocacy is given a greater priority, the extraordinarily difficult security situation is addressed and other areas explored, or a variety of other reasons. The primary implication for USAID is that if the importance of advocacy is to be elevated, the overall program needs to be re-designed to reflect this emphasis. A secondary implication is ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 35 that it may be difficult to bundle generic training, which is designed to jump-start the emergence of civil society with more focused efforts to empower voluntary organizations with the skills and motivation to engage in advocacy. While class-room training relying on standardized models may be useful in teaching such skills as financial management it appears that this approach is less effective in helping an organization learn how to engage in governmental policy making.33 Indeed, one related finding from the Survey was a modest correlation between advocacy activity and direct and personalized visits from ICSP staff where it would have been feasible to develop a tailored approach and provide hands on support and encouragement.34 The CSO Survey validates anecdotal reports that suggest that an active and vigorous Iraq civil society has emerged in the last three years. If this would have occurred in any case after the war and whether and to what extent the phenomenon is directly attributable to the ICSP program is difficult to determine however the ICSP constituted the most important intervening variable and certainly deserves some of the credit. The data validating the emergence of civil society in Iraq is impressive: • The number of CSOs grew from less than 200 before 2003 to a current estimated 2,000. • Weighted estimates of CSO membership suggest that there may be as many as 600,000 CSO members. • An estimated 14,000 Iraqis (37% of them female) served in an official capacity with CSOs last year. • An estimated 200,000 new Iraqi members were recruited to join CSOs in the past year.35 • A large percentage, but not a majority, of new members were women and women’s advocacy was identified by 23% of CSOs as the primary purpose of their organization. This was the largest single percentage among the six ICSP sectors. • Although the level of advocacy activity was not correlated with training provided by the CSRCs, the absolute rate of involvement of CSOs in advocacy work appears to be quite high. • Almost half of the CSOs that were interviewed said they had petitioned government officials or local leaders more than five times regarding the improvement of local services. 36 33 A regression analysis that used five independent variables to predict the number of advocacy events likely shows that the ”number of times an ICSP/CSRC staff member visits a CSO” is the strongest positive factor in predicting increased advocacy events; the ”number of training courses attended” appears to have a negative impact; while the ”number of conference attendances” has a modest positive effect. Workshop attendance and visits by the CSO to the ICSP have no effect on the number of advocacy events. 34This discussion touches on a larger issue inherent in the design of virtually all civil society projects. On the one hand, there is an argument that civil society has an inherent value and that a legitimate USAID objective is to establish and nurture the civil society sector for its own sake. The alternative view is that civil society only has value to the extent that the activities of NGOs or CSOs accomplish desirable results. The first perspective constitutes the philosophy that provided the basis for the original contract with ADF. The second perspective constitutes the philosophy that tends to emerge over time as USAID searches for concrete outcomes and results. 35 Weighted results from the survey data. There will be some double counting possible when persons join as new members in more than one CSO. The individual numbers reported by CSOs were not verified with documentation. Data quality analysis rating is low. 36 It is difficult to assess the effectiveness of advocacy work from the view-point of CSOs. On the one hand, 55% of CSOs reported that they had no or limited influence to change policy or decision making with elected officials or advisory councils. On the other hand, CSOs do not appear to be deterred by these challenges and report a very high effort level. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 36 • Some 40% of CSOs joined with other public institutions to advocate for a common cause during the past six-months • Although more than a third of the CSOs interviewed do not feel they could influence public policy, the remaining organizations that argued that they did have an influence were able to identify and be quite precise regarding the positive results that they had accomplished. This suggests that a significant number of CSOs are targeted and professional in their advocacy work. The CSO sector in Iraq appears to be going through a healthy period of consolidation and maturation as indicated by the following results: • From 20% to 25% of CSOs in the survey were not found and believed to have closed or merged or consolidated with other CSOs. This is a relatively high drop out rate and is consistent with the evolution of the civil society sectors in other countries. • CSOs have been good at partnering with other CSOs working jointly for a common cause. Although there are no dramatic differences across the control groups, CSOs who visited the CSRCs appear to have engaged in networking and have partnered more frequently with other CSOs. • A surprisingly large number (42%) of CSOs reported that they received fees from members. If correct, this suggests a strong and fairly well developed membership base and a significant degree of community outreach and support. • When faced with a management problem, most CSOs look first to their Board of Directors for support and guidance. This finding suggests that Boards perform a valuable role and that they are informed and constructive participants in the activities of their organizations. • CSOs appear to possess a relatively high level of concern for the substantive quality of their work and listed technical help as the second most important management challenge. The CSO Survey validated the fact that the funding base for the emergence of civil society in Iraq is narrow and fragile and based heavily on the personal commitment of volunteers as indicated by the following results: • Fund raising was seen, by a wide margin, as the single most difficult problem facing CSOs. • The development of fund raising skills was identified as the single most important competency that CSOs wanted to improve. • The most important current source of revenue was seen as personal contributions. Corporate, small business or foundation giving was a virtually non-existent source nor was it anticipated that these traditional sources of support for the CSO sector would emerge. • Looking ahead, CSOs believe that local government will constitute the most significant source of future income. While not altogether a negative finding, the fact that CSOs believe that local government will eventually support their activities may be unrealistic, however, at the same time they are at odds with the principle of independence and autonomy so critical to the viability of the sector. • On a positive note, a surprisingly small number (7%) of CSOs believe that future funding will come predominantly from international donors while a relatively large number (23%) ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 37 believe that future funding will come from their own ability to generate revenue. This sense of self reliance compares favorably with other countries where the CSO sector has been fueled by and becomes heavily dependent on the largesse of the international donor community and where the self-generation of income is rare. The perceived importance of the CSRCs is quite positive and the absolute magnitude of the work that emanates from the Centers is very high. However, there is some evidence that the high level of CSO participation in CSRC activity is episodic and perhaps transitory as shown below: • About 1,500 Iraqi CSOs have participated in some sort of training workshop since the program was initiated. Aside from any judgment regarding quality or impact, this is an impressively high number and at a minimum illustrates the potential power and influence of the CSRC structure. • Regardless of whether they had received training, roughly three – quarters of all respondents said that they had visited a CSRC during the last year. This suggests that the CSRCs may have a perceived intrinsic value that goes beyond the training programs that they offer. • The reported level of interaction between the CSRCs and regional CSOs is significant: extrapolating from Survey results, CSO staff members have visited the CSRCs a total of 19,000 times in the past one year while Regional Center staff have visited participating CSOs a total of 6,000 times. While this data is not completely reliable it does suggest an impressively large volume of activity. • 47% of CSOs that were interviewed said they attended one or more ICSP sponsored conferences last year. • There appears to be a potential reservoir of interest and willingness among CSOs to provide financial support to the CSRCs. Some 73% of CSOs said they would be willing to make a contribution towards future training programs and 65% of CSOs said they would be willing to make a financial contribution to sustain the work of the CSRCs. While this finding needs to be viewed with considerable caution, it supports a theme in this evaluation that emphasizes the importance and feasibility of local ownership and the necessity of decentralizing authority and governance to the local CSRCs. • For those CSOs that take advantage of the resources offered by the CSRCs the level of participation is very high: when training sessions, workshops and conference attendance are cumulated, last year the average CSO attended from 20 to 21 events. This finding has mixed implications. On the one hand it suggests that the CSOs deeply value the programs and support that are offered by the CSRCs. On the other hand, the level of participation seems to be abnormally high which suggests a lack of selectively and perhaps a perfunctory attitude toward training and conference participation. While difficult to determine, CSOs may be attending these courses and events solely because they exist in a routinized manner and perhaps because they believe that training will be positively related to the prospect for receiving grant support. Some 43% of CSOs said they had petitioned for improved services to the community (44% said these petitions were successful or mostly successful). 4.5. Efficiency This section deals with the efficient use of resources, both financial, human and material. The consultants were not able to come to any conclusions with respect to financial and budgetary ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 38 considerations as the budgets were not provided to the evaluation team. These would have given the team some perception of the priorities of the program and the degree to which financial inputs went to the CSOs. 4.5.1. Budget The current budget through to June 30, 2007 is $60,880,157.00. Summary budget allocations are shown in the table below. An additional $2 million was provided by the International Narcotics League (INL) in September 2006 and used for the anti-corruption program. It is understood that these funds were used for grants and a national conference to address illegal narcotics. Some gaps with respect to budgeting identified during interviews are: • The budget for M&E is too low to allow for enough field visits; • There is no budget for curriculum revision which is now necessary; • The thematic sections do not have enough budget for field visits; • Libraries received funding but are poorly equipped; • Budgets for CSOs are occasionally over estimated or under estimated. In some cases, the turnout for workshops and conferences is much higher than budgeted for; • CSOs in rural areas need more support to make their budgets; and • ADF-Baghdad exerts high control over any budget expenditures; this takes time to process and activities get delayed. The table shown below indicates the present operating budget of ICSP: Table 8: ICSP Operating Budget CLIN 0001 & CLIN 0003 – COMBINED BASE SERVICES & OPTION 2 August 16, 2004 through June 30, 2007 CLIN 0001 & CLIN 0003 COST US$ General Program Expenses $16,008,389.00 Civil Society Centers & Program $16,925,476.00 Civic Education $4,456,614.00 Anti-Corruption $7,114,677.00 Women’s Advocacy $6,460,980.00 Media Assistance $5,942,287.00 Sub Total $56,908,423.00 Fee $3,971,734.00 Cost Plus Fixed Fee $60,880,157.00 Source: Amendment of Solicitation/Modification ADF Contract with USAID 27.09.07 4.5.2. Efficiency – ICSP and the USAID Performance Structure The results framework briefly outlined in an earlier section of the report has a number of characteristics that may have complicated consensus on the goals of the civil society ICSP and made it more difficult to reach agreement on whether or not the Program was effective and successful. It is not that there is a direct contradiction between the USAID results framework and the project objectives but rather that there is a lack of clarity and occasional ambiguity regarding primary goals and areas of emphasis. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 39 Inevitably tensions of this sort arise in virtually every contracted USAID activity. This is particularly true in the case of civil society projects where there is an inherent conflict between the value of building civil society simply for the sake of having that sector in place and building civil society in order to accomplish a specific policy objective. However in Iraq, because of the difficulty of field observation, the complex bureaucratic structure and the intense pressure to achieve results, the consequences of ambiguity are more serious. These difficulties appear to have been exacerbated by the fact that the procurement instrument was a contract with the rigidity inherent in that instrument rather than a cooperative agreement which would have encouraged thematic adjustment as USAID priorities evolved. Tension points include the following: • A lack of clarity regarding the appropriate balance between promoting advocacy on the one hand and building the generic institutional capacity of the civil society sector on the other. While it is certainly the intent that both objectives are valid and to be pursued, ambiguity with respect to which is more important can be problematic because the approach taken in each instance and the measurement of success is quite different. • A similar tension between thematic areas of concern (such as women’s issues) on the one hand and a broader effort to establish civil society as a vibrant sector on the other. • The uncomfortable location of the central thrust of the civil society program under Strategic Objectives 9 and 10, improved capacity of national governments, since civil society is inherently or at least initially a local phenomenon. • The difficulty of finding solid indicators that could gauge the vitality and long term sustainability of civil society. Understandably, the emphasis in the current Results Framework on immediate, tangible results and indicators such as progress on organizational test scores would appear to be appropriate. But these short term measures are easily biased and may miss or neglect the more important factors such as the emergence of a culture of philanthropy that, in the long run, will be essential if civil society is to thrive in Iraq. • Tension between a short term set of objectives focused on results and social stabilization and longer term emphasis on supporting the emergence of an enabling environment and a structure of attitudes that would enable the emergence of a durable structure of civil society organizations. As developed in the balance of this evaluation, these points of ambiguity although tolerable in most instances have in this case created an underlying and sometimes incorrect impression that the ICSP is off target, moving in the wrong direction or inconsistent with other USAID priorities. 4.5.3. Efficiency - The ADF Contract Taken in its entirety, the initial contract sets forth a coherent and balanced vision of what was to be accomplished and what was to be left in place at the completion of this activity. Underlying the complexity of the technical language, the themes of sustainability, institutional competence, focused advocacy, local ownership and democratic participation are clear. The consultants make several observations with respect to the efficiency of the ADF contract: • As initially envisioned, the strategic model implicit in the contract involved two points of intervention. The first was to build capacity from the top down through the development of CSRC’s, the provision of training taken from an Egyptian model and a centrally managed grant program. The second was to build up local ownership by developing sustainability plans that reflected community needs, creating local governance councils ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 40 and gradually decentralizing authority to the CSRCs. In retrospect, these two points of intervention were contradictory, at least in the time frame envisioned in the contract. • In the best of worlds, the target dates for activity completion were wildly unrealistic. In particular, the plan to establish, staff, operate and be in a position to develop a long range sustainability plan for the four CSRCs within 60 (later 90) days of inception was impossibly ambitious and likely to force precipitous decision making. • Presumably, because of time constraints coupled with the assumption that previous work by other implementing partners had established an understanding of civil society in Iraq, the contract did not call for a civil society sector survey or for a needs assessment that would identify institutional deficiencies. • The use of the contract format, coupled with an emphasis on time-specific deliverables appears, in retrospect, to be inconsistent with the type of capacity building work that was to be carried out under the contract. The extraordinary volatility of the Iraq situation, the inherent difficulty of measuring impact and the consequent necessity of continual flexibility and adaptation suggest the preferability of a cooperative agreement. • In general, the extent and depth of detailed USAID oversight appears to have created a managerial burden that has forced and perhaps legitimized the establishment and maintenance of a powerful central office in Baghdad. Underlying the initial contract is the philosophy of decentralization, local empowerment and local “ownership”. By implication, this type of management structure requires delegation of authority with the center acting in a facilitating and mentoring manner. • The inclusion of a fee for payment linked to completed level of effort is inconsistent with the inherent nature of the work. While effort expended may be a good measure of intent, it is a poor measure of impact and would appear to establish a set of incentives that encourage activity at the expense of substance. 4.5.4. Efficiency: Recommendations The consultants make the following specific recommendations for Efficiency: • Budgeting expertise has to be better developed in the thematic areas so that they can better work with the grants section when studying proposals by CSOs. • Revision of some budget procedures to decentralize spending and speed up procurement and spending at CSRC levels. • The hiring of more Iraqis in place of expatriates will greatly reduce personnel expenditures. This means a process of recruitment of Iraqis with attractive packages has to be introduced, alongside more staff development. • It is recommended that USAID modify its Results Framework to better accommodate a civil society program. Although the Evaluation Team does not have specific recommendations on the content of this revised structure, Section 6.0. in this report identifies ambiguities that need to be addressed in order to provide a coherent and logical structure that will provide guidance to implementing partners. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 41 CHAPTER 5: SUSTAINABILITY This chapter of the evaluation addresses the difficult subject of Sustainability.37 While sustainability is an important component of subjects addressed throughout this evaluation, we deal with it here in a consolidated fashion because of its central importance to an assessment of the ICSP model and because it is critical to the long term success of any effort to strengthen civil society in Iraq. 5.1. Contractual Obligation – Sustainability The original contract with ADF placed very heavy emphasis on sustainability. • “The objective of this activity is to promote an informed, sustainable and active, indigenous Iraq civil society…. • The contractor is required to staff and operate five sustainable civil society resource centers…. • The contractor has to increase the stake of local partners in the future of the Centers and their prospect for sustainability. • Each Center…will have a distinct path to community ownership and sustainability. • The Centres need to be increasingly “owned” by the leading CSOs…The ultimate goal is that the Centers become sustainable over time. • The Contractor has to initiate a USAID/Iraq-approved sustainability plan that transfers control of the Centers to five Boards of Directors…. • From the outset, emphasis has to be placed towards developing institutional sustainability….the contractor has to quickly examine other potential resources….” In the May 2005 modification to the contract, the emphasis on sustainability was reduced with introduction of the following clarifying language together with a number of similar modifications through the balance of the document: “It is recognized, however, based upon past experience that it is highly unlikely that the centers will be self sustaining after a mere 18 months. The point is for eventual sustainability to be built into the program from the very beginning and that the contractor lay the appropriate foundation to encourage that sustainability.” Despite the attenuation of the goal of sustainability, the emphasis on local ownership was retained: 37 The word “sustainable” is overused and misused and perhaps for this reason was not defined in the contract nor to the knowledge of the evaluators in subsequent dialogue between ADF and USAID. The word can be employed to refer either to an individual organization, a group of organizations such as women’s rights groups or an entire sector, in this case the civil society sector. Normally there are two types of sustainability: organizational sustainability and financial sustainability. The former refers to such things as the quality of leadership, the existence of systems and procedures, the effectiveness of governance and, most importantly, the ability of an organization or a group of organizations to establish and nurture a constituency. The latter implies a continuing capacity to raise funds and by implication a diversified funding base and normally a limited and declining reliance on USAID for support. The concept of sustainability as it is applied to non-profit organizations or sectors do not mean “self sustaining” nor does it mean “perpetually sustained”. By definition, no non-profit organization is “self sustaining” since it has to seek voluntary contributions and no social purpose non-profit would be “perpetually sustained” beyond the life of the issue they are attempting to address. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 42 “The responsibility for the management and operations of the civil society resource centers need to be increasingly taken over by the CSOs in each region.” The sustainability plans [required of ADF] will be developed with the advice of and input from the respective Informal Advisory Council of each of the… [CSRCs]. 5.2. Sustainability Plans An initial “Status Report and Sustainability Plan” was prepared for each of the four CSRCs in August 2005. These documents consisted of a description of the structure and operation of the CSRCs, a “SWOT” analysis of strengths and weaknesses, a vision statement and an implementation plan. These documents outlined, in a preliminary way, an illustrative vision for the CSRCs based primarily on input from the Regional Directors. They were similar in nature, quite general in content and prepared by Baghdad headquarters staff. These initial planning documents did not pretend to be comprehensive or reflect unique local needs. A caveat in each of the reports noted: A definitive vision developed by Iraqi staff for each of the ICSP-funded CSRCs has not been fully developed. [However] in consultation with Iraqi CSRC staff, Iraqi general program staff, and with a sampling of Iraqi citizens from communities targeted by the centers, elements of a generic five-year vision for the centers has surfaced…. In addition, it was emphasized that: The focus during the first year of the Iraqi Civil Society Program has been to establish functioning CSRCs in four key regions of Iraq. During the next 18 months, a general objective of ICSP is to launch a program of concerted and focused initiatives that will render the four CSRCs self-sustaining from a managerial, programmatic and financial point of view, and lay the foundation for replication by identifying and strengthening the capacity of Key CSOs to serve as service providers to a broad section of Iraqi civil society organizations throughout Iraq. Detailed CSRC sustainability plans will be integrated into flexible and master ICSP planning documents, which will serve as a monitoring and quality assurance tool to ensure sustainability imperatives and milestones under each sustainability initiative are achieved. Although these plans were generic in nature they did identify four critical imperatives that needed to be addressed in order to promote sustainability of the CSRCs: • Strengthen Internal Management/Finance/Administration/Governance: Sound management and governance are a prerequisite for sustainability. Strengthen Capacity Building Program: Essential to sustainability is the investment in human capital and product. • Market CSRC Products and Services Delivery: Involves opening a market research and development unit that develops and implements a marketing strategy to improve delivery of CSRC products and services. • Strengthen CSO Enabling Environment: Addresses the need for the CSRC to contribute to legal and policy reform by engaging in increased public awareness, advocacy, and coalition building, and promoting democratic values. A second and more definitive set of sustainability plans was prepared and submitted to USAID in March 2006.These plans involved a phased approach with increasing degrees of decentralization. The plans contained the following key elements: ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 43 • An emphasis on staff development with a special emphasis on developing the managerial performance of senior management. • Gradual decentralization of authority for local implementation from the ICSP main office to the CSRCs with established benchmarks to be achieved before each further step is taken. • Establishment of regional Boards of Directors that are representative of the religious, ethnic and gender diversity of the region and the development of by-laws and other materials required for registration. • Assistance to Board Members to help them develop their own internal governance, policies and structure. • The opening of business development and marketing units at each CSRC to develop fee-for-service models. • Exploration of the marketability of a certification program for CSOs based on the OAT. • Provision of staff training on business development and marketing, the management of consultancy services, negotiation, budget development and the running of a business center. • Improvement in the CSRC public Image by developing media programs, a logo, brochures, documentations, presentations and websites. • Development of detailed implementation plans for each region involving phased delegation and the drafting of business plans for revenue-producing activities. These sustainability plans were rejected by USAID on the basis that they were generic in nature and did not reflect local differences. Following this rejection, ADF employed an outside consultant to help them address the issue of sustainability. A draft report was prepared in June 2006 and a meeting of Regional Directors was held in July in Erbil. This meeting was designed to think through alternative sustainability models and to reach agreement on a plan of action to respond to the mandate in the contract. The Director’s retreat produced consensus around a sustainability model for the ICSP program with the following attributes: • Unification of the ICSP program as a single entity under the direction of a central Baghdad office and four CSRCs with the understanding that the Baghdad CSRC and headquarters office may be accommodated in one building. • The registration of a new national entity to be called the Iraqi Civil Society Institute (ICSI). • The establishment of a central “General Assembly” to govern the new entity to be comprised of ICSP staff, members of advisory boards and carefully selected civil society activists. • Centralized accounting, revenue generation, human resource, information technology and database management. • Standardized CSRC staffing, management and organizational structures. • The establishment of a transitional period of nine months through June 2007 in order to effect the transition from ICSP to ICSI. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 44 • The transfer of furniture and equipment to the new organization. • The employment of a senior local staff member to handle the establishment of the general assembly and the designation of a financial officer to handle financial matters. 5.3. Sustainability of Civil Society in Iraq Regardless of the laudable goal of sustainability whether organizational or financial, the Iraqi context is deeply problematic for the following reasons: • The CSO sector has not yet established the habits and practices of seeking financial support from individuals, business or foundations. • While the CSOs that were interviewed for this evaluation have an emphatic vision and a strong sense of social purpose, very few yet manifest an understanding of how to identify, build and nurture a constituent base of support that could provide the energy and financial support necessary to sustain the work of the organization. • In talking with many CSOs, there is a pervasive and counter-productive view that Iraqi society will not contribute to social purpose organizations and that funds can only be obtained from international donors. • Repeatedly, CSO leaders argued that they did not have the capacity to approach individual donors or to raise money in the community because their constituents and members lacked funds. • There has been virtually no attempt to raise funds from the business community and CSOs believe that this is currently infeasible because of negative economic conditions and the lack of a tradition of business giving. • While a few CSOs have initiated income generating activities such as a hair salon in a women’s group, the incremental funds from this source are limited and the time and energy needed to operate these initiatives may deter the group from work more directly connected to their social purpose mission. • Very few CSOs are of a size or level of technical skill to contract with local government. • CSOs are reluctant to approach the mosques for charitable support for fear of being drawn into sectarian disputes. • Although funds are available from the political parties, these resources inevitably draw the recipient into the political arena. • CSOs appear to have a mindset of growing reliance on the international donor community despite the fact that most of them understand that the donors will not be in Iraq forever. However, on the positive side, the following factors suggest that sustainability is possible: • There is a deep humanitarian and philanthropic tradition in Islam that in the long run may provide the attitudinal structure necessary for charitable giving to flourish. • There are roughly two million Diaspora Iraqis that could be cultivated for purposes of supporting Iraqi-based social purpose groups. • When and if stability returns, multi-national firms can be expected to supply significant amounts of community based giving. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 45 • Most of the CSOs that were interviewed for this report have a group of founding members that contribute considerable time and energy to the work of the CSO. While this is rarely monetized, it does illustrate the potential for voluntary giving in the future. • In a few cases, CSOs with a membership base receive a small monetary contribution from their members. 5.4. Role of CSRCs in Sustainability In the context of these challenges, and in addition to providing training, CSRCs can be instrumental in taking on the additional role of building a sustainable civil society sector for reasons stated below: • CSRCs can help validate the role of civil society, advocate on behalf of members and help them achieve their objectives. • CSRCs can act as important sources of public information regarding the role and activities of CSOs and in this way address public concerns regarding the utility of these organizations. • CSRCs can act as a broker between the donor and the CSO by identifying areas of common and by helping applicants prepare proposals. • CSRCs can help legitimize the CSO sector by establishing codes of conduct and by disseminating standards of performance. 5.5. Sustainability of the ICSP Program Although the ICSP has been successful in many ways, the goals related to sustainability have not been achieved. This is in part because the objectives set forth in the contract were impossibly ambitious and in part because sustainability planning was not immediately initiated. In addition, there were differences of interpretation with regard to the intent of the contract and how much responsibility ADF had to insure sustainability of the CSRCs and the ICSP program. Some findings of the evaluators are: • Aside from the CSRCs themselves, which for all intents were training centers, the contract – design contained no emphasis on cultivating philanthropy in general. There was an underlying premise that civil society was blossoming and that training was needed. The foundational need for the attitudes and institutions needed to support durability of the sector was not addressed. • The absence of a baseline civil society survey and the fact that a careful needs assessment was not performed helped perpetuate the idea that CSOs had the inclination and capability to reach out to others for financial support. The fact that there was virtually no competence in fund raising and constituent building and that aside from external donors there were no sources of charitable funding was not known and therefore ignored. • A sustainability plan for the ICSP program in entirety has not been put in place. Although the structure for a new entity that might house the program (the “ICSI”) has been recently registered with the Ministry of Civil Society, it is very unlikely that this entity could be functioning by June 2007, when the contract is due to end. • Even if the new “ICSI” was functional by June 30, the model that has been developed has some serious defects as outlined below: ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 46 o The centralized structure is not a model that is likely to lead to “local ownership” or to the establishment of CSRCs that respond to the unique needs of the local community. o The approach appears to rest on the untested premise that continued funding would either come from USAID or from other international donors who would prefer to work through a centralized system. Thus the model does not address the challenge of long term sustainability that would eventually be rooted in the community o The approach relies on unrealistic expectations regarding the ability of CSRCs to generate funds through fee for service activities. Experience from other countries indicates that if the provision of support services to NGOs is an income generating venture, commercial organizations will enter the market with an equal or better product. • ADF has not developed an exit strategy or a transition plan keyed to the termination of the project on June 30, 2007. While the registration of the ICSI is a step in that direction, the detailed work that needs to be done in order to preserve the investment of energy and financial resources has not been put in place. As a result, the CSRC infrastructure is at serious risk of collapse once ADF leaves in June of this year. 5.6. Sustainability of the CSRCs ADF has expended a considerable amount of effort on the development of individual sustainability plans for the CSRCs. However, the evaluators make the following observations: • A local governance structure has not been put in place and there is very little evidence of local ownership or community participation in the life of the CSRCs. Advisory Boards either do not exist or are at an adolescent stage of development. • Long range planning for the creation of a sustainable group of CSRCs appears to have been done primarily at headquarters without participation from the staffs of the CSRCs who were best equipped to comment on local realities. • The ICSP model and approach (as mandated in the contract) is problematic with respect to the establishment of autonomous and financially viable CSRCs.38 • The reluctance to delegate authority to the Regional Directors has handicapped these individuals and prevented them from developing creative approaches that would reflect community priorities. • The ICSP model is very effective at delivering training support and in quickly enhancing the organizational capacity of a large number of CSOs in a short period of time but it is not effective if the goal is to establish a sustainable structure of local, indigenous organizations. This is because of the following: o There is an inherent contradiction between the top down delivery of training and the bottom-up building of local ownership. The former implies a command structure and headquarters control, the later implies a gradual process of listening to the needs and aspirations of community organizations. o The CSRCs, and in particular the CSRC Directors, lack the authority to adjust their programs to respond to community interests and concerns. 38 As noted elsewhere, the CSRCs are essentially training centers rather than being broad based and reflect the diverse and unique needs of individual communities. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 47 o The imposition of a “one size fits all” model is inconsistent with developing a sense of local “ownership”. o The centralization of grant making and the absence of peer review at the local level neglects a rich opportunity to broaden and deepen community participation in priority setting and program design. 5.7. Sustainability Planning USAID and ADF did not define what is meant by sustainability and to address the fundamental contradictions in the ICSP model that have made decentralization difficult has seriously hampered efforts to develop sustainability plans that are realistic, workable and responsive to the intent of the contract. Neither USAID nor those implementing the ICSP have attempted to clearly define sustainability, to identify the attributes associated with sustainability or to develop benchmarks to measure organizational or sectoral progress toward sustainability. The lack of clarity regarding this pivotal objective has allowed the ICSP to proceed in a vacuum of naiveté with the result that much of the investment that has been made to date may be lost in the future. From staff interviews and from an IRG report on sustainability that was prepared in August 2006, it is clear that there were, and still are, important differences of view regarding ADF’s responsibility to ensure that the CSRCs are sustainable.39 It is unfortunate that these differences of perspective were not addressed and resolved through dialogue between USAID and ADF.40 39 USAID Iraq Civil Society Program: Sustainability Summary Assessment of the Organizational Sustainability of Iraq Civil Society CSRCs. September 2006 40 As noted in the IRG Report: “The concern that emerged [at the Erbil Workshop] and remains unclear is ADF’s institutional responsibility, mandate and policies regarding the centers’ sustainability. Significant questions were: Does the responsibility of ADF’s contract extend beyond submitting 4 individual sustainability plans for each centre? What are ADF’s contractual obligations concerning its SOW with respect to the introduction of possible new organizational models?” The language in the [contract modification] has been diversely interpreted as meaning that ADF has to deliver “sustainability plans” for the CSRCs on the one hand, or as having to actually establish one or more independent Iraqi NGO(s) out of the CSRCs, on the other. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 48 CHAPTER 6: THE ICSP PROGRAM: THE PROS AND CONS OF ALTERNATIVE FUTURE DIRECTIONS This chapter of the report examines future directions and discusses the advantages and disadvantages associated with various alternatives. This discussion is designed to stimulate thinking, identify important trade-offs and focus attention on the critical importance of agreement on objectives. The chapter is divided into the following sequential sections: • Background considerations. • The identification of alternative models. • A discussion of the factors to be considered in evaluating the pros and cons of alternative approaches. • The pros and cons of alternatives. 6.1. Background Considerations The ADF contract will terminate on June 30, 2007. Whether or not it will be extended and for how long is not known at this time. There is no exit strategy currently in place. Although a new entity, the ICSI, has been registered, there does not appear to be agreement between USAID and ADF as to whether this entity will carry on the program and what shape that program, if continued, will take. While ADF has engaged the Regional Directors in planning discussions and has hired a consultant to facilitate a planning process, consensus with USAID has not been reached nor have other alternative donors been successfully identified. For reasons that are unclear, neither the central Baghdad headquarters staff nor the CSRC staff appears to be deeply concerned about this dilemma. Local staff and the CSRC Directors have limited and differential knowledge of the implications of contract completion and the new Institute model. None of the CSRCs have taken steps to put in place the structures, competencies or governance mechanisms that will be needed to continue operations if the program is discontinued. 6.2. The Identification of Alternative Strategies In this context, there appear to be three alternative Program strategies that could be employed. Identification of alternative strategies was not part of the SOW but the following are offered as possible options for the Mission to consider as they continue to develop strategies for civil society in Iraq. Each option would imply a different management and organizational structure. In addition there is the fourth option of terminating the ICSP program pursuant to the terms of the contract. Aside from the termination, all of these approaches would necessitate additional funding from USAID or from another donor. This is not an exhaustive list of options on moving forward. However, if the Mission chooses to retain the support center methodology as an activity, these options could apply: These four options are described briefly below: 1. Terminate the program as scheduled on June 30, 2007. Under this approach, ADF would immediately prepare an exit plan designed to preserve the accomplishments of the program. In the absence of an alternative, an urgent effort would be made to operationalize the governing structure for the new ICSI, establish by-laws and conduct ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 49 an initial board meeting. A concerted effort would be made to locate and secure other donor funding so that all or part of the Program could continue. 2. Continue the current program approach. Under this approach the program would continue to be focused on the generic strengthening of civil society with emphasis areas in women’s rights, anti-corruption, etc. Generalized sector wide training, provision of technical assistance and reliance on the OAT rating system would continue. The organizational structure would continue to be centralized and could be housed in the newly established ICSI. (A modification of this approach would be to continue the current program for approximately 18 months in order to ensure a second round of organizational assessment and provide the basis for a more focused program as outlined below.) 3. Shift from generic capacity building to focused support for a limited number of CSOs in priority areas. Under this approach, ADF would select a limited number of CSOs (12 to 24) for concentrated support based on priority subjects such as conflict resolution, reconciliation, women’s rights, anti-corruption, etc. Selection could be either closed or competitive and based in part on OAT performance scores. Collaborative organizational assessments would be done and training, technical assistance and grant support would be tailored to the individual needs of the chosen organizations. The roles and functions of headquarters and the CSRCs would be adjusted accordingly. (A modification of this approach would be to sustain the current program for approximately 18 months in order to ensure a second round of organizational assessment to better inform the selection process and solidify the work of Phase 1 of the ICSP program.) 4. Place primary focus on the establishment of strong CSRCs. Under this approach, a sustained effort would be made to establish CSRCs rooted in the community. A needs assessment would be conducted in each region to identify the range and types of services that a CRSC would provide if it were to adequately serve its constituents. The current CSRCs (i.e. Training Centers) would be individually re-structured to reflect this new approach and an effort would be made to identify revenue generating activities and membership structures that might in the long run provide a sustainable source of income. The CSRCs would remain under a central umbrella during the transition period but would become ultimately independent. A concerted effort would be made to identify and cultivate a group of community leaders who could function as members of a Board of Directors. (A modification of this approach would be to add a media/public relations component so that CSRCs could also function as information sources to educate the public regarding the role and function of civil society in Iraq.) 6.3. Factors to be Considered in Evaluating the Pros and Cons of Alternative Approaches The evaluation team found that there are four important questions that need to be taken into account by USAID in evaluating the pros and cons of these different approaches: • Purpose. Is the primary purpose to build the broad capacity of the civil society sector or is the primary purpose to have a programmatic impact in a particular area of high importance? • Feasibility. In view of the nascent state of civil society and the high level of instability and insecurity in Iraq, is the approach doable within the constraints of the USAID system and current resource limitations? ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 50 • Sustainability. Will the approach yield results in the short term and at the same time put in place structures or competencies that will constitute the basis for long term development of Iraq? • Complementarities. Is the approach consistent with and complementary of other USAID and/or Iraqi development projects and are there potential synergies that can be exploited? 6.4. The Pros and Cons of Alternatives 6.4.1. Terminate the program as scheduled on June 30, 2007. (Modifications might include a 3, 6 or 9 month no-cost extension.) Advantages • Constitutes a frank recognition of the immense difficulties of building civil society in a war-torn country that lacks the institutions and traditions needed to establish CSO institutions. • Encourages a re-examination and re-articulation of the USAID Results Framework to more clearly position a civil society initiative. • Provides greater latitude for USAID to design and put in place an alternative program that is more clearly in line with current USAID priorities. • Recognizes that generic capacity building of CSOs has been partially accomplished and that a potential CSRC structure has been put in place. • Feasibility and further concern with sustainability are not issues. Disadvantages • Unless other donor funding becomes available, the CSRCs are likely to collapse. • Terminates the capacity building program and the organizational assessment mechanism in mid stream and voids the opportunity to benefit from the completion of this effort. • Terminates support to a small number of important organizations that depend on ICSP and risks their collapse. • Removes a headquarters and regional structure that could be effectively utilized in a second phase of the civil society program. (The loss of highly competent Iraqi staff would be exceedingly unfortunate.) 6.4.2. Continue the current program approach Advantages • Allows completion of the organizational assessment process and establishes a valuable data base that will allow delivery of targeted assistance to CSOs that require assistance. • Retains valuable local Iraqi staff and permits continued staff building and mentoring support to the CSRCs. • Is a strategy most consistent with an approach that would merge the current program into the already established ICSI. • Is feasible and avoids the risks of adopting a new strategy. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 51 Disadvantages • Risks ignoring internal contradictions in project design and management difficulties encountered to date. • Maintains a centralized structure that discourages CSRC autonomy and long term sustainability. • Continues a “cookie cutter” approach to training and technical assistance at a time when the civil society sector is become more differentiated and mature with differing needs. • Does not address important ambiguities in the USAID results framework. 6.4.3. Shift from generic capacity building to focused support for a limited number of CSOs in priority areas. Advantages • May build a cadre of strong, sustainable CSOs that will constitute a replicable model for other emerging groups. • Is the approach most likely to have a significant impact in emphasis areas such as anti￾corruption and women’s rights. • Is the approach that would permit experimentation in the development of a group of CSOs that would concentrate on conflict mitigation and reconciliation. • Constitutes an approach that is most consistent with USAID’s results planning framework and that would ameliorate some of the conceptual difficulties inherent in a civil society program. Disadvantages • A precipitous program re-direction would sacrifice the benefits associated with completion of the current program strategy. • Might jeopardize the role of the CSRCs before these organizations can cultivate a local constituency and move toward local ownership and sustainability. • Re-design would probably require a hiatus in current program activity with consequent costs. 6.4.4. Concentrate primary and focused effort on the establishment of four strong CSRCs. 41 Advantages • If successful, this approach is likely to have the most significant long term impact on the development of civil society in Iraq. • Capitalizes on the initial investment in the establishment of a fabric of CSRCs that has already been made • Likely to retain the valuable services of local Iraqi staff. • Provides significant opportunities for synergies with other USAID projects including the CAG's and RTI’s local governance efforts. 41 This is the approach which is most consistent with the original scope of work for the Project that envisioned the creation of 4 distinct entities that would be locally “owned”. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 52 Disadvantages • The creation of a truly decentralized and community based structure will be difficult to administer, manage and monitor. Accountability could be an issue. • The long-term sustainability of CSRCs is uncertain and will depend on the viability of the civil society sector as a whole. • Results are difficult to measure and do not fit easily into the current USAID results framework. • This strategy requires a significant alteration in project direction. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 53 CHAPTER 7: CONCLUSIONS The following conclusions can be drawn from the findings: 7.1. Program Effectiveness 1. With the exception of the area of sustainability, measuring staff performance, evaluation mechanism and conducting staff training programs (TOF, M&E, TOF), ADF has done a very good job in complying with the deliverables set forth in the initial and subsequently modified contract. 2. The overall ADF model was effective in the rapid delivery of training and technical assistance support to a large number of Iraqi CSOs in a short time period. 3 The application of a training curriculum drawn from another country was appropriate and effective in view of the requirement for urgency and quick impact. The training programs are well designed and well taught to meet the needs of trainers 4 The organizational tool is an effective device for measuring organizational capacity and identifying strengths and weaknesses. 5 A valuable data base of information (which if drawn upon more can identify the strengths and weaknesses of individual CSOs) has been developed. 6 There is early but persuasive evidence that the organizational capacity and institutional maturity of those CSOs that have received managerial support has improved. 7 The grants program has supported worthy projects but has not been linked to the training program as originally envisioned. 8 As a result of the ADF program, a nascent civil society sector appears to have been stimulated in Iraq. 9 A small but important number of highly competent and effective CSOs, particularly in the areas of civil and human rights appear to have emerged. 10 The four CSRCs that have been established have the potential to make a significant contribution to the further development of civil society in Iraq. 11 The ICSP has not come close to constructing a sustainable structure of institutions that could function in an independent way. 12 Adaptations in the design of the training curriculum needs to be transferred to the CSRCs to reflect unique local needs 13 The gains that were made are in jeopardy because neither a transition nor an exit plan has been prepared. 14 The survey indicates the ICSP has had some positive impact with respect to advocacy activities. The overall conclusion of the evaluators is that positive impact was low, but positive outcomes are apparent. 15 Grant activities supported advocacy and other activities which would otherwise not have taken place. 7.2. Program Efficiency 1. The targets and time lines set forth in the original and subsequently amended contract were excessively ambitious. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 54 2. The use of a contract procurement mechanism was an error because it focused attention on whether or not deliverables were being produced instead of developing an adaptive program strategy. 3. Ambiguities in the USAID Results Framework coupled with deterioration in communications with ADF have hampered project implementation. 4. Program implementation could have been smoother if ADF and USAID had agreed on a clear, operationally relevant agreement on the meaning of “sustainability”. 7.3. Management 1. The local and expatriate staff of the ICSP is competent, motivated and professional. 2. The ICSP management structure is too centralized and has not delegated authority commensurate with the growing competence of the CSRCs. 3. Poor communications between ADF and USAID have hampered creative adaptation of the ICSP to changing local circumstances. 4. Retention of the Regional Director layer beyond the point of necessity has prevented the CSRCs from gaining the autonomy they need in order to move toward sustainability. 5. ADF’s confusing internal communication system has created tensions that have undercut morale and hampered progress. 6. Effective regional Advisory Boards have not been created in large part as a consequence of the top down, centralized nature of the ICSP. 7. The Evaluation Team found that the content of the reports provided as reference materials matched the findings in the field in terms of accuracy and being a true reflection of the program. 8. The newly established PRT structure is fully compatible with the goals and objectives of the ICSP. There is no inherent structural conflict. 7.4. Sustainability 1. The primary obstacle to creating a culture of philanthropy in Iraq is to change the attitude of Iraqi CSOs so that they will pro-actively seek financial support and begin to slowly cultivate community donors. In building civil society in Iraq or in any other country, the first and most difficult step is to begin asking for support regardless of how large that amount may be. While certainly some of the Iraqi CSOs are adept at writing grant proposals as a result of their ICSP training, none that were interviewed believe that they will be successful if they go out into the community and make a case for the importance of the work they do. This negative attitude needs to change. 2. Lack of momentum toward the establishment of a sustainable structure at the national and local level constitutes a serious project failure when measured against the objectives set forth in the contract. 3. Both USAID and ADF have been derelict in their disinclination to grapple with this deficiency, to define what is meant by sustainability, to recognize contradictions in program design and to make mid-course corrections.42 42 To be balanced, the Evaluation Team recognizes that in view of the nascent state of philanthropy in Iraq, it is doubtful that much progress could have been made in this short time in any case. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 55 4. The ICSP was designed and implemented in a way that was anathema to the goal of establishing sustainable, locally “owned” CSRCs. 5. An immediate and high priority is to put in place an implementation plan that will at least optimize the prospects that the CSRCs will be able to continue to function. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 56 CHAPTER 8.0: RECOMMENDATIONS The following recommendations are divided into two categories. The first deals with fundamental strategy; the second identifies specific action steps that need to be taken to implement this strategy. It is understood by the evaluation team that the specific action steps are illustrative and will need to be modified according to changing conditions. The details are included in order to illustrate the “spirit” of what needs to be done. 8.1. Fundamental Strategy Recommendations It is recommended that the ICSP Program transition from generic capacity building to a focused concentration on a core group of CSOs that are working in areas of high priority to USAID. This transition needs to be phased over the next 12 to 18 months. • The ICSP current generic training and technical assistance program to be sustained during this transition period in order to complete the valuable work that has begun and to establish a core structure of competent CSOs. • The current ICSP contract to be placed “on hold” and funds to be re-programmed and allocated to organizations working in USAID priority areas. • The ICSP Baghdad structure to be gradually incorporated into the newly established ICSI. The CSRCs need to evolve into locally owned and independent entities with a local Boards of Directors, local staff and individualized programs that reflect the needs of the local community. • The current ADF program to be systematically handed over to the CSRCs over a period of from 12 to 18 months. • ADF expatriate personnel need to train and mentor replacements and all expatriate positions to be phased out during the transition period. • Within the first six months of the transition period, the Regional Directors have to gradually transfer operational and managerial authority and responsibility to the Directors of the Resource Centers. • During this period, responsibility for curriculum design to be gradually delegated to the CSRCs. • During this period, responsibility for identifying the training and technical assistance needs of client CSOs is to be delegated to the CSRCs if not done so already. The current ADF organizational structure is recommended to be gradually compressed and ultimately consist of a small financial office, an M&E unit and a public relations office devoted to nation-wide advocacy for civil society. This entity is recommended to be transferred to the ICSI and have the authority to receive grants from USAID and other donors and make sub-grants to the CSRCs. • USAID (through ADF) to work with the CSRCs to install financial control systems compliant with USAID regulations so that these entities can receive direct grant support from the Mission if appropriate. • Administration of the OAT is recommended to be continued for approximately one year or until ICSP has conducted at least two assessments for each active CSO. ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 57 • Training and the provision of technical assistance is to be modified over the transition period to focus more sharply on tailoring assistance to the specific and individualized needs of the CSOs. • At the completion of the transition period USAID is recommended to consider awarding modest seed-capital grants to the now independent CSRCs. • USAID is recommended to develop a separate project to support substantive initiatives that they are interested in supporting such as women’s rights, violence mitigation and reconciliation. (These are important areas but the management and implementation of a program of this sort is quite different and the location of efforts of this sort within the USAID Results Framework is distinct from the location of civil society sector building.) It is recommended that this new project incorporate the following: • Use a cooperative agreement procurement vehicle. • Be designed by November 30, 2007, issued by December 30, 2007 and awarded by March 2008. • Be designed to that it will utilize the services and competencies of the CSRCs to the extent feasible. • Select participating CSOs on the basis of competition within areas of substantive interest to USAID. • Use the data base of information on organizational capacity prepared by ICSP to inform the selection process. • Include a modest allocation of funds to support new and emerging CSOs outside of USAID priority areas. • Include a modest amount of funds that could be sub-granted to the new ICSI for purposes of public information regarding the role and function of civil society in Iraq. In their proposals, applicants would be required to explain: • How they would establish strong working relationships with the PRTs and the CAP and LGP programs. • The administration of a transparent and competitive sub-grant program that would serve as a model for grant giving in Iraq. 8.2 Higher Level Strategic Options The evaluators also made several recommendations for higher level strategic options which could be taken in Iraq. These options are suggested pending an improved security situation as the successful participation of CSOs is dependent on an enabling environment. • Conflict and mitigation interventions to be implemented in all regions to address conflict resolution based on gender, religion and clan. This could be carried out in a separate program in regions where there is religious conflict for example. The Mission could rely on existing strategies such as the Danish Do no Harm methodology in which community motivators promote reconciliation. The new program could be linked to other regional programs which have been successful. The Mission should link with relevant research institutions to conduct research on how to better engage with CSOs and at what level of government. For example, depending on the level of government, medium sized CSOs can engage to reduce poverty and protect the environment, ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 58 or smaller sized CSOs (comprised of academicians) may engage to change laws viewed to be discriminatory. 8.3 Detailed Recommendations 8.3.1. Transition Planning ADF is required to immediately prepare a Transition plan to include the following elements and which would be in place on or before June 30, 2007. • A plan to complete at least the second round of organizational assessments in order to establish a solid data base of information regarding the civil society sector and identify those CSOs that would warrant grant support under a separate USAID Project. • A detailed outline of steps that will be taken to vest meaningful autonomy in the CSRCs including a training program for Regional Directors if this appears to be warranted. • The outline of a structure of workshops on the future of the CSRCs. The first of these workshops are recommended to be held by September 30, 2007 and include all regional staff. The second and/or subsequent workshops would be individualized and regional specific. (It is understood that these workshops may need to be held out of country.) Workshops facilitation is recommended to be carried out by an external consultant who has extensive experience in organizational development and in the creation of independent entities previously dependent on USAID support. The first workshop would be inclusive of the following elements: • Build shared consensus on the attributes and functions of community based resource centers. • Deepen understanding of financing and revenue generating tools used by NGO CSRCs. • Identify governance models and explore the role and function of Board of Directors and develop clarity regarding Board/staff relationships. The second set of four workshops would be inclusive of the following elements: Develop separate and distinct organizational development and transition plans for each of the Regional Centers and inclusive of the following: • Outline a governance structure. • Describe how the CSRC will complete the OAT process. • Identify new services that the CSRCs can offer to be relevant to community needs and how these will be integrated into their activities • Include an individualized staff development plan for each of the CSRCs to be implemented over the next 18 months. • Outline the design an individualized civil society needs assessment for each region. • Focus on steps that need to be taken to re-design the training curriculum for each CSRC. 8.4. Recommendations for the USAID Results Framework USAID is recommended to modify its Results Framework to better accommodate a civil society program. Although the Evaluation Team does not have specific recommendations on the content of this revised structure, Section 4.0 in this report identifies ambiguities that need to be ICSP Evaluation – Consolidated Annexes IBTCI Consortium Iraq Monitoring and Evaluation Performance Program, Phase II (MEPP II) 59 addressed in order to provide a coherent and logical structure that will provide guidance to implementing partners. ICSP Evaluation – Consolidated Annexes IBTCI Consortium