USAID COMMUNITY STABILIZATION PROGRAM (CSP) COUNTERINSURGENCY (COIN): RAPID PROGRAMMATIC ASSESSMENT April 5, 2008 This publication was produced for review by the United States Agency for International Development. It was prepared by Harvey Herr and Rich Mason. USAID COMMUNITY STABILIZATION PROGRAM (CSP) COUNTERINSURGENCY (COIN): RAPID PROGRAMMATIC ASSESSMENT International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Program, Phase II (MEPP II) 8614 Westwood Center Drive Suite 400 Vienna, VA 22182 Contracted under 267-C-00-05-00508-00 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Community Stabilization Program Rapid Programmatic Assessment Table of Contents LIST OF ACRONYMS ..................................................................................................................II EXECUTIVE SUMMARY ..............................................................................................................1 INTRODUCTION ..........................................................................................................................4 DEVELOPMENT OF THE CSP ....................................................................................................4 ORIGINS OF THE COMMUNITY STABILIZATION PROGRAM ...................................................................................4 THE REQUEST FOR APPLICATION (RFA) ..........................................................................................................5 THE DEVELOPMENT HYPOTHESIS – BASED ON ASSUMPTION...............................................................................5 DEFINING STABILITY – WHERE IS THE IMPACT?..................................................................................................6 LINKS TO THE COMMUNITY ACTION PROGRAM (CAP).......................................................................................7 CONFLICT MITIGATION .....................................................................................................................................8 CSP PROGRAM IMPLEMENTATION .........................................................................................8 CHANGES IN THE COOPERATIVE AGREEMENTS.................................................................................................8 WORKING WITH ALL STAKEHOLDERS................................................................................................................9 CONFLICT ASSESSMENTS.................................................................................................................................9 DEVELOPING CITY PLANS................................................................................................................................9 RAPID START UP...........................................................................................................................................10 TRAINING FOR WORK .....................................................................................................................................11 APPRENTICESHIPS.........................................................................................................................................11 BUSINESS DEVELOPMENT PROGRAM ..............................................................................................................11 TRAINING FOR LOCAL BUSINESSES AND ENTREPRENEURS ...............................................................................12 LINKING BDP GRANTEES TO OTHER SOURCES OF FINANCE .............................................................................12 TARGETING BDP GRANTS .............................................................................................................................12 PROVISION OF LONG-TERM ECONOMIC DEVELOPMENT SUPPORT TO THE PROVINCIAL RECONSTRUCTION TEAMS13 MISSING YOUTH FORA...................................................................................................................................13 RECOMMENDATIONS ..............................................................................................................14 CONCLUSION............................................................................................................................14 ANNEX 1. USAID RESPONSE TO IBTCI RAPID PROGRAMMATIC ASSESSMENT REPORT OF NOVEMBER 14, 2007 ..........................................................................................................16 Community Stabilization Program Rapid Programmatic Assessment List of Acronyms AC Advisory Committee AoR Area of Responsibility (of IPs) BDP Business Development Projects BDPO Business Development Program Office BOQ Bill of Quantity Co Ag Cooperative Agreement CERP Commander’s Emergency Relief Program CHRR Commander’s Humanitarian Relief and Reconstruction Fund CIES Community Infrastructure and Essential Services CMT Community Mobilization Team (of IPs) CSP Community Stabilization Program DAC District Advisory Council EBDP Economic and Business Development Program (of IRD) GAC Grants Advisory Committee IACCI Iraqi American Chamber of Commerce and Industry IBTCI International Business & Technical Consultants, Inc. IEDC Iraq Economy Development Center IR Intermediate Result IRD International Relief and Development IRRF Iraq Relief and Reconstruction Fund LG Local Government M&E Monitoring and Evaluation MEPP II Monitoring and Evaluation Performance Program, Phase II MNFI Multi-national Forces in Iraq MOLSA Ministry of Labor and Social Affairs MOT Ministry of Trade MSI Management Systems International MSME Micro, Small or Medium Enterprise NAC Neighborhood Advisory Council NGO Non-Governmental Organization OTI Office of Transition Initiatives PAC Project Approval Committee PMP Performance Management Plan PC Provincial Council PRT Provincial Reconstruction Team RFA Request for Application SME Small or Medium Enterprise SO Strategic Objective SOW Scope of Work TA Technical Assistance US United States USAID United States Agency for International Development USG United States Government USM United States Military International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 1 Community Stabilization Program Rapid Programmatic Assessment EXECUTIVE SUMMARY Through the USAID-Iraq funded Monitoring and Evaluation Performance Program, Phase II (MEPP II) initiative, International Business & Technical Consultants, Inc. (IBTCI) has conducted monitoring of International Relief and Development’s (IRD) Community Stabilization Program (CSP) at the local level in all geographical areas of project implementation and across each of the three major program areas. IBTCI has monitored Community Infrastructure and Essential Services (CIES), Business Development Program (BDP) and Employment Generation and Youth (EGY) projects in Anbar, Mosul, Kirkuk and Baghdad. Based on discussions with the USAID / Iraq Strategic Objective (SO) 7 team held on October 8, 2007, at the USAID compound, it was agreed that a report providing broader project and strategic recommendations would be presented separately from the individual monitoring reports. IBTCI’s rapid programmatic assessment of the CSP is presented here. Key findings of the rapid assessment include: 1. The current set of CSP Intermediate Results (IR) indicators are output indicators based on the untested assumptions that implementing these activities results in stability. 2. Without impact indicators, IRD program staff and USAID will be limited in their ability to monitor project progress and results in order to make timely, responsive, and appropriate adjustments to project implementation. 3. There is a probable need to provide a testable definition for “stability” in the Iraq context so that “stability” can be configured as an intermediate result with measurable IR indicators. 4. IRD’s current method of reporting violent incidents does not support a causal link between IRD activities and the reduction in violence in areas where the CSP is active. . 5. In their application, IRD noted that performance monitoring is their most important management tool; the current Monitoring and Evaluation (M & E) plan lacks impact indicators that support it as a valuable management tool. 6. Conflict mitigation is the fourth pillar of the CSP program. Concrete conflict mitigation activities are not apparent. However, the hypothesis is that the activities being carried out by CSP will lead to conflict mitigation. 7. Changes in objective 4 (from IRD MOD 3) shows that the original objective to “mitigate ethnic and religious conflict”, has now become conflict mitigation through Youth Programs. This reinforces the perception that conflict mitigation has been diminished in importance. This change appears out of alignment with SO 7: to reduced incentives for participation in violent conflict in selected communities. 8. One of the Cooperative Agreement (Co Ag) statements is that IRD will use proven methods to work with all stakeholders to achieve near-term and sustained impact. MEPP II monitoring does not find this to be occurring on a consistent International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 2 Community Stabilization Program Rapid Programmatic Assessment basis. Given the synergies among U.S. Government (USG) programs, this is imperative. 9. It is not clear whether or not disenfranchised youth and women are particularly targeted for BDP grants. Given the overall objective of decreased conflict, beneficiary selection is critical. 10. IRD staff has reported that Youth Fora (YF) have been deemphasized in CSP programming due to their resource-intensive nature compared with other youth activities that can reach broader audiences. 11. According to the Co Ag, IRD was to include training modules on civic education and ethnic and religious tolerance with each training course. It does not appear that these modules have been developed or are being implemented. 12. There is opportunity to work with vocational trainees and apprentices to receive business skills training followed by grants with which to start a business. This does not appear to be happening based on the sample of respondents surveyed. 13. The Co Ag addresses the follow-on for small business development saying that they “will link CSP activities to other USG-funded economic development programs to promote integrated and sustainable results.” Little evidence was found to show that this was happening. 14. The Co Ag proposes that the IRD Team, along with the PRTs in their areas of focus, will make conflict assessments at the outset of the program to ensure that proposed plans do not contribute to increasing political, religious and ethnic tensions. It is not clear if this has been done, and if it has whether or not these assessments were used to inform project design and implementation. 15. A component of the CSP is the development of “city plans.” This does not appear to have been done. 16. Initial indications seem to show that there has been some abuse of the system within the apprenticeship program. The MEPP II team makes the following recommendations for the CSP program: 1. As an essential precondition, IRD should map out the specific causal assumptions their programs are predicated upon in trying to reach the IRs and SOs of the project by building a logical framework from activities to desired results. 2. IRD should consider revising its current method of reporting violent incidents. 3. Once IRD has developed a clear rationale for how activities should work to achieve higher-level results, the project should revise and implement its monitoring and evaluation (M&E) plan to assess change at multiple levels between outputs and “impact”. 4. Missing impact data can be integrated into IRD’s existing monitoring system by augmenting existing project monitoring forms and the survey instruments currently in use. 5. IRD should attempt to measure the impact of short-term employment in terms of achieving local stability and whether there has been a reduction in propensity to violence by the participants. This might be done through political polling or with attempts to measure empowerment. Existing survey instruments might be modified to include such measures. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 3 Community Stabilization Program Rapid Programmatic Assessment 6. Close attention might be paid to beneficiary selection moving forward. IRD should seek to find disenfranchised participants who are considered leaders within their communities – for good or bad – and bring them into their programs with the caveat that participation in any program includes a concrete conflict mitigation component which is outlined in the Co Ag but not yet being implemented. 7. The process where apprentices are employed by contractors with payment subsidies by the CSP program needs to be closely monitored. There are some initial indications that this relationship may have been abused. 8. Given the issues surrounding disenfranchised youth, IRD should revisit its decision to de-emphasize their YF projects. 9. USAID might consider facilitating strategic planning sessions among partners with the specific deliverable of inter-team coordination of projects to ensure synergies among programs are maximized and for effective use of economies of scale. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 4 Community Stabilization Program Rapid Programmatic Assessment Community Stabilization Program Rapid Programmatic Assessment INTRODUCTION Based on discussions with USAID/Iraq’s SO7 team on October 8, 2007 at the USAID compound, it was agreed that broader project and strategic recommendations would be presented separately from the individual monitoring reports. In response, MEPP II has prepared a series of detailed recommendations based on the overall findings of monitoring visits as well as strategic and programmatic considerations related to the Iraq country strategy statement (Iraq Transition Strategy Statement 2006-2008), the mission Strategic Objectives (SOs) and Intermediate Results (IRs), the Request for Application, the CSP project Cooperative Agreement (and Mods), the CSP project Performance Monitoring Plan and the CSP Monitoring and Evaluation Plan. In preparing these recommendations, IBTCI has reviewed these documents and met with CSP program and M&E staff to review existing systems. Background Through the USAID-Iraq funded MEPP II initiative, IBTCI has conducted monitoring of IRD’s Community Stabilization Program (CSP) at the local level in all geographical areas of project implementation and across each of the three major program areas. IBTCI has monitored CIES, BDP and EGY projects in Anbar, Mosul, Kirkuk and Baghdad. IBTCI has had the opportunity to monitor CSP projects over the past year. Our findings have been conveyed to USAID and IRD. At the project level IBTCI has found that a few projects raised concern, and overall IRD is implementing the CIES, BDP and EGY projects in a suitable manner and are responsive to monitoring issues raised. As IBTCI moves beyond project monitoring to programmatic issues it is less well grounded in the reports issued and actual field practice that are the foundation of a comprehensive program evaluation. The concerns raised here are based on examination of compliance with the Co Ag, issues raised during the project monitoring, and weaknesses in the development hypothesis. DEVELOPMENT OF THE CSP Origins of the Community Stabilization Program The CSP has its origins in the Iraq Transition Strategy Statement of November 2005. Noteworthy in that document are the stated objectives of the transition strategy: “USG objectives cannot be accomplished through security interventions alone. The USAID strategy provides a focused approach for addressing the non-security issues of International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 5 Community Stabilization Program Rapid Programmatic Assessment governance and market led growth. It is important to increase resources for these two vital areas which, in the short-term, will stabilize areas impacted by the insurgency and mitigate the appeal of insurgent recruitment efforts...” Stabilization was an important platform for the strategy. The concept was to focus on communities that have been severely affected by insurgent action by “stabilizing” them. The Transition Strategy Statement goes on to imply that specific activities undertaken will achieve the desired stability leading to a transition to more traditional development programs. “USAID will focus on employment generation, infrastructure rehabilitation, youth programs, assistance to municipal governments and conflict mitigation” in the cities targeted as being in the strategic USG interest. It is presumed that following the achievement of stability, these “cities will be integrated into USAID’s longer-term development initiatives in health and education, agriculture, micro-credit and building the capacity of communities and civil society organizations for advocacy, and the capacity of local government to provide basic services.” The Request for Application (RFA) The CSP (referred to initially as the Focus Stabilization in Strategic Cities Initiative - FSSCI) was solicited to provide the transition activities underlined above. The Request for Application (RFA) diverted slightly from the country strategy statement although maintained the overall purpose of supporting social and economic stabilization by carrying out certain activities. Activities specified in the RFA are: 1) to create jobs and develop employable skills with a focus on unemployed youth (EGY); 2) revitalize community infrastructure and essential services (CIES); 3) support established businesses and develop new sustainable businesses (BDP); and 4) mitigate ethnic and religious conflict. The objective of FSSCI is to “support the objective of the Transition Strategy entitled Focused Stabilization: Reduce the Incentive for Participation in Violent Conflict by developing and implementing activities that support the social and economic stabilization of communities in ten strategic cities. Cities are targeted due to high levels of insurgency activities and significant populations of unemployed and disenfranchised people which help fuel the insurgency.” SO7 in the Mission PMP restates this objective. The development hypothesis – based on assumption CSP appears to be based on the development hypothesis that carrying out the stated activities leads to social and economic stability resulting in a reduced incentive for participation in violent conflict. The critical and apparently untested assumption is that there is a linkage and attribution from the activities => to stability => to desired result. Evidence to support the critical assumption is not provided in any documentation attached to CSP, but MEPP II does recognize that there is a common assumption in the development arena that non-military, counterinsurgency measures such as employment generation, infrastructure rehabilitation, youth programs, assistance to municipal International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 6 Community Stabilization Program Rapid Programmatic Assessment governments, and conflict mitigation do work to stabilize communities in conflict. Unfortunately, to date, MEPP II has found little-to-no in-depth studies or documented reports in the United States that prove this supposition. Some justification for this critical assumption can be found in USAID’s “Fragile States Strategy” document where there is a reference to USAID research. Conceptually, in the terms used in the Fragile States Strategy, the CSP is to “fill the breach between relief and development.” The CSP was formed, in part, with USAID’s Office of Transition Initiative (OTI) in mind and can be seen as an OTI project on a grand scale. Annex F for the RFA “Conflict Assessment/Rapid Response Mechanism” refers to the role of OTI in the Iraq Transition Initiative (ITI) suggesting that the OTI concept Annex was re-tooled to become FSSCI. Annex F notes that OTI’s “’gap filling’ activities in cooperation with the military have focused primarily upon short term employment opportunities for young adult males, the provision of essential services, and support for the repair and rehabilitation of basic infrastructure where needed.” In Iraq, OTI coordinated its activities with military field commanders and civil affairs staff and leveraged its own resources with field Commanders’ Emergency Relief Program (CERP) funds. Such high levels of coordination and cooperation with the military are also part of CSP operations, although in the CSP context the coordination takes place through the CSP staff on the ground and the USAID PRT and E-PRT Representatives. For justification, Annex F refers to the “linkages and methodologies field tested by OTI that are critical for future programming and form the basis for the Missions’ strategic approach to economic and social stabilization in Iraq.” This is possibly the basis for the underlying assumption of the CSP; however, the linkage from the actions taken to the desired interim result (a stable community ready for sustainable development) is not elaborated. Defining stability – where is the impact? There is a need to provide a testable definition for ” stability” in the Iraq context so that ”stability” can be configured as an intermediate result (IR) with measurable IR indicators.1 The current set of CSP IR indicators are output indicators based on the untested assumptions that implementing these activities results in stability. Stability is talked about in terms of activities and outputs; not as a perception that things are returning to ”normal”. According to the development hypothesis, the result of achieving stability is the reduction in violence and the return of legitimate government measured at the SO level. The CSP SO level indicator for measuring the impact of achieving stability is 1) the level of incidents reported in the focus stabilization area, and 2) citizen perception of the ability of local government to provide services. [NB the IRD method of reporting incidents does not support attribution to the activities undertaken.] 1 Whether providing new IR indicators is mission critical is a decision for the Cognizant Technical Officer (CTO) and the Program Office (PRO). International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 7 Community Stabilization Program Rapid Programmatic Assessment As an essential exercise, IRD should show how CSP activities will work to achieve higher-level results – the SO and IRs of the program. By building a logical framework from activities to desired results, IRD staff can identify where the project should focus its monitoring resources to ensure that it is on track at each step and be able to identify breakdown points in the event of shortfalls. Once IRD has developed a clear rationale, the project should revise and implement its M&E plan to assess change at multiple levels between outputs and “impact”. This would require additional indicators, performance indicator reference sheets, and measurement and reporting frequency requirements. IBTCI believes that much missing impact data can be integrated into IRD’s existing monitoring system by augmenting existing project monitoring forms and the survey instruments currently in use. Without impact indicators IRD program staff and USAID will be limited in their ability to monitor project progress and results in order to make timely, responsive, and appropriate adjustments to project implementation. In addition, it will be difficult to explain why the project succeeded or failed to meet its overall objectives, which will make decisions about allocation of funds and resources more challenging. In their application IRD noted that performance monitoring is their most important management tool; yet the current PMP lacks impact indicators that support it as a valuable management tool. Links to the Community Action Program (CAP) There is some overlap between certain CSP activities and those found in USAID’s Community Action Program. The same “set pieces” of CIES and BDP are taken from the CAP program and applied to the focus stabilization areas – municipal areas of intense insurgent activity. There is inherently nothing wrong with this as the CAP model is well tested, and according to a recent evaluation, is achieving its desired result (citizens advocating with the local government to attain the projects they want; different from reducing incentives to join the insurgency). IRD has extensive field experience with CIES and BDP projects. The CSP initiates its activities in a focused area with short-term employment generation (usually neighborhood clean-up campaigns under CIES) as the first step. Emphasis is always intended to be on the youth cohort and this is used as a measure in their M&E plan. This is followed by projects that are elicited from groups in the community. Similar to CAP, CSP works to ensure that there is community ownership for projects and that local authorities are involved. The BDP programs are intended to be the capstone of sustainable development in the communities laying the foundation for market driven development. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 8 Community Stabilization Program Rapid Programmatic Assessment Conflict mitigation CSP is weak in the area of conflict mitigation, the fourth pillar of the program. It appears to have been given less emphasis than the other three pillars. In the CSP PMP and the CSP M&E plan, conflict mitigation is measured through IR 7.2 “Conflict mitigated through increased community activities.” Indicators under this IR are: 7.2.1: “# of youth participating in non-formal education programs”, and 7.2.2: “# of activities completed through integrated decision-making between LG and CSOs.” A lack of specific references to the content and causality of non-formal education programs did not permit an assessment of whether these programs were directed specifically towards conflict mitigation or that conflict mitigation was just a hoped for outcome or side effect of such programs. Youth programs seemed to be almost exclusively soccer tournaments, wall painting or other types of activities designed to keep youth “busy” rather than change knowledge, attitudes or practices related to conflict. Direct conflict mitigation training or awareness was not apparent; and the generation of YF seems not to have been productive. The indicators under 7.2 are output indicators and do not measure changes in the propensity of youth to engage in violence, or to engage productively with counterparts in disparate communities. CSP PROGRAM IMPLEMENTATION Changes in the Cooperative Agreements Since program inception, the CSP Co Ag has had several modifications. The comments provided reflect the program description as provided in Modification 01 of September 30, 2006, and subsequent modifications noted in the table below. Table 1 Contract Modifications Modification Number Date Purpose of Modification 01 Sep, 30, 2006 Increase budget for Baghdad Scenario, revise program description, and incorporate reporting matrix. 02 Oct, 9, 2006 Change to reflect fiscal data; no change in agreement. 03 Oct, 26, 2006 Change city scenarios; restructure CSP’s four objectives. 04 Jan, 31, 2007 MNFI Badging. 05 May 1, 2007 Revise language of City Working Budget. 06 May 14, 2007 Incremental funding. Program Modification 03 (26 October 2006) restructures the four program objectives to: 1. Public Works Programs and Employment Generation; 2. Vocational Training and Apprenticeship Programs; 3. Micro, Small-Medium Enterprise (MSME) Development Program; and 4. Youth Programs (conflict mitigation). International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 9 Community Stabilization Program Rapid Programmatic Assessment Changes in objective 4 shows that the original objective to “mitigate ethnic and religious conflict”, has now become conflict mitigation through Youth Programs. This reinforces the perception that conflict mitigation has been diminished in importance. This modification appears out of alignment with the overall SO 7 to reduce incentives for participation in violent conflict in selected communities. Working with all stakeholders One of the Co Ag statements is that IRD will use proven methods to work with all stakeholders to achieve near-term and sustained impact. Stakeholders stated in this regard are USAID, PRTs, US Military, Government of Iraq (GoI), Non-Governmental Organizations (NGOs) and community organizations. A rapid assessment of the level of communication among the stakeholders could be considered. Missing in the stakeholder list are the other USAID programs such as Research Triangle Institute’s (RTI) Local Governance Program (LGP) II, which has developed productive relationships with local and provincial councils, and the microfinance programs funded by grants from Izidhar and implemented by Cooperative Housing Foundation (CHF), ACDI/VOCA and Relief International. Another relevant program is USAID’s INMA, implemented through the Louis Berger Group, which works to develop Iraqi agriculture and related businesses. These additional programs are presumably linked to the PRT, but linkages of microfinance to specific BDP grants, or of specific local councils to CSP operations might be facilitated by direct regular contact. Conflict assessments The Co Ag proposes that the IRD Team with the PRT will make conflict assessments at the outset to ensure that proposed plans do not contribute to increasing political, religious and ethnic tensions. It wasn’t clear that this had been done, and if it had how it was used to scale or shape the program. Developing City Plans A component of the CSP is the development of “city plans.” These are to be developed through a “participatory planning” approach that includes strategies and action plans that take into account local priorities. Assessments are done with local stakeholders (different from the stakeholders listed above) that include: government leaders, youth groups, women groups, university faculty and professionals, and religious and political leaders. The city action plans are the guide for the development of municipal service projects that will result in employment generation. But the plan also is to identify opportunities to support small scale businesses, the identification of candidates for training and technical assistance, start-up grants, business skills improvements and business association development. The city action plan also is to identify linked program interventions from other USG funded programs. There is little evidence that this has been done. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 10 Community Stabilization Program Rapid Programmatic Assessment The selection of program beneficiaries is informed by the city plan. A best practice would be to select beneficiaries who are seen as community leaders with influence for good, or bad, bringing them together to work towards the implementation of community￾based projects. An integral part of this project should be a very strong, structured conflict mitigation component. It now seems that beneficiaries come from Ministry of Labor and Social Affairs (MOLSA) rolls, word of mouth and advertising, thus suggesting a less than rigorous approach at finding those who are prone to joining militias or the insurgency. In other words, there is a risk of treating the entire population of Iraq as an homogenous conflict prone society. The city plan was plainly seen as an important aspect of the CSP program. However, it is uncertain whether plans were consummated as they do not seem to be referred to in reports. Given their apparent importance as vehicles for incorporation of local priorities in CSP activities, their possible absence is noteworthy. As a corollary, the LGP has been working with the provincial councils to develop the provincial development strategy (PDS). This too is intended to be the result of a participatory process that includes inputs from lower level councils. Whether the city plan is a parallel effort that might be seen to compete with the PDS is a possible issue. As part of CSP coordination with other partners, however, these plans, where they exist, should be harmonized with the PDS and other local development strategies, necessitating a high-level of partnership with LGP II. Rapid Start Up Near-term impact in the RFA/Co Ag context means the generation of short-term employment. In the design of the program, 60 days after the Multi-National Force in Iraq (MNFI) has secured an area, the CSP deploys to generate short-term employment through public works projects (PWPs). These are typically clean up campaigns that use high levels of minimally skilled labor (unskilled youth), and youth programs. The level of short-term employment is a clear output indicator. Even though generating short-term employment is a transition strategy with no intended sustainability, it would seem prudent to attempt to measure the impact of this employment in terms of achieving local stability and to measure whether or not participants are less prone to violence. This might be done through political polling or with attempts to measure empowerment. Existing survey instruments might be modified to include such measures. The generation of short-term employment segues into activities where sustainability can be achieved. These sustainable activities/projects include the expansion or restoration of public buildings that allow persons to return to work. However, the BDP activities are the cornerstone of sustainable market driven development. Here the purpose is the development of long-term jobs in sustainable enterprises. It is not clear whether or not disenfranchised youth and women are particularly targeted for BDP grants. While there is opportunity to work with vocational trainees and apprentices to receive business skills training followed by grants with which to start a business, it is not apparent that this is being done. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 11 Community Stabilization Program Rapid Programmatic Assessment Training for work This assistance is intended to improve prospects for long-term employment. Working with MOLSA, CSP seeks to improve their vocational training facilities and expand their skills training programs. As part of this support, CSP was to include training modules on civic education and ethnic and religious tolerance with each training course. These modules are designed to introduce trainees to basic political, constitutional, conflict mitigation and human rights concepts. MEPP II has interviewed vocational education trainees, but was not aware of these modules at the time so reference to them was not included in the interviews. The monitoring instrument can now be revised to include reference to these issues. However, MEPP II learned that these modules have actually not yet been included in the courses. This reinforces the suggestion that conflict mitigation has been sidelined. Apprenticeships “Training for work” participants are given an opportunity to apply for an apprenticeship. CSP’s contracting mechanism gives preference to contractors who agree to employ apprentices. The CSP Team provides payment subsidies for these apprentices while they work with the contractor. Apprenticeships are either short or long term. Some of these apprenticeships take place on CSP project sites where contractors are implementing infrastructure rehabilitation or essential service provision projects. At the end of the apprenticeship period CSP anticipates that the apprentice will be hired by the contractor. The process where apprentices are employed by contractors with payment subsidies by the CSP needs to be closely monitored. There are some initial indications that this relationship may have been abused. Weekly reports show that the CSP helps with job placement. However, the students interviewed by MEPP II didn’t seem to have an idea where they would work after completion of their course. It would seem to be a logical conclusion to the training for work program. Business development program The Business Development Program (BDP) within the CSP introduces an element of sustainability into the CSP by issuing grants in materials up to $100,000 and by providing business skills training for entrepreneurs and for aspiring unemployed youth. The BDP is intended to support grantees that have been identified through other, short￾term activities or through references from those working in the CSP cities. It was assumed that many of the training-to-work graduates would be likely candidates for BDP grants. As with job placement this is a logical conclusion for some graduates of the training-for-work program. Program documentation, however, does not reveal that graduates become BDP grantees. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 12 Community Stabilization Program Rapid Programmatic Assessment Training for local businesses and entrepreneurs CSP provided technical assistance to the Iraqi Center for Business Community Development (ICBCD), an Iraqi non-governmental organization (NGO), and the Iraqi American Chamber of Commerce Institute (IACCI) to design training curriculums for small businesses. IACCI became a partner with IRD and provides the BDP training. According to the IRD BDP grants manual, the business management training program concentrates on building the capacity of the trainees teaching them the essential business skills and knowledge needed to engage them in employment opportunities. The training program is also linked with the grants program by encouraging the trainees to apply for the available CSP micro-, small- and medium-sized enterprise (MSME) grants. The BDP manual suggests that “this linkage will give the business development component more means of success to the entrepreneur who lacks adequate funds to start a business and the essential managerial skill requirements to successfully operate a private business.” IBTCI’s monitoring of BDP projects in Baghdad showed that when grantee projects were implemented by IACCI, the grantees tended to receive business management training; whereas with IRD-implemented grants this was seldom the case. This seems to be an anomaly, and should be explained or rectified. IRD has indicated that their recruitment for BDP grants starts at the community level whereas IACCI starts theirs in the classroom. All grantees should receive this training and it could be a condition for qualifying to receive a new business grant. A safe assumption is that the probability for a successful business outcome increases when there has been successful completion of business management training. Linking BDP grantees to other sources of finance The Co Ag addresses the follow-on activities for small business development noting that CSP “will link activities to other USG-funded economic development programs to promote integrated and sustainable results.” Little evidence was found that this was happening. None of the grantees interviewed during MEPP II monitoring of BDP grantees were aware of other sources of finance from other programs. It is recommended that CSP insert a module into the business management training program that outlines opportunities for small business finance. CSP BDP officers can be trained to facilitate grantees who wish to pursue financing from other sources. Targeting BDP grants Business development programs are to primarily target vulnerable populations including youth and women. To achieve this, IRD is to work with IEDC and local NGOs in the business sector. The Contemporary Iraqi Women’s Association and the Iraqi Youth Business Association are cited as examples. For Baghdad just 10% of the grantees were women (from among 1660 grants). This certainly needs improvement and should be monitored. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 13 Community Stabilization Program Rapid Programmatic Assessment The Co Ag says that the BDP will focus on micro-enterprises; however the majority (53%) of BDP grants in Baghdad was classified as ‘small.’ The significance of this for program management is uncertain, but it may be important for grant management perhaps indicating that grants are being issued at a higher value than was planned. Additionally, IRD notes that the CSP will support entrepreneurs and micro-enterprises through three programs focusing primarily on the women and youth populations. The three programs are: 1) BDP grants to private enterprise; 2) grants to Cooperative Societies, and 3) Technical Assistance. We did not find evidence of grants to cooperative societies (although this was done in the CAP program). Similarly, we did not find that technical assistance was reported. Overall this may not be significant to program impact. Provision of long-term economic development support to the Provincial Reconstruction Teams In the Co Ag program description, IRD proposes the provision of long-term economic development support to the PRTs. This is to be provided through a four-person team that liaises with the PRTs on all issues related to long-term economic development. While admirable this was not specified in the RFA and seems inconsistent with the transition nature of the project. In addition, there are two other USAID-funded projects perhaps better placed to provide such advice: Izdihar and the Economic Governance II program. There was, however, no evidence to suggest that IRD was providing this support to the PRTs in the areas in which the CSP is currently operational. Missing Youth Fora Referencing Youth Fora (YF), the Co Ag states: . . . [i]n coordination with the Ministry of Youth and Sports (MoY), USAID/PRTs, and the ACs, IRD Team will establish and support existing Youth Forums (YFs) in strategic cities. IRD Team will work with the YFs to develop training modules on life skills, conflict mitigation, and youth activism / advocacy. Life skills include civic responsibility, religious and ethnic tolerance, and health”…”the main mission of the YFs is to promote the active role of youth in society, and enhance their participation in the development process. The YFs will empower young people to think creatively and critically about themselves and their roles in society and give them the opportunity to participate more actively in the development process in Iraq. IRD Team will invite local youth NGOs to design specific activities for the YFs, including training aimed at developing conflict mitigation and management skills among youth. Using a Training of Trainers methodology, youth organizations will transfer skills to training participants who will then be able to further transfer the acquired skills and knowledge to new YF attendees. The Training of Trainers methodology will allow for greater involvement of local International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 14 Community Stabilization Program Rapid Programmatic Assessment Iraqis and will serve to improve the sustainability of conflict mitigation and management program interventions. Discussions to date indicate the Ministry's willingness to participate in organizing youth. Indications are that none of this is happening. In fact, IRD staff has reported that YF have been deemphasized in CSP programming due to their resource-intensive nature compared with other youth activities that can reach broader audiences. It appears, however, as detailed above, that the preferred youth activities are much less focused on conflict mitigation. RECOMMENDATIONS • IRD should map out the specific causal assumptions their programs are predicated upon in trying to reach the IRs and SOs of the project. • IRD should consider revising its current method of reporting violent incidents. • Once IRD has developed a clear rationale for how activities should work to achieve higher-level results, the project should revise and implement its M&E plan to assess change at multiple levels between outputs and “impact”. • Missing impact data should be integrated into IRD’s existing monitoring system by augmenting existing project monitoring forms and the survey instruments currently in use. • IRD should attempt to measure the impact of short-term employment in terms of achieving local stability and whether there has been a reduction in propensity to violence by the participants. This might be done through political polling or with attempts to measure empowerment. Existing survey instruments might be modified to include such measures. • Close attention might be paid to beneficiary selection moving forward. IRD should seek to find disenfranchised participants who are considered leaders within their communities – for good or bad – and bring them into their programs with the caveat that participation in any program includes a concrete conflict mitigation component outlined in the Co Ag as additional training however not yet being implemented. • The process where apprentices are employed by contractors with payment subsidies by the CSP program needs to be closely monitored. There are some initial indications that this relationship may have been abused. • Given the issues surrounding disenfranchised youth, IRD should revisit its decision to de-emphasize their youth fora projects. • USAID might consider facilitating strategic planning sessions among partners with the specific deliverable to develop a framework for inter-team coordination of projects to facilitate synergies across USAID programming and maximize economies of scale. CONCLUSION International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 15 Community Stabilization Program Rapid Programmatic Assessment While IRD is implementing many aspects of the initial proposed program well – and the desired outputs are visible, it is also clear that several components outlined in the proposal have not yet been implemented. This is understandable given that, at the outset, the push was for rapid implementation showing rapid short-term results to augment the military surge. It is unlikely, however, that the current activities will meet the desired objectives of the proposed program given that the causal link between the activities and desired results are tenuous at best. The CSP has many stakeholders within the USG attempting to shape its programming based on priorities that might be somewhat different than the traditional USAID development process. The challenge is to find the right balance within the mix of projects and their implementation to achieve the desired results. It might be prudent to revisit the original SOW and revise it as necessary to address the current realities on the ground and resulting operational demands. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 16 Community Stabilization Program Rapid Programmatic Assessment Annex 1. USAID Response to IBTCI Rapid Programmatic Assessment Report of November 14, 2007 USAID Response to IBTCI Rapid Programmatic Assessment Report of November 14, 2007 Note: FSO met with each other to discuss the recommendations on 21 February 2008. On 27 February, FSO met with IBTCI to review the report findings. Note: Recommendations are listed in order of priority as established by USAID. Recommendations: USAID Response: Action: Timeline: Follow-Up: 1 IRD should map out the specific causal assumptions their programs are predicated upon in trying to reach the IRs and SOs of the project. Agreed. USAID needs to meet with IRD to conduct an in￾depth review of the IRs in relation to the SO. This exercise should be part of the M & E revision process. As part of the M & E revision process, include this IR and sub-IR review. The final, revised M & E plan should be done by early April 2008. USAID met with IRD in March ’08 to review the M & E plan and to consider each IR in relation to the SO. Changes will be incorporated in the revised plan. 2 Once IRD has developed a clear rationale for how activities should work to achieve higher￾level results, the project should revise and implement its M&E plan to assess change at multiple levels between outputs and “impact”. Agreed. This will be part of the M & E revision process along with the exercise identified in priority 1. As part of the M & E revision process, revise the M & E plan to better measure outputs and impact. The final, revised M & E plan should be done by early April 2008. During the M & E revision process (in March ’08), USAID and IRD designed four research activities that will be implemented to more closely capture program outputs and help determine CSP’s impact. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 17 Community Stabilization Program Rapid Programmatic Assessment Recommendations: USAID Response: Action: Timeline: Follow-Up: 3 Close attention might be paid to beneficiary selection moving forward. IRD should seek to find disenfranchised participants who are considered leaders within their communities – for good or bad – and bring them into their programs with the caveat that participation in any program includes a concrete conflict mitigation component outlined in the Co Ag as additional training however not yet being implemented. Beneficiaries are targeted based on geographic location. USAID will work with IRD to improve reporting and data collection to better identify CSP beneficiaries and will consider an increased initiative to include more women during the program revision process in March 2008. USAID is comfortable with the modified fourth objective: “conflict mitigation through youth programs.” USAID will consider enhancing this objective during its program revision in March. As part of the Program Review and Revision in March 2008, consider enhancing conflict mitigation activities for youth through possible civic education and mediation training as part of life skills classes for youth and in the Votech curricula. The program review and revision process began in mid-March and will progress over the course of the next two months with a revised program description and modification to the cooperative agreement. USAID will be providing IRD with recommended guidance in April ’08 to modify the program description for the next modification to increase reconciliation activities for youth and to consider more activities to engage women. 4 Given the issues surrounding disenfranchised youth, IRD should revisit its decision to de￾emphasize their youth fora projects. USAID will consider increased youth activities during the program review and revision in March 2008, and will delete the reference to youth forums in the new program description. During the program review and revision process, references to the youth forums will be deleted and new youth activities will be considered. (See point 3 above.) The program review and revision process began in mid-March and will progress over the course of the next several months with a revised program description and modification to the cooperative agreement. The SOW for the youth focus groups activity will be written in April ’08 and implementation will move forward from there. USAID has provided IRD with recommended guidance to modify the program description for the next modification to increase reconciliation activities for youth. In addition, a new research activity will be implemented to assess youth’s attitudes toward conflict through Iraqi￾wide youth focus groups. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 18 Community Stabilization Program Rapid Programmatic Assessment Recommendations: USAID Response: Action: Timeline: Follow-Up: 5 Missing impact data can be integrated into IRD’s existing monitoring system by augmenting existing project monitoring forms and the survey instruments currently in use. USAID will include this as part of the M & E revision process, but would like to meet with IRD, IBTCI and Lincoln Group to discuss ways to improve the existing surveys. Organize a meeting with IRD, IBTCI, and Lincoln Group to review the existing surveys and discuss possible ways to improve the surveys and the survey process. This meeting was deemed unnecessary once the M & E revision took place and the existing surveys were reviewed and amended. See Recommendation No. 6 below No follow-up needed. 6 IRD should attempt to measure the impact of short-term employment in terms of achieving local stability and whether there has been a reduction in propensity to violence by the participants. This might be done through political polling or with attempts to measure empowerment. Existing survey instruments might be modified to include such measures. USAID will consider modifying the surveys as part of the M & E revision process. However, USAID does not have the capacity to conduct the comprehensive objective analysis needed to determine whether or not CSP’s short￾term employment activities have reduced a propensity toward violence. USAID would like to conduct this in-depth analysis through a separate SOW and award. As part of the M & E revision process, discuss how survey instruments might be modified to collect data relevant to linking job creation with reductions in violence. During the M & E revision meetings in March ’08, USAID and IRD agreed to add two new questions to the surveys to better capture Iraqis’ perceptions and possible perceived links between a reduction in violence and improved community services. The survey results should be reviewed and analyzed once enough surveys have been conducted across the CSP AOs and over time. 7 The process where apprentices are employed by contractors with payment subsidies by the CSP program needs to be closely monitored. There are some initial indications that this relationship may have been abused. USAID has requested more information from IBTCI to determine the extent and nature of the alleged “abuse.” and will follow up with IRD to determine the exact procedures for placing apprentices and what they are doing to prevent waste, fraud and abuse. USAID discussed this recommendation with IBTCI on 27 February and agreed that a closer analysis of this program was needed. A separate research activity will be considered that will focus on the apprenticeship program. During the M & E revision process in March ’08, USAID and IRD agreed to add a research activity that will focus on the apprenticeship program. In early April ’08, USAID plans to write the Statement of Work for this new activity and will request that IBTCI conduct this in-depth analysis. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 19 Community Stabilization Program Rapid Programmatic Assessment Recommendations: USAID Response: Action: Timeline: Follow-Up: 8 USAID might consider facilitating strategic planning sessions among partners with the specific deliverable to develop a framework for inter￾team coordination of projects to facilitate synergies across USAID programming and maximize economies of scale. The CSP Communications Protocol directs and encourages communication between the CSP teams and their PRT and E-PRT counterparts and, overall, communication and cooperation among CSP staff and PRT and E-PRT reps is very good. Any broader strategic planning sessions would be the jurisdiction of the program office, not FSPO. No action needed. None No follow-up needed. 9 IRD should consider revising its current method of reporting violent incidents. USAID would like to make this part of the discussion during the M & E revision to revisit the goal of reducing violence by 25% and how we might measure this differently, if at all. As part of the M & E revision process, discuss whether or not CSP needs to measure a reduction in violence, and if so, how to better capture the data for this analysis. During the M & E revision meetings in March ’08, USAID and IRD agreed to eliminate the baseline figures and targets for this indicator because it is not possible to accurately assess. No follow-up needed.