This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by Ghazia Aslam, Marsin Alshamary, Pragati Godbole, and Kerry Jules-Smith with EnCompass LLC for the USAID/OTI M&E Support Task Order (IDIQ No. AID-OAA-I-15-00021, Task Order No. 7200AA19F00017). The views expressed in this publication do not necessarily reflect the views of USAID or the United States Government. IRAQ REGIONAL PROGRAM FINAL EVALUATION FINAL EVALUATION REPORT March 2022 PHOTO | USAID/OTI IRAQ COMMUNITY RESILIENCE INITIATIVE i | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV CONTENTS ACRONYMS AND ABBREVIATIONS ........................................................................................................................II EXECUTIVE SUMMARY.................................................................................................................................................III INTRODUCTION ............................................................................................................................................................1 Background and Purpose.............................................................................................................................................1 Context ...........................................................................................................................................................................1 Program Approach........................................................................................................................................................2 Evaluation Methods Overview...................................................................................................................................2 FINDINGS...........................................................................................................................................................................5 Contextual Findings......................................................................................................................................................6 Durability of Returns and Social Recovery ...........................................................................................................10 ICRI and Durability of Returns................................................................................................................................15 ICRI, Information Sharing, and Social Media.........................................................................................................23 ICRI’s Strategy and Approach ..................................................................................................................................29 CONCLUSIONS AND RECOMMENDATIONS....................................................................................................32 ANNEXES .........................................................................................................................................................................41 Annex A: Evaluation Questions...............................................................................................................................41 Annex B: Methodology ..............................................................................................................................................43 Annex C: Limitations..................................................................................................................................................53 Annex D: Consent forms..........................................................................................................................................54 Annex E: Data Collection Instruments..................................................................................................................61 Annex F: Linking Findings to Evaluation Questions............................................................................................87 ii | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV ACRONYMS AND ABBREVIATIONS ICRI Iraq Community Resilience Initiative IDP Internally displaced person IOM International Organization for Migration IRP Iraq Regional Program ISIS Islamic State of Iraq and Syria KII Key informant interview OTI Office of Transition Initiatives PDO Program Development Officer PO Program objective SO Sub-objective UNDP United Nations Development Programme USAID United States Agency for International Development iii | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY The United States Agency for International Development (USAID) Office of Transition Initiatives (OTI) commissioned EnCompass LLC to conduct a final evaluation of its Iraq Community Resilience Initiative (ICRI) program. ICRI, or “Ta’afi” (“recovery” in Arabic), sought to contribute to stability in Iraq (program goal) by improving durable returns in areas liberated from the Islamic State of Iraq and Syria (ISIS) (program objective 1) and by mitigating discrete challenges that threaten overall stability (program objective 2). This evaluation focuses on the first program objective of improving durable returns. The program aimed to provide durable returns in Iraq through three sub-objectives: 1) improving delivery of public services prioritized by communities, 2) expanding livelihood opportunities, and 3) improving social conditions related to general feelings of safety, individuals’ ability to go about their daily lives, and individuals’ ability to interact positively with other community members. Although ICRI was implemented in four geographic areas, this evaluation examines ICRI’s work in just two areas―Mosul and the Ninewa Plain―and aims to support OTI, USAID, and other local and international stakeholders to better understand how, why, and under what circumstances the program activities implemented in Mosul and the Ninewa Plain led to durable returns. This evaluation was conducted as the program reached the end of its first phase (June 2018–February 2022) and during the transition to the next phase (September 2021–2024). 1 Therefore, the goal of the evaluation was both to reflect on successes and lessons learned and to inform future programming. EVALUATION QUESTIONS The questions that guided this evaluation (see box below) focused on exploring the program’s contribution to promoting social recovery and, therefore, durable returns. “Social recovery” is defined as the feeling a returnee has of being at home in their own community, living with a sense of safety, and interacting positively with other members of the community. “Durable returns” is a term implying that community members plan to remain in their community for the long term, whether they were displaced by the conflict with ISIS and returned to their community of origin, or have never left. The evaluation also focused on ICRI’s communication strategy (largely social media) and examined its effect on awareness of ICRI, perceptions of social recovery, and encouragement of returns. 1 While the second phase of ICRI has a different strategic focus than the first, this evaluation provides some lessons that might be useful to inform its strategy. iv | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV METHODS The evaluation team selected three communities each in Mosul and the Ninewa Plain for primary data collection. These included al-Islah al-Zirai, the Old City, and al-Tanak in Mosul and Bashiqa, Bartella, and Qaraqosh in the Ninewa Plain.2 The term “community” in this report refers to people who share a locational commonality, regardless of shared or varying ethno-religious identity. The evaluation team used the following three data collection methods to answer the evaluation questions. Other data sources were used when necessary and are referenced throughout the report. ● Key informant interviews (KIIs): Phone and in-person interviews with 46 program participants, community members, and in-country and headquarters program staff ● Quantitative surveys: Surveys with 170 residents (including stayees, returnees, and currently displaced) ● Document review: Review of documentation related to program strategy and to 114 ICRI activities undertaken in Mosul and the Ninewa Plain, including those related to media and communications FINDINGS AND CONCLUSIONS The findings describe the program’s impact on communities in Mosul and the Ninewa Plain, and help stakeholders understand whether and how the program contributed to its goal of creating conditions for durable returns. The data indicate that the displacement patterns in Mosul and the Ninewa Plain were different, with residents of the Ninewa Plain having been displaced for longer than those from Mosul. Returnees in both areas plan to stay in their towns of origin and a majority want to raise their children in these areas as well. ICRI’s theory of change and the relationship between durability of returns and social recovery: The program’s theory of change states that three factors affect durability of returns and social recovery: access to basic services, access to livelihood opportunities, and improved social 2 The Ninewa Plain is a region in Ninewa governorate that has a diverse population and consists of the districts of Hamdaniya, Shekhan and Tal Kayf. Evaluation Questions Question 1: To the extent that social recovery is a necessary condition for durable returns, how effectively has ICRI understood and addressed social recovery needs in areas liberated from ISIS? 1.a.: To what extent and how did the program adapt its strategy and theories of change (as they related to social recovery for durable returns) over the course of three years of programming? 1.b.: To what extent did activity outcomes contribute—intentionally or unintentionally, directly or indirectly—to allow for durable returns? What were the key pathways to change? Question 2: Whether and how has ICRI influenced perceptions of recovery among residents of areas liberated from ISIS? To what extent do these perceptions differ across various groups (region, gender, religious/ethnic group, etc.)? What are the pathways of change? 2.a. How have ICRI’s social media presence and community engagement efforts during activities contributed to this effort? v | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV conditions. Evidence largely confirms ICRI’s theory of change that access to services and livelihood opportunities contributed to durability of returns. Access to basic services is a key factor in residents’ decisions to return, and livelihood opportunities were key motivators for people to stay in their towns of origin in the long run in both Mosul and the Ninewa Plain. Social conditions, however, were not found to be obstacles to returns or durability of returns. Residents of the two areas characterized relations among neighbors, even if from different ethno-religious groups, as strong and positive. However, it is important to note that demographic change and perception of residents as ISIS sympathizers in the Ninewa Plain threaten these positive changes. This emerging discord needs to be managed to ensure that these positive feelings remain. Moreover, the program missed a few opportunities to encourage community members to return and stay. For example, the evaluation found that residents’ attachment to their ancestral homeland is a significant motivator for return and stay, which the program did not take advantage of. Additionally, shared trauma, while extremely harmful, can also act as a bonding agent that can strengthen positive relations among community members from different communities, yet the program did not take advantage of this opportunity for increased bonding. ICRI contribution to durability of returns: ICRI substantially improved community members' access to basic services, livelihood opportunities, and sense of well-being and normalcy. This contributed to community members' decisions to stay in their community, even though ICRI's effects on social conditions were limited. The data also suggest that livelihoods are the most significant determinant in residents’ decisions to stay. It is important to note, however, that unemployment continues to be the greatest and most important challenge to people’s plans to stay in both districts, and there is still a great need for livelihood activities, especially in the Ninewa Plain. Another important achievement is that respondents believe that the program dealt equitably with various ethno-religious groups. In a context like Iraq, where multiple ethno-religious groups exist and conflicts frequently emerge concerning distribution of resources across these group, this achievement is significant. ICRI was also able to manage local communities’ and partners’ expectations. Community members accurately understood ICRI’s role and capabilities. ICRI’s social media strategy: ICRI’s social media strategy aimed to raise awareness among internally displaced persons (IDPs) and residents about the recovery process and progress in Mosul and the Ninewa Plain, with the theory that this awareness would also encourage returns. The data show that social media platforms were an important channel for ICRI to reach community members. An overwhelming majority of residents in Mosul and the Ninewa Plain were aware of ICRI’s presence in their community, largely due to social media. Many residents recognized ICRI’s hashtags, particularly #Taafi and #USAID. There is some evidence that ICRI’s social media presence had a minor effect on promoting returns.3 However, this finding is tempered by data indicating that social media had limitations. First, residents mainly based decisions about when to return on information shared by family and friends, not through social media. ICRI’s social media content had an effect on returns when residents shared content created by ICRI on their social media platforms, encouraging their own circle of family and friends to 3 It is important to note that since the survey conducted for this evaluation was conducted via social media, there is a definite bias toward people who use and/or are savvy on social media. Therefore, findings likely overestimated the popularity of ICRI-related social media among residents of Mosul and the Ninewa Plain. vi | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV return by showcasing improved conditions in the community. Second, some data suggested that many residents might not have access to social media or might not be familiar with using social media. There are also gender differences in how individuals receive information about ICRI. While fewer females in the Ninewa Plain received information about ICRI through social media than men, the reverse is true in Mosul. The data also indicate that ICRI’s offline outreach complemented its online popularity, i.e., community members recognized ICRI on social media because it had a presence on the streets. ICRI strategy and approach: ICRI’s layering and sequencing of its program and sub-objectives at the program strategy level were effective. ICRI layered series of activities that provided public services, improved opportunities for income generation, and improved intra-society relationships to produce a multiplier effect on durability of returns. The program also adopted a sequencing strategy so that public services and livelihood needs were accommodated before the program targeted intra-society relationships or social conditions, with the assumption that social condition interventions are effective (or prioritized) only when the community has access to livelihood opportunities and critical basic services. However, activity reports did not provide evidence of sequencing and layering at the activity level i.e., the activity reports did not link activities or provide rationales for how and why activities built on previous activities. Still, the program’s iterative learning and adaptation, as well as the built-in flexibility in collaborating with local partners, allowed it to build on past successes. ICRI lack of gender integration in programming: ICRI did not seem to have integrated a gender perspective into the preparation, design, implementation, and monitoring and evaluation (M&E) phases of programming. Gender integration is defined as strategies applied in program planning, assessment, design, implementation, and M&E to consider gender norms and to compensate for gender-based inequalities and usually goes beyond consultations with women during various phases of programming.4 One of the observable results of the lack of gender integration is that ICRI programming did not focus on providing services and livelihood opportunities directly to women. Some of its interventions, for example, its outreach strategy through social media, reached fewer women than men; as stated above, more men received information from social media about ICRI in the Ninewa Plain than women, while the opposite was true in Mosul. Compared with men, women control fewer political and economic resources, and have less decision-making power in their daily lives. Because of these differences, without integrating gender perspectives it would be difficult for any program to respond to the needs of women as thoroughly as those of men, impeding the maximum impact of the program. Worse, programming that does not acknowledge and integrate inequalities can perpetuate, exacerbate, or create them.5 Therefore, integrating gender perspectives not only makes sense from a human rights perspective, but also maximizes impact of development interventions. In other words, to the extent that gender-based inequalities or culturally gender-based roles pose challenges to improving durability of returns, researching and integrating gender-aware considerations might have enhanced the impact of ICRI programming. Taking it a step further, integrating gender in the program’s theory of change by acknowledging the range of connections between gender dynamics, conflict, and stability would have led to a more holistic 4 The Interagency Gender Working Group (funded by USAID) has developed a continuum of gender integration. The term “gender blind” refers to programs that are designed without prior analysis of how gender can affect program outcomes. In contrast, “gender aware” programs/policies deliberately examine and address the anticipated gender-related outcomes during both design and implementation. 5 This evaluation did not analyze whether any of these scenarios occurred in Iraq during ICRI programming. The only finding the evaluation team observed was the lack of programming focused on women, which likely weakened the effectiveness of some of its interventions (i.e., social media strategy, as mentioned before). vii | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV approach to promoting stability in Iraq.6 The current context of recovery from post-war displacement in Iraq has created a window of opportunity in which gender relationships and norms can be redefined and re-negotiated in ways that are conducive to trust and address gendered drivers of conflict. 7 RECOMMENDATIONS Capitalize on both pragmatic and emotive motivators of stay. Resettlement programs for displaced persons should focus on livelihoods and services and build on residents’ love for their land as strong motivators for return and stay. Use the healing power of collective recovery. Shared experiences of displacement and return have bolstered social conditions in ICRI communities. Programs can build on the bonds that come from shared experiences to promote collective healing and recovery. Focus on improving social conditions across communities. Programs in Mosul and the Ninewa Plain should focus on improving social conditions between communities, more so than those within communities, which already have strong bonds built on shared experiences, even among those from different ethno-religious groups. Prioritize the eradication of the signs of war. In war-torn areas, improving basic services and physical structures positively affects residents’ sense of well-being and their decision to stay in their community. Advertise the role of the local and/or central government in supporting programming. The objectives of the project did not include improving perception of government by citizens, citizen-state relationship, or trust in the government. However, literature suggests that citizens’ perceptions of a state’s ability and willingness to provide services plays an important role in maintaining a citizen-state relationship conducive to state stability, especially once the work of international organizations transforms or ends.8 It can be argued, therefore, that citizens’ perceptions of their state should have been an important consideration during programming development and monitoring. The program should make an effort to illustrate the role of government in providing services and livelihood opportunities to the citizens to indicate the state's ability and willingness to provide for its citizens. At the very least, the program should ensure that its successes do not undermine public confidence in government 6 Not only does conflict affect men and women differently, but there is also substantial evidence that gender inequality contributes to violent conflict and has a significant negative influence on stability. By contrast, evidence suggests that improving gender equality can indirectly increase a country’s stability through its impact on wealth/income. For example, see: Herbert, S., “Links between women’s empowerment (or lack of) and outbreaks of violent conflict,” GSDRC Helpdesk Research Report: 2014, http://www.gsdrc.org/docs/open/hdq1170.pdf. Also see: Dietrich, L. and Carter, S., “Gender and conflict analysis in ISIS affected communities of Iraq,” Oxfam: 2017, who discuss how gender norms have fueled tensions and insecurity in conflict-affected areas of Iraq in concert with other causes, including sectarianism, perceived corruption, exclusionary policies, and injustice, and underlying structural tensions, such as lack of services, education, and employment. 7 For example, see: O'Driscoll, D., “Emerging trends of conflict and instability in Iraq,” Helpdesk Report, UK Department for International Development: 2018. 8 For example, see: Kooy, M., Wild, L., and Mason, N., “Doing Things Differently: Can Water Supply, Sanitation, and Hygiene Services Support Peace-and State-Building Processes?” Development Policy Review 33, no. 4 (2015): 433–456; McLoughlin, C., “When Does Service Delivery Improve the Legitimacy of a Fragile or Conflict-Affected State?,” Governance 28 no. 3 (2015): 341–356; Denney, L., Mallett, R., and Mazurana, D., “Peacebuilding and service delivery,” New York: United Nations University, Center for Policy Research, 2015. viii | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV organizations, as the public tends to compare the performance of government actors with that of international organizations. Tailor livelihood programming to further encourage durable returns, when possible, by focusing on the community’s traditional livelihood sectors. While ICRI substantially improved livelihood activities in the communities where it was implemented, even more livelihood activities are needed to encourage durable returns, especially in the Ninewa Plain. Rehabilitation of local pre-war industries in traditional livelihood sectors (for example, agriculture in the Ninewa Plain) encourages residents to remain in their communities and prevents emigration by not only providing livelihood opportunities but also by protecting the identities of the communities.9 However, support to traditional livelihoods should aim to—at the very least—do no harm to women or other populations whose human rights and status might be negatively affected under the traditional status quo. There is also a need for longer-term livelihoods and economic recovery efforts in Iraq. While a full-fledged economic recovery program might not be within OTI’s manageable interests, progress that OTI can support in livelihood sectors in stabilization contexts deserves a more intentional discussion. Maintain ICRI’s record of largely unbiased and equitable programming to date, but beware of nepotism and favoritism. Maintaining perceptions of unbiasedness is especially important in Iraq’s context, where social tensions among various ethno-religious groups remain. While the program has been largely able to manage equitable programming, there are reports of and potential for nepotism and favoritism at the local level. Programs should proactively monitor and mitigate opportunities for nepotism and favoritism, and require accountability of both staff and local partners. Manage stakeholders’ expectations by providing clarity when conveying program capabilities and roles. To maintain a good reputation within the country and with beneficiaries, all program staff should be able to articulate the goals, capabilities, and constraints of the program when interacting with the Iraqi government and stakeholders and/or partners. Complement online communications with offline outreach efforts in a comprehensive strategy to reach a greater number and more diverse groups of residents. Social media is an effective tool to disseminate information about the progress of a community’s recovery. However, to increase reach to women and/or residents who cannot or choose not to use social media, the program should explore alternate strategies to communicate with them. Such a strategy might include leveraging traditional media such as TV, radio, and, especially, in-person communication opportunities, such as town halls and community meetings. Offline outreach activities can also help amplify social media outreach by increasing awareness of ICRI’s social media presence. Enlist community leaders to support program outreach. Future programming can build partnerships with community leaders and influencers to disseminate information and conduct outreach through popular social media networks. Integrate gender considerations throughout the Program Cycle, including during planning and proposal development; development of theory of change; program design; implementation and management; monitoring and evaluation; reporting, documentation and dissemination and research utilization. The program should reference existing gender analyses of local communities or, if need be, 9 It is also important to recognize that it may not always be possible to rely on the traditional livelihood sector. Modern market conditions and the direction of the country’s economy may require a shift. However, this still remains an important observation. ix | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV conduct its own comprehensive gender analysis to understand factors that contribute to gender-based marginalization and that are found across social structures at the individual, interpersonal, community, and policy/institutional level. Use the space opened in the aftermath of war to renegotiate gender norms in ways that further the goal of stability in Iraqi communities. The program should partner with civil society organizations led by people other than men to benefit from their experience navigating social structures and institutions in Iraq. 1 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV INTRODUCTION BACKGROUND AND PURPOSE This evaluation report presents the findings, conclusions, and recommendations from an analysis of the United States Agency for International Development (USAID) Office of Transition Initiatives (OTI) Iraq Regional Program (IRP), referred to within Iraq as the Iraq Community Resilience Initiative (ICRI), or “Ta’afi” (“recovery”) in Arabic. The evaluation examines ICRI’s work in Mosul and the Ninewa Plain, looking particularly at al-Islah al-Zirai, the Old City, and al-Tanak in Mosul and Bashiqa, Bartella, and Qaraqosh in the Ninewa Plain.10 The evaluation’s purpose is to support OTI, USAID, and other local and international stakeholders to better understand whether, how, why, and under what circumstances the program activities implemented in Mosul and the Ninewa Plain led to durable returns. In particular, the evaluation elucidates the pathways of change and how each of the sub-objectives—improving livelihoods, services, and social conditions—have affected the durability of returns. This evaluation also marks the end of the first phase of the ICRI program (June 2018–February 2022) and the transition to the next phase (September 2021–2024).11 Therefore, the goal of the evaluation is both to reflect on successes and lessons learned and to inform future programming. The report first summarizes the current Iraqi context. Then, it reviews the evaluation questions and presents findings, conclusions, and recommendations. CONTEXT Iraq has a long history of displacement, with the current and perhaps most widespread wave starting after the Islamic State of Iraq and Syria (ISIS) occupation and the resulting military campaigns. According to the International Organization for Migration (IOM), around 80 percent of the total displaced population in Iraq had returned as of July 31, 2021, and around 1.2 million people remained displaced. The majority of these internally displaced persons (IDPs)—980,000 (53 percent)—originated in four districts in Ninewa (Mosul, Sinjar, Telafar, and Ba’aj), followed by Salah al-Din (13 percent) and Anbar (11 percent).12 Key obstacles to return include damaged and destroyed houses, lack of basic services, loss of livelihoods, illegal occupation of homes and public institutions by squatters, multiplicity of armed actors, lack of security, movement restrictions, fraught intra-societal relationships, and fears of arbitrary detention. In Mosul, which accounts for 22 percent of the total returns in Iraq, there is also concern that the western side that sustained greater damage during the war is marginalized during reconstruction efforts. West Mosul is a majority Sunni area, with the exception of a Christian population in the Old City, while East Mosul comprises more minorities. There is a perception among Iraqis that West Mosul was supportive of ISIS because of the religious affiliation and the people of West Mosul believe that they are being punished for this perception.13 The Ninewa Plain, which is the most diverse 10 The Ninewa Plain is a region in Ninewa governorate that has a diverse population and consists of the districts of Hamdaniya, Shekhan, and Tel Keyf. 11 While the second phase of ICRI has a different strategic focus than the first, this evaluation provides some lessons on its strategy. 12 International Organization for Migration, “Iraq,” Displacement Tracking Matrix web page. 13 ISIS claims to represent Sunni Islam. 2 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV area in Iraq, is contending with continuing demographic change and the presence of competing armed actors. These factors complicate not only the desire to return, but the ability to remain after return. In light of these conditions, a number of international organizations are operating in communities recovering from ISIS occupation. This includes international organizations like IOM and the United Nations Development Programme (UNDP), governmental aid agencies like GIZ (which is part of the German Federal Ministry for Economic Cooperation and Development), and other nongovernmental organizations like Malteser International (which is the relief corps of the Sovereign Military Order of Malta). UNDP programs have focused on supporting micro, small, and medium-sized enterprises in various areas across Iraq and on promoting reintegration in post-conflict communities. GIZ works on a number of projects within Iraq, including reconstruction in Mosul and stabilization efforts in the Ninewa Plain and in other post-conflict communities. In the Ninewa Plain, Malteser International has a reconstruction program, which focuses on rehabilitating homes damaged from war, supporting small and medium-sized enterprises, promoting vocational training, and working on social cohesion. PROGRAM APPROACH Amid these many challenges, ICRI sought to contribute to stability in Iraq by improving living conditions in the communities most affected by ISIS’s occupation and supporting social recovery and durable returns of IDPs. OTI initiated the Iraq Program in June 2018 in communities newly liberated from ISIS in northern and western Iraq. Initially implemented as a Regional Ramp-up Option from the OTI Syria task order implemented by Chemonics International, the program responded to a U.S. foreign policy imperative to support persecuted minorities. In late 2018, OTI awarded Chemonics a task order for ICRI with an end date of February 2022. The program had two program objectives (POs): the first was to improve durable returns in areas liberated from ISIS (PO I), and the second was to mitigate discrete challenges that threaten overall stability (PO 2). These objectives contributed to the overall stability of Iraq (Program Goal), which is in the interest of United States foreign policy because durable returns of IDPs are expected to thwart disruptive demographic and social changes that undermine stability. PO 2 provides the program with flexibility to fill in immediate gaps and to respond to unexpected crises as they emerge. PO 1 is the focus of this evaluation. The program aimed to improve durable returns in Iraq through three sub-objectives: improving delivery of public services prioritized by communities (SO 1), expanding livelihood opportunities (SO 2), and improving social conditions related to general feelings of safety and individuals’ ability to go about their daily lives and interact positively with other community members (SO 3). While ICRI was implemented in four geographic areas, the evaluation examines ICRI’s work in just two areas: Mosul and the Ninewa Plain. EVALUATION METHODS OVERVIEW The evaluation aims to examine ICRI’s contribution to improving durability of returns in Mosul and the Ninewa Plain by providing basic services and livelihood opportunities and improving social conditions. The evaluation also examines the program’s communication strategy, focusing on social media, to explore its role in perception of recovery and returns. Evaluation questions are presented below, and Annex A provides more detail. 3 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV The evaluation is informed by document review and primary data collection using qualitative and quantitative methods. Qualitative data collection in Mosul and the Ninewa Plain included key informant interviews (KIIs) with ICRI staff, community leaders and members, government officials, and activity participants. The evaluation team also conducted an online survey. All instruments were developed in English and translated to Arabic. Annex B presents the detailed methodology, Annex C presents the limitations, and Annexes D and E present the English version of the consent forms and data collection instruments. Data collection was conducted September–December 2021 and focused on six communities: Bashiqa, Bartella, and Qaraqosh in the Ninewa Plain and the Old City, al-Tanak, and al-Islah al-Zirai (abbreviated as al-Islah in all exhibits) in West Mosul (Exhibit 1). 14 Most of the KIIs were conducted virtually, although some KIIs in Mosul were conducted in person. The quantitative survey was disseminated through OTI networks, data collector networks, and networks from Think Bank, a local firm subcontracted to recruit respondents from Mosul. The interviewees from Mosul were solicited from the list of local partners provided by OTI staff (from which the data collectors snowballed) and supplemented with interviews conducted by ThinkBank, a data-collection subcontractor. The KIIs from the Ninewa Plain were solicited through personal networks of the data collectors. 14 The Evaluation Team used purposive sampling methodology to identify research sites and respondents; therefore, the sample is not representative of the Ninewa Plain or Mosul. Evaluation Questions Question 1: To the extent that social recovery is a necessary condition for durable returns, how effectively has ICRI understood and addressed social recovery needs in areas liberated from ISIS? 1.a.: To what extent and how did the program adapt its strategy and theories of change (as they related to social recovery for durable returns) over the course of three years of programming? 1.b.: To what extent did activity outcomes contribute—intentionally or unintentionally, directly or indirectly—to allow for durable returns? What were the key pathways to change? Question 2: Whether and how has ICRI influenced perceptions of recovery among residents of areas liberated from ISIS? To what extent do these perceptions differ across various groups (region, gender, religious/ethnic group, etc.)? What are the pathways of change? 2.a. How have ICRI’s social media presence and community engagement efforts during activities contributed to this effort? 4 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Exhibit 1. Areas of data collection Note that Bakhdeda is referred to as Qaraqosh throughout the report, and al-Islah al-Zirai is abbreviated as al-Islah in all exhibits. Bakhdeda represents the older name that locals use in speaking in their Aramaic dialect. The KII sample was able to achieve a balanced representation of the six geographic communities, as shown in Exhibit 2). Exhibit 2. KIIs sample COMMUNITY COMMUNITY INTERVIEWEES The Old City 14 Al-Tanak 6 Al-Islah al-Zirai 5 MOSUL TOTAL 25 Qaraqosh 4 Bartella 7 Bashiqa 8 THE NINEWA PLAIN TOTAL 19 TOTAL ACROSS ALL SITES 44 Moreover, it was able to achieve ethno-religious representation as well (see Annex B). The survey reached at least 14 individuals in each geographic community (Exhibit 3). However, it is skewed more toward the Ninewa Plain than Mosul, largely due to the enthusiasm of the residents of Bashiqa (n=80). 5 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV However, the evaluation team found it difficult to reach women. Men agreed to speak with data collectors at three times the rate of women. Exhibit 3. Survey sample LOCATIONS COUNT The Old City 20 Al-Tanak 14 Al-Islah al-Zirai 14 MOSUL TOTAL 48 Qaraqosh 20 Bartella 22 Bashiqa 80 THE NINEWA PLAIN TOTAL 122 TOTAL ACROSS ALL SITES 170 Once collected, different team members coded the KIIs and the documents while maintaining inter￾coder reliability, and analyzed them using Dedoose qualitative data analysis software.15 Data analysis summaries were then created to reflect emergent themes from the various sources of data. The data were then organized into findings, discussed at various internal and external (with OTI) analysis workshops, and synthesized into the findings that are presented in this report. Annex C discusses the evaluation’s limitations and strengths. FINDINGS The findings below tell a story about the program’s impact on communities in Mosul and the Ninewa Plain, and help stakeholders understand whether, to what extent, and how the program contributed to its goal of creating conditions for durable returns. The findings are divided into five thematic areas and begin with contextual findings that provide background for understanding ICRI’s role in promoting durability of returns in Mosul and the Ninewa Plain. The second theme examines ICRI’s theory of change and tests the relationship between durability of returns and social recovery, where social recovery is defined by (1) access to services, (2) access to livelihood opportunities, and (3) general feelings of safety, individuals’ ability to go about their daily lives, and positive interactions with other community members. The third theme examines whether, to what extent, and how ICRI has contributed to social recovery, and, therefore, to durable returns.16 The fourth theme relates to ICRI’s communication strategy (largely social media) and examines its effect on awareness about ICRI, perceptions of social recovery, and encouragement of returns of those currently displaced. The last 15 Dedoose is a cross-platform application for analyzing qualitative and mixed-methods research. 16 To examine the impact of the program on durable returns through social recovery and to analyze the pathways of change, it is imperative to first examine the relationship between social recovery and durable returns. In other words, we first examine whether social recovery (as understood by the program team) encourages durability of returns, and then go on to analyze the program’s contribution to social recovery, and therefore, to durable returns. The ability to understand and explore these relationships allows us to provide a more nuanced understanding of how the program has influenced durable returns. 6 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV theme identifies program approaches that have been instrumental in achieving success. Annex F provides a mapping of findings to evaluation questions. CONTEXTUAL FINDINGS Finding 1: Mosul and the Ninewa Plain had different patterns of displacement. In the Ninewa Plain, displacement occurred en masse in the summer of 2014, as ISIS invaded the area. In contrast, displacement from Mosul was largely a result of the war to liberate Mosul in 2017. Residents of the Ninewa Plain returned in the early days of liberation, while those from Mosul returned after the dust of the war for liberation had settled. In June 2014, because of the military operations that began after ISIS took control of large parts of Iraq, thousands of families fled to other parts of Iraq or to other countries. At one point, the number of IDPs reached more than 5 million. 17 In recent years, many displaced persons have attempted to return to their places of origin, even as Iraqi authorities closed displacement camps. The first finding provides contextual information about displacement patterns of communities where the ICRI project undertook activities: the Ninewa Plain and Mosul. Interview data reveal different patterns of displacement and return between the Ninewa Plain and Mosul. KII respondents mentioned that residents of Bashiqa, Bartella, and Qaraqosh in the Ninewa Plain were nearly all displaced in the summer of 2014 by the encroachment of ISIS into their territories. Most respondents were internally displaced, with Shabaks (the majority of whom are Shi’a) fleeing to southern Iraq while other ethno-religious groups such as Assyrians and Yazidis fled to the Kurdistan Region of Iraq. “The entire area left, 99 percent left, we all had the same experience and when we came back, life went back to normal.” (Community Member Direct Participant, the Ninewa Plain, Male) Although all the interview respondents from the Ninewa Plain had been displaced, nearly all had returned to their community of origin, though some described a partial return in which they work in one city and live in another (for example, holding public employment in Mosul while residing in the Ninewa Plain). This pattern of return in the Ninewa Plain is supported by data from survey respondents. While the patterns of displacement were different across Mosul and the Ninewa Plain (Exhibit 4), more than a third of respondents reported that most people in their towns are back, and a very small percentage have partially returned with no trends regarding ethnic distribution (Exhibit 5). Survey respondents from Bartella were the most likely to report that almost everyone is back in their town; the residents of Bartella are currently largely Shabaks. Bartella has experienced demographic change in the last few years, during which the number of Shabaks increased relative to Assyrian and Yazidi residents. Having been displaced in 2014, the residents of the Ninewa Plain were displaced for longer than the residents of West Mosul. Consequently, during liberation, they followed the news closely. Some 17IOM, Iraq Displacement Matrix. 7 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV respondents described visiting their cities as soon as they possibly could in the earliest days of liberation in 2016 and 2017. By contrast, not all residents of West Mosul were displaced by the ISIS incursion. Interview respondents were more likely to report being displaced during the war of liberation, largely going to Baghdad or places in the Kurdistan Region of Iraq. Some respondents decided to stay despite the war and were never displaced at all. According to the survey (Exhibit 4), overall, 21 percent of respondents from Mosul were not displaced, the highest percentage being in the Old City (35 percent), compared to al￾Tanak where only 7 percent of respondents said they were not displaced. In addition, nearly half of respondents said that everyone or almost everyone had returned (Exhibit 5). This holds true across towns (al-Islah al-Zirai has a slightly higher percentage, by seven points). Exhibit 4. Patterns of displacement (n=168) Displaced respondents of Mosul, like those in the Ninewa Plain, followed the situation closely and returned as soon as they could: “The issue of return is a very necessary and inevitable issue. I was following the liberation operations with passion, and I was waiting for the hour when I heard my family be liberated. Immediately after my family was released, within a week, I returned to Mosul.” (Community Member, ICRI Participant, Mosul, Male) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain No Yes, inside Iraq Yes, outside Iraq 8 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Exhibit 5. Community perceptions of return (n=170) Finding 2: Returnees in Mosul and the Ninewa Plain plan to stay in their towns of origin and the majority want their children to grow up in these areas as well, suggesting durability of returns. However, there are some differences across ethno￾religious groups and genders. When asked if they plan on staying in their area for the foreseeable future, nearly 77 percent of survey respondents from the Ninewa Plain reported that they plan to stay in their current location for a few years; 63 percent said they would stay for the rest of their lives (Exhibit 6); and 65 percent said they wanted their children to raise their families in their town. However, there are some differences across communities. The percentage of residents looking to emigrate is lowest in Bartella, which has recently become a Shabak-majority town (even though it has been more diverse in the past). In Qaraqosh, however, the percentage of residents looking to emigrate is the highest; the desire to emigrate in Qaraqosh is so palpable that even Bartella residents mentioned Qaraqosh residents’ desire to emigrate. The desire to migrate and a history of previous migration are not exclusive to the city of Qaraqosh, but are also common to Assyrian Christians, many of whom are from Qaraqosh: “The ones who have a desire to migrate are our Christian brothers. They have the intention to leave the area, but we don’t know the reason. Most Christian families have not come back to Bartella, their houses are abandoned.” (Community Member, Non-Participant, the Ninewa Plain, Male) “Christians don’t want to return, I don’t know why, even though their houses are available, they came back at only 50 percent. Some people want to migrate, the Christians want to migrate.” (Community Member, ICRI Participant, the Ninewa Plain, Male) In contrast to findings from the interviews, in the survey, Yazidis and Sunni Arabs most frequently said their community is seeking to emigrate. Given that Sunni Arabs are few and far between in Qaraqosh, Bartella, and Bashiqa, this finding may be shaped by Sunni Arabs’ disproportionately high response rate. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Not many people have returned Some people/families have returned Almost half the people have returned Most people have returned Almost everyone has returned I don’t know 9 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV In the Ninewa governorate, Sunni Arabs are residents of Mosul and outlying villages in the Ninewa Plain, but not Qaraqosh, Bartella, and Bashiqa. Yazidis, however, are among the major ethno-religious groups residing in these areas and their professed desire to migrate (as expressed in the survey) is surprising. Exhibit 6. Respondents who plan to stay by city and gender (n=163) In Mosul, 43 percent of survey respondents said they would stay there for the rest of their lives and 19 percent said that they would stay for a few years, proportions that are lower than those observed in the Ninewa Plain. Almost 26 percent said that they do not know. The interviews showed respondents’ desire to remain in Mosul, but the respondents hedged by indicating that inadequate access to services or livelihood could affect their decisions. In other words, people want to stay in their towns, but they are not sure they will be able to. There is also a difference among genders in both Mosul and the Ninewa Plain. In both districts, a higher percentage of men than women said they would like to stay in their town of origin for the rest of their lives. Women tended to respond “I do not know” more frequently than men when asked about their decision to stay. Though there are no definitive data on why women answer in this way, a possible explanation is that women are less often heads of household in a male-dominated society, and accordingly, they view the decision to move or to remain as being outside of their control. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Plan to stay by city (n=163) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Male Female Male Female Mosul Ninewa Plain Plan to stay by gender (n=163) I do not know I want to leave soon Yes, for a few years Yes, for the rest of my life I do not live in my original area 10 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV DURABILITY OF RETURNS AND SOCIAL RECOVERY In this set of findings, we explore ICRI’s theory of change and explore the relationship between durability of returns and social recovery as defined by the program. Specifically, we examine how and in what ways residents’ access to public services, livelihood opportunities, and social conditions led to durability of returns. Social conditions are defined by the ICRI strategic framework as how people feel about living in their communities, their general feelings of safety, and their ability to go about their daily lives, practice their religion and culture, and interact positively with other community members. In essence, the program encourages returnees to stay in their communities by creating an atmosphere in which people feel they have access to public services, can earn a living, and feel safe and welcome to go about their daily lives. Finding 3: Factors encouraging returns and durability of returns in both regions include improved access to basic services, removal of rubble, and returnees’ love for their hometown. In the Ninewa Plain, improvement in security was an additional critical factor in encouraging return. Survey data suggest that improvement in services was critical to promoting returns and increasing durability of returns; two-thirds of respondents from the Ninewa Plain strongly or somewhat agreed that they moved back to the area because services in their town were improving. Similarly, 72 percent of Mosul respondents strongly or somewhat agreed that they moved back because of improved services, with the highest percentages from al-Tanak (83 percent), and with al-Islah al-Zirai and the Old City following at 58 percent and 50 percent, respectively. “Yes, I will stay. Right now, the situation is very good, services are available from the health and education and clean drinking water.” (Community Member, ICRI participant, Mosul, Male) “I think if there is a relationship [between services and emigration] the reason is that without a hospital and medical team, a person may be afraid of getting sick one day or his parents and that there isn’t enough time for them. So, this may motivate people to leave. Ten days ago, we had a young man who had a stroke and died, but if there is a hospital in the region, we could have saved him. But there is not enough time to drive them to the next hospital. My own uncle died because of this, he died in a few hours because no hospital is nearby.” (Community Member, ICRI Participant, the Ninewa Plain, Male) Interview respondents also identified the removal of rubble and rehabilitation of destroyed homes as another reason contributing to their decision to return and as motivators to stay. Removal of rubble and rehabilitation of homes not only served their immediate purpose, but also improved residents’ feeling of well-being (Finding 5). Another recurring theme in interview and survey data was respondents’ sense of loyalty and love for their town of origin as a reason for return and motivation to stay. In the Ninewa Plain, 83 percent of survey respondents strongly agree or somewhat agree that missing their hometown was a significant factor that motivated their return. Respondents’ love for their land was shared across genders and ethnicities, and across the three towns of the Ninewa Plain. Interview data show that love of town of origin also plays a significant role in respondents’ decision to stay. In interviews from the Ninewa Plain, Yazidi and Assyrian respondents described their love of their community as an attachment to an ancestral homeland. For example, in describing his decision to return, one Assyrian respondent from Bashiqa said: 11 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV “From the first moment we got out of the area, we were destined to return to the area, this is our land and the place of our birth. The first person in this world was there.” (Local Partner, Non-Participant, the Ninewa Plain, Male) Similarly, in Mosul, 76 percent of respondents strongly or somewhat agreed that they moved back to their town of origin because they missed it. This percentage was highest in the Old City (91 percent), followed by al-Islah al-Zirai (75 percent) and al-Tanak (67 percent). There was also a significant difference between genders in Mosul, with a larger percentage of men agreeing with the statement (almost 72 percent) than women (56 percent). Sunni Arabs, the majority residents of the area, tended to agree most (87 percent). This sense of loyalty and affection for their land sometimes even emerged as more important than other factors. “As holders of higher degrees, we were offered job opportunities outside Iraq, but the sense of belonging to this city prevented us from leaving it. On the contrary, we were motivated to build and rebuild it again.” (Local Partner, ICRI Participant, Mosul, Male) While improvement in services, removal of rubble, and love of land emerged as dominant reasons for return and motivators to stay, difficult conditions at places of displacement and improved security also played an important role in motivating return. Difficult conditions at places of displacement were a more significant motivator to return in Mosul than in the Ninewa Plain. Almost half of the Ninewa Plain respondents and 68 percent of respondents from Mosul at least somewhat agreed that they returned because the conditions where they had been displaced were bad. In interviews, Mosul residents mentioned difficulties such as water shortages in summer and not having enough money to rent a house at the displaced destination. In the words of a female community member: “Displacement is difficult, services are not available, children are homeless. I lived with 75 people inside a house when I was displaced during the days of fear. There was bombing and lack of water in the summer. The motives for returning: the area was left behind, the house was there, and we came back.” (Community Member, Non-Participant, Mosul, Female) While most motivators for return are similar between Mosul and the Ninewa Plain, one significant difference is security. Interview data show that improved security (including de-mining) played a significant role in respondents’ decision to return to the Ninewa Plain after the liberation of the territories by Iraqi forces. Further, 86 percent of survey respondents from the Ninewa Plain at least somewhat agreed that they returned because of improving security conditions. In Mosul, security did not emerge as an important motivator in interviews and a lower percentage of survey respondents (68 percent) agreed that improvement in security was a variable in their return. Finding 4: Economic and livelihood opportunities are key factors in motivating people to stay in their towns of origin in both districts, and lack of opportunity is a significant driver of migration, especially in the Ninewa Plain. Livelihood opportunities emerged as one of the most important factors for respondents in deciding whether to stay in their communities of origin, even though economic factors did not appear as a dominant reason for return in either the Ninewa Plain or Mosul. Still, livelihood opportunities remained important, as 50 percent of the Ninewa Plain respondents and 42 percent of Mosul respondents agreed with the statement that they returned to their town of origin because they had a business to take care 12 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV of. A lack of livelihood and economic opportunities also emerged as the dominant motivator for emigration for all current residents, especially in the Ninewa Plain. In interviews, lack of or limited employment opportunities for individuals was highlighted by community members and local leaders in the Ninewa Plain, with one community leader calling it the greatest challenge. A Shabak community member mentioned that employment opportunities were better in Bartella than Qaraqosh, where most people were either salaried or retired. The lack of employment opportunities, according to respondents, was the largest motivator for migration. In qualitative interviews, nearly all respondents described youths’ desire to emigrate as being motivated by the high unemployment rate, particularly for graduates. “Migration is wanted by most youth of all ethnic groups. They graduate at 22 and they face a harsh reality: no public employment and no private sector employment. They have high degrees and go work crazy jobs for 10 thousand dinars a day [about $7], of course he will prefer migration. We would like to stay if we can, but there’s no work. You saw the protests in Baghdad in October, people have no job opportunities. This is true of all of Iraq. The economic situation is very bad. We want migration, doesn’t matter if it’s outside or within Iraq, we want to invest our degrees.” (Local Partner, ICRI Participant, the Ninewa Plain, Male) Unlike in the Ninewa Plain, qualitative interview respondents from Mosul less frequently cited livelihood or employment as a factor in promoting durability of returns, or lack of opportunities as motivating emigration. However, the sentiment was still there: “Yes, some do [want to emigrate], and the reason for emigration at the present time is the lack of job opportunities, before there were security reasons, but the search for work is not easy. The weakness of the private sector is one of the reasons for the scarcity of job opportunities” (Community Member, ICRI Participant, Mosul, Male) Respondents were careful to note, however, that the economically driven desire for emigration was not restricted to residents of the Ninewa Plain and Mosul, but one that was exhibited by Iraqis in other areas that had been untouched by ISIS. They understood that the devaluation of the dinar, the general crisis of public-sector employment, and the financial crisis brought on by the COVID-19 pandemic and the subsequent crash in oil prices have affected the country as a whole. Finding 5: Social conditions, defined as residents’ general feelings of safety and their ability to go about their daily lives and have positive interactions with community members, do not seem to be a significant obstacle to returns or durability of returns. Residents of Mosul and the Ninewa Plain characterize relations among neighbors, even if from different ethno-religious groups, as strong and positive. However, demographic change and perception of residents as ISIS sympathizers in the Ninewa Plain threaten these positive feelings. When asked about general feelings of safety, individuals’ ability to go about their daily lives, and interaction among community members, residents from Mosul and the Ninewa Plain describe relations between neighbors as being strong and positive and their social lives as safe: 13 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV “Communication exists, thank God, with the neighbors, and we help each other, just as before. And our neighbors are the main reason why we went back to the city. After displacement, communication with neighbors increased” (Community Member, ICRI Participant, Mosul, Male) "Personally, my relationship with my neighbors and members of my community is very good and I can say it is excellent, I helped with a lot of voluntary activities inside my community … Whenever aid would come, we would distribute it voluntarily inside the community, me and the mukhtar and other important individuals in the community." (Community Member, ICRI Participant, Mosul, Male) Respondents from Mosul and the Ninewa Plain indicated that one factor that generated and, in some cases, strengthened relationships between neighbors was the collective trauma of war and displacement. They described relations between neighbors as positive because they “left together and returned together” and shared a common story of victimhood, displacement, and return. They mentioned that this experience strengthened their relationships and facilitated cooperation and increased optimism in dealing with the recovery process. “Social relations are strengthened and social bonds are strengthened because the city has reached the bottom; the bonds are strengthened between the classes of society and the people of the same neighborhood. Relations have become stronger because of the circumstances that have occurred” (Local Partner, ICRI Participant, Mosul, Male) “When they [residents] came back, they came back together… People left Bartella together and came back gradually together, so the relationships are good." (Local Partner, ICRI Participant, the Ninewa Plain, Male) Another factor that seems to have played a role in Mosul is the further homogenization of the city that began during the civil war (which took place from 2005 to 2008 and lasted through the war with ISIS). The sectarian civil war drove non-Sunni Arabs out of the city in waves (including Shia Shabaks and Christians). This displacement was then exacerbated by ISIS’s predatory tactics toward non-Sunnis. By the time liberation occurred, the once diverse population of Mosul was homogenized to just Sunni Arabs, with few minorities (Shabaks, Christians, Kurds) feeling comfortable enough to remain. As a result of the homogeneity of the city, issues in the last two to three years have centered on social conditions among different geographic groups in Mosul rather than among ethno-religious groups. Specifically, West Mosul, which sustained more damage during the war and from where ISIS had made its inroads and incursions into the city, suffered social stigmatization. Residents of West Mosul felt marginalized by fellow Mosulis and by the government and other actors working in reconstruction. In the Ninewa Plain, the three sites of Bartella, Bashiqa, and Qaraqosh remain more diverse than Mosul. The collective trauma of war has resulted in stronger relationships among these diverse neighbors, with Yazidis, Christians, Kakais, and Shi’a Shabaks speaking positively and fondly of their life with their neighbors, who, in many cases, are from a different ethno-religious background than their own. However, it is important to note that some fault lines rooted in demographic change exist within the communities and need to be managed to ensure that these positive feelings remain in the future. In the Ninewa Plain, qualitative interview data revealed that respondents are concerned with demographic change favoring Shabak migration. Assyrian and Yazidi respondents in the Ninewa Plain expressed discomfort about demographic changes in their region that have resulted in an increased number of Shabaks moving into the urban centers of the Ninewa Plain. Bartella is described as witnessing the most demographic change, and Bashiqa residents worry that their town is following in its footsteps. The fear 14 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV is so palpable that one Christian community leader in Bashiqa described the creation of a local council that regulated the buying and selling of property in order to preserve the demographic contours of the area, though this supposed council was not recognized by others interviewed from Bashiqa. Residents of the Ninewa Plain have previous experience with population change, though the current degree of displacement post-ISIS war is unprecedented. “Yes, 100 percent, I believe that the identity of my city is changing slowly. I can feel it, it is tangible, there is a lot of migration from Shabaki villages―Shi’a Muslims―they are trying in every way to come live in Bashiqa and change it demographically, same as they did in Bartella. Their numbers are increasing and it is tangible and it is not in the best interest of our area. We respect them, we are all Iraqis. But their intentions are not clean.” (Community Member, Non-Participant, the Ninewa Plain, Male) These findings are also substantiated by the document review, which found that fault lines exist between Christian and Shabak communities in Bartella. Data from the survey (Exhibit 7) underscore the demographic change experienced in the Ninewa Plain. More than 50 percent of the Ninewa Plain respondents mentioned that there were still groups of people that had not returned to the area, with the highest percentage in Qaraqosh. It is important to note that in qualitative interviews some of the more informed respondents were able to temporally trace the history of displacement patterns and asserted that demographic changes in the Ninewa Plain preceded the war with ISIS and were not solely because of it. These reactions imply that some community members understand the complexity behind displacement and the multiplicity of factors and actors that led to it. In other words, they do not see demographic change as stemming solely from the ISIS invasion, but they understand that it has happened over waves historically and, on because of that, do not bear animosity toward any ethno-religious group for being a participant in it. Exhibit 7. Demographic changes noted by survey respondents in the Ninewa Plain and Mosul (n=166) Another fault line expressed by respondents is distrust, especially in the Ninewa Plain, of both new and longtime residents who were or may have been ISIS sympathizers. Some respondents in the Ninewa Plain expressed fear of a resurgence of the mentality that brought about ISIS. This is particularly true of Yazidis and Christians who were victimized during ISIS's rule. 0% 10% 20% 30% 40% 50% 60% 70% 80% All Ninewa Plain Qaraqosh Bashiqa Bartella All Mosul The Old City Al-Tanak Al-Islah Ninewa Plain Mosul Nothing has changed New groups of people have moved in People live in different parts There are still groups who have not returned I do not know Other 15 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV “Our society is a conservative rural tribal society, and they maintain their relations as they are, except for the families of ISIS, they became a rift and enmity, not only that I, of course, even prevented them from entering the neighborhood, and whoever entered the neighborhood in any way was deported. Until the state ordered them to return and they returned to their children, killed or convicted; the people who intervened are the Imams of the mosques that did not accept them in society.” (Community Member, Non-Participant, Mosul, Male) ICRI AND DURABILITY OF RETURNS The following set of findings focuses on understanding the impact of ICRI programming in contributing to durability of returns by improving access to services and livelihood opportunities and by affecting social conditions. These findings examine whether ICRI responded to communities’ needs for basic services and livelihoods, thereby encouraging durability of return. They also analyze the contribution ICRI activities have made to improving social conditions for community members. Finding 6: ICRI is praised for improving basic services in both Mosul and the Ninewa Plain, which residents report has improved community members’ quality of life and feelings of well-being. Respondents especially appreciated infrastructure activities, including rubble removal, and viewed these improvements as contributing to durability of returns. Residents of both Mosul and the Ninewa Plain praise ICRI for having provided basic services in their districts. Document review data show that ICRI activities focused on providing a range of critical services. These activities included provision of crucial equipment to enable or enhance functionality of various city departments such as health centers, civil registry records, schools, waste management, fire control and water services, and public libraries. Primary data from interviews and surveys also highlight ICRI’s contribution to providing basic and critical services to the residents of Mosul and the Ninewa Plain. Generally, respondents were highly appreciative of the services provided by ICRI. They often compared ICRI’s provision of services to the central government in Baghdad, which is seen as doing far less work.18 Respondents differentiated between the work of the central government and of the local government; they viewed the latter as being much more engaged and productive. Respondents mentioned services related to roads, water, electricity, street lighting, provision of materials to hospitals and schools, and rehabilitation of parks and 18 The project objectives did not specifically include state perception of government, citizen-state relations, or trust in the government even though ICRI collaborated with local government officials for programming. If improvement of citizens’ perception toward government or citizen-state relations is important to the program, this is a potentially important finding. The literature suggests that citizens’ perception of the state’s ability and willingness to provide services plays an important role in maintaining citizen-state relations conducive to state stability. For example, see: Kooy, M., Wild, L., and Mason, N., “Doing Things Differently: Can Water Supply, Sanitation, and Hygiene Services Support Peace-and State-Building Processes?”; McLoughlin, C., “When Does Service Delivery Improve the Legitimacy of a Fragile or Conflict-Affected State?”; Denney, L., Mallett, R., and Mazurana, D., “Peacebuilding and service delivery.” 16 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV public spaces when mentioning ICRI activities that were beneficial to the community.19 A quote from a government official from Bartella sums up this sentiment: “They have given a lot to Bartella, they created a park that ties various neighborhoods, it helps the municipality but it also helps the citizens primarily. They also rehabilitated industrial stores. Painting shops, etc. They did more than one project in Bartella and their projects are successful.” (Government Official, Non-Participant, the Ninewa Plain, Male). "The biggest victory in Bashiqa is the municipality of Bashiqa, in the last two years, they worked amazing work, they revived the area from the start. They paved the roads, they created parks, in every area there is a park―especially women, they don’t have public areas for them to go to, now families can go and meet up and have place to breathe." (Community Member, ICRI Participant, the Ninewa Plain, Female) Survey data also reinforce these findings; the majority of respondents from both Mosul and the Ninewa Plain report that infrastructure and services have largely improved in the last two to three years (Exhibit 8). In the Ninewa Plain, survey respondents most frequently reported that the greatest improvement was in infrastructure. Eighty-three percent of respondents said that the infrastructure had at least somewhat improved, followed by electricity (65 percent), water (62 percent), health care (60 percent) and education services (43 percent). In Mosul, quantitative survey respondents most frequently reported improvements in water services (83 percent), followed by infrastructure (80 percent), and then electricity (83 percent). 19 While respondents were not always able to link certain activities to ICRI, they mentioned USAID or, more generally, “organizations.” The evaluation team was able to link activities that respondents shared with ICRI with WHOSE database of activities implemented in each location. 17 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Exhibit 8. Perceptions of recovery across sectors 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Infrastructure (n=160) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Electricity (n=159) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Water (n=160) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Healthcare (n=157) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Education (n=156) Got worse Has not improved Has somewhat improved Improved a lot I don't know 18 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV In addition to responding to residents’ immediate needs, these services have also improved their everyday lives and feelings of well-being, which has clearly motivated people to stay.20 Respondents, especially women, cited rubble removal and restoration of parks and other social spaces as supporting a sense of well-being as they replaced images of war and provided an avenue for families to spend their time. One respondent in the Ninewa Plain speculated that opportunities for family recreational time also led to a reduction in domestic violence, which has risen during the COVID-19 pandemic. Street lighting has led to improved security and increased residents’ ability to gather, leading to a better social life. Respondents also mentioned that rehabilitation activities have led to increases in property values for those living in close proximity to these projects in both Mosul and the Ninewa Plain. The cumulative effect of ICRI-financed infrastructure and service provision has been motivation to return and stay. “The activities helped people who needed help and it gave a push to those who wanted to stay to stay in their homes and to be attached to their land.” (Government Official, Non-Participant, Mosul, Male) “These projects helped to return this area because they are considered central to the return of people. I can’t return to my area without water and electricity and clean and paved roads.” (Community Member, Direct Participant, Mosul, Male) Another significant success of ICRI is that, according to most respondents, these activities benefited all residents equally and with little bias. Respondents did not believe that some benefited at the expense of other communities: “It is not necessary that I personally get the benefit. It is for the benefit of all people. Those who have children, their kids have to go to school. People received support to rebuild their houses. Rubbish had been removed. Who else benefited from this activity? All people benefited from it, especially those in need.” (Community Member, Non-Participant, Mosul, Female) While ICRI was largely praised for its work, a few negative observations emerged. A few respondents viewed USAID as focusing disproportionately on Christians and less on other communities. Others were disenchanted with local ICRI staff and implementing partners, saying there is an aspect of nepotism. These respondents provided specific examples of ICRI activities where they thought that nepotism and favoritism had occurred, such as cases where individuals received resources when they should not have, according to the selection criteria put in place for the activity. This seems to be a functional issue not related to beneficiary selection processes. Female respondents also noted the absence of programming focused on women. For example, a female respondent suggested that the program could focus livelihood opportunities on providing support for businesses related to cooking food because many women can benefit from it. While there have been many changes in the past two to three years both in recovery of public services and improvements in infrastructure in Mosul and the Ninewa Plain, a high percentage of respondents felt that the pace of recovery in services and infrastructure has been very or somewhat slow (Exhibit 9). These findings were corroborated by interview respondents who mentioned poor water supply and inaccessibility of proper health care services (especially in Bashiqa) in the Ninewa Plain. 20 Improvement in well-being is defined as improvement in social conditions (SO 1.3), suggesting that contribution toward SO 1.3 was achieved not only through activities labeled under the sub-objective but through the cumulative effect of all activities. Also see Finding 8. 19 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Exhibit 9. Pace of recovery in services vs. infrastructure Finding 7: Respondents from both the Ninewa Plain and Mosul have appreciated ICRI’s livelihood activities and perceive these activities as creating further employment opportunities, thereby promoting durability of returns. Many residents expressed a desire to see more of these types of activities, with unemployment commonly cited as the greatest challenge in both districts (especially the Ninewa Plain). ICRI’s programming included several livelihood activities geared toward providing sustainable economic assistance to help with the recovery process. Many partners involved in the implementation of these activities perceived implementation as being important for creating further employment opportunities, which is expected to contribute to residents’ decision to stay in their community of origin. According to document reviews, ICRI’s livelihood activities in the Ninewa Plain included assistance to aid the restoration of car shops, fodder-selling shops, veterinary pharmacies, chicken hatcheries, poultry shops, and processing plants. Two of the most memorable livelihood projects implemented by ICRI, as reported by the respondents, were the rehabilitation of the industrial sector in Bartella and support for poultry farms in various areas, which was viewed as equitable across ethnoreligious groups. ICRI’s livelihood activities in Mosul included rehabilitation of several shops and businesses such as shops in the Blacksmith market, Al-Sarjkhana market, and Al-Sha’areen, Al-Makkawi, and Al-Farooq markets in the Old City. In addition to direct support, ICRI also provided entrepreneurial, project management, and networking training to increase participants’ business skills. Given the importance of livelihood 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Services (n=161) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Infrastructure (n=162) Very slow Somewhat slow Neither slow nor fast Somewhat fast Very fast I don't know 20 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV opportunities in community members’ decision to stay (Finding 2), it is safe to suggest that ICRI has positively contributed to durability of returns through these activities. Survey results provide additional nuance to this finding in that while both Mosul and the Ninewa Plain residents at least somewhat agreed that job opportunities have improved (Exhibit 10), the percentage of respondents agreeing was much higher in Mosul (60 percent), with the highest in the Old City (almost 75 percent), than the Ninewa Plain (almost 30 percent). This is especially surprising because the program implemented a higher number of livelihood activities under SO 1.2 in the Ninewa Plain than in Mosul. Exhibit 10. Improvements in job opportunities (n=159) ICRI was largely able to maintain the perception of equity in the implementation of livelihood activities across ethno-religious groups. Most respondents described livelihood activities as equitable and benefiting all ethno-religious groups equally. When targeting was needed because of scarce resources, respondents suggested that the targeting strategies made sense. In the words of a respondent who benefited from support to poultry farm activities: “I benefited through a project to restore the poultry project, which was one of the projects that provided a service to the region in general. They worked equally in the area for Muslims and for Christians, they choose two projects from each sect.” (Community Member, Non-Participant, the Ninewa Plain, Male) One grantee noted that the activity that rehabilitated the shops was designed strategically to ensure diversity of the beneficiary pool and the non-redundancy of services provided. “The shops were selected based on specific criteria by the grantee to diversify the market and to avoid any duplication. The rehabilitated shops are varied like shops for selling cloth, dairy products, house items, foodstuff, coffee, vegetables, and fruits, and shops for fluffing wool, sewing curtains, repairing electrical appliances and air conditioners, barber, electrical appliances, and electronic games, bicycle repair, and a refreshment shop.” (Program activity evaluation reports, the Old City, Mosul activity) However, the evaluation team did come across at least one incident where there was evidence of some corruption. Two respondents from Qaraqosh, referring to an activity that provided recipients with funds to purchase equipment for starting a business, suggested that it was abused by many individuals. One of the respondents who received funds for this activity described his experience as follows: “Organizations [implementing ICRI program], not all of them, also have “wastas” (connections). Someone tells people they will register other people, and some people don’t get registered. There is not enough 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Got worse Has not improved Has somewhat improved Improved a lot I don't know 21 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV equality sometimes, in front of me two times, I saw a guy in Qaraqosh who runs these workshops, he registered his friend who is employed and not another guy who was unemployed and needed it more.” (Community Member, ICRI Participant, the Ninewa Plain, Male) Despite the efforts at improving livelihood opportunities, respondents from both districts indicated that more needs to be done in this area, especially in the Ninewa Plain. Respondents from the Ninewa Plain specifically mentioned the need for more livelihood opportunities targeted at agriculture, the historically dominant livelihood sector in the area. “Our regions depend on agriculture and livestock, and the organization did not take this seriously. A group of projects proposed by local organizations were not sufficient or broadly covering the region.” (Community Member, ICRI Participant, the Ninewa Plain, Male) Another gap mentioned by female respondents was the lack of livelihood activities targeted at women. One respondent, for example, suggested supporting women in using their cooking skills to generate and sustain income. “There are a lot of women at home who are housewives who are excellent cooks. They don’t have an income, a really bad income, the organization can open a plant for them―a really simple one―a factory for foodstuffs, these women can support their homes with it. The factory is useful for society. Activities can be designed for women in mind. We always import from Qaraqosh, but why not create our own stuff? We can empower women and you don’t need to give them handouts.” (Community Member, ICRI Participant, the Ninewa Plain, Female) Finding 8: ICRI’s impact on social conditions is difficult to determine. The absence of evidence likely emanates from the fact (Finding 5) that aspects of social conditions the program targeted were not an obstacle in promoting durability of returns in the first place. ICRI implemented activities that targeted improving general feelings of safety, individuals’ ability to go about their daily lives, and interaction among community members. These activities focused on cultural awareness, peaceful coexistence workshops, festive/cultural activities, restoration of recreational facilities, sporting tournaments, reconstruction of churches/public facilities, commemoration of traumatic and historic events (e.g., Five Years of Resilience: Commemorating the Yazidi Genocide in Hamdaniya and Memorial for the Victims in the Ferry Disaster in Mosul). ICRI implemented a substantially greater number of activities in Mosul than in the Ninewa Plain under S.O. 1.3 (23 in Mosul and 6 in the Ninewa Plain in the communities selected for this evaluation). In addition to activities implemented under SO 1.3, activities implemented under other strategic objectives were also aimed at contributing to bridging ethno-religious barriers. For example, the program encouraged interaction among members of different groups through the Improving Pedestrian Conditions in (Qaraqosh) activity. The survey results show that social conditions and interactions among community members have improved over the last two years (Exhibit 11). Seventy-nine percent of respondents from Mosul indicated that social conditions have at least somewhat improved in the last two to three years, with the highest percentage reported in al-Tanak (92 percent), followed by al-Islah al-Zirai (85 percent). The lowest level of agreement was reported in the Old City (65 percent). By comparison, a survey of residents of the Ninewa Plain indicated almost 69 percent of respondents agreed that social conditions, particularly social interactions, have improved in the last two to three years, with Bartella reporting 22 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV slightly better relations than the other two towns. More women reported improvement in social conditions than men (78 percent of women and 66 percent of men). Exhibit 11. Improvement in social relations (n=163) However, it is difficult to determine how ICRI contributed to these perceptions because interview respondents rarely identified ICRI activities aimed at improving social conditions. The respondents did not talk about any ICRI activities in their areas (including the ones that were identified by program activity evaluation reports) 21 as having a positive impact on social conditions, even when prompted. 22 Only three community members and one local partner talked about workshops and seminars focused on peaceful coexistence between different ethno-religious groups (see quotes below). Additionally, some activities that the program implemented under SO 1.3, which focused on improving social conditions, did not contribute to positive interaction, but instead contributed to feelings of well-being (Finding 6). "The youth in the city have become very aware of everything and the spread of awareness among the youth, they overcame the things that the events [of ISIS] planted in their hearts and they are more aware now; Peaceful coexistence that the organizations participated in and cultivated in them through seminars and conferences; The city has changed positively and improved on the cultural and social level" (Local Partner, ICRI Participant, Mosul, Male) "To be frank, when we first went back, it was very tense. If we heard someone coming from Fadliya, we would call the national police immediately or we would take care of it … But there have been a lot of co￾existence workshops – we tried to create workshops with our youth and with youth from Fadliya, they can’t all be ISIS sympathizers … this isn’t going to take in the next two years, but there is a huge difference. When Fadliya sheikhs come to Bashiqa, there is a welcoming of them and they are being proactive. I think there is improvement." (Community Member, ICRI Participant, the Ninewa Plain, Female) 21 These reports are produced by the program’s monitoring, evaluation, and learning team and our mandatory for each activity. 22 Document review revealed multiple instances where post-activity evaluation pointed to positive impact of activities on social conditions. However, the evaluation team was not able to substantiate these impacts during data collection. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain Got worse Has not improved Has somewhat improved Improved a lot I don't know 23 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV "One of the very important achievements … Many of the projects and activities that took place inside Mosul, such as the Peace Festival, the Book Sidewalk, and others."(Local Partner, ICRI Participant, Mosul, Male) The evaluation team conjectures that the absence of evidence of ICRI’s influence on social conditions emanates from Finding 5 on the relationship between social conditions and social recovery. Finding 5 explains that improvement in social conditions was not a contributor to durable returns in Mosul and the Ninewa Plain. In fact, the quality of interaction among community members within their own neighborhoods (even when multiethnic) was not a critical issue. Respondents attributed positive relations to the bonds created by the collective trauma of war and displacement. Since the returns took place before ICRI started operations in both districts, there was less of a need to focus on positive communal interactions. Therefore, improvement in social conditions and interactions with others in their community as a result of ICRI did not come across as a strong theme in interviews. Given these observations, the program team’s decision to move the improvement of social conditions to a program sub-objective rather than keeping it as a main program objective made sense, and seemed to be the right decision at the time. However, as this report’s conclusions and recommendations explain, focusing on different aspects of social conditions (e.g., issues related to demographic change and attitudes toward perceived ISIS sympathizers) may be beneficial for the program. ICRI, INFORMATION SHARING,AND SOCIAL MEDIA Finding 9: Most individuals in the Ninewa Plain and Mosul have heard about ICRI, primarily from social media, and recognize associated hashtags (especially #Taafi and #USAID). However, many are not able to connect certain activities in their communities to ICRI. The overall aim of ICRI’s media strategy was to raise awareness among IDPs and residents about the progress and recovery process in Mosul and the Ninewa Plain with the hope that this awareness would also encourage returns. Survey and interview data sources from both Mosul and the Ninewa Plain suggest that ICRI is popular among residents of these regions. According to interviews in Mosul and the Ninewa Plain, both community members and local partners were impressed with the work ICRI did. Even if most respondents were not able to relate certain activities to ICRI or name specific activities implemented by ICRI, most respondents identified ICRI as a key implementer in the recovery process in their communities. Others who did not recognize ICRI were still knowledgeable about USAID’s work in general. Survey evidence substantiates this result (Exhibit 12). Almost 92 percent of respondents from Mosul reported that they were aware of ICRI. This percentage dropped to 78 percent in the Ninewa Plain, but this is still substantially high. ICRI is least well-known in Bashiqa, where 59 percent of respondents said they were aware of ICRI. Another important difference is between genders. A higher percentage of men reported awareness of ICRI than women in both Mosul and the Ninewa Plain. Women in the Ninewa Plain are the least aware of ICRI (45 percent). 24 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Exhibit 12. ICRI awareness by city and gender In terms of sources of information, respondents indicated they had seen information about ICRI’s work on social media (especially Facebook, Snapchat, and Instagram). Some had viewed highlights of ICRI’s work on satellite channels or on its website, and others heard about it through word of mouth via grantees or through visibility in the city, such as seeing logos on equipment or vehicles. The survey findings reinforce this, showing that 64 percent of respondents in the Ninewa Plain and 47 percent of respondents in Mosul who had heard about ICRI had heard about it from social media (Exhibit 13). The highest percentage of respondents who relied on social media came from Bartella (68 percent) while the smallest percentage came from al-Islah al-Zirai (33 percent). Other sources of information included other people; 33 percent of respondents in Mosul and 20 percent in the Ninewa Plain relied on other people. Stickers and flyers also played a role. Evidence from the document review stressed the importance of social media, highlighting the Facebook pages of the Ninewa Media Center and the Directorate of Electricity. Besides social media, key informants from both Mosul and the Ninewa Plain mentioned obtaining information about the recovery process and ICRI’s work via traditional satellite channels as well. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain By City (n=169) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Female Male Female Male Mosul Ninewa Plain By Gender (n=169) No Yes 25 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Exhibit 13. Sources of information about ICRI by city and gender There is also a significant difference across genders; men and women, especially in the Ninewa Plain, receive information differently. Women in the Ninewa Plain rely predominantly on social media, while men get their information from varied sources, including from other people. Women in Mosul, albeit a small percentage, rely more on TV, radio, and other media to receive information about ICRI. Delving deeper into the use of social media for ICRI awareness, the evaluation team also analyzed survey data on awareness of different hashtags that ICRI has employed to promote its work on social media (Exhibit 14 and Exhibit 15). ICRI’s hashtags are recognizable to residents of Mosul and the Ninewa Plain, particularly #Taafi and #USAID. Interview respondents recognized these two hashtags most frequently as well. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain By City (n=161) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Female Male Female Male Mosul Ninewa Plain By Gender (n=161) Signs and flyers Other media (TV, radio) Other people Social media Elsewhere 26 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Exhibit 14. Hashtag awareness in Mosul (n=48) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% comebackandlearn keepitclean fortheiraq rejoiceyouareback unitedstateagencyforinternationa alhamdania_recovering backtolife recover mosul_recovering All Mosul The Old City Al-Tanak Al-Islah 27 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Exhibit 15. Hashtag awareness in the Ninewa Plain (n=122) However, these findings must be tempered by the acknowledgement that social media has limitations. Some interview respondents mentioned that many residents may not have access to social media or may not be familiar with using social media, and that only individuals who have been directly involved in ICRI or are involved in development of their communities follow ICRI on social media. Since the survey conducted for this evaluation was conducted on social media, there is a definite bias toward people who are savvy on social media and it has likely overestimated the popularity of ICRI-related social media among residents of Mosul and the Ninewa Plain. The following quotes capture these limitations: “The people who work in civil society know these pages but the ones who do not work in this field, only a few of them know about these pages.” (Community Member, ICRI Participant, the Ninewa Plain, Male) “It depends on the culture and awareness of the person. Simple people, I don't think there is a possibility they follow these pages, but conscious people consider them and have a desire to look forward.” (Local Leader, ICRI Participant, Mosul, Male) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% comebackandlearn keepitclean fortheiraq rejoiceyouareback unitedstateagencyforinternationa alhamdania_recovering backtolife recover bartella_recovering mosul_recovering All Ninewa Plain Qaraqosh Bashiqa Bartella 28 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV "Last two years, they had a great media presence, but I don’t follow a lot. They worked more than anyone here, Ta’afi [ICRI] was very active" (Community Member, ICRI Participant, the Ninewa Plain, Male) Another theme that emerged from the interviews was that ICRI’s offline outreach complemented its online popularity, i.e., community members recognized ICRI on social media because it had a presence on the streets. In the words of a community member: “These pages are known in the areas where they work like USAID on Facebook is known and followed by many people because USAID is one of the organizations helping our area in all of Mosul, and that cannot be denied. If you tour Mosul today you will hear a lot about USAID from people, even if you are just driving around in a car, I see the municipality cars having the USAID logo, the sanitation workers, the paving of the streets even some of the emergency vehicles.” (Community Member, ICRI Participant, Mosul, Male) Street presence and offline activities are therefore important to maintain this awareness and to build recognition, which could contribute to social media awareness. Finding 10: Most residents base their decision on when to return to their town of origin on information from family and friends. Many returnees have encouraged their family and friends to return, some with positive results. There is some evidence that ICRI’s social media presence had an indirect, minor, positive effect on promoting returns as well. Survey results suggest that when making decisions about returning, most residents either depended on information received from family and friends or visited the area themselves to understand the state of their towns of origin (Exhibit 16). KIIs suggest that returnees’ communication with displaced residents played a large role in motivating individuals to return to their communities of origin by providing constant updates about the progress of liberation and the process of reconstructing, de-mining, and returning livelihoods and services. Survey data corroborate trends emerging from KIIs. Nearly 86 percent of survey respondents from the Ninewa Plain and 90 percent from Mosul asserted that they had encouraged friends and family to return, reinforcing the finding that family and friends remain the most effective and trustworthy sources of information where decisions to return are concerned. Another significant source of information is social media. Almost 38 percent of survey respondents from Mosul and 22 percent of respondents from the Ninewa Plain reported that they depended on social media for information about their town of origin when making a decision to return. These two factors also interact significantly, as some KII respondents reported using social media to convey information about their town of origin. As a result, respondents mentioned seeing information about ICRI on Instagram and Snapchat even though ICRI does not operate on either of these social media platforms. ICRI’s primary impact on returns seems to have been through this interaction, where residents shared content created by ICRI on their social media platforms encouraging their own circle of family and friends to return by showcasing the success. 23 “Yes, as friends, I had a lot of Christian friends abroad and when I went back I used to post Instagram stories, and they would ask me about Bartella and is life back to normal? After a year or so, they were 23 The pathways from ICRI’s communication strategy to encouraging durability of returns are not clear. Data collected as part of this evaluation also do not indicate any pathways. 29 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV surprised. I told them to come back and some of my family went to the south. I encouraged them to return.” (Community Member, ICRI Participant, the Ninewa Plain, Male) “Yes of course, I didn’t just encourage them―I used to always post on my Facebook, show photos of the area, I would livestream the area. I would support them mentally and materially. Even the religious holidays, we would host religious holidays and we would get drummer and we took photos, we tried as much as possible to return life to normal. People would get excited to return. We encouraged them as much as possible and a lot of people did return.” (Community Member, Non-Participant, the Ninewa Plain, Male) “There were photographs and videos through social medias, mainly through Facebook. This was the case for most people in the Ninewa Plain and all of Iraq.” (Local Partner, ICRI Participant, the Ninewa Plain, Male) Exhibit 16. Sources of information for decision to return (n=10) ICRI’S STRATEGY AND APPROACH Finding 11: ICRI’s program strategy was sequenced and layered to support durability of return. However, there was no evidence of sequencing and layering at the individual activity level. ICRI’s program strategy used layering and sequencing, which is inherent to OTI’s model of implementing development programs. Layering refers to multiple activities simultaneously aiming to produce the same or similar outcomes. Sequencing is temporal. ICRI layered providing public services, improving opportunities for income generation, and improving intra-society relationships to produce a multiplier effect on durability of returns. The program also adopted a sequencing strategy so that public services (SO 1) and livelihood needs (SO 2) were accommodated before the program targeted intra-society relationships or social conditions, with the assumption that social condition interventions are effective (or prioritized) only when the community has access to livelihood opportunities and critical basic services. Respondents did not comment on activity sequencing and layering even when interviewers defined the concepts of sequencing and layering. However, when ICRI’s sequencing strategy—from direct service outputs to supporting livelihoods to improving social conditions—was described, some respondents 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Al-Islah Al-Tanak The Old City Total Mosul Bartella Bashiqa Qaraqosh Total Ninewa Plain Mosul Ninewa Plain From family and friends Elsewhere From TV From social media I don’t know Went myself to see From the radio 30 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV suggested that it was “rational” and “logical.” For example, respondents remarked that sequencing of rubble removal and deconstruction before implementation of other activities made sense, implying that the former were first necessary to enable the latter to have the intended effects. Respondents did not comment on ICRI’s layering strategy even when prompted. The review of the activity documents, including final evaluation reports for activities, also did not allude to how activities were sequenced or layered. Finding 12: ICRI effectively applied internal learning that improved the program’s efficiency and effectiveness. Data from document review and KIIs show that the program used lessons learned, recommendations, and relationship building to increase efficiency and effectiveness of newly implemented activities. For example, by partnering with grantees who had previously and successfully implemented activities, ICRI was able to benefit and build on the experience and established relationships it had gained from previous activity implementation. This approach also ensured that there was continuity between activities with little duplication of effort. The activity documents provide an example of this dynamic. ICRI partnered with a newly established group in Bartella, which had previously worked with another ICRI partner organization in the Ninewa Plain. The program was able to use the experiences and lessons learned from the Hamdaniya project while implementing the new activity. Similarly, on an education project, the ICRI procurement team proactively combined the in-kind items of the activity with another related education-focused activity for efficiency purposes and to ensure smooth delivery of items. Another example comes from the area of procurement; document review revealed several areas of improvement with respect to ICRI procurement and needs assessment in Mosul. Noor Al Fanar Voluntary Group is a newly established group in Bartella comprising Shabaki youth from Bartella. This group once was part of the Al Rafad Organization for Economic Development, which successfully implemented two ICRI activities. Following its establishment, the group brought and used the experience it gained partnering with ICRI to implement two additional activities. (Program activity evaluation reports, various activities). “The projects are happening gradually and they focus on the basics first and it seems they cooperate with many organizations, but it also depends on who applies for it.” (Community Member, ICRI Participant, the Ninewa Plain, Male) Finding 13: Collaboration with local partners and the built-in flexibility in ICRI’s strategy allowed ICRI to better understand and quickly respond to communities’ needs. During implementation, ICRI was confronted with two urgent resident needs that served to demonstrate the project’s ability to quickly pivot in collaboration with local partners and its ability to prioritize needs to improve the social recovery process for residents. The most significant example of quick pivoting was ICRI’s response to the COVID-19 pandemic while aiding social recovery for durable returns. Many respondents from both Mosul and the Ninewa Plain praised ICRI’s response to the pandemic regarding the provision of medical supplies and hygienic necessities, provision of items to increase social distancing, and support to local partners and government officials to care for affected persons. This was echoed during interviews with several respondents. 31 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV “In what I have seen during covid, there was a rationale behind the projects, there was prioritization for urgent needs, like covid-19. They supported Bashiqa with Covid-19 because it was spreading rapidly, so they responded rapidly.” (Local Partner, ICRI Participant, the Ninewa Plain, Male) Similarly, distributing food baskets to vulnerable communities in Karamles, Qaraqosh, and Bartella was very significant in mitigating the economic burden of the pandemic and was repeatedly mentioned during interviews. Furthermore, in Bashiqa, where the main health center was only partially rehabilitated, ICRI’s assistance in providing medical and operational supplies to a group of local volunteers responding to COVID-19 enabled people to stay in their homes and receive medical care and health consultations, and prevented further spread of the virus. Another situation demonstrating ICRI’s flexibility was its response to the Pope’s announcement of his plan to visit the Ninewa Plain. ICRI recognized the importance of this event to the people and assisted in infrastructure restoration to ensure residents were ready for the event. Respondents mentioned this project frequently during the interviews. This ICRI initiative had multiple benefits, including increasing returns to the districts. “The Pope project in Hamdaniya was also very successful and people were worrying because they knew the Pope was coming and they felt the area was not ready. Hamdaniya’s people loved the Pope and didn’t want him to see the city in this way, so the creation of sidewalks and the fence and the preparation of the streets really helped them.” (ICRI Program Staff, ICRI Participant, the Ninewa Plain, Male) “Of course, a lot of families returned to Hamdaniya after the Pope’s visit and after they saw the work inside Hamdaniya, the progress there in terms of services, especially Ta’afi [ICRI] encouraged a lot of Christian families to return to Qaraqosh.” (ICRI Program Staff, ICRI Participant, the Ninewa Plain, Male) 32 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV CONCLUSIONS AND RECOMMENDATIONS As the first iteration of the ICRI program reaches its conclusion, this evaluation report aims to tell the story of its progress and achievements in stabilization. Although it has been years since the liberation of these areas from ISIS, these communities still require extensive investment in services, livelihoods, and improving social conditions to recover. Below, we draw conclusions that build on our gained understanding of ICRI’s impact in the Iraqi context. We also reflect on how some of these findings could inform future iterations of the program or similar programs that seek to expand stabilization and recovery efforts in Iraq. CONCLUSION I: Evidence largely confirms ICRI’s theory of change that access to services and livelihood opportunities contributes to durability of return. However, the role of social conditions, especially within communities, in encouraging people to stay is not supported by the data. Moreover, there are a few missed opportunities the program could have taken into account with regards to the pathways to durable returns, including residents’ attachment to their ancestral homelands. The program’s theory of change undertook a transformation in September 2020. The first iteration of the theory of change was that a trifecta of variables can contribute to the goal of durable returns: (1) improved services, (2) improved livelihoods, and (3) improved social conditions. While the first two are straightforward, the term “social conditions” is taken to mean general feelings of safety and an individual’s ability to go about their daily lives and interact positively with other community members. Midway through implementation, the program team demoted social conditions from a PO to an SO, which is in line with the findings of the evaluation, suggesting that the theory of change and its adaptive iterations were responsive to reality. The evaluation presents strong evidence that the goal of durable returns is well served by investment in services and livelihoods. Moreover, as services have improved in the last two to three years (albeit slowly in some communities), the need for livelihood opportunities is even more prominent. The link between durability of return and livelihoods (or conversely, of migration and unemployment) is strong across the data. The importance of social conditions in achieving ICRI’s goal of durable returns, however, is less straightforward. First, evidence suggests that social conditions within neighborhoods were favorable from the beginning and may have even been strengthened by dealing together with collective trauma faced during the war. Second, there is not enough evidence to suggest that ICRI’s intervention in this area (focused within geographical communities) was needed. In fact, the evidence suggests that residents fear or distrust groups that exist outside of their own community, even when geographic communities are internally ethnically and religiously diverse. In other words, there are more social tensions between community members and outsiders (e.g., from nearby villages or across the governorate) than there are within a neighborhood among community members. The key issue when it comes to social conditions is that of demographic change in Bartella and Bashiqa, which residents describe as not being merely an ethno-religious issue, with Shabaks moving into traditionally Yazidi and Assyrian communities, but also a socio-economic one where Shabak villagers are moving into larger cities and causing demographic 33 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV change. Future iterations of the program or follow-on programming would be well-advised to distinguish between tensions within communities and across communities, and focus on relations across communities, especially between rural and urban areas, to allay fears regarding other groups and decreasing mistrust. Finally, the program missed an opportunity to engage with community members’ love of their land as a motivator for return. In the case of the Ninewa Plain, it is deeply rooted love for an ancient and ancestral homeland. Additionally, in thinking about social conditions, the program could have addressed shared trauma and built on collective healing to create strong community bonds, although doing so comes at a cost when and if reconciliation is adapted at a more macro level (governorate-wide or nation-wide) in the future. RECOMMENDATIONS Capitalize on both pragmatic and emotive motivators of stay. Other programs focusing on durability of returns can learn from ICRI’s emphasis on livelihoods and services. In addition, though, the evaluation has demonstrated that residents’ love for their ancestral land is an untapped motivator for return and stay. These emotive factors (affection for one’s hometown or, in the case of the Ninewa Plain, a sense of ancestral birthright) can be used in complementarity with material incentives, livelihoods, and services to promote durable returns. This recommendation is also timely for the next iteration of the program as it focuses on building a narrative of unity. Use the healing power of collective recovery. The evaluation has also demonstrated that the shared experience of displacement and return, particularly in areas of the Ninewa Plain, has helped bolster social conditions in communities. Although shared trauma can be extremely harmful, causing mental distress that leads to transgenerational effects, shared trauma among a group of people can also help build bonds and increase cooperation among group members.24 Shared traumatic experiences can also help groups “redefine who they are and where they are going.” 25 Future programming, including follow-on programming to ICRI, can use the healing power of collective recovery from trauma to build positive connections between and across communities. The interventions in this area can range from building a narrative of unity based on collective trauma through social media to providing members of different groups platforms to share their trauma. Focus on improving social conditions across communities. Social conditions within geographical communities (within neighborhoods, towns, and villages) are not a major source of concern, but social conditions across communities―shared goals of ICRI and follow-on programming―are critical to social recovery and stability in the long run. Individuals from different ethno-religious groups who have lived in the same community before and after the ISIS invasion do not report a need for intracommunity programming. However, there is a need to focus on social conditions across communities, which fall along ethno-religious lines; for example, between Assyrians and Yazidis living in the Ninewa Plain and Arab Sunnis living in outlying villages. Programming for the latter form of relationship is more pressing than the former, according to our research. Programming that addresses these social conditions can take the form of basic post-conflict peace-building and/or transitional justice that addresses the need for 24 Bastian B, Jetten J, Ferris LJ., “Pain as social glue: shared pain increases cooperation,” Psychol Sci. 25 no. 11 (2014):2079–2085. doi:10.1177/0956797614545886 25 Hirschberger, G., “Collective trauma and the social construction of meaning,” Frontiers in psychology 9 (2019):1441. 34 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV an apology and an acknowledgement of the crimes committed against minority communities and the silence of other Iraqis regarding those crimes. Grievances and fears are different in each community and understanding their layout is essential before organizing programming. CONCLUSION 2: ICRI had a substantial impact on improving community members’ lives by improving access to basic services, providing sources of livelihoods, and enhancing the sense of well-being and normalcy. All these achievements contributed to encouraging community members to stay in their community. ICRI’s impact on social conditions was limited. The lives of residents of Mosul and the Ninewa Plain were improved by the provision of services, including water, electricity, education, health, paved roads, lighting, and public works projects like parks and recreational spaces. Different areas and sectors experienced rehabilitation of services to varying degrees, sometimes leading to uneven service provision. For example, while residents of Bashiqa were happy with electricity rehabilitation, they were worried that they did not have a hospital. These services are essential to creating a sense of normalcy and a return to pre-war life. Respondents reported that the rehabilitation of public spaces enhanced their feeling of well-being. Services that improved as a result of the program, even when described as not improving fast enough, tended to be described as superior to the rest of Iraq. Livelihood opportunities were essential to the cities of the Ninewa Plain and Mosul because residents consider the cities commercial and agricultural hubs. The return of commerce and industry in these communities, in addition to providing jobs, also restores the image of the city and protects it from becoming overly securitized. Future iterations of the program and other projects focusing on rehabilitation of the economy should keep in mind that restoring communities’ traditional industries not only provides employment, but also restores the identity of a city, which is always challenged by militarization during war. RECOMMENDATIONS Advertise the role of the local and/or central government in supporting programming. The project’s goals did not include improving perception of government by citizens, citizen-state relations, or trust in the government. However, literature suggests that citizens’ perception of the state’s ability and willingness to provide services plays an important role in maintaining citizen-state relationship conducive to state stability, especially once the work of international organizations transforms or ends.26 Therefore, it can be argued, citizens’ perception of the state should be an important aspect of this type of programming. The next phase of ICRI should make an effort to highlight the role of government in providing services and livelihood opportunities to citizens to indicate the state’s ability and willingness to provide for its citizens. At the very least, the program should ensure that its successes do not undermine public confidence in government organizations, as the public tends to compare the performance of government actors with that of international organizations. 26 For example, see: Kooy, M., Wild, L., and Mason, N., “Doing Things Differently: Can Water Supply, Sanitation, and Hygiene Services Support Peace-and State-Building Processes?”; McLoughlin, C., “When Does Service Delivery Improve the Legitimacy of a Fragile or Conflict-Affected State?”; Denney, L., Mallett, R., and Mazurana, D., “Peacebuilding and service delivery.” 35 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Prioritize the eradicating of the signs of war. In war-torn areas, improving the physical aspect of the city—by paving roads, installing lighting, removing rubble, and creating public spaces—improves residents’ sense of well-being and contributes to the feeling of a return to normalcy. Eradicating the signs of war should be prioritized in future programming because it has a significant impact on residents’ feeling of well-being and their decision to stay in their community. CONCLUSION 3: Livelihoods are the most significant determinant in residents’ decision to stay. There is still a great need for livelihood activities, especially in the Ninewa Plain. Livelihoods have a direct bearing on community members’ ability and willingness to remain in their place of origin. The lack of livelihood opportunities corresponds with desire for migration, not only in areas recovering from ISIS but across all of Iraq. The lack of livelihood opportunities also explains why some residents live in their community but commute elsewhere. Broadly, there are two types of livelihood programming that have proven successful in the study sites: capacity-strengthening (e.g., skill training) and rehabilitation and financing support (e.g., the rehabilitation of the car repair shops). A combination of both is needed, sometimes applied together, across both Mosul and the Ninewa Plain. Iraqi graduates, for example, often complain about not having opportunities and resources to use existing skill sets. A difference between Mosul and the Ninewa Plain is that the latter is an agricultural area that, despite programming from ICRI, still requires a lot more livelihood programming to complement and enhance its traditional sectors. Mosul is an urban center with commercial activity that has been reinvigorated by ICRI programming. Large-scale programming may not serve the needs of all communities, and tailored approaches that build on existing resources and strengths, like agriculture, may be better at providing livelihood opportunities, without necessarily having to invest in retraining individuals. Finally, there are different types of unemployment in these communities with different implications for social recovery going forward. For example, if skilled graduates do not find sustainable employment, their absence may create brain drain in the community, with long-term repercussions. RECOMMENDATION Tailor livelihood programming to further encourage durable returns, when possible, by focusing on the traditional livelihood sectors of the community. Although ICRI has focused on livelihoods as a central tenet of its program, more livelihood programming is needed and should be a priority for other international organizations and future iterations of the program. Improving livelihoods, as mentioned before, reduces the desire for migration among returnees, particularly among youth. Livelihood activities that are more tailored, for example, activities that provide capacity-strengthening or training or provide resources for those who already have skills, are needed in future programming. Tailored approaches are important because they target different types of livelihoods needs that are required for the recovery of the community, from graduates trained in medicine, law, or education to laborers, artisans, and farmers. 36 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV While it may not always be possible to rely on the community’s traditional livelihood sectors because of modern market conditions and the direction of the country’s economy, rehabilitating local industries not only provides jobs, it also restores the identity of the community and undermines incentives to join militias. This is of special concern in the Ninewa Plain where more livelihood activities in agricultural areas can be helpful. Wherever possible, rehabilitation of local pre-war industries encourages residents to remain in their communities and prevents emigration. There is also a need for a longer-term livelihoods and economic recovery program in Iraq. While a full￾fledged economic recovery program may not be within the manageable interests of OTI, what OTI can do in the livelihood sector in stabilization contexts deserves more intentional discussion. CONCLUSION 4: ICRI was able to appear equitable in its dealing with various ethno-religious groups, and was also able to manage the expectations of the local communities and local partners it worked with, which is difficult to accomplish in a context like Iraq where social tensions remain. Many beneficiaries and local partners know ICRI to be part of USAID, and the American affiliation is difficult to manage in Iraq. This is not only because of the history between the two countries, but also the perception of the United States as being able to accept refugees and asylum seekers, which may burden local staff with having to manage expectations of residents who seek to migrate. The evaluation team found that despite these contextual challenges, community members understood ICRI’s role and their understanding of ICRI’s capabilities was accurate. Moreover, community members also largely noted the equity and unbiasedness of ICRI toward different ethno-religious communities. Although it occurred infrequently, there were complaints in the Ninewa Plain about local staff behaving nepotically. RECOMMENDATIONS Maintain ICRI’s record of largely unbiased and equitable programming toward various ethno-religious groups but beware of favoritism and nepotism. Given the social tensions in Iraq, it was critical that ICRI maintain unbiasedness and equity toward different ethno-religious groups in its programming. The data show that it was able to do that, as beneficiaries and community members largely viewed ICRI as not favoring one group over the other. It is essential that future programming maintains this image of equity and unbiasedness, especially because the program is now larger and deals more directly with division and polarization. Although ICRI was able to largely maintain the perception of equitability, there is evidence of and potential for nepotism and favoritism at the local level, and these challenges are likely to remain. Therefore, future programming should rely on mitigation strategies related to accountability of local partners and should continue to be intentional in training and inculcating staff and implementing partners against nepotism and favoritism. Manage stakeholders’ expectations by providing clarity when conveying program capabilities and roles. It is essential that staff at all levels of the program—dealing with district-level, governorate-level, and national-level officials and with community leaders and members—can clearly and uniformly convey the program’s goals, capabilities, constraints, and roles to community members, 37 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV leaders, and beneficiaries. In addition, ensuring that implementing partners and local partners also have a grasp on what is within and outside of the program’s capabilities is essential to maintain a good reputation in the country and among beneficiaries. CONCLUSION 5: Social media is an important tool for ICRI to reach community members, especially for men and residents who participate in community building and development activities, including civil society. However, community members receive information in many different ways and these dynamics differ by gender. It comes as no surprise that social media is an important tool for knowledge dissemination. Social media platforms allowed residents to interact during liberation and to share critical information about the state and habitability of their city. Residents and civil society activists are aware of ICRI’s social media presence and track it to keep up-to￾date on ICRI’s activities. Social media has been more successful than traditional media in creating a brand. Even so, the only hashtags that are highly recognizable are #Taafi and #USAID. Social media recognition of ICRI’s specific activities is limited to activists and interested individuals, and not everyday citizens, who only recognize the program name. Offline communication strategies used by the program have complemented its social media activities to build program recognition and social media awareness. ICRI’s social media presence played an indirect role in promoting returns by being a platform where residents disseminate messages about the recovery progress within their own family and social network, which then influences community members’ decision to return. More direct effects of the program’s social media strategy on returns are difficult to gauge, as many returnees made the decision to return before the ICRI program came into effect, having returned immediately following liberation. Fewer women have heard about ICRI from social media than men, which is surprising, since the Internet tends to be a space of equal access in Iraq. RECOMMENDATIONS Complement online communications with an offline outreach effort in a comprehensive strategy to reach a greater number and more diverse groups of residents. ICRI’s current social media strategy has been effective at making the program recognizable and disseminating information about activities. Social media, in general, is an effective tool to disseminate information about the progress of a community’s recovery and has been used by returnees to spread information within their social networks. Future iterations of the program should learn and build from the social media strategy ICRI used and should explore alternative means of communication for women and residents who do not have access to social media or are unable to access it because of technology issues. These strategies can include traditional media such as TV and radio and in-person strategies such as town halls and community meetings. Offline outreach activities can also help magnify the social media outreach by increasing awareness of ICRI’s social media presence. Additionally, the next iteration of the program should determine communication preferences across the spectrum of age groups to facilitate greater reach. It can also focus on developing content that is more sharable through community members’ own social networks to capitalize on information dissemination through trustworthy news sources. 38 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Enlist community leaders to support program outreach. Future programming can also rely on community leaders to disseminate information through communal social media networks, like Facebook or WhatsApp groups, or even through community meetings and gatherings. ICRI’s official social media accounts are known to local activists and leaders, but not necessarily to community members. When there is a critical need for information dissemination, relying on the social media networks of the community itself is a useful tool to consider. CONCLUSION 6: ICRI’s lack of focus on gender dimensions while designing and implementing activities was a missed opportunity. ICRI does not seem to have integrated a gender perspective into the preparation, design, implementation, and monitoring and evaluation phases of programming. Gender integration is defined as strategies applied in program planning, assessment, design, implementation, and M&E to consider gender norms and to compensate for gender-based inequalities, and usually goes beyond consultations with women during various phases of programming.27 It is clear that women and men perceive and respond to issues related to durable returns and stability in Iraq in different ways. First, the man in traditional male-led households is likely the one deciding whether to stay; this decision is calculated differently in female-led households. Second, the livelihood opportunities available to and empowering of women are entirely different than those available to men in Iraqi society. Third, services affect women’s everyday lives in different ways. For example, Iraqi women, particularly in lower income brackets, carry most of the burden of household work and are therefore more affected by shortages in water, electricity, and education services than men. Women also receive information differently than men. Women control fewer political and economic resources than men and have less decision-making power in their daily lives. These gender differences warrant gender integration in every aspect of ICRI programming. One of the observable results of the lack of gender integration is that ICRI programming did not focus on providing services and livelihood opportunities directly to women; this absence is felt by civil society. Additionally, some of its interventions, for example, its social media outreach strategy, had an unintended differential impact on women. For instance, more men received information from social media about ICRI in the Ninewa Plain than women, while the reverse was true in Mosul. Without integrating gender perspectives, it would be difficult for any program to respond to the needs of women as thoroughly as of men, impeding the maximum impact of the program. At worst, programming that does not acknowledge and integrate inequalities can perpetuate them.28 Therefore, integrating gender perspectives not only makes sense from a human rights perspective, but also maximizes the impact of development interventions. In other words, to the extent that gender-based 27 The Interagency Gender Working Group (funded by USAID) has developed a continuum of gender integration. The term “gender blind” refers to programs that are designed without prior analysis of how gender can affect program outcomes by understanding culturally defined set of economic, social, and political roles, access to resources, etc. In contrast, “gender aware” programs/policies deliberately examine and address the anticipated gender-related outcomes during both design and implementation. 28 This evaluation did not analyze if any of these scenarios occurred or not in Iraq during ICRI programming. The only impact that the evaluation team observed was the lack of programming focused on women, which likely weakened the effectiveness of some interventions (e.g., social media strategy, as mentioned before). 39 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV inequalities pose a challenge to improving durability of returns, integrating gender can enhance the impact of ICRI programming. Taking it a step further, integrating gender in the program’s theory of change by acknowledging the range of connections between gender dynamics, conflict, and stability would lead to a more holistic approach to promoting stability in Iraq.29 The current context of post-war displacement in Iraq has created a window of opportunity where gender relationships and norms can be redefined and re￾negotiated in ways that are conducive to trust and that address gendered drivers of conflict. 30 RECOMMENDATIONS Integrate gender considerations throughout the Program Cycle, including planning and proposal development; development of theory of change; program design; implementation and management; monitoring and evaluation; reporting, documentation and dissemination and research utilization. The program can reference existing gender analyses of local communities or, if need be, conduct its own comprehensive analysis of factors that contribute to the marginalization of women and opportunities to enhance their participation in activities that contribute to stability. These can be found across social structures at the individual, interpersonal, community, and policy/institutional level, and can be deeply embedded in different spheres (for example, laws, policies, and institutional practices; cultural norms, beliefs, and perceptions; roles and responsibilities; assets and resources; power and decision￾making). Understanding these dynamics will not only allow ICRI to devise more effective programming for women, but at the very least will prevent any unintended consequences for women. Taking it a step further, stabilization programs can focus on integrating gender into the program’s theory of change to maximize the impact of the program. Use the space opened in the aftermath of war to renegotiate gender norms in Iraqi communities to further the goal of stability in the country. The program should partner with female-led civil society organizations, being careful not to miss opportunities to also engage groups less formally constituted as “organizations,” “associations,” etc., to renegotiate gender norms. Female activists have unique ideas, understanding, and capabilities within their communities, especially concerning nuances of how laws, cultural norms, and informal institutions affect gender dynamics. Through their networks and through their experience navigating social settings, female activists can especially help reach women the program has not been able to reach. Moreover, female activists understand how to present and promote programming that is directed at improving women’s lives to Iraqi stakeholders, including Iraqi community leaders, local and central governments, and religious 29 Not only does conflict affect men and women differently, there is also substantial evidence that gender inequality contributes to violent conflict and negatively influences stability. Better gender equality can also indirectly increase a country’s stability through its impact on wealth/income. For example, see the following for a review of literature: Herbert, S., “Links between women’s empowerment (or lack of) and outbreaks of violent conflict,” GSDRC Helpdesk Research Report. 2014. Also see: Dietrich, L. and Carter, S., “Gender and conflict analysis in ISIS affected communities of Iraq,” Oxfam: 2017, which discusses how gender norms have fueled tensions and insecurity in conflict-affected areas of Iraq in addition to and intertwined with other causes, including sectarianism, perceived corruption, and exclusionary policies and injustice and underlying structural tensions including lack of services, education, and employment. 30 For example, see: O'Driscoll, D., “Emerging trends of conflict and instability in Iraq,” Helpdesk Report, UK Department for International Development: 2018. 40 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV authorities. They are important partners not only because of their ties to women and their understanding of what Iraqi women need, but also because they know how to communicate and get support for those needs from powerful actors within the Iraqi community. 41 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV ANNEXES ANNEX A: EVALUATION QUESTIONS Question 1: To the extent that social recovery is a necessary condition for durable returns, how effectively has ICRI-Ta’afi understood and addressed social recovery needs in areas liberated from ISIS? Question 1.a.: To what extent and how did the program adapt its strategy and theories of change (as they related to social recovery for durable returns) over the course of three years of programming? Question 1.b.: To what extent did activity outcomes contribute—intentionally or unintentionally, directly or indirectly—to allow for durable returns? What were the key pathways to change? This question analyzed the program’s contribution to promoting social recovery and, hence, durable returns. Social recovery in the simplest terms for this evaluation is defined as the feeling a returnee has of being at home in their own community, living with a sense of safety, and interacting positively with other members of the community. Durable returns implies that both those who were displaced by the conflict with ISIS and returned to their communities of origin and those who never left plan on remaining in those communities for the long term. This question presumes a relationship between social recovery and durable returns; in other words, social recovery encourages returnees to stay and therefore increases durable returns or durability of returns. For this evaluation, to address the main question, it was imperative to examine this relationship and understand the necessary and sufficient conditions for social recovery to occur, as well as how the program has affected these conditions. The ability to understand and explore these relationships allowed the evaluation team to explore pathways of change and provide a more nuanced understanding of how the program has affected durable returns, including understanding the significance and contribution of layering and sequencing activities, thereby distilling lessons for future programming. It also allowed the evaluation team to analyze how contextual factors came into play by framing social recovery in practical terms that may differ across different contexts. The evaluation team also sought to examine any indirect and unanticipated outcomes of program activities that have contributed to durable returns (by improving social recovery but also independently). The evaluation team also examined how the program’s strategy has changed over the course of the last three years and across regions to adapt to changing contextual conditions and needs and how pathways to change have been calibrated based on prior experiences. Question 2: Whether and how has ICRI-Ta’afi influenced perceptions of recovery among residents of areas liberated from ISIS? To what extent do these perceptions differ across various groups (region, gender, religious/ethnic group, etc.)? What are the pathways of change? Question 2.a.: How have ICRI-Ta’afi’s social media presence and community engagement efforts during activities contributed to this effort? Through this question, the evaluation team sought to learn how the program affected the perceptions of recovery, through what mechanisms, and how these perceptions and mechanisms differed across groups. In addition to exploring other pathways, the evaluation team specifically examined whether and 42 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV through which channels the information disseminated affected these perceptions. Because the ICRI program focused on social media, the evaluation examined the impact of social media tools on perceptions and the pathways through which they work. In addition to its value for the ICRI project, understanding the link between social media presence and perceptions of recovery is important for the next iteration of this program, launched in February 2022. Perceptions of recovery are important for returnees and current residents because these perceptions play an important role in their decision to continue to stay. Currently displaced persons presented another intended audience for these messages, because their perceptions of improved recovery significantly factor into their decision to return. Currently displaced individuals were not a significant portion of our sample; however, we tried to answer this question indirectly for this group by using a snowballing approach. 43 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV ANNEX B: METHODOLOGY The evaluation team employed a mixed-methods, complexity-aware evaluation approach that recognized that the nature of any improvement in social recovery (in the form of basic services, livelihoods, and social conditions/intra-society relationships) associated with the program cannot be measured conclusively. In a complex intervention, measuring the nature of social change or the extent to which activity outcomes have led to program objectives is challenging. The evaluation questions focused on perceptions of higher-level outcomes that are central to the ICRI program strategy, including fostering durable returns and social recovery as well as understanding and identifying incremental changes and exploring pathways of change. Based on the above, the evaluation team used a non-experimental and mixed evaluation methodology that collected both qualitative and quantitative data through document review, surveys, and KIIs. The evaluation methodology, therefore, reflects qualitative data collection (in-depth and semi-structured KIIs; document review) and quantitative data collection (survey). QUALITATIVE DATA COLLECTION Qualitative methods made up a key component of the evaluation. The evaluation team used qualitative methods to build a narrative of how and through what pathways of change the program had affected the lives of community members; especially as it related to social recovery and durable returns. Qualitative methods also surfaced a description of how the program was implemented, including the sequencing and layering of activities, and allowed the evaluation team to trace the changes it brought about in focus communities. SITE SELECTION For qualitative data collection, the evaluation team sampled geographically by selecting a set of neighborhoods in Mosul and towns and villages in the Ninewa Plain to examine the accumulated impact of the program on residents’ lives. Geographic sampling allowed the evaluation team to consider the cumulative effect of the project on people in a defined geographic space and synchronicity (including sequencing and layering) between program activities. It also allowed the evaluation team to construct a narrative of social recovery in selected communities. Geographic sampling did not come without potential confounding variables. Critical among these was the fact that many of the post-conflict communities in which OTI has worked have also been beneficiaries of programs from other international organizations like the United Nations Development Programme (UNDP) and the International Organization for Migration (IOM), which have worked on similar activities. In this way, in speaking to beneficiaries and surveying them, the team may have captured the accumulated effect of many organizational and programmatic efforts; not solely OTI. In some communities, OTI had been purposeful in designing activities that complemented work being done by others. ACTIVITY SELECTION For this evaluation, the following guidelines were used in the selection of activities.: 44 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV For Mosul 1. The evaluation team identified all activities implemented within Mosul municipality using the OTI database, only selecting activities that were cleared, completed, or closed. 2. The team identified the location of the activities using activity summaries. It identified if the activities were implemented governorate-wide, Mosul district-wide, or in East/West Mosul, or if they were focused on specific neighborhoods. If ICRI implemented activities in specific neighborhoods, the evaluation team identified them. 3. The evaluation team then collated this data to identify the number of activities identified in each neighborhood across the three sub-objectives. It asked the Program Development Officers (PDOs) and MEL Officers to validate these lists and confirm that the locations identified were the actual focus of the activities. Discussion with the program staff led to recategorization of some locations. For example, the program team suggested consolidating several neighborhoods into one location—the Old City—for the purpose of this evaluation.31 4. In collaboration with the PDOs and MEL officers, the evaluation team then used the following criteria to identify locations in Mosul: a. Locations where the greatest number of activities were implemented to allow evaluation to assess the cumulative effect. b. Locations that provided variance across sub-objectives so that cumulatively, the selection locations covered all sub-objectives. c. Locations that represented different contexts. For example, some of the neighborhoods in Mosul are old and located in the city’s center (i.e., the Old City) while others are peripheral and more remote (i.e., al-Islah al-Zirai). d. The selected locations represent opportunities to examine the layering and sequencing of activities, which reflected instances where activities designed to serve specific sub￾objectives had contributed directly or indirectly to other unintended sub-objectives, leading to insights about project complementarity overall. 5. The evaluation team also identified activities that were implemented across West Mosul. It further selected activities from the Mosul city-wide activities for the sample. Exhibit 17 provides the distribution of activities across selected locations and sub-objectives, as identified through activity descriptions in the database and conversations with program staff. Exhibit 17. Sampled locations in Mosul LOCATION NUMBER OF ACTIVITIES SO 1 SO 2 SO 3 The Old City 8 5 3 0 Al-Tanak 6 5 0 1 Al-Islah al-Zirai 6 4 0 2 West Mosul 13 10 1 2 Mosul-wide 37 17 2 18 Total activities* 65 36 6 23 *The total is not the sum of the rows above because one activity may have been implemented in multiple locations. This number signifies total unique activities in Mosul. 31 The following neighborhoods were included in the Old City location: Al-Farooq, Al-Makkawi, Al-Sarjkhana, Al￾Sha’areen, Bab Jadeed, Bab Ligish, Khazraj, Ras a-Jadda, and Ras al-Koor. 45 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV For the Ninewa Plain 1. The evaluation team used the OTI database to identify closed, cleared, and completed activities implemented in two districts within Ninewa Governorate: Hamdaniya and Tal Kayf. District Al Shikhan also falls within the Ninewa Plain, but the initial review showed that very few activities were conducted in Al Shikhan and it was therefore not included in the remaining location selection process. After further consultations to keep the sample manageable, the evaluation team, in consultation with the program team, decided to focus only on Hamdaniya district. Excluding Tal Kayf from the evaluation allowed the evaluation team to reduce the sample size without compromising breadth, as Tal Kayf and Hamdaniya share many contextual factors that may have mediated the effect of the program on durable returns. Regarding demographics, both districts have sub-districts that contain diverse populations of Assyrians, Yazidis, Shabaks, Kaka’is, Turkmen, Arabs, and Kurds. Arguably, Hamdaniya is more diverse, which made it the clear choice in site selection. Both districts also border the Kurdistan Region of Iraq, with Hamdaniya bordering Erbil Governorate and Tal Kayf bordering Duhok Governorate, placing them at the center of Baghdad-Erbil tensions. Hamdaniya was occupied by ISIS in early August 2014 and was liberated by the Iraqi Army in October 2016. Tal Kayf was also occupied by ISIS in early August 2014, but it was not freed until January 2017, likely due to its proximity to Mosul. 2. The evaluation team used the activity summary section to identify the location of the activities and identified whether the activities were implemented governorate-wide or district-wide. 3. The evaluation team then collated these data to identify the number of activities in each neighborhood across the three sub-objectives. The evaluation team asked the PDOs and MEL officers to validate these lists and confirm that the locations identified were the actual focus of the activities. The discussion with the program staff led to recategorization of some locations. 4. The evaluation team also included the town of Bashiqa when selecting towns from within Hamdaniya even though the district was not part of the Ninewa Plain. Discussion with the PDOs revealed that it was an important town for ICRI, and the OTI database identifies activities within Bashiqa as having occurred within Hamdaniya. 5. The evaluation team then identified locations in Hamdaniya in collaboration with the PDOs and MEL officers using four criteria: a. A significant number of activities were implemented in the locations to allow the evaluation to assess the cumulative effect. b. Locations provided variance across sub-objectives so that, cumulatively, the selection locations covered all sub-objectives. c. Selected locations represented different contexts; for example, urban and rural. d. Selected locations represented opportunities to examine the layering and sequencing of activities, which reflected instances where activities designed to serve specific sub-objectives had contributed directly or indirectly to other unintended sub-objectives, leading to insights about project complementarity overall. Exhibit 18 provides a distribution of activities across selected locations and sub-objectives, as identified through activity descriptions in the database and conversations with program staff. Exhibit 18. Sampled locations in Hamdaniya LOCATION NUMBER OF ACTIVITIES SO 1 SO 2 SO 3 Qaraqosh 13 8 4 1 Bartella 12 5 3 4 Bashiqa 8 3 4 1 46 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV LOCATION NUMBER OF ACTIVITIES SO 1 SO 2 SO 3 Total unique activities 25 11 8 6 *The total is not the sum of the rows above because one activity may have been implemented in multiple locations. This number signifies total unique activities in a location. Media and Community Strategy Activities In addition to activities that targeted specific locations, the evaluation team also included activities specifically related to media and communication strategy implemented at the national, governorate, district, or neighborhood level. Inclusion of selected activities was based on the following steps: 1. The evaluation team identified all activities tagged as media and communication activities in the OTI database that were cleared, completed, or closed. 2. Out of the activities selected in the above, the evaluation team identified the activities that were most relevant to media and communication strategies. 3. The evaluation team included activities that were more focused on national and Ninewa governorate level and on ones that were focused on Mosul and Hamdaniya. DATA COLLECTION METHODS The evaluation team used the following methods to collect data within the selected communities: Document Review: Document review included all documents related to selected activities. This included documentation from the activity database, ICRI strategic framework iterations, and final evaluation reports. See the document review sample in Exhibit 19. Exhibit 19. Document review sample for qualitative data collection REVIEW MOSUL THE NINEWA PLAIN MEDIA AND COMMUNICATIONS TOTAL Documents Coded and Analyzed 65 SO 1.1 = 36 SO 1.2 = 6 SO 1.3 = 23 25 SO 1.1 = 11 SO 1.2 = 8 SO 1.3 = 6 24 114 Virtual key informant interviews (program level): The team conducted in-depth, semi-structured KIIs with key stakeholders, including OTI staff, implementing partners, and program staff at the headquarters level. These interviews were carried out via Zoom, telephone, WhatsApp, and face-to￾face. The respondents were selected purposively in consultation with the program team and included staff members who had been involved in program strategy design and implementation at the national and district level. These individuals included Country Representatives (former and current), the Program Manager, Transition Advisors, the ICRI Chief of Party, PDOs for Mosul and the Ninewa Plain, MEL Officers, and the MEL Manager. Virtual key informant interviews (district and local level): The team also conducted KIIs with stakeholders at the district and local level in the selected locations. The categories of stakeholders for KIIs at the local level included (1) government officials at governorate and district level, (2) local 47 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV implementing partners, and (3) community members, including participants and beneficiaries of ICRI program activities. At the district level, the evaluation team targeted local government officials who had participated in ICRI activities in Mosul and in Hamdaniya. Other respondents included senior staff at the technical level from the Directorate of Mosul Municipality and Hamdaniya District Office. Among persons excluded were mayors, since pre-design interviews and conversations indicated they had been over-interviewed by evaluators and tended to speak very positively about the program, which had potential to bias the results. In addition, the evaluation team interviewed local government officials representing Mosul and the Ninewa Plain. The evaluation team also interviewed seven local partners of ICRI from the selected locations. The team selected these partners from a list provided by the PDOs. The selection criteria included whether they had partnered with ICRI during project implementation in the selected locations. For community members, the evaluation team targeted activity participants, community leaders, and community members. The evaluation team used purposive and snowball sampling to select participants for KIIs. A sampling frame was constructed from the list of activity participants, beneficiaries, and community leaders compiled by the program staff in each of the selected communities. From this initial list consisting of nearly 40 individuals, the evaluation team randomly chose 8–10 members per location. Using the initial list, the evaluation team built the sampling frame within each community, which was further extended using the snowballing technique for the identification of participants. Interviewees in Mosul were largely selected through this framework; in the Ninewa Plain, the evaluation team also relied on snowball sampling through other networks in the area, outside of the names provided by the OTI team. The selection criteria for participants specified their role in the community, their role in ICRI, their status (stayee, returnee, IDP), and their ethno-religious identity. The goal was to achieve as close to equal gender representation as possible (which was achieved by asking an interviewee if their sister/wife/mother could be interviewed, and having them connect the interviewer to other women in the area). With regards to status, the priority was to speak mainly to stayees and returnees. With regards to ICRI participation, we were interested in people who were directly involved and indirect beneficiaries. As for ethno-religious identity, the evaluation team was interested in representing the diversity of the Ninewa Plain more than Mosul, which is more homogenous. However, to do so with sensitivity, the team phrased it as introducing the interviewer to an acquaintance from a different ethno￾religious group than the interviewee. The exclusion criteria included anyone under 18. Further, since culturally, women do not speak to men who are not related to them on the phone, all the interviews with female respondents were conducted by a female data collector (the subject-matter expert). The interviews with women that were conducted through Think Bank, a contracted partner, were also conducted by a female data collector. Recruitment of primary respondents: Following the random selection of individuals from the list of participants provided by the OTI PDOs, a recruitment message was sent via WhatsApp describing the purpose of the research and the fact that they had been recommended by the OTI team. Once the interviewee followed up after the first message had been sent, the in-country data collector asked for an interview that night or the next night, always selecting the first available date. If the interviewee did not respond, the data collector called them after 24 hours. If there was no response, then the next strategy was to find an alternate candidate through the list OTI had provided. 48 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Recruitment of secondary respondents: After each interview, respondents were asked to provide the interviewer with contacts who could be potential respondents. The interviewer shared the desired characteristics of the potential respondents (age, sex, status, ethno-religious group, level of participation in the activity as described above) and asked them if they would be willing to be interviewed. Once the potential secondary respondents consented to be contacted by the data collection team, the primary respondent would be asked to provide the contact information of the individuals who had agreed to be contacted and their preferred means of communication. The referred individuals would be added to the sampling frame and selected for interviewing using the same criteria used for the primary respondents described above. Ultimately, the evaluation team did not acquire anyone’s contact information without that person consenting to their information being shared. In addition, the primary respondent would only know that the secondary respondent may be contacted by the evaluation team and would not know if they were actually selected, thereby protecting the confidentiality of the secondary respondent. Exhibit 20 provides the distribution of interviews conducted across locations. In total, the evaluation team conducted 46 KIIs. See Exhibit 21 and Exhibit 22 for distribution of participants by ethno-religious identity and other disaggregation. Exhibit 20. Stakeholder groups for qualitative data collection at the community level COMMUNITY COMMUNITY INTERVIEWEES PERCENTAGE MOSUL The Old City 14 56 (of Mosul respondents) Al-Tanak 6 24 (of Mosul respondents) Al-Islah al-Zirai 5 20 (of Mosul respondents) TOTAL MOSUL 25 57 (of total respondents) THE NINEWA PLAIN Qaraqosh 4 21 (of the Ninewa Plain respondents) Bartella 7 37 (of the Ninewa Plain respondents) Bashiqa 8 42 (of the Ninewa Plain respondents) TOTAL THE NINEWA PLAIN 19 43 (of total respondents) TOTAL 44 (Excluding 2 Local Staff) Exhibit 21. Distribution of respondents for qualitative data collection by ethnicity ETHINICITY INTERVIEWEES PERCENTAGE Assyrian 6 13 Shabak 7 15 Yazidi 3 7 Kaka’is 1 2 Arab Sunni 19 41 Other/Don’t know/Program Staff 10 22 TOTAL 46 100 Exhibit 22. Distribution of participants for qualitative data collection by gender, participation, and position INTERVIEWEES NUMBER PERCENTAGE GENDER Female 9 20 49 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV INTERVIEWEES NUMBER PERCENTAGE Male 37 80 PARTICIPATION Participants 26 57 Non-Participants 18 39 Program Staff 2 4 POSITION Community Members 40 87 Community Leaders 4 9 Program Staff 2 4 QUANTITATIVE DATA COLLECTION The evaluation team complemented qualitative data collection with a concurrent survey. The objective of the survey was to learn about the impact of the program on social recovery, durable returns, and perceptions of social recovery from a broader population of people in Mosul and the Ninewa Plain. The evaluation team collected data on contextual factors, which provided information to guide the team’s understanding of the changes inspired by the program. The evaluation team collected 170 survey responses, surpassing the target sample size of 150. Not all respondents fully answered the survey questions on gender and ethnicity. As a result, there is a slight difference in the number of responses for gender and ethnicity breakdown in Exhibit 24 and Exhibit 25. During the preparation of the survey, there were no available population estimates for the targeted areas within Mosul and the Ninewa Plain districts. However, the evaluation team was still able to roughly estimate the margin of error, since after a certain point, population has a minimal impact on it. The 170-response sample size provided a margin of 7.41 points with a confidence level of 95 percent and 50 percent population percentage. 32 Sampling and recruitment: The evaluation team targeted all residents (stayees, returnees, and currently displaced) of the selected locations through the survey. The team constructed the initial list of potential respondents by requesting contact information of participants and beneficiaries of OTI activities, as well as of community leaders and other stakeholders in the selected communities from PDOs. The team used a snowball strategy to distribute the survey in two ways. First, it asked the community leaders and civil society organizations identified from the initial lists to disseminate the survey link along with a brief description to individuals who fulfilled the criteria of the respondent groups within their social circles, for example, on WhatsApp or Facebook. This request was made during interviews with the community leaders. The second strategy was to include the question at the end of the survey, “Do you know anyone from the Old City, al-Tanak, al-Islah Al-Zirai, Bartella, Bashiqa, Qaraqosh?” If respondents answered yes, the evaluation team asked them to forward the survey link to 32 Confidence interval: represents how often the true percentage of the population who would pick an answer lies within the confidence interval. The 95 percent confidence level means you can be 95 percent certain. Population percentage refers to percentage of the sample who would select a particular answer. Worst case scenario is when population is evenly divided, i.e., the percentage is 50 percent. It is possible to calculate margin of error for responses to specific questions after data collection is complete. 50 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV their contacts. See Exhibit 23, Exhibit 24, and Exhibit 25 for disaggregation of survey respondents by location, gender, and ethnic group. Exhibit 23. Distribution of participants for quantitative data collection by location AREA LOCATIONS COUNT PERCENTAGE Mosul Al-Islah al-Zirai 14 29.17 Al-Tanak 14 29.17 The Old City 20 41.67 Mosul Total 48 100 The Ninewa Plain Bartella 22 18.03 Bashiqa 80 65.57 Qaraqosh 20 16.39 The Ninewa Plain Total 122* 100 *Eight respondents did not respond to this question. Consequently, only 122 responses were used when doing analysis by town/city. Exhibit 24. Distribution of participants for quantitative data collection by gender AREA GENDER COUNT PERCENTAGE Mosul Male 35 73 Female 13 27 Mosul Total 48 100 The Ninewa Plain Male 101 78 Female 29 22 The Ninewa Plain Total 130 10 Exhibit 25. Distribution of participants for quantitative data collection by ethnicity THE NINEWA PLAIN MOSUL Ethnic group Number Percentage Number Percentage Assyrian 27 21 1 2 Shiite Turkmen 2 2 0 0 Shiite Shabak 25 20 0 0 Sunni Arab 5 4 29 60 Yazidi 7 6 0 0 Arabic Shiite 3 2 0 0 Kakai 3 2 0 0 Sunni Shabak 4 3 1 2 Sunni Kurdish n/a n/a 1 2 Sunni Turkmen n/a n/a 10 21 Other 40 32 1 2 Refused to answer 10 8 5 10 TOTAL 126 100 48 100 The survey included demographic questions about location, gender, age, ethnicity, status (returnee, stayee, IDP, refugee), and participants’ role in OTI activities. This information facilitated a better understanding of the representativeness of the respondents and allowed the evaluation team to make sure specific groups were included so that the strategy to reach potential participants could be adjusted if certain groups were not well represented. 51 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV The evaluation team ensured that the questionnaire was simple and easy to understand. As mentioned previously, the team used multiple platforms for different audiences to disseminate the survey, including platforms that are popular in Iraq like WhatsApp, Telegram, Messenger, and Viber. The survey itself was hosted on Google Forms with a link that respondents could forward to contacts. DATA COLLECTION PROCESSES Qualitative data collection was primarily conducted by a subject-matter expert and one in-country data collector. Both were Iraqi and fluent in Arabic, including colloquial Arabic. The data collectors were also responsible for engaging virtual survey respondents in collaboration with the PDOs and MEL Officers. The evaluation team had trouble recruiting respondents with particular characteristics for KIIs in Mosul and Hamdaniya (primarily Mosul). A snowballing approach was not able to provide the desired sampling frame. The evaluation subcontracted a data collection firm with extensive experience in collecting data in sensitive contexts in Iraq. For this specific set of data, participants were recruited via subcontractors’ “recruitment hubs,” which cover a range of different approaches to recruiting target samples. Before data collection, the evaluation team developed data collection instruments and the data collection guide. The KII data collection instruments were specifically tailored to each respondent group and approved as appropriate to gather information on respondents’ perspectives. The team facilitated data comparison by asking similar questions to different respondent groups when possible. The data collection instruments, consent forms, and related materials (for example, incident report forms and adverse event forms) was developed in English and translated into Arabic by a translation firm. Several revisions were done to the English-Arabic translations for accuracy. The Arabic-speaking evaluation team member reviewed all translated instruments for accuracy and did spot checks by conducting back/reverse translation of selected instruments to ascertain accuracy. In addition to the spot checks, native Arabic￾speaking team members (subject-matter expert and in-country data collector) thoroughly reviewed the translated materials to ensure that the translated materials conformed to the message of the original documents. The program team also reviewed instruments to ensure that they were context specific. The instruments were further refined through the pretesting and piloting process for reliability, validity, and cultural sensitivity. GUIDE TO DATA COLLECTION AND DATA COLLECTION TEAM TRAINING The data collection guide provided comprehensive guidelines for selection of participants for KIIs and protocols for using snowballing techniques, including profiles of potential respondents, criteria for selection of each contact for further data collection, methods of recruitment of survey respondents, and a review of best practices for data collection. The guide also specified detailed roles and responsibilities of all team members. The team conducted a virtual data collection training to inform the instrument piloting process and ensured that the data collectors were thoroughly trained on the details of each instrument and on appropriate ethical data collection best practices. The KIIs were conducted in a mix of Modern Standard Arabic (the language in which the consent form and the questions are written) and colloquial Iraqi Arabic, which was used to clarify any questions and to communicate with the respondents. In Mosul and the Ninewa Plain, there are many ethno-linguistic groups with their own languages, but everyone speaks, reads, and writes Arabic (both the Iraqi dialect and Modern Standard Arabic). Local variations in Arabic dialect are small enough that the respondent and interviewee are able to understand to each other (they can be compared to speaking North American and British English). Both the subject-matter expert and the qualitative data analyst were 52 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV fluent in written Arabic (Modern Standard Arabic) and the central Iraqi dialect and underwent training, which included how to explain questions and the consent form. QUALITY ASSURANCE The evaluation team established comprehensive processes for monitoring data collection. Throughout data collection, the SME and evaluation specialist provided supervision and technical support to the data collectors to ensure adherence to protocols, including systematic reviews during data collection to promptly identify and correct issues. The subject-matter expert and in-country data collector met 2–3 times a week to review the data collection tracker, transcripts, and sampling frame as it developed through snowballing. In addition to checking on quality, these check-ins were also used to identify additional respondents to be interviewed. Moreover, the subject-matter expert was available to the in￾country data collector at all times for trouble-shooting. The whole team met weekly to review the data collection tracker, the sampling frame, and selection of participants. The evaluation team lead provided quality assurance by periodically reviewing data collected and processes used during data collection. Both she and the subject-matter expert updated the OTI team on progress during the scheduled biweekly meetings and in data collection reports. 53 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV ANNEX C: LIMITATIONS It is accepted in the social sciences that every evaluation has limitations, because it is unrealistic to control for every variable in a dynamic social environment as one would in a laboratory setting. Limitations associated with this evaluation include: ● Perception-based data are sometimes not reliable because of the possibility of inaccuracy and self-report bias. For example, respondents may have selective memory about experiences, especially when some time has passed between the event and the actual interview; or sometimes a respondent may focus on a particular traumatic or pleasurable event due to the relative significance for them. Comparatively, a respondent may not focus on or remember or may downplay a particular event or information based on its relevance to them. There is not much that can be done in these situations. However, being able to triangulate responses with quantitative data was useful to support the findings. ● Because data collection was conducted during the COVID-19 pandemic, it was decided that nearly all data collection would occur over the phone or virtually. Therefore, the process of recruiting interviewees took longer than anticipated because the data collectors were not able to simply walk around communities and request to speak with people, but had to rely on snowball sampling to identify potential interviewees, which can also create selection bias. Further, interviewing virtually presents unique challenges. Virtual sessions may deter respondents from connecting and “opening up” to interviewers. They also prevent the interviewer from viewing non-verbal cues useful for determining the need for probing. ● The evaluation team disseminated the survey with support from OTI local staff and from Think Bank. This meant that interviewees were not randomly sampled, although efforts were made to be as representative of various ethno-religious groups, genders, and ages as possible. Fortunately, the sample was representative of the various ethno-religious groups. However, the sample sizes among locations were different, with Bashiqa being by far the highest. This still allows us to compare numbers across locations. However, findings from locations with a higher sample size are more representative of the opinion of residents of that location than residents of the locations with a smaller sample size. ● There was also a concern about the potential for social desirability bias in the KIIs, where respondents may have been incentivized to give socially acceptable responses. For example, respondents may have felt pressured to be positive about programming in their community and to exaggerate the changes to appease the local team. This is particularly challenging when it involves social relations, where respondents may feel pressured to paint a positive picture of relations in their community. To mitigate against this bias, our data collectors were trained to be trauma-conscious and empathetic, careful to not lead respondents, and to assure them of the anonymity of their responses. ● Additionally, social desirability bias was a concern because respondents may want to gain support from USAID or hope for resources for their community if they answered in a certain way. To mitigate against this, the evaluation team was very honest about its inability to affect programming and emphasized the reflective nature of the evaluation exercise. 54 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV ANNEX D: CONSENT FORMS INFORMED CONSENT FORM FOR VIRTUAL KII (12TH GRADE READING LEVEL) Principal Investigator: Dr. Ghazia Aslam Organization: EnCompass LLC Sponsor: United States Agency for International Development This informed consent has two parts: (1) information sheet and (2) provision of oral consent. PART I: INFORMATION The United States Agency for International Development, Office of Transitions Initiatives has recently awarded a contract for an independent evaluation of their Iraq Regional Program, also referred to as the Iraq Community Resilience Initiative (ICRI Ta’afi). Our objective is to collect the knowledge and experiences you feel comfortable sharing about your communities. The overall goal of the evaluation is to derive evidence to inform strategy and future programming. Before you decide to participate, you can talk with or ask questions of anyone you feel comfortable with. If you have any questions regarding this content, please let me know. If you have questions later or in the future, you can contact any member of the evaluation team. Information about how you can reach the evaluation team is included in this information sheet. METHODS OF DATA COLLECTION For this study, we are collecting information by conducting virtual interviews, focus group discussions, and with ICRI staff, local government officials, civil society organizations, community leaders, and community members, as well as an online survey with additional community members. This interview will last approximately one hour. Data will be collected throughout September 2021. RESPONDENT SELECTION You have been chosen for a key respondent interview because you either have knowledge of and/or experience with the ICRI program or within the communities that ICRI has worked. VOLUNTARY PARTICIPATION AND RIGHT TO WITHDRAW Your participation in this evaluation is entirely voluntary. If you choose not to participate, we will remove the contact information we have for you from our data set. If you decide to participate, you may change your mind and withdraw from the study at any time, and we will remove your contact details and any other information you have provided up to that time. Choosing not to participate in this 55 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV study will not affect your relationship with United States Agency for International Development or EnCompass LLC. If you decide to participate, you may ask to skip any question or stop the interview at any time. You do not have to share anything you are not comfortable sharing. After the interview, if you decide you do not want your remarks/answers to be used, you may let us know and we will immediately destroy all information and it will not be used. PERMISSION TO RECORD We would like to take notes and record this discussion via the online platform used to conduct the virtual interview (e.g., Zoom, Jitsi, Signal). This is so that we can develop an accurate transcript of what you say that will help us in analyzing the data. The recording will be kept on EnCompass’ secure password-protected web-based platform specific to this evaluation, which is only accessible to the EnCompass team conducting this evaluation. You will not be identified by name in the recording file name, and any identifying information will be removed during transcription. Once the transcription is complete, the recording will be deleted and any written notes will be destroyed, and only the transcript will be kept. In the meantime, no one outside of the EnCompass assessment team will have access to the recordings or notes. RISKS The study team will keep all of your answers and personal information in a secure location, and will not share this information with anyone not involved in the study. Your responses will not be shared with colleagues or your supervisor or anyone else. Your personal information, including your name, will not be included in the final report or any presentation of findings. Any risk of breaching your privacy or confidentiality has been significantly reduced by the strict rules and ethical principles all our team members have agreed to observe. In terms of risks: · Some questions may cause discomfort or distress. If that is the case, we will give you a list of people you can call or meet for assistance. You can ask us at any time for this information—now, during, or after we finish the interview. You can also leave the interview for a while or leave and not come back. · There is a chance that the conversation could be intercepted online from the platform we are using to conduct the virtual interview (e.g., Zoom, Jitsi, Signal). We are taking many steps to prevent this from happening, such as using password-protected meeting links and choosing encrypted platforms when possible. However, it is still possible that the discussion could be intercepted through a security breach in the online platform. · There is a chance that the information you share will not stay private due to a data leak. We are doing many things with the data we collect to prevent the leak from happening, but it could happen. · You may be worried someone will learn what you said and not like that you joined the meeting. You may not feel safe. Again, we will do everything we can to keep your answers safe and private. If someone does find out, somehow, you may not feel safe. You can contact people listed on this consent form. We will give you a copy of this form, if participating in this study affects your safety. There are actions we can take to keep your answers safe. 56 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV BENEFITS You may receive no direct benefits from participating in this study, but your participation is likely to help inform USAID’s programing. REIMBURSEMENTS No compensation is provided for this interview. CONFIDENTIALITY The information you share will be kept confidential. We will not share the information you provide to us with anyone who is not part of the evaluation team. We will store information, such as recordings, transcripts, and notes, electronically in password-protected and secure web-based data storage and analysis platforms specific to this evaluation. The interview transcripts, notes, and recordings will be de￾identified, with each participant assigned a number ID. The ID linked to each participant will only be available in a participant tracker, which is saved separately on the password-protected platform specific to this evaluation that is only accessible to the EnCompass assessment team. Information analyzed will be anonymous. We will destroy all the information you provide three years after the publication of the final report. SHARING SUMMARY RESULTS The evaluation team will combine the information you provide with that from other interview respondents, focus group participants, survey respondents, and reviewed documents; we will present it in a final report that we will share with the USAID and other stakeholders. An external version summary will also be produced and we can send it to you. This report will also be available to other data collection participants and members of the public. WHOM TO CONTACT This study has been reviewed and approved by the EnCompass LLC Review Board [insert reference number], whose members are tasked with making sure participants are protected from harm. If you wish to find out more about the ethics review, contact Dr. Sarah Lunsford at ssmith@encompassworld.com. If you have any questions about this research study in the future, please contact me using the following information: Name and phone # of interviewer: _________________________________________________ If you have complaints about our team or the study, please contact: Dr. Ghazia Aslam Team Lead, Iraq Regional Program Evaluation Email: gaslam@encompassworld.com 57 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV PART II: PROVISION OF ORAL CONSENT, CONSENT TO RECORD AND CONSENT TO BE CONTACTED TO RECEIVE EXTERNAL SUMMARY Say to the respondent: I want to confirm that you have read the consent form or I have read it to you. Do you have any additional questions? After answering respondent’s questions: Do you voluntarily consent to participate in this study? Following verbal consent, note the response of the respondent with your initials: Yes ___ No ___   Interviewer’s initials: ________ If the respondent answers no:   Thank them for their time and end the interview.    If the respondent answers yes: Ask them for their verbal consent to audio and video record the conversation using the virtual platform: I/We would like to both take notes and audio and video record our conversation. This is so we can develop an accurate transcription of what you say. The recording can be stopped at any time, if you choose to do so. We will delete the recording and destroy written notes immediately after creating a transcription of the conversation. The transcription will be deleted three years after the publication of the final report. In the meantime, no one outside of the study team will have access to this information. Do you give me/us permission to record the audio and video of this virtual interview, using the record feature of this platform? Following verbal consent to audio and video-record, note the response of the respondent with your initials: Yes ___ No ___ Interviewer’s initials: ________ If the respondent answers yes: Continue with the interview. If the respondent answers no: I understand your decision not to have our conversation recorded. Would you be comfortable proceeding with this interview if I take written notes only and do not use the record feature on this platform? If the respondent agrees to hand-written notes, proceed with the interview. Would you like us to send the report using the same contact information that we used to contact you? Yes ___ No ___ Interviewer’s initials: ________ If the respondent does not consent to hand-written notes, end the interview. Signature of the interviewer: _______________________________________ Date: _______________________________________ 58 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV INFORMED CONSENT FORM FOR VIRTUAL KII (5TH GRADE READING LEVEL) Principal Investigator: Ghazia Aslam Organization: EnCompass LLC Sponsor: United States Agency for International Development, Office of Transitions Initiatives PART I: INFORMATION REASONS WE ARE DOING THIS STUDY AND YOU ARE INVITED TO PARTICIPATE: Thank you for speaking with me today. My name is [name of data collector]. I work for an American company called EnCompass. I am here to learn from people like you to better understand the local conditions in your community. This will help provide insights for development programs run by the United States Agency for International Development, Office of Transitions Initiatives (USAID/OTI). The OTI team in [Mosul/NP] suggested that we talk to you because you are a member of this community. Do you have any questions? (Answer questions, if any, then continue.) METHODS FOR COLLECTING DATA: For this study, we are collecting information by conducting interviews with OTI staff, with local government figures in Mosul and Hamdaniya, with religious leaders, civil society leaders, and as well as community members. If you decide to join this study, we will ask you about different topics, such as your perception of recovery in your city or your work and experience with OTI. We will listen to you and write down your answers. This will take approximately one hour. NOW, I WILL DESCRIBE YOUR RIGHTS: You do not have to be part of this study. You can say “no” and end the call. We will not bother you again. If you decide to participate, you can change your mind and stop at any time for any reason. You have the freedom to choose. You may consider some of the questions to be sensitive. You can also choose to skip any question you do not want to answer. You can let us know if you want us to keep your answers or not. If you do not want us to keep your answers, we will destroy them before they are included in the study. NOW, I WILL DESCRIBE RISKS AND BENEFITS: We use strict rules to keep your information safe and secure. We will not share your private information, like your name, with people who are not on our team. We will protect your privacy and only report your answers within a group of many answers from a lot of other people, so that your privacy is protected. We keep information safely stored for one year. We destroy all your answers one year after we finish the report. 59 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV In terms of risks: · Some questions may not feel good to answer. If you feel things you don’t like, we will give you a list of people you can call or meet for assistance. You can ask us at any time for this information, now, or during or after we finish the interview. You can also leave for a while or leave and not come back. · There is a chance that the information you share will not stay private due to a data leak. We will do everything we can to keep your answers safe and private, but a leak could happen. · There is a chance that our conversation could be intercepted online from the platform we are using for this conversation (e.g., WhatsApp). We are taking many steps to prevent this from happening. We are using password-protected and encrypted meeting links and choosing safe and reliable platforms, when possible, but an online security breach could happen. · You may be worried someone will learn what you said and not like that you joined the meeting. You may not feel safe. Again, we will do everything we can to keep your answers safe and private. If someone does find out, somehow, you may not feel safe. You can contact people listed on this consent form. We will give you a copy if this form, if participating in this study affects your safety. There are actions we can take to keep your answers safe. In terms of gains, there is not a direct gain for you personally by joining this meeting. However, your participation will provide a general benefit to your city and its residents, those who are living here now and those who would like to come back, because it will help us understand how projects can help communities improve their quality of life and how they can help rebuild after war. We can take your opinions and use them to make better projects for Ninewa. Do you have any questions for us now? (Answer questions, if any.) PART II: ORAL INFORMED CONSENT, CONSENT TO RECORD, AND CONSENT TO BE CONTACTED TO RECEIVE THE EXTERNAL SUMMARY Now we want to ask your verbal permission to be part of the study. Would you like to participate in this study? Note the response of the respondent with your initials: Yes ___ No___Interviewer’s initials: ________ If they say no: Stop asking questions, thank them. If they say yes: Thank you. We would also like to record this meeting to have an accurate account of your statements. You can ask us to stop the recording any time for any reason. Is it okay with you if we record this interview? Note the response of the respondent with your initials: Yes ___ No___Interviewer’s initials: ________ If the respondent says no: 60 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Ask them if they are okay participating without recording, limit data collection to written notes. If the respondent says yes: Note the response of the respondent with your initials: Yes ___ No___Interviewer’s initials: ________ The evaluation team will combine the information you provide with that from other interview respondents, focus group participants, survey respondents, and reviewed documents; we will present it in a final report that we will share with the USAID and other stakeholders. An external version summary will also be produced that will also be available other data collection participants and members of the public. Would you like us to send the report using the same contact information that we used to contact you? Yes ___ No___Interviewer’s initials: ________ Do you have any other questions or comments? (Answer them, if any.) If you have questions after this interview, you may can contact us using the information on this form. Would you like a copy? (Share if they would like a copy.) Now we are ready to begin. ADDITIONAL INFORMATION: This study has been reviewed and approved by the EnCompass LLC Institutional Review Board. This committee reviewed this study. They asked for improvements. We made these improvements. This was done to make sure you and others who choose to join the study are protected from harm. To learn more, contact Dr. Sarah Lunsford, email: ssmith@encompassworld.com. If you have any questions about this research study, please contact me using the following contact information: Name and Phone # of Data Collector: _______________________________________________ If you have comments, questions, or complaints about our team or the study, please contact: Dr. Ghazia Aslam, evaluation team Lead, email: gaslam@encompassworld.com 61 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV ANNEX E: DATA COLLECTION INSTRUMENTS INTRODUCTORY MESSAGE FOR VIRTUAL INTERVIEWS Hi [Name). I hope you’re doing well. My name is […] and I work with a company called EnCompass that is doing an evaluation for OTI’s work in Iraq. I got your number from […] and am messaging to ask you if I can interview you for the evaluation. The goal from this evaluation is to understand how the activities of the OTI programming have impacted recovery in your town. We are trying to interview many residents and individuals who are impacted by this program so we can evaluate how well it is working in places like Mosul and Hamdaniya. The interview will be private and if you chose not to do it, it will have no effect at all on you, even if you have a relationship with OTI. The interview would be over the phone at whatever time works for you. If you would like to be interviewed, please respond to this message and let me know what time I can call. I will also send you some background information before the interview. Please let me know if you have any questions. Best, Name [EnCompass LLC) FOR IN-PERSON INTERVIEWS Hi [Name). I hope you’re doing well. My name is […] and I work with a company called ThinkBank that is doing an evaluation for OTI’s work in Iraq. I got your number from […] and am messaging to ask you if I can interview you for the evaluation. The goal from this evaluation is to understand how the activities of the OTI programming have impacted recovery in your town. We are trying to interview many residents and individuals who are impacted by this program so we can evaluate how well it is working in places like Mosul and Hamdaniya. The interview will be private and if you chose not to do it, it will have no effect at all on you, even if you have a relationship with OTI. The interview would be in-person at whatever time works for you. If you would like to be interviewed, please respond to this message and let me know what time I can call to confirm the details. I will also send you some background information before the interview. Please let me know if you have any questions. Best, Name [EnCompass LLC) 62 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV KEY INFORMANT INTERVIEW GUIDE – ICRI PROGRAM STAFF Key Informant Interview (KII) Guide Interviewer’s Name: Interview Date: Respondent Number: Respondent’s Position [Give rank and denote whether the position is supervisory]: Years in Current Position: Respondent’s Sex: Interview Start Time: Interview End Time: PARTICIPANT CRITERIA QUESTIONS ( ) Has given oral consent to participate in the interview RESPONDENT BACKGROUND 1. To start, can you please give a brief description of your job responsibilities within the ICRI team? How long have you worked on social recovery issues in Iraq? a. What geographical region do you serve in your capacity? [Note to interviewer: ask if the respondent is from the area that they work in, generally are they from the Ninewa Plain, Mosul, etc.?] ICRI PROGRAM CONTEXT 2. In your view, what are the main issues that the ICRI program sought to address in the areas of Mosul and/or the Ninewa Plain where you worked (tailor according to respondents’ responsibilities)? a. [Ask only if time allows] What is unique about the [Mosul/the Ninewa Plain] context, compared to other regions in which ICRI implemented activities? What challenges did this present, if any? Please be specific and discuss variations within [Mosul/the Ninewa Plain] if there are variations. b. To what extent have previously displaced persons/refugees returned to and stayed in different areas of [Mosul/the Ninewa Plain]? In your view, why have/haven’t durable returns occurred? [note interviewer: ask where have most often displaced to] 63 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV PROGRAM APPROACH IN MOSUL/THE NINEWA PLAIN 3. In your view, how did ICRI activities improve social recovery in the community? [Note to the interviewer: Social recovery is understood in terms of improvement in provision of critical services, availability of livelihood opportunities, and social conditions. Some of these outcomes may have already been covered in the question above]. Can you give any examples, including activities that built on one another or together made a bigger difference than they would have by themselves? a. How did these activities contribute to community members’ life? (E.g., in terms of having access to basic services in their communities, being able to live peacefully, being accepted in the community)? What did that look like in practice? How are the changes different for different groups? [keep the question broad for them to be able to identify all aspects of change] b. How did these activities change the relationships community members have with each other (community members from different ethno-religious groups, government officials)? What did those changes/behaviors look like in practice? [For each effect – ask how, why, what were the mechanisms of change; and most importantly, what do you see/hear that makes you say that? What is the evidence that this change occurred, and how do these changes vary across different locations and/or groups]? c. Did the activities have any unintended consequences (positive or negative)? If so, what are these consequences and why/what do you think caused them to occur? 4. What challenges did you or other stakeholders face in implementing ICRI activities? [Ask the respondent to think of specific activities, as well as in general]. a. To what extent do these challenges still exist and/or how did you overcome them? If these challenges still exist, are they trending upwards or downwards and why? UNDERSTANDING AND ADDRESSING SOCIAL RECOVERY NEEDS IN [MOSUL/THE NINEWA PLAIN] 5. To what extent, if at all, do you believe that activities were responsive to communities’ needs and contributed directly or indirectly to social recovery? [Interviewer note: give examples of the types of activities were are referring to here] a. How did the program ensure that the activities were responsive to communities’ needs? [Prompt if needed: Were needs formally agreed upon or were they informally acknowledged? How frequently were these strategies applied (in every activity, some activities etc.)? How did these strategies evolve? b. To what extent, in your opinion, did the program ensure that various groups (ethnicities, religions, age, gender, etc.) equally participate in or benefit from activities that promoted social recovery? What, if any, efforts were made to ensure that all groups participated or benefited equally? 6. To what extent, in your view, did program participants and beneficiaries feel that the activities helped with social recovery, and what are the pathways of change? [Probe for how they define social recovery and the role that service delivery, livelihoods and social conditions play in social recovery] 7. What work still needs to be done to support social recovery in [Mosul/the Ninewa Plain]? [Probe: ask about needs and challenges that persist and how have they evolved over time] 64 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV a. Who has or should have a role in addressing these needs (probe about community leaders, community members, local government officials, national government, someone else)? ALLOWING FOR DURABLE RETURNS IN [MOSUL/NINEWA PLAIN] 8. To what extent (and through what pathways) do you feel that the program activities contributed to previously displaced persons returning and investing in their communities? What does that look like practice? a. Does this vary by group (age, location, etc.)? Do you have any examples? [note to interviewer: we are trying to gauge if people returning are planning on staying and investing in their communities] b. How has the program's strategy played a role? What aspects of the strategy are most effective in your community [Probe for sequencing and layering, and deeply contextualized activities]? 9. In your view, what are the remaining challenges to durable returns? a. Why do these challenges persist? [Note to the interviewer: ask why ICRI was unable to address these needs] b. Have these challenges evolved over time? If so, in what ways? c. What can be done to address these challenges? PERCEPTIONS OF SOCIAL RECOVERY IN [MOSUL/THE NINEWA PLAIN] 10. In your view, to what extent do you think different groups of people perceive that social recovery has been progressing in the last 2-3 years? a. How, if at all, these perceptions vary across different groups (based on gender, age)? Locations? b. What role do you think these perceptions play in determining whether or not a previously displaced person will return to and stay in their community of origin? 11. To what extent do you think that ICRI activities influenced different community members’ perceptions of social recovery in the last 2-3 years (including perceptions of service delivery and intra-society relationships)? a. What strategies did the ICRI program employ to influence communities’ perceptions of recovery? b. Which strategies/activities were more effective or less effective than others? c. What role do you think ICRI media activities or media components of other activities played in influencing perceptions of recovery? d. What strategies and/or approaches did the ICRI program employ to amplify information about project activities? What audiences did the program target with these strategies or approaches? What, if any, challenges did the program face with respect to ensuring audiences received accurate and timely information about activities? [note to interviewer: ask about both traditional and social media strategies] e. How did ICRI-Ta’afi’s strategy or activities interact with other international assistance organizations’ work in the same locations? [Probe: To what extent did ICRI leverage or complement other organizations’ activities, or vice versa?} 65 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV LOOKING AHEAD 12. Is there anything you would change about how the program [or a specific activity or type of activities] was designed or conducted? a. What worked well and what didn’t? [note to interviewer: ask about how the ICRI program coordinated internally] b. What aspects of the program would you like to have seen amplified/replicated or expanded? 13. Finally, thinking specifically of your own involvement in the program, if you could recommend changes to improve future assistance strategies or activities in [Mosul/the Ninewa Plain], what do you think is the single most important lesson that should be written down? 66 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV KEY INFORMANT INTERVIEW GUIDE – LOCAL GOVERNMENT OFFICIALS Key Informant Interview (KII) Guide Interviewer’s Name: Interview Date: Respondent Number: Respondent’s Position/Title: Respondent’s Sex: Respondent’s Town/Community: [Also note if this is the community they have returned to] Interview Start Time: Interview End Time: PARTICIPANT CRITERIA QUESTIONS ( ) Participant is over the age of 18 ( ) Has given oral consent to participate in the interview RESPONDENT BACKGROUND 1. To start, can you please give a brief description of your position? a. What are your responsibilities in this position? b. How did you come into this position (e.g., is it an elected position, an assigned position or an inherited one?) c. What geographical region do you serve in your capacity? SOCIAL CONDITIONS 2. In your view, what are the top 3 key successes that communities in [Mosul/the Ninewa Plain] have experienced since 2019? [Ask for ranking] Who has contributed to these successes? a. How have neighborhoods or areas of the city changed over the past several years [note to interviewer: ask about how the city/neighborhood has physically evolved in terms of infrastructure and availability or access to services]? 67 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV b. Has the way that community members interact with each other changed? How? Can you give any examples? c. How have the (religious/ethnic) identities of residents and demographics of neighborhoods or areas of the city changed over the past several years? 3. In your opinion, how do most residents perceive the following? (l = very slow; 2=somewhat slow; 3=neither fast nor slow; 4= somewhat fast; 5= very fast) [If interviewee does not respond, please mark as N/A] Question Very Slow Somewhat slow Neither slow nor fast Somewhat fast Very fast N/A The pace of infrastructural recovery in your city in the last 2-3 years? 1 2 3 4 5 N/A The delivery of critical public services in your city in the last 2-3 years? 1 2 3 4 5 N/A The progress toward a return to more normal social activities and interactions among residents in the last 2-3 years? 1 2 3 4 5 N/A The pace of economic recovery in your city in the last 2-3 years? 1 2 3 4 5 N/A The pace of economic recovery in Iraq overall in the last 2-3 years? 1 2 3 4 5 N/A 4. Are these perceptions shared by older and younger individuals? Are they shared by men and women? Are there ethnic or religious groups who may have a different perception? 5. In your opinion, how do residents form these opinions? What tools do they rely on to form their opinions about what is happening in their own community versus what is happening in Iraq overall? [Probe: do they trust others? Do they rely on social media to see what is happening in their city? Do they rely on word of mouth? On the opinion of trusted community leaders DURABLE RETURNS: 6. Please tell me about your community’s experience with displacement during ISIS’s incursion and during the war? a. To what extent did people leave the community? 68 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV b. To what extent have previously displaced persons returned to [Mosul/the Ninewa Plain]? c. In your view, why did individuals choose to return? [If the respondent indicates that some people go back and forth between the community and other cities where they partially live, ask why this is and what would make them stay.] d. In your view, are those who returned or never left likely to stay? Why or why not? If they leave, where would they likely go, and why? e. In your view, to what extent are residents from your city looking for a chance to emigrate? In your opinion, are residents in your town just as likely, more likely, or less likely to seek emigration than other Iraqis? f. [Ask only if time allows] Where did displaced people from your community ultimately settle in (if not in Mosul/NP)? Why? [Ask which factors played a role service delivery, economic or intra-society relationship] GOVERNMENT AND PROGRAM ACTIVITY 7. How has your office (through activities and policies) tried to promote durable returns? What do you think are the pathways through which your office can promote durable returns? [Probe about provision of critical services, livelihood and economic opportunities, and social conditions]. Has your office partnered with or received help from any international organization to promote economic opportunities and improve communal ties in your city? 8. What has your experience been like with Ta’afi program? Can you identify some activities that Ta’afi has implemented in your area? For each activity they identify: e. Were you involved in designing the activity? Who else was involved? f. Who benefited from the activity? Did any group benefit more than the other? g. In your view, what is successful about the [insert activity title or grouping] activity? h. How did this activity help social recovery? How did it contribute to durable returns? i. Did activities supported by Ta’afi build on other activities, policies, or improvements that your office, other government offices, NGOs, or others had done before? [If yes] Can you tell me about this story? Did the activities ICRI supported enable further activities, policies, or other community improvements to begin or to be more effective? [If yes] Can you tell me more about this? For the activities that the respondent is not able to identify: 9. What can you tell me about [insert activity title or grouping] activity? a. Were you involved in designing the activity? Who else was involved? b. Who benefited from the activity? Did any group benefit more than the other? c. In your view, what is successful about the [insert activity title or grouping] activity? d. How did this activity help social recovery? How did it contribute to durable returns? e. Did activities supported by Ta’afi build on other activities, policies, or improvements that your office, other government offices, NGOs, or others had done before? [If yes] Can you tell me about this story? Did the activities ICRI supported enable further activities, policies, or other community improvements to begin or to be more effective? [If yes] Can you tell me more about this? f. Is there anything you would change about how the activity was designed or implemented? g. Did the activity have any unintended consequences (positive or negative)? 69 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV LOOKING AHEAD 10. [Ask only if time allows] If you had three wishes for your community, what would they be? 11. [Ask only if time allows] Is there anything else you wish I had asked you or anything else you would like to share? 70 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV KEY INFORMANT INTERVIEW GUIDE – COMMUNITY MEMBERS AND PARTICIPANTS Key Informant Interview (KII) Guide Interviewer’s Name: Interview Date: Respondent Number: Respondent’s Sex: Respondent’s Age: Respondent’s District/City: Respondent’s Occupation: Respondent’s Religion/Ethnicity: Interview Start Time: Interview End Time: PARTICIPANT CRITERIA QUESTIONS ( ) Participant is over the age of 18 ( ) Has given oral consent to participate in the interview RESPONDENT BACKGROUND 1. Can you please give us a brief description of your social situation and your family situation? [Probe for: What type of work do you perform? What is your role within your family? Do you support a family? If so, of what size? Do they live with you? Are you married? Do you have children? What level of education?] DECISION TO RETURN AND DURABILITY OF RETURNS 2. Were you displaced from the town you were living in during the conflict with ISIS? If so, where did you go? If since returned, when did you return/partially return? [ONLY for those who were displaced, but have since returned] 71 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV 3. Can you describe the moment that you decided to return? What led to that decision? [Probe for: triggers to return (was it proactive or reactive) vs. conditions at places where they were displaced] a. When you returned to your community, were there many others who had returned or were you among the first? b. When you decided to return, did you have any expectation or idea of what your community/city looked like? To what extent have these expectations been fulfilled? [Probe for: what the expectations were about including access to services, livelihood and economic opportunities, as well as intra-society relationships, feelings of being accepted in the community] c. Where did you get that information? [Probe for: information sources, such as family, friends, social media, etc.] 4. Have you encouraged displaced friends or family to return? Why or why not? Have you been successful? Why or why not? [Probe if they say they have encouraged people: Has someone returned because of your encouragement? Do you know if there was something in particular that convinced them? What are the main reasons that people are unconvinced?] 5. Do you plan on staying in [community/city name] for the foreseeable future? Why or why not? [Probe for: services available, infrastructure, employment opportunities, and general sense of acceptance/inclusion] a. Would you like your children or youth in your family to grow up here and raise their own families here? Why or why not? [ask about services available, employment opportunities, and general sense of acceptance/inclusion] 6. In your view, to what extent are residents from your city looking for a chance to emigrate? Why or why not? a. In your opinion, are your citizens just as likely, more likely, or less likely to seek immigration than other Iraqis? [FOR IDPS ONLY] 7. Are you still currently living in the town where you first displaced? [Probe for secondary displacement and reasons for it] a. What are your reasons for not returning? [Probe for: services availability, infrastructure, employment opportunities, and general sense of acceptance/inclusion] 72 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV [ALL RESPONDENTS] 8. What, if anything, have you heard about Ta’afi on social media or other media? Have you seen the hashtags [#ICRI #USAID #NinewaRecovery]? What platforms are they on? What have you learned from social or other media about ICRI? a. How well-known are these pages in your community? PERCEPTION OF SOCIAL RECOVERY [ONLY for those who live in the community – returnees or stayees] 9. What did your city look like 2-3 years ago? [Note to interviewer: ask about the availability of services, such as jobs, healthcare, infrastructure, etc.] a. What have been the three greatest successes in your city in the last 2-3 years? Which was most important, and why? What has contributed to these successes? b. How have your interactions with your neighbors and other members of the community changed in the last 2-3 years? What has contributed to this change? c. Do you feel that the identity of your city has changed in the last 2-3 years? Is this a positive or negative change? What, in your opinion, caused this change to occur? (ALL RESPONDENTS) 10. In your opinion, how do most residents perceive the following? (l = very slow; 2=somewhat slow; 3=neither fast nor slow; 4= somewhat fast; 5= very fast) [If interviewee does not respond, please mark as N/A] Question Very Slow Somewhat slow Neither slow nor fast Somewhat fast Very fast N/A The pace of infrastructural recovery in your city in the last 2-3 years? 1 2 3 4 5 N/A The delivery of critical public services in your city in the last 2-3 years? 1 2 3 4 5 N/A The progress toward a return to more normal social activities and interactions among residents in the last 2-3 years? 1 2 3 4 5 N/A The pace of economic recovery in your city in the last 2-3 years? 1 2 3 4 5 N/A 73 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Question Very Slow Somewhat slow Neither slow nor fast Somewhat fast Very fast N/A The pace of economic recovery in Iraq overall in the last 2-3 years? 1 2 3 4 5 N/A The progress of national dialogue and reconciliation in Iraq overall in the last 2-3 years? 1 2 3 4 5 N/A USAID & INGOS ROLE [ONLY for those who live in the community – returnees or stayees] 11. Have you noticed any public works projects, livelihood projects or service delivery projects in your area? Do you remember who was responsible for these projects? IF they identify Ta’afi, ask for each activity a. What was your role in the activity? How did you come to know about this activity? b. Do you know who is responsible for this activity? How do you know who is responsible for these activities (e.g., social media, volunteer word of mouth?) 12. How, if at all, you benefited from the activity? Who else benefited from the activity? Did any group benefit more than the other? Who might feel left out or negatively affected? a. In what ways, if at all, did the activities have in your community members’ motivation to stay in this community? b. Did this activity build on other activities or improvements in your community? [If yes] Can you tell me about this story? Did this activity enable further activities, policies, or other community improvements to begin or to be more effective? [If yes] Can you tell me more about this? c. In which ways could this activity have been better designed or implemented to meet your community needs? 13. For the activities that the respondent did not identify: Are you familiar with any of the following activities - [list ICRI Ta’afi activities in city]? Which have you participated in or are familiar with? If they are familiar with any activity, ask the following for each activity. a. What was your role in the activity? How did you come to know about this activity? b. Do you know who is responsible for this activity? How do you know who is responsible for these activities (e.g., social media, volunteer word of mouth?) 14. How, if at all, you benefited from the activity? Who else benefited from the activity? Did any group benefit more than the other? Who might feel left out or negatively affected? a. In what ways, if at all, did the activities have in your community members’ motivation to stay in this community? b. Did this activity build on other activities or improvements in your community? [If yes] Can you tell me about this story? Did this activity enable further activities, policies, or 74 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV other community improvements to begin or to be more effective? [If yes] Can you tell me more about this? c. In which ways could this activity have been better designed or implemented to meet your community needs? 15. How have the above activities [refer to Ta’afi activities] affected the quality of life in your city in the last 2-3 years? Can you give any examples? LOOKING AHEAD (ALL RESPONDENTS) 16. If you had three wishes for your community, what would they be? [For IDPs, ask about the community they displaced from, and what would make them return] 75 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV KEY INFORMANT INTERVIEW GUIDE – LOCAL PARTNERS Key Informant Interview (KII) Guide Interviewer’s Name: Interview Date: Respondent Number: Respondent’s Position: Respondent’s Sex: Respondent’s Town/Community: Interview Start Time: Interview End Time: PARTICIPANT CRITERIA QUESTIONS ( ) Participant is over the age of 18 ( ) Has given oral consent to participate in the interview RESPONDENT BACKGROUND 1. To start, can you please give a brief description of your organization? a. What position do you hold within this organization? What does this position entail? b. What geographical region do you serve in your capacity? c. What, if any, demographic group(s) do you mostly work with or help? SOCIAL RECOVERY CONTEXT 2. In your view, how has life changed in communities in [geographical region you serve] since 2019? a. How has the community evolved over the past several years [Probe for: how the city/neighborhood has physically evolved in terms of infrastructure, service delivery and economic activity]? Can you give any examples? b. Has the way that community members from different groups interact with each other evolved? How? Can you give any examples? 3. What are the top three challenges that the [geographical region you serve] is currently facing? 76 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV a. What work still needs to be done? [Probe for: infrastructure, economic growth, intra-society relationships, government services, etc.]? In your view, why do these challenges persist? Who has a role in addressing these challenges?] DURABILITY OF RETURNS 4. To what extent have previously displaced persons returned to [the geographical region you serve]? a. From your perspective, are those who returned likely to stay? Why or why not? b. In your view, why or why not have these returns occurred? [note interviewer: ask where displaced persons most often went] c. To what extent do you think the challenges you mentioned [recall respondents’ answers for question 3] impact people’s decision to return and stay? 5. In your view, to what extent are residents from the geographical region you serve looking for a chance to emigrate? In your opinion, are the citizens in the [geographical region] you serve just as likely, more likely, or less likely to seek emigration than other Iraqis? 6. In your opinion, how do most residents in the geographical region you serve perceive the following? (l = very slow; 2=somewhat slow; 3=neither fast nor slow; 4= somewhat fast; 5= very fast) [If interviewee does not respond, please mark as N/A] Question Very Slow Somewhat slow Neither slow nor fast Somewhat fast Very fast N/A The pace of infrastructural recovery in your city in the last 2-3 years? 1 2 3 4 5 N/A The delivery of critical public services in your city in the last 2-3 years? 1 2 3 4 5 N/A The progress toward a return to more normal social activities and interactions among residents in the last 2-3 years? 1 2 3 4 5 N/A The pace of economic recovery in your city in the last 2-3 years? 1 2 3 4 5 N/A The pace of economic recovery in Iraq overall in the last 2-3 years? 1 2 3 4 5 N/A a. How do these perceptions vary across different groups? Locations? 77 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV b. What factors are most likely to influence perceptions of recovery? [note to interviewer: ask about traditional and social media and communications campaigns] 7. In general, how do people receive/send information or news about the [geographical region you serve]? [Probe for sources other than media]. [When media] Which platforms do people tend to engage with most frequently [Probe for: trust in this platforms/information and what each platform is used for]? PROGRAM ACTIVITIES 8. Can you tell me about this activity? [How did you get involved in the activity? With the ICRI program? 9. Did community members generally know about these activities? How did you conduct outreach about the activity? How did community members learn about the activities? 10. In your view, what is successful about the [insert activity title or grouping] activity? a. To what extent do you think that community members benefited from this activity? Did all groups benefit equally? Was any group missing? Who might feel left out or negatively affected? [Probe: Were people equally represented by ethno-religious group, age, gender, etc.?] b. In what ways, if at all, did the activities have in your community members’ motivation to stay in this community? c. Did this activity build on other activities or improvements in your community? [If yes] Can you tell me about this story? Did this activity enable further activities, policies, or other community improvements to begin or to be more effective? [If yes] Can you tell me more about this? 11. In which ways could this activity have been better designed or implemented? 12. Did the activity have any unintended consequences (positive or negative)? 13. Are there any other lessons from being part of this activity that you would like to share with us? 78 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV KEY INFORMANT INTERVIEW GUIDE – COMMUNITY LEADERS (TRADITIONAL AND NON-TRADITIONAL) Key Informant Interview (KII) Guide Interviewer’s Name: Interview Date: Respondent Number: Respondent’s Sex: Respondent’s Age: Respondent’s District/City: Respondent’s Occupation: Respondent’s Religion/Ethnicity: Interview Start Time: Interview End Time: PARTICIPANT CRITERIA QUESTIONS ( ) Participant is over the age of 18 ( ) Has given oral consent to participate in the interview RESPONDENT BACKGROUND 1. Can you please give us a brief description of your role in your community? How long have you held this role? 2. How long have you lived in [community/city name]? SOCIAL CONDITIONS 3. How has the community evolved over the past several years [note to interviewer: ask about how the city/neighborhood has physically evolved in terms of infrastructure]? Can you give any examples? a. Has the way that community members interact with each other evolved? b. How has the configuration of different groups in terms of ethnicity changed over time? [Probe if people live in different areas than they previously did, or new groups who 79 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV didn’t previously live there and have moved in or one or more groups of people who had been displaced have not returned] c. What are the top three challenges that [Mosul/the Ninewa Plain] is currently facing? d. Who has a role in addressing these challenges? PERCEPTION OF SOCIAL RECOVERY 4. In your opinion, how do most residents perceive the following? (l = very slow; 2=somewhat slow; 3=neither fast nor slow; 4= somewhat fast; 5= very fast) [If interviewee does not respond, please mark as N/A] Question Very Slow Somewhat slow Neither slow nor fast Somewhat fast Very fast N/A The pace of infrastructural recovery in your city in the last 2-3 years? 1 2 3 4 5 N/A The delivery of critical public services in your city in the last 2-3 years? 1 2 3 4 5 N/A The progress toward a return to more normal social activities and interactions among residents in the last 2-3 years? 1 2 3 4 5 N/A The pace of economic recovery in your city in the last 2-3 years? 1 2 3 4 5 N/A The pace of economic recovery in Iraq overall in the last 2-3 years? 1 2 3 4 5 N/A a. How do these perceptions vary across different groups? Locations? b. What factors are most likely to influence perceptions of recovery? [note to interviewer: ask about traditional and social media and communications campaigns] DURABLE RETURNS 5. Please tell me about your community’s experience with displacement during ISIS’s incursion and during the war. [Probe for the extent to which people stayed or left]. 6. To what extent have previously displaced persons returned to [community/city name]? 80 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV a. In your view, why did individuals choose to return? b. In your view, are those who returned or never left likely to stay? Why or why not? If they leave, where would they likely go to? c. In your view, to what extent are residents from your city looking for a chance to emigrate? In your opinion, are your citizens just as likely, more likely, or less likely to seek immigration than other Iraqis? USAID & INGOS ROLE 7. Have you noticed any public works projects, livelihood projects or service delivery projects in your area? Do you remember who was responsible for these projects? IF they identify Ta’afi, proceed with these questions: a. Were you involved in any of these activities? If so, what was your role? b. How helpful have these activities and the Ta’afi program been in improving the quality of life in your area during the last 2-3 years? For your community? c. How helpful have these activities and the Ta’afi program been in improving the quality of relationships and community ties in your city in the last 2-3 years? d. In which ways can these activities be more responsive to your community needs? e. IF they do not identify Ta’afi, proceed with these questions: f. Are you familiar with [list ICRI activity] that is supported by the Ta’afi program? IF YES, proceed to the following questions: g. How helpful have these activities and the Ta’afi program been in improving the quality of life in your area during the last 2-3 years? For your community? h. How helpful have these activities and the Ta’afi program been in improving the quality of relationships and community ties in your city in the last 2-3 years? i. In which ways can these activities be more responsive to your community needs? IF they done identify any Ta’afi activities or miss any Ta’afi activity 8. Are you familiar with any of the following activities - [list ICRI activities in city]? Which have you participated in? For each activity that you participated in: a. How did you become involved? What other stakeholders were involved? b. How helpful was this activity and the Ta’afi program been in improving the quality of life in your area during the last 2-3 years? [Probe for improvement in delivery of services and livelihoods]. c. How helpful have these activities and the Ta’afi program been in improving the quality of relationships and community ties in your city in the last 2-3 years? d. In which ways can these activities be more responsive to your community needs? 81 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV COMMUNITY MEMBERS AND BENEFICIARIES SURVEY We at Encompass are carrying out an evaluation study of USAID projects in Nineveh Governorate. In this study, we collect information by conducting a survey with the residents of Mosul and the Nineveh Plain and we would be grateful if you would participate in this survey. If you decide to participate in this study, we will ask you about different topics, such as your perception of recovery in your city or work and your opinion of the operation of a USAID-funded recovery program. We will not ask you for any personal information, and no one outside the EnCompass team will see the answers to the survey. BACKGROUND AND DEMOGRAPHICS 1. What year were you born? 1. [drop-down menu] 2. Decline to answer/Don’t know 2. What is your gender? 1. Female 2. Male 3. Decline to answer/Don’t know/Other 3. Governorate of birth? 1. List all governorates, plus an option for outside of Iraq 4. District of birth? 1. List of all districts and an option for ‘other’ 5. Where do you currently live? [the options here are the sampling sites] 1. Old City of Mosul 2. Al-Islah Al-Zirai 3. Al-Tanak 4. Qaraqosh 5. Bartella 6. Bashiqa 7. Other [please list] 6. What is the highest level of education you have completed? 1. No education 2. High School education or below 3. Vocational/technical training 4. Bachelor's degree 5. Professional Degree 7. Do you have a job that enables you to earn money? 1. Yes, Public sector/government employee 2. Yes, private sector employee or NGO employee 3. Yes, various intermittent and temporary jobs 4. Yes, self-employed 5. Yes, other type of employment. 6. No 7. Decline to answer 82 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV DECISION TO RETURN AND DURABILITY OF RETURNS 8. Were you displaced? 1. Yes, within Iraq, where […] 2. Yes, outside Iraq, where […] 3. No 4. Decline to answer 9. Have you returned to the town you were displaced from as a result of the ISIS conflict? 1. Yes, I have fully returned 2. Yes, I have returned partially 3. No, I am still displaced (Skip to Q. 15) 4. No, I was never displaced (Skip to q.18) 5. Decline to answer 10. Which of the following factors motivated you to return? Factor Strongly Disagree Somewh at disagree Disagr ee Neither agree nor disagree Somew hat Agree Strongly Agree Services in my town were improving Security situation in my town was improving Social life in my town was returning to normal I had a business I wanted to start operating I had a job opportunity to return to I missed my town The conditions in my displaced destination were very bad 11. What, if anything according to your information, is different about the population composition compared to when you were displaced? 1. People live in different areas of the town than they previously lived in 2. One or more groups of people who had been displaced still mostly have not returned. 3. New groups of people who didn’t previously live there have moved in. 4. Something else (not listed here) 5. Nothing has changed about the groups that live in my area. 6. Decline to answer/ Don’t know 12. Have you encouraged displaced friends or family to return? 1. Yes 2. No 3. Decline to answer 83 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV 13. If you are still displaced, do you ever plan to return to the city you were living in before you were displaced by ISIS? 1. Yes, I plan on returning. 2. No, I do not plan on returning. 3. I am not displaced; this question is not relevant to me. 4. 5. Decline to answer/ Don’t know 14. Did/have people already residing in the town you were displaced from encourage you to return? 1. Yes 2. No 3. I was never displaced; this question is not relevant to me. 4. Decline to answer 15. If you were displaced or ae still displaced, how do/did you get reliable information about the state of your city/town? 1. Through family and friends 2. Through social media (please specify platforms) 3. Through newspapers 4. Through radio 5. Through TV 6. I went there myself to see 7. Other, please list […] 8. Decline to answer 16. Do you plan on staying in your area for the foreseeable future? 1. Yes, for the rest of my life 2. Yes, for a few years 3. No, I want to leave soon 4. Decline to answer/ Don’t know 17. According to your estimation, how many people have returned to the town you were displaced from as a result of the ISIS conflict? 1. Not many people have returned 2. Some people/families have returned but not many 3. Almost half the people have returned 4. Most people have returned 5. Almost everyone has returned by the time I returned 6. Decline to answer / Don’t know 18. Would you like the children or the youth in your family to grow up in the town you were displaced from during ISIS conflict and raise their own families t/here? 1. Yes 84 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV 2. No 3. Decline to answer/Don’t know 19. In your view, to what extent are residents from the town you were displaced from during ISIS conflict are looking for a chance to emigrate? 1. Many residents are looking for a chance to emigrate 2. Some residents are looking for a chance to emigrate 3. Few residents are looking for a chance to emigrate 4. Decline to answer/Don’t know 20. In your view are residents of the town you were displaced from during ISIS conflict just as likely, more likely, or less likely to seek immigration than other Iraqis? 1. More likely than other Iraqis 2. Just as likely as other Iraqis 3. Less likely than other Iraqis 4. Decline to answer 21. Does your town government have social media pages? [if still displaced, please answer about town you were displaced from] 1. Yes 2. No 3. Decline to answer 22. Have you heard about Ta’afi? [check all that apply] 1. Yes, from social media 2. Yes, from other media (TV, Radio, etc.) 3. Yes, from people I know 4. Yes, from signs or flyers 5. Yes, from somewhere else 6. No 7. Decline to answer 23. Have you encountered the following hashtags on social media? 1. #ICRI 2. #USAID 3. #NinewaRecovery PERCEPTION OF RECOVERY 1. How has the state of the following services changed in your area from 2019 to today? ([if still displaced, please answer about town you were displaced from] Service Has gotten worse Has not improved at all Has improved a little Has improved somewhat Has improved a lot Decline to answer/Don’t know Infrastructure (e.g., roads and buildings) Electricity Water Healthcare 85 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Service Has gotten worse Has not improved at all Has improved a little Has improved somewhat Has improved a lot Decline to answer/Don’t know Education Job opportunities Social life in my town was returning to normal End the survey if currently displaced [Answer to Q. 9 is option 3] 24. In your opinion, how do most residents perceive the pace of the following? ([if still displaced, please answer about town you were displaced from] (l = very slow; 2=somewhat slow; 3=neither fast nor slow; 4= somewhat fast; 5= very fast) [If interviewee does not respond, please mark as N/A] Question Very Slow Somewhat slow Neither slow nor fast Somewhat fast Very fast N/A Infrastructural recovery in your city in the last 2-3 years? 1 2 3 4 5 N/A Restoration of delivery of critical public services in your city in the last 2-3 years? 1 2 3 4 5 N/A A return to more normal social activities and interactions among residents in the last 2-3 years? 1 2 3 4 5 N/A Economic recovery in your city in the last 2-3 years? 1 2 3 4 5 N/A Economic recovery in Iraq overall in the last 2-3 years? 1 2 3 4 5 N/A USAID & INGOS ROLE 25. How helpful has the ICRI Ta’afi program been in improving the following in 2-3 years… 86 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV Factor Very Helpful Helpful Neutral Unhelpful Very unhelpful Decline to answer/ Don’t know The quality of life in the town you were displaced from during ISIS conflict Public services in the town you were displaced from during ISIS conflict Economic and employment opportunities in the town you were displaced from during ISIS conflict. Communal ties in the town you were displaced from during ISIS conflict 26. What is your ethno-religious background? 1. Shia Arab 2. Sunni Kurd 3. Assyrian 4. Armenian 5. Yezidi 6. Sunni Shabak 7. Shia Shabak 8. Sunni Turkmen 9. Shia Turkmen 10. Kaka’i 11. Kikia 12. Decline to answer 13. Other [….] 87 | IRAQ REGIONAL PROGRAM EVALUATION REPORT USAID.GOV ANNEX F: LINKING FINDINGS TO EVALUATION QUESTIONS FINDING EVALUATION QUESTION I To the extent that social recovery is a necessary condition for durable returns, how effectively has ICRI Ta’afi understood and addressed social recovery needs in areas liberated from ISIS? EVALUATION QUESTION II Whether and how has ICRI-Ta’afi influenced perceptions of recovery among residents of areas liberated from ISIS? To what extent do these perceptions differ across various groups (region, gender, religious/ethnic group, etc.)? What are the pathways of change? Finding 1 (contextual finding on different patterns of displacement) X Finding 2 (factors of return and durability of return) X Finding 3 (Importance of economic opportunities) X Finding 4 (Social conditions) X X Finding 5 (Services activities) X X Finding 6 (Livelihood opportunities) X X Finding 7 (Social conditions activities) X X Finding 9 (Social media and decision to return) X Finding 10 (Layering and Sequencing) X Finding 11 (Built-in Flexibility)