SOMALIA PROGRAM SUPPORT SERVICES TRANSITION INITIATIVES FOR STABILIZATION PLUS (TIS+) FINAL SYNTHESIS REPORT DELIVERABLE #25 IDIQ AID-623-I-14-00009 TASK ORDER AID-623-TO-16-00015 Resubmitted August 12, 2019 This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by International Business & Technical Consultants, Inc. (IBTCI), under IDIQ AID-623-I-14-00009, Task Order AID-623-TO-16-00015. 2 SOMALIA PROGRAM SUPPORT SERVICES (SPSS) TRANSITION INITIATIVES FOR STABILIZATION PLUS (TIS+) SYNTHESIS REPORT DELIVERABLE #25 IDIQ AID-623-I-14-00009, TASK ORDER AID-623-TO-16-00015 Resubmitted August 12, 2019 Evaluation Team: Sarah Wood, Team Leader Dr. Lucas Malla, Quantitative Specialist Patrick Okwarah, Qualitative Specialist Suleiman Omar, Subject Matter Specialist Mathias Kjaer, Contribution Analyst Additional Contributions: Dr. Abdourahmane Ba, Tom Muga, Jeffrey Swedberg Submitted by: International Business & Technical Consultants, Inc. (IBTCI) In US: 8618 Westwood Center Drive, Suite 400 Vienna, VA 22182 USA +1.703.749.0100 In Kenya: 8th Floor Block C Westcom Point Building Mahiga Mairu Road, off Waiyaki Way Nairobi, Kenya +254 (0) 737 544 912 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development (USAID) or the United States Government. 3 TABLE OF CONTENTS ACRONYMS .......................................................................................................................................... 5 EXECUTIVE SUMMARY....................................................................................................................... 8 I. INTRODUCTION.......................................................................................................................16 Project Background ................................................................................................................ 16 Task Order Purpose................................................................................................................ 16 Assessment Questions............................................................................................................ 17 III. METHODS..............................................................................................................................18 Secondary Literature Review................................................................................................. 19 Household Survey ................................................................................................................... 19 Key Informant Interviews....................................................................................................... 20 Focus Group Discussions ........................................................................................................ 20 Index Tool................................................................................................................................ 20 Political Economy and Stabilization Assessment (PESA) Tool.......................................... 20 III. CONTRIBUTION ANALYSIS AND TIS+...............................................................................22 The Role of Contribution Analysis ........................................................................................ 22 Testing the Theory of Change for TIS+................................................................................ 22 Contribution Analysis Stories: Tests for Contribution in Complex Contexts.................. 24 IV. LIMITATIONS................................................................................................................................25 Limitations and Operational Challenges .............................................................................. 25 V. DATA ANALYSIS .........................................................................................................................26 Influencing Factors and Alternative Explanations ............................................................... 26 Developing Findings, Conclusions, and Recommendations: Synthesis (Phase IV) .............. 26 VI. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ..............................................27 Analyzing Stabilization Interventions Using Contribution Analysis .................................. 28 VII. FINDINGS ...............................................................................................................................29 TIS+ Interventions .................................................................................................................. 29 Methodological Considerations ............................................................................................. 31 Key Findings............................................................................................................................. 32 VIII. CONCLUSIONS .....................................................................................................................44 IX. RECOMMENDATIONS .........................................................................................................50 IX. CONTRIBUTION CLAIMS AND TIS+ ..................................................................................54 CONTRIBUTION STORY 1: COMMUNITY PLANNING AND OVERSIGHT ............... 54 CONTRIBUTION STORY 2: TECHNICAL COORDINATORS AND TIS+ .................... 66 4 CONTRIBUTION STORY 3: SPORTS FOR DEVELOPMENT AND PEACE.................. 76 CONTRIBUTION STORY 4: GENDER EQUALITY AND SOCIAL INCLUSION .......... 86 X. INFLUENCING ACTOR TABLES – BY CLUSTER...................................................................104 WAVE 1 FINAL REPORT .................................................................................................. 117 WAVE 1 FINAL REPORT-ANNEXES ................................................................................. 190 WAVE 1I FINAL REPORT..................................................................................................... 234 WAVE 1I FINAL REPORT-ANNEXES................................................................................. 369 5 ACRONYMS AMISOM African Union Mission in Somalia AS Al-Shabaab AWD Acute Watery Diarrhea BRA Banadir Regional Administration CA Contribution Analysis CDD Community-Driven Development CPO Community Planning and Oversight CSR Corporate Social Responsibility CVE Countering Violent Extremism DARS Data and Research Solutions DC District Commissioner ENDF Ethiopian National Defense Forces FGD Focus Group Discussion FGS Federal Government of Somalia FMS Federal Member States GESI Gender Equality and Social Inclusion GEWE Gender Equality and Women’s Empowerment Strategy GGI Good Governance Index GIZ Gesellschaft für Internationale Zusammenarbeit HHS Household Survey IBTCI International Business and Technical Consultants, Inc. IDP Internally Displaced Persons IFOC Infrastructure Facility Oversight Committee IGAD Intergovernmental Authority on Development INGO International Nongovernmental Organization IOM International Organization for Migration JSS Jubaland State of Somalia KAQ Key Assessment Questions KDF Kenya Defense Forces KII Key Informant Interview M&E Monitoring and Evaluation MIDA Migration for Development in Africa MOIFAR Ministry of Interior, Federal Affairs, and Reconciliation MOYS Ministry of Youth and Sports NDP National Development Plan NGO Nongovernmental Organization NISF Nordic International Support Foundation O&M Operations and Maintenance OTI Office of Transition Initiatives P/CVE Preventing/Countering Violent Extremism PESA Political Economy and Stabilization Analysis POC Project Oversight Committees PIKA Pool of In-Kind Activities PSS Puntland State of Somalia SDP Sports for Development and Peace SFO Somalia Field Office SHIIR Somalis Harmonizing Inter- and Intra-Communal Relationships SIF Somalia Infrastructure Fund SNA Somali National Army 6 SPSS Somalia Program Support Services SSF Somalia Stability Fund SSI Somalia Stabilization Initiative STTA Short-Term Technical Assistance SWS South West State TC Technical Coordinator TIS Transition Initiative for Stabilization TIS+ Transition Initiative for Stabilization Plus ToC Theory of Change UNDP United Nations Development Programme UNHCR United Nations High Commissioner for Refugees UNJPLG United Nations Joint Programme on Local Governance UNOCHA United Nations Office for the Coordination of Humanitarian Affairs USAID United States Agency for International Development USAID/SFO USAID/Somalia Field Office USG United States Government WASH Water, Sanitation, and Hygiene 7 Figure 1 WASH: TIS+ Water, Sanitation, and Baseline/Midline Assessment Geographical ClustersHy 8 EXECUTIVE SUMMARY Project Background The Transition Initiatives for Stabilization Plus (TIS+) activity is a five-year, US$66.9 million project implemented by AECOM that will run from May 2015 to April 2020. TIS+ seeks to implement community-centric stabilization activities that (1) are implemented with multiple levels of government (i.e., caretaker and interim administrations, new states, and the Federal Government of Somalia [FGS]); (2) support inclusive participation of stakeholder groups in community decision-making; and (3) encourage shared asset management between citizens and the government to demonstrate the government’s interest in an emerging capacity to contribute positively to citizens’ daily lives. TIS+ activities are designed to contribute to improvements in government service delivery, leading to strengthened governance and community cohesion. USAID Somalia’s development hypothesis reads as follows: “If Somali governance systems and processes that enable inclusive governance are strengthened, and the Somali people benefit from expanded, tangible dividends of the state-building project, then the foundation will be built for Somalia’s longer-term recovery.” TIS+ contributes to USAID Somalia’s Transition Objective 1: Systems and Processes that Enable Inclusive Governance Strengthened. Specifically, it contributes to Sub-Transition Objectives 1.1 and 1.2: 1.1: Create Enabling Conditions through Stabilization of Key Communities, and 1.2: Strengthen Effective, Accountable, and Representative Governance Institutions. Task Order Purpose On September 30, 2016, USAID Somalia awarded the TIS+ Baseline-Midline Assessment Task Order to International Business and Technical Consultants, Inc. (IBTCI), under the Somalia Program Support Services (SPSS) IDIQ. The primary purpose of the TIS+ baseline and midline assessment are to generate a better understanding of the extent to which TIS+ interventions have achieved their objectives and contributed to USAID’s goal of increased stability in Somalia through participatory processes that promote good governance and community cohesion. Specific clusters or geographic areas have been strategically chosen to assess the influence that the TIS+ program activities have had on mitigating the effects of violent extremism and insurgency in these high-risk areas of Somalia. During the midline assessment, data was collected in two waves. In Wave I, data was collected in Lower Juba and Lower Shabelle, and in Wave II, data was collected in Banadir, Gedo/Bay, and Bakool. This report focuses on synthesized analysis of Wave I and Wave II data. Table 1 below shows the questions asked at this stage. Because there was no TIS+ implementation in Bakool, the assessment team, in consultation with USAID Somalia, removed this cluster from the across cross-cluster analysis in this report. Table 1: Synthesis Report Assessment Questions No. Synthesis Questions (Phase IV) 1 In the aggregate, did TIS+ contribute to any of the following anticipated results? What range of results can be observed across assessed clusters? A: Community ownership of development projects and processes B: Effective shared asset management C: Improved delivery/satisfaction of services D: Reduced conflict over resources E: Greater citizen confidence in government F: Reduced support for violent extremism 2 Which TIS+ implementation processes contributed most significantly to the cluster-level results identified? Methods Both baseline and midline assessments employed a mixed-methods approach that integrated a suite of quantitative and qualitative data collection methods. Data collection methods consisted of desk review/secondary literature review, a household survey (HHS), Key Informant Interviews (KIIs), and Focus Group Discussions (FGDs). Assessment questions were answered using a combination of analyses of data from these four collection methods and the use of contribution analysis to estimate TIS+’s unique contribution to observed midline results. For this report, the assessment team analyzed data from across clusters to answer questions 1 and 2 above. 9 About Contribution Analysis Contribution Analysis (CA) is an analytical approach to estimating program contribution to observed effects in complex contexts with many factors and actors influencing intervention outcomes. CA is a good option for a stabilization program because it examines stabilization as a continuum and explores conflict dynamics. CA works within a Theory of Change (ToC) model to look at contribution from several angles. It employs an applied ToC approach to look beyond high-level outcomes (goal/objectives) to more specific activities and interventions across diverse project spaces. CA is a pragmatic and methodologically rigorous approach to situations for which the activity is one of many potential contributory factors and where it is not easy to measure attribution directly, such as is often the case in stabilization programming. Security and Access Somalia is a difficult country in which to gather data, and the team had to consider security limitations as well as the dynamic nature of districts and areas under varying stages of control by Al-Shabaab (AS). AS is a dynamic and increasingly networked group, at both the transnational and local level (cluster/district). It has continued to exert both direct and indirect control in most regions in Southern Somalia, including Banadir, Lower Juba, Lower Shabelle, Gedo, and Bay. The Assessment Team designed the approach and management plan to mitigate anticipated effects as much as possible through means such as multiple data points for each question, confirmation of facts, customized data collection protocols that helped the team collect data in remote and insecure areas, and staggered data collection time frames. SUMMARY OF KEY FINDINGS Q1: Did TIS+ contribute to any of the following anticipated results? What range of results can be observed across assessed clusters? Q1A: Community Ownership of Development Projects and Processes 1. Across most clusters, the level of awareness and participation generally increased for all groups, except clan elders. 2. Across clusters, there generally was a positive association between increased awareness and increased participation, with the exception of Gedo/Bay, likely explained by political difficulties with government staff and clan relations. 3. In the aggregate, there were stronger results since the baseline assessment on community perceptions that projects were selected based on a community collaborative planning approach, and that selected projects met community needs. 4. Results were influenced by a high number of factors and actors in addition to TIS+, though those interventions were generally limited across clusters. Q1B: Effective Shared Asset Management 1. There was increased awareness of TIS+ Project Oversight Committees (POCs) across all clusters probably linked to the presence of other service-level POCs, with the exception of Gedo/Bay due mainly to political tensions around high-profile infrastructure projects. 2. There were mixed results on the perceived effectiveness of POCs likely explained by varied mandates and ways of operating of the POCs. Q1C: Improved Delivery of/Satisfaction with Services 1. There were mixed results on perceptions of service provision by the state, non-state actors, and the business community. However, across clusters, there was a persistent trend of community members being more satisfied with the quality of services provided by international and national nongovernmental organizations (INGO/NGOs). 2. In terms of “security as a service,” there was a clear awareness that this responsibility fell on the Federal Government of Somalia (FGS) and state governments. 10 3. Across clusters, women reported lower improvements in security than men. Q1D: Reduced Conflict over Resources 1. Although there was an increase in perception that large-scale resource-based conflicts had decreased across the clusters in the HHS, qualitative data showed that micro-conflicts (e.g., conflicts over land and water) continue to impact community cohesion. 2. There were large increases across clusters in perceived trust placed in local structures (clan elders, peacebuilding committees, local administration, religious leaders, and Ugaas/sultans). Importantly, the trust placed in the militia groups and the diaspora to resolve conflicts considerably decreased across clusters. Q1E: Greater Citizen Confidence in Government 1. All clusters demonstrated increased scores since baseline on their Good Governance Index (GGI). 2. There was a perceived negative trend in citizen confidence toward state government across both TIS+ and non-TIS+ areas since baseline. There were non-statistically significant changes for both federal and local governments amid citizens’ apathy on a host of issues. 3. There was little documented evidence that TIS+ implemented its strategic communications plan. Q1F: Reduced Support for Violent Extremism There was an increase across clusters of the perception of youth vulnerability to recruitment by extremist groups. Q2: Which TIS+ implementation processes contributed most significantly to the cluster-level results identified? Q2A: Cross-Cutting Findings 1. Gender and Social Inclusion: Across clusters, approaches to gender and social inclusion are not contributing to improvements in community cohesion as anticipated in the theory of change of TIS+. 2. Political Economy and Stabilization Analysis (PESA): Across clusters there was evidence, that conflict mapping, Do No Harm, and/or political analysis is not influencing intervention planning. Q2B: TIS+’s Community Planning and Oversight (CPO) Approach The use of CPO is at the core of TIS+’s programming approach. As developed in the program’s ToC, this approach is the heart of the TIS+ program model. Improving the relationship between government and communities is central to achieving an improved stabilization space. 1. TIS+ took active steps to ensure that communities were supported to collectively identify and prioritize their shared community needs. 2. TIS+ implemented most of its intended CPO steps but substantially deviated from the original design when it came to the establishment, training, and use of the POCs. 3. There was an increase in the reported roles of state and local governments in managing shared assets, particularly in the management of roads and sports arenas, which were Pools of In-Kind Activities (PIKAs). 4. There was a wide and diverse set of groups of actors implementing activities in each of the TIS+ clusters reviewed, most of whom reported utilizing some aspect of community engagement, planning, and oversight. 5. Qualitative interviews (both KIIs and FGDs) conveyed that the unique socioeconomic characteristics of TIS+ settlements in these clusters played a large role in the results seen during the midline. 6. Interviewees also explained that the building of trust between communities and respective levels of government following decades of conflict is a long-term endeavor and requires generational change. 7. POC training was not systematic and uniform, which contributed to the mixed effects observed. Q2C: TIS+’s Technical Coordinator (TC) Approach 11 1. TCs have helped improve coordination between the government and TIS+ as well as increase capacity by the government to engage with stabilization actors and donors and encourage better planning by the government. 2. TCs’ contribution to the state-level ministries is visible but there is a mixed picture at the federal level due to multiple actors providing an assortment of technical assistance. 3. TCs benefit from carryover effects from TIS, particularly at the federal level in Mogadishu, where the program was first piloted. 4. The political and security environment had a strong influence on the contribution of the TCs. Insecurity as well as frequent political turnover negatively impacted the work of the TCs. 5. TCs felt that USAID did not adequately understand their role. TCs were also underutilized during decision￾making processes in TIS+ that resulted in programmatic shifts and changes in intervention. 6. TIS+ did not allocate capacity-building resources for the TCs because it was not envisioned as being part of their scope. However, the low capacity within the institutions often diverted the TCs work to capacity￾building and away from their core roles supporting and coordinating stabilization work. Q2D: Findings on Results of the “Cluster” Approach The cluster approach was not fully utilized due to complexities in security, delayed effects of anticipated military campaigns, and insecurity in particular cluster areas. Q2E: Findings on Government Involvement in Shared Asset Management and Provision 1. Across clusters, with the exception of security, the federal government was regarded as having a limited role in the management of shared assets. 2. However, the perception of local and state government’s role in the management of all assets increased from the baseline. There was an awareness by communities that local administrations’ resources and capacities were limited. 3. Across clusters respondents acknowledged the will of various levels of government in asset management, while equally identifying resource and systemic capacity challenges. 4. Respondents stressed that asset management thrives under good governance and is a shared responsibility between the government and the community. They recommended stronger public–private partnerships. Q2F: Findings on TIS+’s Inclusion Approach 1. There was an increase across clusters in those reporting that they believed that women were considered equal to men. 2. There were notable efforts by TIS+ to ensure women’s participation in POCs, though without a targeted training component. 3. Although all POCs reviewed included youth members, youth were primarily only given direct oversight and management responsibility for sport facilities and playgrounds. Youth expressed an interest in broader participation in stabilization activities. 4. There was limited data from TIS+ on how they tracked minority clan involvement or how this influenced programming. 5. Community Cohesion Index scores increased across clusters. Q2G: Findings on TIS+’s Sports for Development and Peace (SDP) Approach 1. There was consensus among key informants working on sports-related activities that conducting or facilitating sports events alone was not enough to deliver the desired results. 2. There may be some inherent benefits for social cohesion through sports events and availability of sports venues in communities. 3. Having sports venues without additional supporting activities (e.g., education, vocational, social, etc.) was regarded by interviewees as insufficient for achieving higher-level results. 12 4. Related to gender, midline interviews suggested that Somali society is generally accepting of men’s participation in sports events. However, women’s and girls’ participation is much more controversial. SUMMARY OF KEY CONCLUSIONS TIS+ CONTRIBUTION TO INTENDED RESULTS: Q1: Did TIS+ contribute to any of the following anticipated results? What range of results can be observed across assessed clusters? Q1A: Community Ownership of Development Projects and Processes 1. TIS+ contributed to a small but positive increase in community awareness and participation in development activities in the settlements where it worked. More broadly, since baseline, Somali populations assessed were increasingly more aware of development projects in their settlements, providing a ripe opportunity to foster active participation in stabilization activities. 2. The settlement of Baardheere was a strong outlier, indicating that political developments and how TIS+ reacts to those developments can significantly influence results even at the cluster level. 3. Failure to manage expectations during community planning events had a negative correlation with communities’ perceptions of whether selected projects meet their needs. Q1B: Effective Shared Asset Management TIS+ has made a small contribution to increased awareness of POCs in areas where it operated. While there was a general increase of awareness of POCs across clusters, this did not translate into improved perceptions of performance likely due to inconsistencies in POC formations and implementation and the types of community infrastructure selected. Q1C: Improved Delivery of/Satisfaction with Services There was little evidence that TIS+ contributed to perceptions of improved service delivery and satisfaction due to varied results and mixed patterns that were observed across different service areas. Q1D: Reduced Conflict over Resources There was no observable evidence of TIS+ contribution to reduced conflict over resources at the cluster level. Notably, these conflicts over resources generally remained stable and/or persisted across clusters since baseline. The mediation of resource-based conflicts appears not to have been a priority for the program. Q1E: Greater Citizen Confidence in Government Based on the evidence, TIS+ did not make a unique contribution to greater citizen confidence given similar trends in non-implementation areas. Q1F: Reduced Support for Violent Extremism TIS+ did not contribute to reduced support for violent extremism across clusters. However, potential results related to countering violent extremism is at the far end of the TIS+ intended results chain. TIS+ can be expected to have indirect and minimal, if any, influence over these results. Q2: Which TIS+ implementation processes contributed most significantly to the cluster-level results identified? Q2A: Cross-Cutting 13 1. Approaches to gender, equality and social inclusion are not contributing to improvements in community cohesion as anticipated in the TIS+ ToC. Inclusion factors are co-related, and targeting is not enough. 2. There is a need for TIS+ to more effectively integrate conflict mapping, Do No Harm, and/or political analysis directly into intervention planning and monitoring of effects. Q2B: TIS+’s Community Planning and Oversight (CPO) Approach 1. There is limited evidence that TIS+’s CPO approach is consistently contributing to higher-order, outcome￾level results related to improved citizen engagement and confidence in government, as well as reduced conflict over resources or support for violent extremism. 2. While the midline assessment found evidence of TIS+ achieving notable results in terms of community awareness and participation, as well as local and state government service provision and management of shared assets, these results were usually only at the individual settlement level and were not consistent across clusters. Q2C: TIS+’s Technical Coordinator (TC) Approach The TCs were not well embedded in intervention planning. There was evidence of impact at the state level, but less evidence at the federal level. Q2D: Findings on Results of the “Cluster” Approach TIS+ attempted to adapt programming to match the reality of the constraints presented by the lack of military campaigns along the anticipated corridors, but this proved to be challenging for a development partner. Q2E: Findings on Government Involvement in Shared Asset Management and Provision 1. The federal government’s role was limited to the provision of security. 2. There was consistent recognition that government involvement at all levels was necessary for shared asset management. TIS+ facilitated the government’s taking a lead in community consensus planning and community contracting. Q2F: TIS+’s Inclusion Approach 1. TIS+ has taken a number of steps to increase women’s participation in its activities and has succeeded in facilitating women’s participation across its POCs, as well as influencing perceptions in its settlements that women were considered equal to men. 2. There was a lack of systematic monitoring of minority clan involvement or at least documentation of such monitoring in TIS+ interventions. Q2G: TIS+’s Sports for Development and Peace (SDP) Approach Any potential claim for TIS+’s SDP activities to have influenced higher-order outcomes related to community cohesion or positive youth development to date are not well substantiated by current evidence. 14 SUMMARY OF KEY RECOMMENDATIONS 1. TIS+ should ensure that the program follows a more consistent and complete community planning and oversight approach. This involves providing timely training and capacity-building to POC members so that they can properly serve in their oversight roles for improved community ownership and effective shared asset management. 2. TIS+ and USAID should ensure that programming approaches are periodically monitored for alignment with TIS+’s overarching Theory of Change. An applied ToC approach requires not only flexibility, but also a strategic alignment based on accurately observed changes in the context informed by regular field monitoring and management. 3. TIS+ and USAID should consider focusing on supporting more day-to-day government-led service delivery activities and infrastructure projects if it aims to influence higher-level results related to increased citizen confidence in government or improved perceptions of service delivery. If the goal is to change perceptions of effective shared assessment management, the type of shared asset that the POCs oversee is critical to shaping community perceptions. 4. TIS+ should consider strengthening its strategic planning within and across program efforts. It is critical to analyze and assess the localized dynamics of violent extremism before tailoring interventions at the cluster level. 5. TIS+ should strengthen its strategic communication efforts to help improve citizen confidence. Targeting narratives, counter-messaging, and influencing perceptions will contribute to higher-level results. In addition, strategic communications, particularly the use of social media, can be utilized to promote positive perceptions in confidence and trust in government. 6. TIS+ should work to increase citizen confidence in district and state levels and not focus on the federal level. Future stabilization interventions should target making an impact at these levels of government. 7. TIS+ should continue to explore ways to improve its Gender Equality and Social Inclusion (GESI) efforts by going beyond simply ensuring increased participation of women, youth, and marginalized groups in its activities. Instead, it should ensure more meaningful participation of these groups in the selection and oversight of projects. 8. TIS+ should ensure regular training forums for women, youth, and minority clan members working in project oversight Project Oversight Committees (POCs) and asset management through a program that is implemented with fidelity. Increases in participation are distinctly linked to citizens feeling more competent to engage in decision-making, management, and oversight. 9. TIS+ needs to plan targeted outreach and campaign events that encourage women and youth participation in SDP events while discouraging stereotyping associated with the participation of women and girls. These events should be more intentional in their design to be fully inclusive of women, girls, and minority clan members, as well as to avoid potential perverse outcomes that may lead to participants being targeted. 10. TIS+ and USAID should explore opportunities to better utilize TCs to encourage the participation of various levels of government in shared-asset management, as well as ensure enhanced coordination and information exchange. TIS+ should expand the scope of the TCs work to effectively support the government’s role as a facilitator in service provision. 15 11. USAID should endeavor to ensure that partners better understand the unique dynamics of countering violent extremism in Somalia and utilize cross-cutting interventions across all its programs to address the localized effects of violent extremism on communities and within specific minority groups. The broad portfolio of USAID programming in Somalia (not just TIS+) should focus on a more coordinated approach to violent extremism and its prevention. 16 I. INTRODUCTION Chronic instability in the late 1980s and the collapse of the Somali state in 1991 resulted in two decades of warfare and violence in Somalia. The decades of instability, intercommunal conflict, and humanitarian crisis that characterized the second half of the 20th century in Somalia were marked by numerous attempts at peacekeeping, stabilization, and international relief. Clan conflict, violent extremism, drought, famine, out-migration, and multiple failed central state governments have typified the obstacles facing a unified, independent Somali state capable of service delivery and crisis response. In 2012, Somalia was finally able to establish a complete central government, after 20 years without a parliament, and held the first presidential election since 1967.1 Despite the continued threats of disorder and violence posed by Al-Shabaab (AS), the Federal Government of Somalia (FGS), in partnership with the international community, has taken impressive strides toward durable peace- and nation￾building. During this period, Somalia also adopted and devolved a federal system that saw the establishment of the new Federal Member States (FMS). Most recently, Somalia, with extensive support from the international community, held an “indirect” election on February 8, 2017, during which Mohamed Abdullahi Farmajo was elected in the second round of voting, unseating incumbent Hassan Sheikh Mohamud. In November 2016, FGS launched a draft of the first National Development Plan (NDP). This plan is organized per the following objectives: poverty reduction, infrastructure repair, strengthening state capacity, and sustaining political inclusivity. Gender mainstreaming and a focus on youth are both features of the NDP. Project Background The Transition Initiatives for Stabilization Plus (TIS+) activity is a five-year, US$66.9 million project implemented by AECOM that will run from May 2015 to April 2020. The TIS+ activity intends to support the achievement of USAID’s goal by fostering participatory processes that promote good governance and community cohesion. Given the fragile nature of the Somali government and the existence of an insurgency in parts of South-Central Somalia, USAID is maintaining a continued focus through TIS+ on quick impact stabilization activities at the district and community levels. At the same time, Somalia is aiming to move from crisis response and stabilization to medium- and longer-term development, so TIS+ is also designed to support this aim. Under Somalia’s New Deal Framework, as well as the recently completed NDP, the FGS and the donor community have committed to advancing inclusive political dialogue, improving security, and increasing economic growth—pillars that TIS+ will also support. TIS+ seeks to implement community-centric stabilization activities that (1) are implemented with multiple levels of government (i.e., caretaker and interim administrations, new states, and the FGS); (2) support inclusive participation of stakeholder groups in community decision-making; and (3) encourage shared asset management between citizens and the government to demonstrate the government’s interest in an emerging capacity to contribute positively to citizens’ daily lives. TIS+ activities are designed to contribute to improvements in government service delivery, leading to strengthened governance and community cohesion. USAID Somalia’s development hypothesis reads as follows: “If Somali governance systems and processes that enable inclusive governance are strengthened, and the Somali people benefit from expanded, tangible dividends of the state-building project, then the foundation will be built for Somalia’s longer-term recovery.” TIS+ contributes to USAID Somalia’s Transition Objective 1: Systems and Processes that Enable Inclusive Governance Strengthened. Specifically, it contributes to Sub-Transition Objectives 1.1 and 1.2: 1.1: Create Enabling Conditions through Stabilization of Key Communities, and 1.2: Strengthen Effective, Accountable, and Representative Governance Institutions. Task Order Purpose On September 30, 2016, USAID Somalia awarded the TIS+ Baseline-Midline Assessment Task Order to International Business and Technical Consultants, Inc. (IBTCI), under the Somalia Program Support Services (SPSS) IDIQ. The primary purpose of the TIS+ baseline and midline assessments is to generate a better 1 Between 2000 and 2012, Somalia had a Transitional Federal Parliament. A full parliament was constituted in 2012. 17 understanding of the extent to which TIS+ interventions have achieved their objectives and contributed to USAID’s goal of increased stability in Somalia through participatory processes that promote good governance and community cohesion. Specific clusters or geographic areas have been strategically chosen to be analyzed to assess the influence that the TIS+ program activities have had on mitigating the effects of violent extremism and insurgency in these high-risk areas of Somalia. This task order is designed to assist USAID in identifying lessons learned from conducting stabilization activities in Somalia. Key Assessment Questions The TIS+ baseline/midline assessment focuses on two levels of analysis: within cluster analysis (cluster-level analysis) and cross-cluster analysis (comparative cluster analysis). USAID identified five clusters for analysis: (1) Banadir (Mogadishu), (2) Lower Shabelle, (3) Bakool, (4) Lower Juba, and (5) Gedo/Bay. During the midline, data were collected in two waves. In Wave I, data was collected in Lower Juba and Lower Shabelle, and in Wave II, data were collected in Gedo/Bay, Banadir, and Bakool. This report focuses on analyzing and synthesizing findings from Wave I and II data collected. The assessment will answer the following assessment questions (see Table 2). Table 2: Synthesis Report Assessment Questions Aggregate and Cross-Cluster-Level Analysis (Phase IV) 1 In the aggregate, did TIS+ contribute to the anticipated results described above? What range of results can be observed across assessed clusters? 2 Which TIS+ implementation processes contributed most significantly to achievement of results? For example: ● Did successful inclusion of women, youth, and minority clans amplify the effectiveness of TIS+ activities? ● Were some kinds of interventions more effective than others in improving citizen confidence or reducing support for violent extremism? 18 III. METHODS Both baseline and midline assessments employed a mixed-methods approach that integrated a suite of quantitative and qualitative data collection methods implemented over two waves to answer the above assessment questions. This methodology affords USAID a greater depth of comparative analysis across the clusters as well as the aggregated analysis used during the final synthesis stage (Phase IV). As shown in Table 3, data collection methods consisted of desk review/secondary literature review, a household survey (HHS) of 4,174, 192 Key Informant Interviews (KIIs), and 137 Focus Group Discussions (FGDs). The synthesis assessment questions were answered using a combination of analyses of data from these four collection methods, via triangulation of evidence and the use of contribution analysis to estimate TIS+’s unique contribution to observed results. The team also used a qualitative data analysis tool, known as NVivo, to code and organize qualitative data for intuitive exploration of correlation, insights, both within and across clusters. Contribution analysis (CA) is an analytical approach to estimating program contribution to observed effects in complex contexts with many factors and actors influencing intervention outcomes. CA is a good option for a stabilization program because it examines stabilization as a continuum and explores conflict dynamics. CA works within a Theory of Change (ToC) model to look at the contribution from several angles. It employs an applied ToC approach to look beyond high-level outcomes (goal/objectives) to more specific changes due effected by activities and interventions across diverse project spaces. CA is a pragmatic and methodologically rigorous approach to situations for which the activity is one of many potential contributory factors and where it is not easy to measure attribution directly, as is often the case in stabilization programming. In the case of this task order, CA was used to assess the contribution of TIS+ activities to observed outcomes based on the totality of evidence gathered. The essential value of contribution analysis, in this case, is that it offers evidence to reduce uncertainty about the contribution of TIS+ to the observed results through an increased understanding of why the observed results have occurred (or not) and the roles played by TIS+ activities and other internal and external factors.2 A quantitative household survey (HHS) was administered to residents living in communities where TIS+ activities were planned to be implemented. The qualitative data collection instruments included secondary literature review, in-depth KIIs with diverse TIS+ stakeholders, and FGDs with persons living in the targeted clusters, designed to better understand within each cluster the status of community infrastructure planning and other interventions. This data was analyzed to create an evidence base that provided comprehensive answers to Key Assessment Questions (KAQs) 1–4 within clusters and KAQs 5–6 across clusters. A description of each data 2 John Mayne. (2008). “Contribution Analysis: An Approach to Exploring Cause and Effect,” http://dmeforpeace.org/sites/default/files/0501_Contribution_Analysis_ILAC.pdf Cluster HHS FGDs KIIs CA KIIs Banadir 1,289 72 42 34 Gedo/Bay 1,371 28 34 6 Lower Juba 885 24 25 13 Lower Shabelle 629 13 22 2 Total 4,174 137 123 55 Table 3: Qualitative Data Coverage, Wave I and II THE STEPS OF CONTRIBUTION ANALYSIS 1. Set out the cause–effect issue to be addressed in a detailed ToC. 2. Identify the risks to the postulated ToC, including rival explanations. 3. Gather evidence (Baseline and Midline). 4. Assemble and assess the contribution claim and challenges to it. 5. Seek out additional evidence. 6. Revise and strengthen the contribution story. Figure 2: Steps of Contribution Analysis 19 collection method is provided below as well as a description of the catchment and cluster areas accessible during data collection. Secondary Literature Review The first data collection method undertaken by the team was to conduct a comprehensive review of internal and external documents of relevance to the TIS+ activity. The IBTCI team worked in coordination with USAID/Somalia Field Office (SFO) and the TIS+ management team to identify, collect, and then review the following documents over the course of the assessments: • Activity designs, training curricula, event agendas, and other key documents for understanding the content of activities • Annual Work Plans (initial and subsequent) • Activity Monitoring, Evaluation and Learning Plan (initial and revised) • Quarterly and annual reports • Reports on TIS+ participatory monitoring and evaluation (M&E) activities • Monitoring, Verification, and Reporting documents • SPSS third-party monitoring and verification reports • Additional documents provided such as Stability Assessments and Community Consensus Reports • External publications by other stabilization stakeholders, donors, academic researchers, and NGOs Household Survey The assessment team contracted with the Somali firm Data and Research Solutions (DARS) to conduct the HHS. During the design phase of the assessments, it was envisioned that a total sample size of 848 (baseline = 424 and midline = 424) per cluster would be large enough to draw statistically meaningful conclusions from the results of TIS+ activities. This sample size was estimated to be sufficient to detect a minimum change of 10 percent at a power of 80 percent and a significance level of 5 percent. However, due to limited implementation by TIS+, it was reasonable to limit the assessment of TIS+’s contribution only to settlements where there were interventions.3 That implied reducing the effective cluster-level baseline sample sizes that would be used, together with the midline data, for comparative analysis.4 There were also significant variations in findings between settlements in all the clusters other than Banadir, which implied limiting generalizability to settlement level. As a result, the assessment team recalculated significance levels at cluster and settlement levels, which widely varied between 14 percent and 58 percent (see Annex D). Significance level may be defined, in simple terms, as the chance to make an error in distinguishing between significant and non-significant findings. What has been used commonly in practice, and might be understood as the standard, is 5 percent. Therefore, 58 percent would imply a huge chance to make an error in assessing significance of findings. Recognizing this limitation, the assessment team used penalized likelihood regression approaches5 to limit bias due to data sparsity and to emphasize findings that were significant at 5 percent. The assessment team did not reject non-significant findings at 5 percent since the estimated p – values were less than the recalculated significance levels. The assessment team interpreted them when necessary. If the threshold was 5 percent, a number of true findings would wrongly be rejected and hence missed. The survey instrument for TIS+ was tailored to ensure that it accurately reflected TIS+ activities and their expected outcomes, while also ensuring that the assessment questions could be analyzed at baseline and midline. 3 For better utilization of data, we still assessed changes in settlements that did not have any TIS+ implementation. They were used for comparison purposes and not assessment of attribution. 4 For instance, to assess TIS+’s contribution in Banadir, we used a baseline sample size of only 77 while maintaining the midline sample size of 424. 5 Cole, S. R., Chu H., Greenland S. Maximum Likelihood, Profile Likelihood, and Penalized Likelihood: A Primer. Am J Epidemiol. 2013 09/16; 179(2):252-60. 20 Key Informant Interviews The assessment team and DARS conducted a total of 178 KIIs from baseline to midline. A major advantage of the KII method, compared with other forms of qualitative research, was the opportunity it provided for in-depth exploration of the knowledge and experience of individuals directly exposed to TIS+ and stabilization work in Somalia. This was particularly important for employing contribution analysis. The selection of the key informants was made in collaboration with USAID/SFO and the TIS+ implementing partner, AECOM, to develop a representative list of informants well-versed in activity design, implementation, and anticipated results. The following represents the universe of key informants who were targeted for data collection: • USAID officials in the USAID/SFO • AECOM senior management, regional office staff, and subcontractor staff • Officials from the Federal Member States (FMS) and relevant government actors at the central and district levels in Somalia • Key staff from other stabilization programs that interact with TIS+ or support programming that shares its objectives and geography • Representatives from the private sector, NGOs, or governmental donor organizations collaborating with or asked to collaborate with TIS+ communities • Participants purposively selected relevant to analyzing the effects of TIS+ program design and activities within the targeted clusters and catchment areas Focus Group Discussions The assessment team conducted a total of 137 FGDs from baseline to midline. A major advantage of the FGD method, compared with other forms of qualitative research, is its flexibility for in-depth exploration of the knowledge and attitudes of the group toward specific cluster-level conditions relevant to TIS+ activities and processes. This includes participation; initial levels of trust in district, regional, and FGS officials; community contracting and planning approaches; and the inclusion of women, youth, minority clans, and members of other marginal groups in decision-making. FGDs were conducted in each selected settlement with the following groups: (1) female elder beneficiaries, (2) male elder beneficiaries, (3) male youth beneficiaries, and (4) female youth beneficiaries. Index Tool The goal for the TIS+ project is to increase stability in the clusters of implementation. For stability to be achieved, two preconditions should be satisfied: (1) increased good governance and (2) increased community cohesion. Annex C explains in more detail the development of index scores to measure these preconditions.6 Household questions were grouped into three themes underlying good governance, including confidence in government, service delivery, and stakeholders’ partnerships as the general constructs defining governance status. Questions were also grouped into three themes explaining community cohesion: network diversity, interpersonal and institutional trust, and social and political inclusiveness. An index tool was developed to allow the team to aggregate a series of interrelated questions from the HHS to establish benchmarks at the cluster level. This index assisted the team in cross-cluster and aggregate analysis as well as providing an analytical anchor to look at the relationship between these factors: key assessment questions, TIS+ interventions, and impacts on stability. These measures were used at midline to assess changes in areas of TIS+ programming and to also look at potential trends evident by comparing baseline measures to midline measures. Political Economy and Stabilization Assessment (PESA) Tool The assessment team also conducted desk research and interviews to understand more completely the political and security conditions evident at the district and cluster level. This quick assessment tool allowed the team to 6 Index score is a single measure derived from a set of variables. 21 assess current contextual conditions under which TIS+ was operating. CA interviews with government officials working on security as well as donors and project staff experiencing varied levels of political will and access helped the assessment team better understand the dynamic environment in which TIS+ staff work. 22 III. CONTRIBUTION ANALYSIS AND TIS+ The Role of Contribution Analysis TIS+ operates within a wider system in which it is only one of several factors that potentially affect activity outcomes. The main purpose of contribution analysis in this assessment is to identify the effect that TIS+ programming had in an environment with other actors and influences in the same geographic area at baseline and in the midline stages. Testing the Theory of Change for TIS+ USAID/Somalia has a Theory of Change (ToC) that summarizes at a high level how change is expected to happen in its focus areas of intervention in Somalia. The TIS+ ToC feeds into this higher-level theory. TIS+ had previously worked on articulating a “development hypothesis” at the overarching program level, including a summary of the intervention logic, mechanisms, and assumptions. Contribution analysis requires a ToC that is much closer to the level of the interventions, or what the program is doing, and that describes the sequence of resultant behavior changes that are expected to lead or catalyze outcomes.7 It was necessary to ask at midline whether the program was implemented as originally planned and to assemble evidence that may have influenced these changes. In response to this need, and as required in the task order, the assessment team built on TIS+ existing work to develop a fuller ToC for these purposes. The development of this ToC was based on a one-day workshop on June 14, 2017, in Nairobi with TIS+ field and headquarters staff, as well as the USAID/Somalia lead for the TIS+ program. Figure 3 shows the overarching ToC workshopped with TIS+ and shows how the CA Contribution Stories map onto it. The ToC development workshop, as well as TIS+ documents, revealed that “The TIS+ model is strongly based on a community-government programming approach.”8 This model is the basis of TIS+ Objective 1 work, which accounts for the program’s largest expenditure, and of the Objective 2 work, which is a modification designed for areas that have achieved a higher level of stability and had a stronger semblance of district government. TIS+ Objective 3, on supporting Somali engagement, is a cross￾cutting objective that feeds into the work under Objectives 1 and 2. Objective 3 allows for activities supporting social reconciliation or cohesion in communities that may be too volatile for Objective 1 activities, where Objective 1 is planning to roll out, or where it is already implementing. It follows a similar process of using community consensus groups but also allows for some activities to be supported without conducting the full community engagement process. Objective 4 integrates gender across all TIS+ interventions. This objective encompasses both the focus on women’s participation in the community consensus process and coordination within and between the district, state, and federal levels of government on gender-sensitive planning and implementation. The assessment team has included TIS+ work on capacity injection in ministries at the state and district levels on the right-hand side of the ToC diagram. Although this work is not covered by a separate objective and is only partly intended to support Objective 1 and 2 activities, it is also intended to strengthen communication across levels of government on stabilization activities. 7 Davies, R. (2012). “Criteria for Assessing the Evaluability of a Theory of Change,” Rick on the Road Blog. http://dmeforpeace.org/learn/criteria-assessing-evaluability-theory-change 8 Transition Initiatives for Stabilization Plus (TIS+) Process of Engagement document (March 2016). 23 Figure 3: Applied Theory of Change for TIS+, showing CA Stories 24 Contribution Analysis Stories: Tests for Contribution in Complex Contexts TIS+ is a complex program that includes several different types of interventions, allowing it to use an adaptive approach to programming in Somalia. In consultation with TIS+, the assessment team broke this overarching ToC down into its core elements for our Contribution Stories. These stories will collectively ensure that our analysis provides good coverage of the overarching TIS+ ToC and TIS+ work plan activities. These Contribution Stories will serve as the core learning tools for the synthesis report, providing detailed examples of the effects of TIS+ stabilization approaches and implementation processes on the anticipated results. In summary, the assessment team selected the following four contribution stories: 1. Community Planning and Oversight Story: This story encompasses the “core offer” of the TIS+ model. This reflects the fact that TIS+ is, at its core, a program delivering infrastructure, but one where the process of planning and delivery engaging community and government is crucial to delivering the expected results. 2. Sports for Development and Peace (SDP) Story: TIS+ has supported several sports-related activities across all its objectives, from constructing and rehabilitating sports infrastructure to setting up or maintaining and managing committees, organizing sports events, distributing sports equipment, and supporting Ministries of Youth and Sport (MOYS) with technical assistance at federal and state levels. To consolidate and further develop this work, TIS+ enlisted a technical consultant specialist on SDP in March 2018 to work on an SDP Strategy and Implementation Framework for the MOYS. 3. Technical Coordinators Story: TIS+ has several Technical Coordinators (TCs) embedded within ministries, designed to support TIS+ objectives 1, 2, and 3. These are roles at least partly designed to enhance the efficacy of TIS+ interventions, and to deliver wider work in supporting communication between levels of government more broadly. 4. Gender, Equality, and Social Inclusion (GESI) Story: This story investigates how the principles of inclusion were applied within the TIS+ model and how this influenced anticipated higher-level results. Specific analysis focuses on the implementation of the TIS+ GESI Strategy and how this manifested in program operations and planning. It also includes analysis of the inclusion of marginalized and minority communities and groups. 25 IV. LIMITATIONS Limitations and Operational Challenges Complex conflict contexts are challenging. The assessment team designed the approach and management plan to mitigate anticipated effects as much as possible through means such as multiple data points for each question, confirmation of facts, customized data collection protocols that helped the team collect data in remote and insecure areas, and staggered data collection time frames. Conducting complex multidimensional data collection requires tight discipline, efficient use of time and resources, close communication between the team and the client, the ability to adapt and solve data collection problems on the spot, and the ability to be an evaluation diplomat when dealing with informants at all levels, especially in senior positions. While considerable, these challenges were not unfamiliar or insurmountable. Anticipating the Somali context, the team identified several important challenges that the IBTCI approach addressed: • The volatility of indicators on Corruption, Governance, and Community Cohesion. These aspects are not directly measured with the data collected but are measured through the perception of stakeholders interviewed. Linked to the security context and other dimensions, perceptions are volatile and may change over time depending on what happened just before the interviews take place. • Difficulty in ensuring rigorous analysis across clusters to the low level of implementation of TIS+ and various partners working in each cluster. Some conclusions from data analysis are more valid at the settlement level because there is variation in conditions and political and cultural composition of these districts. This has an impact on the analysis and conclusions able to be drawn at the cluster level that the Scope of Work (SOW) requested. • Assessing a complex and multifaceted stabilization program operating in areas newly liberated from Al￾Shabaab and often with varying levels of security. Contexts were often highly politicized and changes in government as well as state structure influenced program collaboration with the host country. • The TIS+ program also experienced external shocks from varied funding levels from the USAID Somalia Mission, which also influenced program planning at various stages of implementation. 26 V. DATA ANALYSIS At the synthesis stage, it is evident that several factors will influence changes in these environments, and while TIS+ is the largest program in these clusters, it is not the only program working on improving governance, community cohesion, and stabilization in Somalia. At this stage of the task order the assessment team has mapped carefully the partners, programs, and factors that are most likely to impact and influence the work of TIS+. Interviews were targeted to direct beneficiary groups. The assessment team considered all objective areas of TIS+ and looked at more technical interventions, such as capacity development approaches used within the FGS and at the state level embedding staff in ministries. The interviews conducted at all phases of the assessment and the current synthesis analysis of data reinforce that CA methods are an excellent match for USAID and TIS+. CA is particularly appropriate in situations where “multiple initiatives are working toward the same end” and where complex environments mean that linear attribution of results to any one intervention is “neither meaningful nor accurate.”9 Stabilization programs fit these criteria and are good candidates for the use of CA methods. The application of CA methods is an innovation for USAID/Somalia and TIS+ that can add to improving methods of assessing programming in the future. Influencing Factors and Alternative Explanations It is important to outline the difference between influencing factors and alternative explanations in CA. Influencing factors are used either to support contribution stories or to support alternative explanations for results observed. Influencing factors for stabilization include both contextual factors and other interventions. Contextual factors will be strongly linked to conflict analyses at the intervention level produced by TIS+, other development partners, and academics. Other interventions include donor and partner contributions where relevant in the targeted TIS+ settlements. Other donor programs working on similar activities and overlapping geographically TIS+ areas are also potential data sources. TIS+ maps and actively collaborates with stabilization actors in TIS+ areas, and this work will be analyzed for contribution, effect, and relevance. There have been chronic gaps in mapping donor activity in Somalia. Where possible the assessment team identified the most relevant stakeholder groups collaborating with or influencing TIS+ program interventions. Developing Findings, Conclusions, and Recommendations: Synthesis (Phase IV) TIS+ is a multilayered project, and this synthesis analysis focuses across cluster areas where TIS+ implemented. These are not administrative regions, but rather clusters of settlements where AS has continued influence, security is dynamic, and stability is critical. TIS+ focuses on community planning, contracting, and improving the oversight frameworks that strengthen citizen and government relationships at the district, regional, and national level. Since CA is an analytical approach and not a method of data collection, it relies on the types of data that are being generated to feed the analytical process. TIS+ itself, and the baseline and midline assessments, operated and collected data at the cluster level. Therefore, observed results and contribution stories are generated at this level and have been refined to assess cross-cluster effects. Major results and enduring trends observed across the clusters were turned into these contribution stories that are being used as the primary learning tool to assess the varied approaches utilized by TIS+ to achieve anticipated results. This cross-cluster analysis highlights overall contributions to outcomes of TIS+ to help inform USG future programming. Each contribution story concludes with an overarching assessment of the strength of TIS’s “contribution claim” to the results observed. The contribution stories critically examine the extent to which TIS+ has contributed to improvements in anticipated higher-level effects (see Section IX). 9 Quinn Patton, Michael. (2008). Utilization-Focused Evaluation: The New Century. 4th edition. Thousand Oaks: Sage Publications. 27 VI. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS This synthesis analysis focuses across cluster areas where TIS+ implemented. These are not administrative regions, but rather clusters of districts where AS has continued influence, security is dynamic, and stability is critical. TIS+ focuses on community planning, contracting, and improving the oversight frameworks that strengthen citizen and government relationships at the district, regional, and national level. Activities are structured to foster constructive interaction between citizens and newly established governing authorities, such as basic training of authorities and community leaders, supporting visible small-scale infrastructure, and creating forums for regular dialogue such as town halls and cultural events. TIS+ follows on initial interventions in newly recovered areas with a consensus-based community-driven development model that prioritizes equitable participation in decision-making and management of community assets. Participatory, consensus-building approaches are used to build citizen confidence in the governing institutions and address critical local-level needs by facilitating the construction and/or rehabilitation of vital community infrastructure (such as roads, bridges, community centers, local administration offices, health clinics, schools, water sources, and markets). The findings section of this report provides analysis of Wave I and Wave II data sets, which consists of the HHS data for Lower Juba, Lower Shabelle, Gedo/Bay, and Banadir, and triangulation of this information with the KIIs and FGDs conducted during the baseline. Contribution analysis was the primary approach applied to analyze the relationship between TIS+ planning and coordination activities with other donors and contextual factors. Quantitative analyses of baseline and midline differences were triangulated with qualitative data and supplemental CA interviews with key donors working on stabilization to ascertain these findings. Findings are aggregated across clusters. TIS+ is working in an environment where stabilization is a collaborative effort between multiple actors who are contributing to shared objectives. Moreover, the program is operating in a fluid and unpredictable environment, which sometimes offers opportunities to support stabilization objectives as well as presenting serious obstacles to achieving those objectives. The use of CA is particularly appropriate in situations where “multiple initiatives are working toward the same end” and where complex environments mean that linear attribution of results to any one intervention is “neither meaningful nor accurate.”10 Stabilization programs fit these criteria remarkably well. To answer the synthesis assessment questions, the assessment team has integrated CA into the presentation of findings and recommendations for assessment questions 1 and 2. The suitability of CA for programming in conflict-affected contexts has been recognized in several sources, but a literature search has revealed that few (if any) CA approaches have been used in stabilization programs for USAID.11 Using CA methods in this task order is an opportunity to capture TIS+ effects using the most appropriate methods, and to add to the evidence and knowledge base on stabilization programming. 10 Quinn Patton, Michael. (2008). Utilization-Focused Evaluation: The New Century, 4th edition. Thousand Oaks: Sage Publications. 11 Goldwyn, Rachel, & Chiagas, Diana. (2013). “Monitoring and Evaluating Conflict Sensitivity: Methodological Challenges and Practical Solutions.” DFID, CARE, and CDE Collaborative Learning Projects Paper; Rogers, Mark. (2012). “Selecting Evaluation Approaches: Options in Peacebuilding Evaluation.” CARE International Paper. 28 Analyzing Stabilization Interventions Using Contribution Analysis CA is an analytical approach and not a method of data collection, so the data generated is constantly being assessed at all levels to address gaps in findings as well as assumptions formed. TIS+ itself, and the baseline and midline assessments, operates and collects data at the cluster level. Therefore, observed results and contribution stories are generated at this level. Major results and enduring trends observed across the clusters were turned into contribution stories that are being used as the primary learning tool to assess the varied approaches utilized by TIS+ to achieve anticipated results. This cross-cluster analysis highlights overall contributions to outcomes of TIS+ to help inform USG future programming. To reach robust conclusions, CA contribution stories must meet certain criteria for establishing a plausible link between the intervention and the observed results.12 Each story is structured according to the five criteria identified by John Mayne, the originator of the Contribution Analysis methodology, to analyze the extent of a contribution claim (or plausible association) between TIS+ activities and the results witnessed during the midline assessment. These criteria include the following: 1. Plausibility: Is the theory of change plausible? 2. Implementation per plan: Has the program been implemented with high fidelity? 3. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? 4. Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? 5. Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? The contribution story concludes with an overarching assessment of the strength of TIS’s “contribution claim” to the results relevant to the identified results chain area in the midline assessment. The contribution stories critically examine the extent to which TIS+ has contributed to improvements in anticipated higher-level effects (see Section X). 12 Mayne, J. (2011). Addressing cause and effect in simple and complex settings through contribution analysis. In: Schwartz, R., Forss, K., and Marra, M. (Eds.) Evaluating the Complex: Attribution, Contribution and Beyond. New York: Transaction Publishers. “Stability gains are not sustainable without citizen responsive governance. Early transition work lays the foundation for long￾term development by promoting reconciliation, jumpstarting local economies, supporting emerging independent media, and fostering lasting peace and democracy through innovative programming and evidence-based approaches.” – USAID Administrator Mark Green speaking to the House Appropriations Committee, November 2017 29 VII. FINDINGS TIS+ Interventions The following interventions were the primary activities by which TIS+ sought to engage government and stakeholder partners to ensure that community cohesion and governance improvements were observed. Table 4, shows the Pool of In-Kind Activities (PIKAs) conducted across clusters by TIS+. This gives the reader an idea of the types of activities that TIS+ implements. The majority of project funding goes to infrastructure activities. These activities are highlighted in blue. Table 4: TIS+ Pool of In-Kind Activities (PIKAs), Organized by Cluster, Settlement, and Type BANADIR REGIONAL ADMINISTRATION (BRA) CLUSTER (*Bridging Cluster) Settlement TIS+ PIKA Activity Type Abdiaziz Bridging Plan (BP) Stakeholders Consultation Consultation Mogadishu Book Fair Inclusion Bridging Plans Task Force – Roads and District Stabilization Consultation Districts Sport Facilities and Youth Engagement Youth Bridging Plans Task Force Capacity-Building Capacity-Building Bridging Plans Engineering Technical Assessment Study – Roads Study Abdiaziz District Basketball Mini Stadia Rehabilitation Infrastructure Hawlwadag Mogadishu-Hawlwadag District Sayidka Road Gap Filling/Upgrading Infrastructure Mogadishu Hawlwadag District Capital City Basketball Court Rehabilitation Infrastructure Shangani Department of Urban Planning and Engineering Capacity-Building Capacity-Building Wadajir Wadajir District Administration Complex Reconstruction Infrastructure Mogadishu Wadajir District Dan-Wadagaha Road Upgrading Infrastructure GEDO AND BAY CLUSTER Settlement TIS+ PIKA Activity Type Baardheere Baardheere Capital City Community Consensus-Building Session Consensus Baardheere Capital City Roads Bridge, Airstrip Assessment Study and Design Study Baardheere Capital City Community Orientation Center Construction Infrastructure Youth Motivational Sessions Youth Dinsoor Ministry of Interior and Dinsoor District Community Consensus-Building Session Consensus Ministry of Public Works and Reconstruction Women’s Center Construction Infrastructure Ministry of Public Works and Reconstruction Road and Airstrip Upgrading Survey Study Dinsoor District Capital Football Stadium Construction Infrastructure Dinsoor District Dinsoor Capital City Airstrip Rehabilitation Infrastructure Dinsoor District Dinsoor Capital City Main Road Rehabilitation Infrastructure Baidoa ISWA Regional and District Stability Assessment and Actions Capacity-Building Capacity-Building Ministry of Information and Social Awareness Peace and Stability/Arts and Culture Inclusion SWS Ministry of Women, Family Affairs, and Human Rights Promoting Women’s Roles in Leadership and Stability in ISWA Inclusion Ministry of Interior and Local Governments Stability Department Institutional Capacity-Building Capacity-Building SWS Ministry of Youth and Sports Baidoa Youth Day Youth SWS Ministry of Planning and International Cooperation Furniture and Equipment - SWS Ministry of Interior and Local Governments PIKA Implementation Support Unit Capacity-Building LOWER JUBA CLUSTER Settlement TIS+ PIKA Activity Type Afmadow Afmadow District Community Consensus-Building Session Consensus Afmadow District Commissioner’s Office and Community Hall Construction Infrastructure Afmadow Capital City Increasing Youth Participation and Role in Society Youth Afmadow Capital City Women’s Market Infrastructure Kismayo Regional and District Stabilization Assessments Capacity-Building for Ministry of Interior Capacity-Building Ministry of Interior and Security Institutional Capacity-Building Capacity-Building Ministry of Women, Family Affairs, and Human Rights Increasing Women’s Role in Peacebuilding and Social Reconciliation Inclusion 30 Kismayo Capital City International Women’s Day Support Women Kismayo Capital City Road Construction Validation Consultation Consultation Kismayo Capital City Solar Street Lighting Maintenance Unit Vehicle and Capacity￾Building Capacity-Building Kismayo Capital City Freedom Square Rehabilitation Infrastructure Ministry of Interior PIKA Implementation Support Unit Capacity-Building Kismayo Capital City Market Roads Design Work Study Kismayo Capital City National Youth Day Youth Engagement Youth Kismayo Capital City – Airport Road (section 1) Design Work Study Kismayo Capital City Youth Iftar Youth Jubaland Chamber of Commerce and Industry Project Coordination Office Technical Capacity Capacity-Building Ministry of Women, Family Affairs, and Human Rights Technical Coordinator/Gender TC Ministry of Youth and Sports Technical Coordinator/Sports Capacity-Building Kismayo Capital City Bridging Plan Electrical Power Load Study Study State Ministry of Interior and Security Technical Coordinator/Strategic Governance Advisor TC LOWER SHABELLE CLUSTER (Limited TIS+ Implementation) Settlement TIS+ PIKA Activity Type Afgooye Mogadishu-Afgooye Road Technical Assessment and Survey Infrastructure Barawe Ministry of Youth and Sports Barawe District Capital City Football Stadium Construction Infrastructure Barawe Barawe District Local Administration Urban Roads Assessment Study Infrastructure Wanlaweyn Ministry of Public Works and Reconstruction Wanlaweyn District Infrastructure Survey and Design Works Infrastructure NATIONAL-LEVEL PIKAS TIS+ PIKA Activity Type FGS Ministry of Interior and Federal Affairs Wadajir National Framework Consultation-Launch Consultation FGS Ministry of Interior and Federal Affairs Inter-Ministerial Meeting (July 2016–current) FGS Wadajir Framework Community Contracting Capacity-Building Capacity-Building FGS Ministry of Information, Culture, and Tourism Wadajir Framework Radio Series Production Inclusion FGS Ministry of Interior and Federal Affairs Wadajir Framework Quarterly Inter-Ministerial Meetings Coordination FGS Ministry of Interior and Federal Affairs Stabilization Initiatives TC Technical Coordination FGS Ministry of Women, Family Affairs, and Human Rights Development TC FGS Ministry of Information, Culture, and Tourism Strategic Communications TC FGS Ministry of Information, Culture, and Tourism Strategic Communications TC FGS Office of the Prime Minister Aid Coordination Unit Capacity-Building Salary Injection FGS Ministry of Interior and Federal Affairs Stabilization Clusters Focus Group Workshop FGS Ministry of Interior and Federal Affairs Capacity Building for Community Consensus Building Guide Capacity-Building FGS Ministry of Interior and Federal Affairs Stabilization Coordination Support Capacity-Building FGS Improving Federal and State-Level Actors Stabilization Coordination Capacity-Building FGS Ministry of Public Works and Reconstruction Sustainable Solar Street Light Maintenance Workshop Capacity-Building FGS Improving Federal and State-Level Actors Stabilization Coordination Capacity-Building FGS Ministry of Public Works and Reconstruction Sustainable Solar Street Light Maintenance Workshop Capacity-Building Somali Women’s Basketball Support Regional Training Inclusion Somali National Women’s Basketball Tournament FGS Ministry of Women, Family Affairs, and Human Rights Development Somali Women’s Cultural Festival Women Enhancing Engineering and Community Engagement Capacity of PIKA Portfolio, TIS+ Capacity-Building FGS Ministry of Interior and Federal Affairs National Drought Response Committee Support Coordination FGS Ministry of Humanitarian Affairs Relief Operations Unit Support Coordination FGS Ministry of Interior and Federal Affairs Bimonthly Ministry of Information Inter-Ministerial Meeting Coordination Somali National Men’s Soccer Tournament Support Inclusion Somalia Sports for Development and Peace Framework STTA STTA 31 Other development activities and partners have also been active in TIS+ areas. Prior USG investments are evident, and this has influenced data analysis of these clusters. Section XI shows the actors and activities that were active during the baseline and midline data collection period. There were multiple non-state actors evident in the clusters in which TIS+ implemented, but the ones delineated in this section were anticipated to have effects likely to influence TIS+ programming and approaches in each of the clusters analyzed during the synthesis stage. Methodological Considerations Findings from HHS data compared baseline and midline percentages and the assessment team weighted data to adjust for population differences within the settlements. The team also used regression analysis to identify distinctions between “significant” and “non-significant” results at midline. In addition to the HHS, which surveyed 4,174 citizens the team augmented this data with 137 FGDs, and 192 KIIs which were conducted in the following settlements (See Table 5). TIS+ and non-TIS+ settlements were analyzed for comparison purposes only. Table 5 lists the areas that were assessed, and where data was collected. 32 Table 5: Data Sets and Cluster Coverage (Includes TIS+ and non-TIS+ Settlements) CLUSTER SETTLEMENT HOUSEHOLD SURVEY FOCUS GROUP DISCUSSIONS KEY INFORMANT INTERVIEWS CONTRIBUTION INTERVIEWS Baseline Midline Baseline Midline Baseline Midline Baseline Midline TIS+ IMPLEMENTATION Lower Juba Total 453 432 8 16 11 14 13 Kismayo 304 287 4 12 10 9 11 Afmadow 149 145 4 4 1 5 2 Banadir All Banadir TIS+ settlements 77 424 16 16 3 6 6 Gedo/Bay Total 315 425 4 8 5 9 1 Dinsoor 82 116 0 4 0 4 1 Baardheere 233 309 4 4 5 5 0 LITTLE TIS+ IMPLEMENTATION Minimal implementation of activities, some with only partial completion Lower Shabelle Total 200 429 5 8 9 13 2 Wanlaweyn 124 263 5 4 6 9 0 Barawe 76 166 0 4 3 4 2 NO TIS+ IMPLEMENTATION Banadir All Banadir non￾TIS+ settlements 356 432 20 20 19 14 28 Gedo/Bay Total 185 446 4 12 8 12 5 Elwak 86 200 0 0 0 2 0 Baidoa 0 0 0 4 4 4 0 Luq 0 0 0 4 0 2 0 Garbahaarey 99 246 4 4 4 4 0 Total All Clusters and Settlements 1,586 2,588 57 80 60 63 55 Key Findings Question 1: In the aggregate, did TIS+ contribute to the anticipated results described below? What range of results can be observed across assessed clusters? (a) Community ownership of development projects and processes; (b) Effective shared asset management; (c) Improved delivery/satisfaction of services; (d) Reduced conflict over resources; (e) Greater citizen confidence in government; (f) Reduced support for violent extremism? 1(a): Community Ownership of Development Projects and Processes To examine changes in “community ownership of processes and development projects,” the assessment team analyzed data across three areas: (1) participation of different stakeholders; (2) awareness of projects being selected based on community planning events; and (3) the extent to which community members felt that the selected projects meet priority community needs. 33 Table 6 below shows the trends across clusters relevant to these particular factors.13 Table 6: Changes in community ownership of development projects and processes indicators from 2017 to 2018 Based on the analysis of both the quantitative survey data and qualitative interviews, the team found the following results: 1. Across most clusters, the level of awareness and participation generally increased for all groups, except clan elders. ● Across the clusters, the majority of FGDs conducted confirmed awareness of, and participation in, community planning events. ● In several qualitative FGDs and KIIs there was a concern that clan elders were not as engaged in development planning, prioritization, and interventions. This pattern of response held across clusters surveyed.14 ● It was also evident in KIIs with stabilization partners, which noted that the emphasis primarily is on the participation and targeting of youth and women and not on minority clans or clan elders. ● The fact that participation and awareness of clan elders did not increase might be because these groups were not priority targets of the stabilization approach, which primarily targeted youth and women. 2. Generally, across clusters there was a positive association between increased awareness and increased participation, with the exception of Gedo/Bay. ● Between 2017 and 2018, awareness of community planning events increased significantly in all clusters, with the exception of Gedo/Bay. This trend was similar for measures of increased participation. ● There was evidence in both the HHS as well as in FGDs and KIIs that awareness of planning events increased participation for almost all groups. 13 Table 6 refers to awareness of community planning events, not necessarily the POCs, which are specifically asked about in Table 7. 14 Inclusive processes that emphasize equitable participation of all groups, including marginalized groups, may upset the existing status quo (predominantly male elders). FGDs and KIIs expressed concern that the elders’ feeling of being disengaged may actually affect stabilization in the long-term. 34 ● The cluster approach placed Gedo and Bay as one cluster, but the two areas are quite distinct, as are the districts where interventions took place. ● FGD and KIIs conducted in Gedo/Bay indicated political difficulties with government staff at the district level in this cluster area. ● Clan relations also look to have had a strong influence in Gedo/Bay. From qualitative interviews, there were expressed concerns about clan manipulation by AS and also reported incidences of clannism. 3. In the aggregate, there were stronger results since the baseline assessment on community perceptions that projects were selected based on a community collaborative planning approach, and that selected projects met community needs. ● Since the baseline assessment, there were noted increases across all clusters, in the perception by community members that projects were prioritized based on community needs. ● There was evidence in KIIs and FGDs that most stakeholders working in these clusters have a collaborative approach to project selection with local communities, likely influencing this effect. ● However, some FGD respondents felt that minority group members were often left out. The fact that this was reflected in the data in several locations indicates weak targeting of minority group needs. ● In the aggregate, there was a significant increase in those perceptions of improved community collaboration on project planning. 4. Results were influenced by a high number of factors and actors in addition to TIS+, although those interventions were generally limited across clusters. ● In the aggregate, there was a significant increase in those perceptions of improved community collaboration on project planning. ● There were multiple actors working in the TIS+ clusters, which influenced contribution to findings and effects observed ● Most projects reviewed had participatory approaches to project planning and oversight that also included quotas for women and youth ● The level of intervention by TIS+ was limited and the trends evident across the clusters was also a result of multiple influences and actors working in these areas. 1(b): Effective Shared Asset Management The assessment team examined changes on “effective shared assessment management” along three criteria: (1) community awareness of project oversight committees (POCs); (2) community perceptions of the effectiveness of those POCs; and (3) other actors (apart from POCs) responsible for shared asset management. Table 7 shows the trends across cluster relevant to these particular factors. The pink color represents scores, and that trended moderately negative from baseline. The red color indicates a strong negative trend. The light blue shows moderate positive changes from baseline, and the dark blue shows a strong positive trending at midline. “Every project the government plans passes through a public consultation phase mediated by various teams of committees representing the youth, women, religious council, business owners, and elders. Through this platform, we created a common consensus in the planning and the execution of projects.” – Government official, Garbahaarey 35 Table 7: This table reflects results from two HHS questions: (1) To what extent have the Project Oversight Committees proved to be effective or ineffective in the management of the following in your settlement? (2) How effectively were these services maintained during the past year (by other parties)? 1. There was an increase of awareness of TIS+ POCs, with the exception of Gedo/Bay, due mainly to political tensions around high-profile infrastructure projects. ● From the HHS as well as qualitative data, there was evidence that awareness of POCs across the clusters had increased. ● There was a noted difference in Gedo/Bay in these measurements in TIS+ settlement areas. ● It was evident that political tensions can also influence other anticipated effects and indicators, particularly on higher-profile infrastructure projects. ● Across the clusters, the primary reason for this increase is that POCs are a common unit of oversight utilized by a multitude of partners across service sector areas. ● Schools and health centers regularly have POCs, as do other partners implementing smaller community￾driven development interventions. ● It was evident that the TIS+ approach is not unique in its use of this mechanism to improve community planning, oversight, and ownership. 2. There were mixed results on the perceived effectiveness of POCs. ● Across the clusters, there were mixed perceptions of the effectiveness of POCs. 36 ● Based on the HHS data and FGDs, these variations seemed to reflect a lack of training of POC members. There were numerous interviews requesting increased levels of training for POC members, particularly regarding oversight and maintenance of facilities. ● The type of activity that the POCs are overseeing will in all likelihood affect the frequency with which the POC will engage with the community. It can reasonably be assumed that other POCs responsible for overseeing more direct service delivery-focused projects—for example, the building of hospitals, health centers, and schools—will have more frequent interaction with the community than the TIS+ activities focused on constructing roads and airstrips. ● While the assessment team had expected to see changes in perceptions of oversight committee’s effectiveness in the management of at least roads and meetinghouses in the TIS+ areas given that these were the type of assets that TIS+ PIKAs focused on, these results did not materialize. ● The results indicate that the original approach of establishing district-level POCs was not an effective mechanism for increasing perceptions of the effectiveness of oversight committees, at least in the short term, as there was little evidence that this translated to cluster-level effects. 1(c): Improved Delivery of / Satisfaction with Services To analyze this result area, the team examined community perceptions around (1) service provision by government and other actors; (2) satisfaction with services provided; and (3) perceptions on “security as a service.” 1. There were mixed results on perceptions of service provision by the state, non-state actors, and the business community. However, across clusters there was a persistent trend of community members being more satisfied with the quality of services provided by international and national nongovernmental organizations (INGOs/NGOs). ● While the results vary by cluster, at least two significant trends can be identified: (1) INGOs are seen as having an increased role in the provision of nearly all types of services across the cluster. (2) The role of both state and local government in the provision of services that TIS+ PIKAs worked on did not predictably increase across clusters. ● The central role played by INGOs and NGOs in service provision was further confirmed in qualitative interviews. ● FGD respondents widely preferred INGOs and private sector providers, and were often critical of government services, particularly for roads and health. ● FGD respondents also felt that the government did not do enough to monitor and maintain services. This was consistent in TIS+ and non-TIS+ settlements. ● FGD and KII respondents indicated the existence of other providers, including local and international NGOs, diaspora community, local businesses, humanitarian organizations, telecommunication companies, money remittance companies, banks, and water and electricity companies that were involved in service provision besides the government. ● Illustrative of the general trend across the interviews, an elder woman from an FGD in Baardheere aptly summarized, “There are many NGOs in the district that help in food aid distribution while some offer cash aid for orphans in the community. There are also individuals that set up INGOs that work in various sectors within the community. The most important ones are the youths that mobilize themselves to develop their community. For example, these youth organizations work in schools and offer affordable education to the people living outside the town who cannot afford to attend the well-established schools.” “Humanitarian organizations provide most of the good quality development projects qualities to the people. The government assists them by guiding them to whom and where the interventions should be targeted and by providing security. The government system is not functioning well in the entire country.” – Elder Men FGD, Banadir 37 2. In terms of “security as a service,” there was a clear awareness that this responsibility fell on the FGS and state government. ● Across clusters, qualitative respondents were clear that the primary service provided by the FGS currently is security. ● There was also evidence across the clusters that the Federal Member States (FMS) also have a role to play in this service. ● In some areas where TIS+ worked, there was a larger state security presence, which influenced this perception at the midline and in the synthesis analysis. 3. Across clusters, women reported lower improvements in security than men. ● Across clusters, in the HHS as well as qualitative data, women respondents reported lower improvements in security than men. ● This trend was also evident in perceptions of human security gathered from FGDs. ● There was evidence that women view security at a localized level and that security is directly linked to human security needs, such as freedom of movement and equal access to services. ● The male and female FGD participants varied significantly in their responses, and women’s responses often illustrated links between security improvements and improvements in community cohesion. 1(d): Reduced Conflict over Resources To analyze this result area, the team examined changes in perceptions on (1) the level of conflict over resources and (2) trust across various actors to resolve conflicts over resources. 1. Although there was an increase in perception that large-scale resource-based conflicts had decreased across the clusters in the HHS, qualitative data showed that micro-conflicts (e.g., conflicts over land and water) continue to impact community cohesion. ● Micro-conflicts that influence district-level conflict over resources often also overlap with clan dynamics, impacting community cohesion. ● Across the clusters there was an increased perception that large-scale resource-based conflicts had decreased. This however was not true of micro-conflicts. ● In numerous qualitative interviews, micro-conflicts were seen as a cause of increased tension. These conflicts affect community cohesion at the local level and can also spiral quickly into more serious and systemic conflict. ● While qualitative interviews (FGDs and KIIs) credit the local Peacebuilding Committee, clan elders, and district and state government with helping to resolve conflict, these interviewees equally conveyed a sense that resource-based conflicts continue to linger, particularly around issues related to land, water, and access to pasture land, with the minority clans bearing the brunt of such conflicts. ● This was evident in numerous interviews. As stated by one young man from an FGD in Dinsoor, “Yes, there are disagreements over land disputes, which occur in the district due to annexation of clan lands. This becomes a base of misunderstandings in the district and often affects the minority clans in the district. These minority clans are always looked down upon by the major clans. This happens repeatedly.” 2. There were large increases across clusters in perceived trust placed in local structures (clan elders, peacebuilding committees, local administration, religious leader, and Ugaas/sultans). The trust placed in the militia groups and the diaspora to resolve conflicts considerably decreased across clusters. “International and local NGOs implement the projects because of the funds they have. Women organizations are the mothers and they encourage and give moral support to those NGOs helping the communities. Youths, both males and females, support those who are implementing the project by taking part in the consultations. The government helps anyone who is helping this community by providing security.” – Youth Girls FGD, Afmadow 38 ● This finding was supported by qualitative data (KIIs and FGDs), which emphasized the centrality of local structures (including peace committees) in resolving conflicts. ● As one male elder from Dinsoor stated, “The elders, the chiefs, the clan leaders, the peace committee, and most importantly, the local authority resolve the conflicts.” 1(e): Greater Citizen Confidence in Government To analyze this result area, the team examined changes in (1) the Good Governance Index; (2) citizen confidence toward government; and (3) perceptions on extent of corruption. 1. All clusters demonstrated increased scores since baseline on their “Good Governance Index” (GGI). ● The Good Governance Index (GGI) is a composite measure that considers changes to indicators along three themes: (1) confidence in government, (2) service delivery, and (3) stakeholders’ partnership. The individual scores for each cluster are provided in Figure 4. ● Because the index score was a composite measure, the direction of effects varied, positive and negative between baseline and midline data, in the individual survey questions constituting the good governance index. However, the trend is clearly positive in the areas surveyed. ● The observed positive trends in TIS+ areas were mostly explained by survey questions on service provision, public trust, transparency (the local administration sharing information), accountability (reduction in corruption levels both at federal and local administration levels), and representation (federal officials not being directed by clan interests in decision-making Figure 4: Good Governance Index (GGI) measures 2. There was a perceived negative trend in citizen confidence toward state government, across both TIS+ and non-TIS+ areas, since baseline. There were non-statistically significant changes for both federal and local governments amid citizens’ apathy on a host of issues. • Based on HHS data, confidence in state government decreased in both TIS+ and non-TIS+ areas. • Both areas saw non-significant changes in confidence toward both federal and local (BRA) levels. 39 • FGD and KII respondents reported a long list of frustrations, including perceptions of nepotism and clannism, unfulfilled election promises, corruption, security lapses coupled with fear of victimization by AS, injustices and weak judiciary, poor service delivery, and outsized influence of the diaspora. 3. There were mixed findings reported on perceptions of corruption across clusters at various levels of government. However, there was a yearning among the citizens for wide engagement, participation, information sharing, and transparency. ● Perceptions of government corruption, at both local and federal levels, decreased between baseline and midline across the clusters. However, results were mixed at the state level. ● Qualitative interviews (both KII and FGD) indicate that community members recognized government efforts to increase their transparency, and that this was attributed to helping reduce overall corruption. ● Interviews also emphasized that efforts by the government to improve social service delivery, as well as to help mediate and resolve local conflicts, helped strengthen the relationships between government and communities, validating several of TIS+’s key underlying programming assumptions. 4. There was little documented evidence that TIS+ implemented its strategic communications plan. ● Despite having a strong strategic communications plan there was little documented evidence of the impact of this on intervention planning or in cluster-level data collection. ● Qualitative interviews noted that there was no assigned staff who specialized in strategic communication. ● As gaps in information sharing was an important area noted to increase perceptions of improved governance, this omission seemed relevant to these project factors. 1(f): Reduced Support for Violent Extremism To analyze this result area, the team examined perceptions around (1) youth vulnerability to extremist recruitment and (2) the extent the respondents reported that extremist groups were “not at all welcomed.” Table 8 shows the trends across clusters relevant to these particular factors. 1. There was an increase across clusters of the perception of youth vulnerability to recruitment by extremist groups. ● The household survey showed there were notable increases across the clusters surveyed that youth were still vulnerable to recruitment by extremist groups. ● Female respondents reflected a higher percentage in their perception of youth vulnerability than male respondents. Females from baseline, reported a higher incidence of youth vulnerability than male respondents. ● While this general increase can be attributed to a number of reasons, Al Shabaab has in the past targeted youth disenfranchisement to heavily recruit in urban and remote areas. ● Based on FGDs, there is still evidence that Al Shabaab employs youth. However, this was less evident than in the interviews during the baseline data collection stage. ● There was no evidence found that TIS+ activities contributed to mitigating youth being more susceptible to recruitment in these clusters. “Earlier the management was corrupt, but this transparency and accountability systems have been created to net those doing illegal things. Therefore, we have more confidence on the services of the government” – Female Elder FGD, Kismayo 40 ● This results area is at the far end of the results chain and it is expected that other factors would be more likely to influence the results observed. Table 8: This table reflects results from questions relating to violent extremism. “Proneness” is a measure of susceptibility to recruitment and “Not welcomed” is a measure of the community’s acceptance of AS. Question 2: Which TIS+ implementation process contributed most significantly to the cluster￾level results identified? TIS+ has produced several guiding documents outlining some standardized approaches that the program planned to utilize across its clusters. As these approaches were intended to be program-wide, the assessment team analyzed these approaches through the use of “contribution stories” (see Section X). These stories critically examined the “contribution claims” of the extent to which TIS+ contributed to higher-level results through the use of four specific contribution stories: (1) Community Planning and Oversight (CPO); (2) Technical Coordinators (TCs), (3) Sports for Peace and Development (SPD), and (4) Gender, Equality, and Social Inclusion (GESI). While the contribution stories reference examples across the clusters, this section will focus on highlighting additional examples and influencing factors that impacted TIS+’s approaches across clusters. It will examine TIS+’s CPO approach, which includes the use of community consensus processes, TIS+’s TCs approach, and the use of the “cluster” approach itself for geographically focusing activities. Results from TIS+’s GESI approach will be also be examined. 2(a): Cross-Cutting Findings Gender, Equality, and Social Inclusion: Across clusters, approaches to gender, equality, and social inclusion are not contributing to improvements in community cohesion as anticipated in the theory of change of TIS+. 41 1. Both TIS+’s own M&E reporting, as well as the assessment’s qualitative interviews (KIIs and FGDs) indicate that TIS+ focused mostly on ensuring higher numbers of youth, women, and other marginalized groups participating in events rather than on the quality of participation of these groups. This trend was found to be a consistent shortcoming with other donor programs reviewed, including the Somalia Stability Fund (SSF) and the Somalia Stabilization Initiative (SSI). 2. Qualitative interviews as well as HHS data showed that inclusion factors were correlated. 3. KII and FGD respondents cited concerns about meaningful participation, particularly for women, youth and women. Interviews and a review of TIS+ documents also indicate that TIS+ tended to artificially divide its activities targeting women, youth, and other marginalized groups, rather than recognizing that individuals could have multiple layers of vulnerability and that specific strategies would need to be developed to reach those considered most vulnerable. 4. There was minimal documentation showing how the program set targets and established criteria for participation based on an analysis of inclusion needs. 5. There was a Gender Strategy, but this strategy lacked specific approaches catered to stabilization and conflict sensitive conditions specific to Somalia. Political Economy and Stabilization Analysis (PESA): Across clusters there was evidence that conflict mapping, Do No Harm, and/or political analysis is not influencing intervention planning. 1. There was little documented evidence that the Do No Harm and conflict mapping done during the consensus planning process was integrated into intervention planning. 2. There was also minimal evidence that field staff participated in a regular or rolling analysis of current conflict and political conditions with the potential to affect infrastructure and interventions. 3. There was evidence that this was done informally, but there was no standard or cross-cluster approach that would allow the project to draw on lessons learned to improve stabilization planning or to leverage effects across clusters. 2(b): TIS+’s Community Planning and Oversight (CPO) Approach The use of community planning and oversight is at the core of TIS+’s programming approach. As developed in the program’s ToC, this approach is the heart of the TIS+ program model. As stated in interviews, improving the relationship between government and communities is central to achieving an improved stabilization space. TIS+’s CPO process emphasizes the importance of the process not just the end product and is key to distinguishing TIS+ as a stabilization program rather than a more traditional development and infrastructure program. In short, the approach specifies how the program will engage community and government actors to plan, implement, oversee, and utilize PIKA activities and how this interaction will help the program achieve higher-level outcome results, such as increased citizen engagement and confidence in government, and reduced conflict over resources and reduced support for violent extremism. TIS+’s specific approach to CPO is articulated in a number of guiding program documents and throughout its periodic reports. A critical feature of TIS+’s CPO process is the formation of POCs. The central role that POCs play in the larger CPO processes is documented in a number of TIS+ documents. Findings of Results of CPO Approach “No. They don’t have equal access to public assets and services. The reason is that the local culture gives priority to men. Traditional customs and rules are used to suppress the rights of women. Men pressure women to stay at home and not engage in any work outside of home. So we cannot say that women enjoy equal access to public assets and services especially when there are no equal employment opportunities; men are often preferred even when they compete with highly educated women counterparts, who often far outnumber them. That said, there are certain areas such as schools and hospitals where a substantial proportion of women work and benefit from services. I personally believe that women are the only people currently whom we can entrust to provide us a just leadership. Yet they are not given this role” – Business leader, Kismayo 42 1. TIS+ took active steps to ensure that communities were supported to collectively identify and prioritize their shared community needs. 2. TIS+ implemented most of its intended CPO steps but substantially deviated from the original design when it came to the establishment, training, and use of the POCs. 3. There was an increase in the reported roles of state and local governments in managing shared assets, particularly in the management of roads and sports arenas, which were Pools of In-Kind Activities (PIKAs). 4. There was a wide and diverse set of groups of actors implementing activities in each of the TIS+ clusters reviewed, most of whom reported utilizing some aspect of community engagement, planning, and oversight. 5. Qualitative interviews (both KIIs and FGDs) conveyed that the unique socioeconomic characteristics of TIS+ settlements in these clusters played a large role in the results seen during the midline. 6. Interviewees also explained that the building of trust between communities and respective levels of government following decades of conflict is a long-term endeavor and requires generational change. 7. POC training was not systematic and uniform, which contributed to the mixed effects observed. 2(c): TIS+’s Technical Coordinator (TC) Approach 1. TCs have helped improve coordination between the government and TIS+, as well as increase capacity by the government to engage with stabilization actors and donors and encourage better planning by the government. 2. TCs’ contribution to the state-level ministries is visible but there is a mixed picture at the federal level due to multiple actors providing an assortment of technical assistance. 3. TCs benefit from carryover effects from TIS, particularly at the federal level in Mogadishu, where the program was first piloted. 4. The political and security environment had a strong influence on the contribution of the TCs. Insecurity as well as frequent political turnover negatively impacted the work of the TCs. 5. TCs felt that USAID did not adequately understand their role. TCs were also underutilized during decision￾making processes in TIS+ that resulted in programmatic shifts and changes in intervention. 6. The low capacity of staff and officials within the institutions often diverted the TC’s work to capacity-building and away from their core roles supporting and coordinating stabilization work. 2(d): Findings on Results of “Cluster” Approach 1. The cluster approach was not fully utilized due to complexities in security, delayed effects of anticipated military campaigns, and insecurity in particular cluster areas. 2(e): Findings on Government Involvement in Shared Asset Management and Provision 1. Across clusters, with the exception of security, the federal government was regarded as having a limited role in the management of shared assets. 2. However, the perception of local and state government’s role in the management of all assets increased from the baseline. There was an awareness by communities that local administrations’ resources and capacities were limited. 3. Across clusters respondents acknowledged the willingness of various levels of government in asset management, while equally identifying resource and systemic capacity challenges. 4. Respondents stressed that asset management thrives under good governance and is a shared responsibility between the government and the community. 2(f): TIS+’s Inclusion Approach 1. There was an increase across clusters in those reporting that they believed that women were considered equal to men. 43 2. There were notable efforts by TIS+ to ensure women’s participation in POCs, though without a targeted training component. 3. Although all POCs reviewed included youth members, youth were primarily only given direct oversight and management responsibility for sport facilities and playgrounds. Youth expressed an interest in broader participation in stabilization activities. 4. There was limited data from TIS+ on how they tracked minority clan involvement or how this influenced programming. 5. Community Cohesion Index scores increased across clusters. 2(g): TIS+’s Sports for Development and Peace (SDP) Approach 1. There was consensus among key informants working on sports-related activities that conducting or facilitating sports events alone was not enough to deliver the desired results. 2. Interviews suggested that there are benefits for social cohesion through sports events and availability of sports venues in communities. 3. Respondents often cited that having sports venues without additional supporting activities (e.g., education, vocational, social, etc.) was regarded by interviewees as insufficient for achieving higher-level results, such as deterring youth from engaging in AS activities. 4. Related to gender, midline interviews suggested that Somali society is generally accepting of men’s participation in sports events. However, the participation of women and girls is much more controversial. “Yes, there are very good opportunities for women to contribute to security. I say so because they are very good in collecting information. They are even better than men, and I would suggest that women take over the security of the district.” – Elder Men FGD, Daynile 44 VIII. CONCLUSIONS Question 1: In the aggregate, did TIS+ contribute to the anticipated results described below? What range of results can be observed across assessed clusters? (a) Community ownership of development projects and processes; (b) Effective shared asset management; (c) Improved delivery/satisfaction of services; (d) Reduced conflict over resources; and (e) Greater citizen confidence in government. Q1A: Community Ownership of Development Projects and Processes 1. TIS+ contributed to a small but positive increase in community awareness and participation in development activities in the settlements where it worked. More broadly, since baseline, Somali populations assessed were increasingly more aware of development projects in their settlements, providing a ripe opportunity to foster active participation in stabilization activities. There was a general trend of increased community awareness and participation. However, results were mixed for the increased participation of women, youth, and minority clans. From baseline and across clusters there was an increase in the awareness of development projects. Citizens were more informed of development planning events and there was evidence that partners working in stabilization spaces were utilizing more participatory methods to ensure greater citizen engagement. There were similarities in intervention modalities, showing that most stabilization partners had a method that they used to target citizens living in the areas in which interventions were planned. HHS conducted as well as FGDs and KIIs showed an increased interest in taking part in development planning and intervention over the course of this task order. Citizens were generally more informed, and interest was higher in active engagement in planning, oversight, and maintenance of development projects. 2. The settlement of Baardheere was a strong outlier, indicating that political developments and how TIS+ reacts to those developments can significantly influence results, even at the cluster level. Local politics and clan relations had a strong influence on the success of TIS+’s activities in Baardheere. Interviewees explained that Baardheere is divided between two dominant clans, in part exemplified by an ongoing power struggle between the District Commissioner (DC) and Deputy DC. It was evident during synthesis analysis and findings that political economy and conflict-sensitivity analysis is necessary for program planning and intervention. Political will and cooperation are critical for stabilization interventions, and it is evident that TIS+ works hard in the field to ensure cooperation with local and state officials. In this settlement, the DC proved to be uncooperative and difficult for field staff to work with. This is likely to be the case in multiple areas and across interventions. It was unclear whether TIS+ had a strategy or approach to mitigate these particular political tensions. It was also evident that these tensions greatly influenced results observed by the assessment team. These settlement results also affected the cluster. There is a need for TIS+ to carefully monitor and adapt activities to respond to local political dynamics. TIS+ must have a mitigation strategy for such implementation problems. 3. Failure to manage expectations during community planning events had a negative correlation with communities’ perceptions of whether selected projects meet their needs. This pattern was observed across multiple clusters and settlements. In several cases, community members were thankful for interventions, but were unclear how priority projects were chosen. As in many cases the community consensus events were held long before infrastructure planning was to begin. This discontent may also be a result of weak communication channels between TIS+ and communities during these periods. Delays in establishing 45 PIKA activities and involving community members in planning could have been mitigated by more effective strategic communication as well as interim planning events planned with host communities. Q1B: Effective Shared Asset Management TIS+ has made a small contribution to increased awareness of POCs in areas where it operated. While there was a general increase of awareness of POCs across clusters, this did not translate into improved perceptions of performance likely due to inconsistencies in POC formations and implementation and the types of community infrastructure selected.15 While there was a general increased awareness of POCs, qualitative respondents highlighted gaps in government resource and management capabilities, particularly at the state and local (district) levels. The delay in forming PIKA-level POCs and types of community infrastructure projects selected were identified as contributing factors explaining this result. The mixed results on the perceived effectiveness of POCs depends on each POC’s mandate and ways of operating. Direct training for citizen groups engaged in development planning is important. Some types of infrastructure have more sustained local involvement, such as schools and water points. Infrastructure types that seem to have more community participation and local governance overlays are school repairs, maternal and child health clinics, small-scale culverts, road repair, and markets. Q1C: Improved Delivery of/Satisfaction with Services There was little evidence that TIS+ contributed to perceptions of improved service-delivery and satisfaction due to varied results and mixed patterns that were observed across different service areas. There was a modest increase in the perception of local government providing services, and strong evidence that communities still see INGOs and NGOs as frontline providers of services. Communities, however, had an increased understanding (from baseline to midline) of the challenges that local and state governments face in administering these services. Notably, in cases where the government actively participated in the management/governance of the delivery of the services, respondents had an increased perception of the government as a service provider. Stabilization partners will benefit from a focus on supporting more day-to-day government-led service delivery activities and infrastructure projects if it aims to influence higher-level results related to increased citizen confidence in government or improved perceptions of service delivery. Q1D: Reduced Conflict over Resources There was no observable evidence of TIS+ contribution to reduced conflict over resources at the cluster level. Notably, these conflicts over resources generally remained stable and/or persisted across clusters since baseline. The mediation of resource-based conflicts appears not to have been a priority for the program. There was a decrease in conflicts over resources across clusters. However, qualitative interviews showed a concern about the impact of micro-conflicts (e.g., over land and water) on community cohesion and inter-clan 15 The program has been implemented as required by the project design/contract. TIS+ notes that many community leadership groups have asked for improved airstrips and boundary fences; however, such requests are often not possible to implement. 46 relationships. There was a general increase of reliance on local structures (clan elders, peacebuilding committees, and local and even state administrations) to mediate these conflicts. Q1E: Greater Citizen Confidence in Government Based on the evidence, TIS+ did not make a unique contribution to greater citizen confidence given similar trends in non-implementation areas. Across clusters, there was little or negative change in confidence at all levels of government. The assessment could not establish TIS+ contribution to greater citizen confidence in aggregate in government given mixed evidence across the clusters. The Good Governance Index (GGI) recorded more positive results. However, at the cluster level, there were variations of measures used to establish the Index. In clusters assumed to be further along the stabilization continuum (i.e., Banadir) measures decreased significantly. This was likely a result of increased insecurity in this cluster area. Citizens were more interested in the ability and capacity of the district and state governments to manage affairs (service provision, management of shared assets) and this linked with the levels of confidence they had in the local and state governments. TIS+ missed opportunities to positively influence citizens’ perception of confidence in government through strategic communications. TIS+ would benefit from strengthening its strategic communication efforts to help improve citizen confidence. TIS+ work is more likely to lead to increases in citizen confidence at the district and state levels than work done at the federal level. Future stabilization interventions should target making an impact at these levels of government. At the federal level, TIS+ and future stabilization programs should focus on enhancing the federal government’s coordination capacity. These interventions would help influence more directly evident increases in confidence in government. Q1F: Reduced Support for Violent Extremism TIS+ did not contribute to reduced support for violent extremism across clusters. However, potential results related to countering violent extremism is at the far end of TIS+ intended results chain. TIS+ can be expected to have indirect and minimal, if any, influence over these results. There was no evidence that TIS+ contributed to reduced support for violent extremism. Analysis of program documents indicates a cursory understanding of the dynamics and shows no direct linkage between the dynamics and the design of interventions such as Sports for Development and Peace interventions or youth outreach activities. Potential results related to preventing/countering violent extremism (P/CVE) are at the far end of the TIS+ intended results chain. Most stabilization interventions had only indirect results aimed at reducing youth engagement with AS. TIS+ interventions in this area largely focused on youth engagement and sports/facilities. The majority of youth surveyed, however, wanted interventions focused on increasing leadership and agency, platforms for meaningful participation in decision￾making, and economic opportunity. TIS+ does not have an articulated P/CVE strategy that ensures that interventions are informed of this strategy. Question 2: Which TIS+ implementation process contributed most significantly to the cluster￾level results identified? Q2A: Cross-Cutting 1. Approaches to gender, equality, and social inclusion are not contributing to improvements in community cohesion as anticipated in the theory of change of TIS+. Inclusion factors are co-related, and targeting is not enough. 47 2. There is a need for TIS+ to more effectively integrate conflict mapping, Do No Harm, and/or political analysis directly into intervention planning and monitoring of effects. Q2B: TIS+’s Community Planning and Oversight (CPO) Approach 1. There is limited evidence that TIS+’s Community Planning and Oversight approach is consistently contributing to higher-order, outcome-level results related to improved citizen engagement and confidence in government, as well as reduced conflict over resources or support for violent extremism. TIS+’s CPO approach contributed to increased levels of community awareness and participation but did not result in higher-level outcomes. Deviations from its original design, changes in the process of engagement, and broken linkages between anticipated outcomes and interventions influenced these effects. While there was evidence of TIS+ achieving increased community awareness and participation, as well as local and state government service provision and management of shared assets, these results were usually only at the individual settlement level and were not consistent across clusters. TIS+’s contribution claim is also limited by the lack of a consistent implementation approach across the program. While the community consensus workshops were consistently implemented, the form and timing of other key parts of the CPO process varied across settlements. This includes a difference in the way that community contracting took place across the settlements, and most notably, significant deviations on how POCs were formed, trained, and utilized throughout the process. TIS+’s program design documents highlight the central role that POCs play in the overarching CPO process. However, evidence indicates that TIS+ did not prioritize their role until late in the PIKA process, usually after the construction of the different structures was already complete. 2. While the midline assessment found evidence of TIS+ achieving notable results in terms of community awareness and participation, as well as local and state government service provision and management of shared assets, these results were usually only at the individual settlement level and were not consistent across clusters. TIS+’s contribution claim was limited by the lack of a consistent implementation approach across the program. While the community consensus workshops look to have been consistently implemented, the form and timing of other key parts of the CPO process varied across the clusters. This includes a difference in the way that community contracting took place across the clusters, and, most notably, significant deviations on how POCs were formed, trained, and utilized throughout the process. TIS+’s own program design documents clearly highlight the central role that POCs play in the overarching CPO process; however, evidence indicates that TIS+ did not prioritize its role until late in the PIKA process, almost always only after the construction of the different structures was already complete. Q2C: TIS+’s Technical Coordinator (TC) Approach The Technical Coordinators (TCs) were not well embedded in intervention planning. There was evidence of impact at the state level, but less evidence at the federal level. TIS+ provided technical assistance to state and federal governments. The evidence of this was stronger at the state level regarding shared asset management. TIS+ Technical Coordinators worked with district (local) government to develop and adopt Operations and Maintenance Manuals of shared assets. Stabilization efforts were most felt at the district and state level across the clusters, and TCs seemed to have a greater impact in contributing to citizen confidence in government at this level. The TCs’ contribution at the federal level was less evident, likely because other stabilization programs also embedded coordinators and advisors to carry out similar functions. There was limited evidence that the TIS+ TCs contributed as expected to higher-level results such as increasing confidence in government. Respondents tied service delivery to citizen confidence in government and TCs supporting the facilitation role of government in service provision may contribute to increased confidence. 48 Contribution Analysis found that the contribution claim is limited by the fact that other actors, in particular stabilization programs, also embedded TCs and advisors to carry out similar functions such as supporting state￾and federal-level coordination. There was limited evidence that the TIS+ TCs contributed as expected to higher￾level results such as increasing confidence in government. Q2D: Results of Cluster Approach TIS+ attempted to adapt programming to match the reality of the constraints presented by the lack of military campaigns along the anticipated corridors, but this proved to be challenging for a development partner. The cluster approach was not as effective in some clusters because it did not build in adaptive management processes to address distinct differences between settlement demographics and contexts. This approach was also hampered by the lack of expected military progress of Somali National Army (SNA)/African Union Mission in Somalia (AMISOM) military campaigns and the opening of “corridors” between settlements. Clusters assumed to be further along the stabilization continuum also experienced increased insecurity, calling into question some of the initial criteria for engagement. USAID and implementers should carefully consider the unique local contexts and sociopolitical dynamics within and across settlements when considering future clustering approaches. Q2E: Government Involvement in Shared Asset Management and Provision 1. The federal government’s role was limited to the provision of security. Across the clusters, the federal government was regarded as having a limited role in the management of shared assets, apart from contributing to increased security which enabled interventions. While the federal government was regarded as having a limited role in the management of shared assets, community members perceived state and district government as having increased their role since baseline, with particularly positive results in TIS+ areas (roads and sports facilities) while the federal government played a major role in managing security. TIS+ should invest in improving the capacity of local and state administrations to ensure they have a more effective role in shared asset management. 2. There was consistent recognition that government involvement at all levels was necessary for shared asset management. TIS+ facilitated the government taking a lead in community consensus planning and community contracting. TIS+’s efforts to strengthen the perception of government service provision seems to have been hampered by a variety of factors both within and outside of the program’s control. Local government figured prominently compared to federal and state governments as having the will to manage assets but often limited by capacity, resources and systemic challenges that needed to be filled through a shared responsibility between government and the community as well as stronger public–private partnerships. Q2F: TIS+’s Inclusion Approach 1. TIS+ has taken a number of steps to increase the participation of women in its activities and has succeeded in facilitating women’s participation across its POCs, as well as influencing perceptions in its settlements that women were considered equal to men. However, evidence from the midline also suggests that there is a need for additional capacity-building of POC members, particularly youth and women. While TIS+ has succeeded in facilitating the participation of youth in community discussions and planning for the construction of shared assets, the program has mostly limited youth’s participation in the management and oversight of shared assets to those involving sports facilities and playgrounds. 49 2. There was a lack of systematic monitoring of minority clan involvement, or at least documentation of such monitoring in TIS+ interventions. Across clusters there are significant gender gaps in monitoring and oversight of development assets, leadership in peacebuilding, inclusion in decision-making, and political representation for women and minority clans. There was no evidence in TIS+ interventions that minority clans and/or marginalized groups were systematically engaged. TIS+ staff demonstrated deep knowledge of the local context, conflict factors, and marginalization dynamics that could undermine community cohesion. TIS+ did not adequately document how this knowledge was utilized to inform or adapt program planning and decision-making to mitigate micro-conflicts. TIS+ has taken a number of steps to increase the participation of women in its POCs but does not have a clear criteria for identifying the capacity gaps and needs of minority clans. During the assessment phase at district level and state level there was minimal evidence that TIS+ identified minority and marginalized groups and targeted them for stabilization interventions. Q2G: TIS+’s Sports for Development and Peace (SDP) Approach Any potential claim for TIS+’s SDP activities to have influenced higher-order outcomes related to community cohesion or positive youth development to date are not well substantiated by current evidence. Contribution Analysis found no linkages between the TIS+ SDP gender- and youth-focused activities to its intended higher-outcome-level results. CA uncovered little evidence that the program has succeeded in supporting the quality of participation of women, youth, and other minority groups in its activities. Youth felt marginalized and underutilized. TIS+ primarily engaged youth through its SDP approach. The TIS+ Results Framework and ToC primarily identified youth and women as the target groups to address community cohesion gaps. TIS+ stabilization interventions often targeted one or the other of these groups without identifying shared grievances. However, there is room for optimism as TIS+’s recent SDP Strategy and Implementation Framework not only demonstrated a commitment to more systematic link SDP activities to achieve higher-level results but also provided a concrete road map for how to do so. It is expected that TIS+’s contribution claim around its SDP results is likely to be strengthened in the coming years. 50 IX. RECOMMENDATIONS 1. TIS+ should ensure that the program follows a more consistent and complete community planning and oversight approach. This involves providing timely training and capacity-building to POC members so that they can properly serve in their oversight roles for improved community ownership and effective shared asset management. Increases in participation and capacity are distinctly linked to citizens feeling more competent to engage in decision-making, management, and oversight. Long-term CPO approaches are familiar across Somalia, but the quality of participation is still a challenge for the TIS+ project. While there is evidence of increased awareness of POCs in TIS+ clusters, few results demonstrated the increased effectiveness of the POCs. There is, therefore, a need for TIS+ to train further and support POC oversight and responsibility. POCs should be formed prior to construction. Increased emphasis on training will capacitate community members to engage more meaningfully in the oversight and management of shared assets and lead to higher-quality citizen and government interaction. This recommendation is linked to the conclusion for Q1B: Effective Shared Asset Management: “TIS+ has made a small contribution to increased awareness of POCs in areas where it operated . . . but this did not translate into improved perceptions of performance.” 2. TIS+ and USAID should ensure that programming approaches are periodically monitored for alignment with TIS+’s overarching Theory of Change. An applied ToC approach requires not only flexibility but also a strategic alignment based on accurately observed changes in the context informed by regular field monitoring and management. Stabilization programs require adaptive monitoring systems that complement an applied TOC approach. This, in turn, allows for greater program adaptation over time based on field-based analysis. This recommendation is linked to the conclusion for Q2B: TIS+’s CPO Approach: “There is limited evidence that TIS+’s Community Planning and Oversight approach is consistently contributing to higher-order, outcome-level results related to improved citizen engagement and confidence in government, as well as reduced conflict over resources or support for violent extremism.” 3. TIS+ and USAID should consider focusing on supporting more day-to-day government-led service delivery activities and infrastructure projects if it aims to influence higher-level results related to increased citizen confidence in government or improved perceptions of service delivery. If the goal is to change perceptions of effective shared assessment management, the type of shared asset that the POCs oversee is critical to shaping community perceptions. TIS+ should carefully consider which interventions would contribute to increased perceptions of effective shared asset management and how these interventions provide space for government to facilitate and interact with citizens. This would require more direct coordination with local government partners and is likely to result in more co-created interventions. This recommendation is linked to the conclusion for Q1B: Effective Shared Asset Management: “TIS+ has made a small contribution to increased awareness of POCs in areas where it operated . . . but this did not translate into improved perceptions of performance.” 4. TIS+ should consider strengthening its analysis and assessment of the localized dynamics and effects of violent extremism before tailoring interventions at the cluster level. There was no direct evidence that TIS+ interventions have contributed to reducing violent extremism, and as previously mentioned, P/CVE is at the far end of the TIS+ intended results chain. TIS+ would benefit from having a specific mechanism for analysis of localized effects of violent extremism. 51 This recommendation is linked to the conclusion for Q1F: Reduced Support for Violent Extremism: “TIS+ did not contribute to reduced support for violent extremism across clusters. However, potential results related to countering violent extremism is at the far end of TIS+ intended results chain.” 5. TIS+ should strengthen its strategic communication efforts to help improve citizen confidence. TIS+ currently has one TC16 embedded in the Ministry of Information to support its communication efforts. Primary efforts have been through the Radio Wadajir Program. However, this does not qualify as a key strategic communications intervention. A broader effort driven by senior staff within TIS+ is necessary and the efforts should be fully integrated into policymaking. The focus should be on the core narrative, supported by frequent analysis and research, as opposed to emphasis on products such as radio shows or billboards. These efforts could include improving citizen access to information to ongoing stabilization activities, improving visibility of government’s leading role in stabilization and development initiatives, and improving interaction and feedback between government and citizens. Strategic communications should be at the heart of TIS+’s stabilization efforts. Targeting narratives, counter-messaging, and influencing perceptions will contribute to higher-level results. In addition, strategic communications can be utilized to promote positive perceptions in confidence and trust in government. Strategic communication should cover a wide array of elements (e.g., radio, social media, event-specific content, hardware, and software). In particular, TIS+ should make better use of social media. TIS+ should provide an alternate rather than a counter narrative. Focus on “counter” helps to amplify AS’s message rather than providing a better alternative. This recommendation is linked to the conclusion for Q1E. Greater citizen confidence in government: “Based on the evidence, TIS+ did not make a unique contribution to greater citizen confidence given similar trends in non-implementation areas. . . . TIS+ missed opportunities to positively influence citizens’ perception of confidence in government through strategic communications.”17 6. TIS+ should continue to work to increase citizen confidence in district and state levels and not focus on the federal level. Future stabilization interventions should target making an impact at these levels of government. TIS+ does support federal level coordination efforts; however, TCs’ work at the federal level is not as visible. TIS+ and future stabilization programs should focus on enhancing the federal government’s coordination capacity and not seek to work directly on interventions at the federal level. Citizens have much more direct interaction with district and state government officials and see these levels as the primary provider of services as opposed to the federal government. This recommendation is linked to the conclusion for Q1E: Greater Citizen Confidence in Government: “Based on the evidence, TIS+ did not make a unique contribution to greater citizen confidence given similar trends in non-implementation areas. . . . TIS+ work is more likely to lead to increases in citizen confidence at district and state levels than work done at the federal level. Future stabilization interventions should target making an impact at these levels of government. At the federal level, TIS+ and future stabilization programs should focus on enhancing the federal government’s coordination capacity. These interventions would help influence more directly evident increases in confidence in government.” 7. TIS+ should continue to explore ways to improve its GESI efforts by going beyond simply ensuring increased participation of women, youth, and marginalized groups in its activities. 16 In a July 2016 report titled “Strategic Communications in Conflict and Stabilization Interventions,” the UK Stabilization Unit notes that “strategic communications should be fully integrated into policy making from the earliest stages . . . with the communicators fully engaged in decision-making at the most senior level.” This is not possible with the current TIS+ model where strategic communications is dealt with a TC embedded in a Ministry as opposed to a senior staff member who links directly with USAID on a frequent basis. 17 TIS+ notes that, although it does not have a staff person who specializes in strategic communications, an entire PIKA (SOM008 Strategic Communications Technical Coordinator) is devoted to supporting the implementation of the TIS+ Strategic Communications Plan. 52 Instead, it should ensure more meaningful participation of these groups in the selection and oversight of projects. Often, these groups have shared constraints including limited access to decision￾making platforms, economic exclusion, and lack of capacity. TIS+ should design a targeted training program to ensure meaningful and regular participation for youth and women in POCs. The entrenched status quo favoring elders and male members of the society means that even when women and youth are included in these groups, they often lack the capacity to actively participate. USAID and TIS+ should strengthen M&E efforts to better track minority clan and other disadvantaged groups’ engagement. This recommendation is linked to the conclusions for Q2F. TIS+’s Inclusion Approach: “1. TIS+ has taken a number of steps to increase the participation of women …. However, evidence from the midline also suggests that there is a need for additional capacity-building of POC members, particularly youth and women;” and “2. There was a lack of systematic monitoring of minority clan involvement, or at least documentation of such monitoring in TIS+ interventions.” 8. TIS+ should ensure regular training forums for women, youth and minority clan members working in project oversight committees (POCs) through a program that is implemented with fidelity. Increases in participation are distinctly linked to citizens feeling more competent to engage in decision-making and management and oversight. Long-term community-driven development approaches are familiar in these clusters, but the quality of participation is still a challenge for the TIS+ project. Increased emphasis on training will capacitate community members to engage more meaningfully in the oversight and management of shared assets and lead to higher-quality citizen and government interaction. This recommendation is linked to the conclusions for Q2F. TIS+’s Inclusion Approach - “1. TIS+ has taken a number of steps to increase the participation of women. . . . However, evidence from the midline also suggests that there is a need for additional capacity-building of POC members, particularly youth and women;” and “2. There was a lack of systematic monitoring of minority clan involvement, or at least documentation of such monitoring in TIS+ interventions.” 9. TIS+ needs to plan targeted outreach and campaign events that encourage women and youth participation in SPD events while discouraging stereotyping associated with the participation of women and girls. These events should be more intentional in their design to be fully inclusive of women, girls and minority clan members, as well as to avoid potential perverse outcomes that may lead to participants being targeted. TIS+ should adopt a conflict-sensitivity lens in its sports interventions. As much as bringing people together in events and tournaments could support a sense of shared identity, it is important to understand existing conflict fault lines and whether sports events could trigger them.18 This recommendation is linked to the conclusions for Q2G. TIS+’s Sports for Development and Peace (SDP) Approach: “Any potential claim for TIS+’s SDP activities to have influenced higher-order outcomes related to community cohesion or positive youth development to date are not well substantiated by current evidence. Youth felt marginalized and underutilized.” 10. TIS+ and USAID should explore opportunities to better utilize TCs to encourage the participation of various levels of government in shared asset management, as well as ensure enhanced coordination and information exchange. TIS+ should expand the scope of the TCs’ work to effectively support the government’s role as a facilitator in service provision. TCs can do this by improving the government’s ability to set regulations and standards for private and NGO service providers, which will enhance the role of the government in service provision and shared asset management. TCs should have defined contract periods. Rather than long-term engagement, TCs would be more effective as Short Term 18 There is little evidence to date of TIS+ triggering such fault lines. 53 Technical Assistance (STTA) professionals to help state and local administrations build systems and policies. TCs can be “stabilization champions” in areas where government authority is being devolved to and citizens who have been under AS rule are interacting with government for the first time. STTA provides more flexibility in light of the funding uncertainties for the long-term TCs. STTA also mitigates the risk of long￾term elite capture as highlighted in our TC CA story. This recommendation is linked to the conclusions for Q2C. TIS+’s TC Approach: “The Technical Coordinators (TCs) were not well embedded into intervention planning. There was evidence of impact at the state level, but less evidence at the federal level.” 11. USAID should endeavor to ensure that partners better understand the unique dynamics of countering violent extremism in Somalia and utilize cross-cutting interventions across all its programs to address the localized effects of violent extremism on communities and within specific minority groups. The broad portfolio of USAID programming in Somalia (not just TIS+) should focus on a more coordinated approach to violent extremism and its prevention. An intervention designed around a single program such as TIS+ will not have attributable results as evidenced by this assessment but increasing coordination between implementing partners working in Somalia, particularly in AS-controlled areas, will greatly enhance impact and effects. This recommendation is linked to the conclusion for Q1F: Reduced Support for Violent Extremism: “TIS+ did not contribute to reduced support for violent extremism across cluster. However, potential results related to countering violent extremism is at the far end of TIS+ intended results chain.” 54 IX. CONTRIBUTION CLAIMS AND TIS+ The following contribution stories will critically examine the extent to which TIS+ has contributed to improvements in anticipated higher-level effects. Each story is structured according to the five criteria identified by John Mayne, the originator of the Contribution Analysis methodology, to analyze the extent of a contribution claim (or plausible association) between TIS+ activities and the results witnessed during the midline assessment. These criteria include the following: 1. Plausibility: Is the theory of change plausible? 2. Implementation per plan: Has the program been implemented with high fidelity? 3. Evidentiary confirmation of key elements: To what extent are the key elements of the ToC confirmed by new or existing evidence? 4. Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? 5. Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? The contribution story will then conclude with an overarching assessment of the strength of TIS+’s contribution claim to the results relevant to the identified results chain area in the midline assessment. CONTRIBUTION STORY 1: COMMUNITY PLANNING AND OVERSIGHT TIS+ differs from other more traditional infrastructure programs. While the program aims to identify and address priority community needs through the delivery of quick-impact projects, most of which are infrastructure￾focused, the process for how TIS+ engages community and government actors to plan, implement, oversee, and utilize these activities is what distinguishes it as a stabilization program. Simply put, the “how” (processes used to plan, implement, and oversee activities) is more important than the “what” (specific activities supported in each settlement) for achieving TIS+’s intended higher-level results. These include increased citizen engagement and confidence in government, and reduced conflict over resources and support for violent extremism. PLAUSIBILITY OF THE TIS+ CPO THEORY OF CHANGE The initial step in building a contribution story involves identifying an intervention’s intended Theory of Change and then assessing its plausibility. The majority of the main elements behind TIS+’s ToC for community planning and oversight are derived from the concept of Community-Driven Development (CDD), an approach to supporting international development projects systematized by the World Bank during the 1990s but utilized and refined by a wide range of other international financial institutions and development partners since then. At its core, CDD focuses on establishing partnerships between communities and local government units to improve the transparency, inclusivity, accountability, sustainability, and local capacity to provide basic services and manage shared assets. According to the World Bank, “CDD approaches are particularly prominent in conflict and fragile situations—CDD programs operate in 22 countries on the list of fragile and conflict-affected situations, and an additional seven countries with internally displaced populations, refugees, or conflict zones.”19 TIS+’s CPO Approach TIS+’s approach to CPO is articulated in a number of guiding program documents and throughout its periodic reports. As explained in its “Process for Engagement,” one of the program’s foundational design documents, “The TIS+ model is based strongly on a community/government programming approach. TIS+ activities will work to foster the relationship between community members and government representatives and supporting Somali leadership in those processes. Particularly, supporting Somali self-efficacy in governance is a priority, which supports USAID’s goals in Somalia and New Deal Principles. TIS+ activities will contribute to improvements in 19 The World Bank Group, “Community Driven Development,” https://www.worldbank.org/en/topic/communitydrivendevelopment#2. 55 government service delivery, leading to strengthened governance and community cohesion. Communities will be supported with access to livelihoods, income generation, and basic community infrastructure.”20 According to the “Community Contracting Guide,” a document jointly developed by TIS+ and the Ministry of Interior, Federal Affairs, and Reconciliation (MOIFAR), TIS+’s CPO approach is intended to complement FGS’s ongoing District Council Formation process and emphasizes the “leadership role of the inclusive community, including women, youth, traditional authorities, and minority clans.”21 Communities are expected to collectively prioritize their needs through consensus planning and then undertake a “more detailed planning to directly participate in the implementation and long-term management of the project, as well as to help manage resources with appropriate participation and oversight from the local authorities.”22 The guide posits that “as a result of this process, and with appropriate support and training, the communities gain confidence, and organizational and negotiation skills, enabling them to engage with external partners and the local authorities on an equal footing. The direct involvement of the community leaders, with local, state and federal government counterparts and [TIS+], is intended to build confidence and accountability to address nation-wide stabilization concerns and enhance local ownership of projects and processes. As stabilization projects are successfully implemented in a transparent and accountable process, trust is built between the community and all the three levels of government . . . to deliver basic services and to construct local government infrastructure, in a more participatory and accountable manner.”23 According to the “Consensus Process Guide,” a proper CPO process can also help mitigate and reduce conflict over resources. The guide outlines a 10-step process for implementing the initial community consensus process, which includes initial steps working with communities to conduct conflict and resource maps and introduce the concepts of “Conflict Sensitivity” and “Do No Harm.” The guide then instructs facilitators on how to support communities to identify, prioritize, and reach consensus on activities, form Project Oversight Committees, and how to conduct strategic communication and dialogue events.24 Project Oversight Committees (POCs) A critical feature of the CPO process is the formation of POCs, introduced under TIS+’s predecessor program, “Transition Initiatives for Stabilization” (TIS). As explained in TIS+’s Operational and Management Guide (OM Guide), “A [POC] is a community-based entity created for the Transition Initiative for Stabilization Initiative (TIS+) Program. The POC is a Joint Government–Community Implementation Unit created for a project location to oversee the project delivery from start to finish. The POC works with other stakeholders, including other international organizations, which serve to strengthen government and community interaction. The POC acts as a monitoring and accountability mechanism while mobilizing the community and increasing its awareness of the project for joint implementation. In addition, POCs add an element of sustainability by providing an opportunity for communities to take ownership of project maintenance through the formation of a lasting relationship with state and federal government officials.”25 According to the OM Guide, POCs are instrumental for providing the following: ● A Mechanism for Community Mobilization – allowing “for a range of interests to be addressed and a diversity of stakeholders to have input into a project, which creates a stronger and more cohesive community identity . . . By coalescing local community voices around a project, the new community linkages may remain after the project is finished. . . . POCs serve as an access point between the community and government, 20 AECOM, “Transition Initiatives for Stabilization Plus (TIS+) Process of Engagement,” (March 2016). 21 MOIFAR, “Community Contracting Guide,” (2018). 22 Ibid. 23 Ibid. 24 Federal Government of Somalia Ministry of Interior and Federal Affairs, “Consensus Process Guidebook.” 25 AECOM, “General Principles of the O&M Plans for TIS+ Supported Infrastructure Activities in Somalia,” (February 2018). 56 allowing communities to better respond to issues affecting the community as a whole and in the future. POCs also help build social capital that will enable a community to develop better public services and/or resist negative influence.” ● Monitoring and Accountability – “POCs serve as a monitoring body, verifying that the work on a project is completed, according to basic specifications. . . . POCs lay the foundation for the real-time monitoring and community’s accountability of the tender work . . . since the community members have constant access to the project site, while government officials and TIS+ representatives are usually based elsewhere, community￾based members provide an invaluable first-line monitoring capability.” ● Facilitating Government–Community Interaction – “By providing a forum for community and government representatives to interact, TIS+ ensures that there is regular communication during project implementation and, hopefully, after project completion. POC meetings encourage active government involvement in project implementation and increase visibility among key community stakeholders. In addition, for the government to receive adequate recognition for its involvement, it is the responsibility of the POC to actively communicate project developments to the rest of the community.” ● Improving Project Implementation – “POCs help TIS+ implement quicker and better projects. POCs resolve issues that pose obstacles to the swift completion of implementation.” ● Operations of the Infrastructure – POCs should help lead the running of facilities and should include “a representative from the District Administration, for a better liaison and understanding.” ● Maintenance – “The POC is the main mechanism to ensure the maintenance as in most cases there is an evolving governance structure and it requires the ownership, legitimacy, and support from the communities. . . . The POC provides that platform to be utilized, constituted and supported, in terms of the capacity to be the main institution providing the maintenance service after the project completion.” The centrality of the POCs as the key nodal point in the CPO process is aptly summarized in Figure 5, which also provides a useful summary of TIS+’s overarching CPO process. Figure 5: Overview of TIS+’s CPO Process 57 Training and Capacity-Building of POCs Given the importance of POCs for ensuring the quality of community participation and oversight, the OM Guide explains, “It is important for TIS+ to carry out the aforementioned trainings, before the groundbreaking of the project to enable, and encourage, the POC to play its vital role in an effective way. During these training sessions, identifying the specific focal persons for the technical and operational/financial aspects of the O&M will establish the needed link between the investment and its sustainability right from the start” (emphasis added).26 The OM Guide further outlines five specific areas where TIS+ will aim to build the capacity of POCs (see Figure 6). Figure 6: Areas for TIS+ Capacity-Building of POCs No Training Support Required Proposed Steps 1. Training on their roles and responsibility (training on how to conduct POC meeting, take minutes, and keep records, including notes on decisions made, schedule of meetings, and overall agreements/outcomes). Provide training and facilitate sessions to build the capacity of the POC to understand and meet the demands of the role (including record keeping). 2. Increase POC knowledge of Public Finance Management concepts to support fundraising efforts and ensure that community contribution is used for the betterment and sustainability of projects. For example, funding to support the implementation of Operations and Maintenance (O&M) of the projects, post￾completion. Train POCs on the financing modalities and resource mobilization aspects for project sustainability and accountability. 3. Ensure POCs understand Do No Harm principles. POC should avoid assuming a parallel role to the local administration, but rather limit themselves to their roles on the project to support the local governance structure. Following and engage Do No Harm principles with POC, throughout project implementation phases, to curb community conflict that may arise during and after the project handover. 4. Train POCs on legitimate government documentation needed for the community contracting processes. This will enable POCs to play an active custodianship role during the community contracting process. Train POCs on correct and legitimate documentation requirements, particularly for construction awards (e.g., local administration, Ministry of Public Works, or federal government certificates). This will prepare POCs to be watchdogs during the implementation process. 5. Train POCs on the respective O&M arrangements for each project. This shall include two key components: the community– government partnership for the resource mobilization required for the maintenance and the technical O&M part. Before the project is handed over, it is important for TIS+ to establish this important liaison role and working partnership between the local administration and the POC, preferably in writing. This will help with long-term sustainability and support long-term O&M prospects. The above capacity-building plan directly responds to one of the key recommendations from the final evaluation of TIS, which found that POCs play a vital role in ensuring community participation and oversight. Specifically, the evaluation recommended that “the POCs’ role at the community level should be strengthened and institutionalized. POCs should be provided with adequate training to allow adequate monitoring of projects. The contractor and the community must properly understand the POC’s role.”27 THEORY OF CHANGE Building on the elements described above, the midline assessment team conducted a one-day workshop in June 2017 with TIS+ field and headquarters staff as well as the USAID/Somalia lead for the TIS+ program to clearly articulate the intended results, causal logic, and underlying assumptions guiding TIS+’s overarching activities, but also those specifically focused on CPO. 26 Ibid. Pg. 8. 27 IBTCI, “Final Performance Evaluation of the “Transition Initiatives for Stabilization” (TIS) Project,” ed. Somalia Program Support Services (June 2016). 58 Figure 7 below outlines that Theory of Change with the light blue boxes depicting intended results and the grey boxes highlighting some of the underlying assumptions that need to be fulfilled to progress along the causal logic. 59 Figure 7: Theory of Change for TIS+’s Community Planning and Oversight 60 Assessment of Plausibility: Moderate While TIS+’s CPO approach is informed by decades of similar CDD work implemented by the World Bank and a wide range of other development partners, a major recent study by the International Initiative for Impact Evaluation (3ie) questions many of the principles behind the CDD approach and the extent to which it leads to tangible results. The study, a meta-analysis of 25 impact evaluations covering 23 programs across 21 low- and middle-income countries, finds that while “CDD programs have made a substantial contribution to improving the quantity of small-scale infrastructure,” they achieve few of their intended higher-level outcomes, with the exception of results around improved water supply.”28 Furthermore, and particularly problematic for TIS+, “CDD programs have little or no impact on social cohesion and governance. This synthesis study shows that the assumption that the entire community participates in the program is not valid. Data show that participation in decision-making is limited to a small number of community members. There is a clear ‘funnel of attrition’; many people may be aware of the program and the community meeting, but few participate in the meeting and fewer still speak or participate in decision￾making. People participated in making bricks, not decisions.”29 As expected, the meta-analysis study has set off a fierce debate within the development community. The report has received both widespread praise and criticism, leading the authors to publish a series of clarifications of their original findings. The authors readily acknowledge the success of CDD approaches for delivering small-scale infrastructure and the immediate benefits that these structures bring to those community members who use them. However, the authors restate that there is limited evidence of “higher-order social outcomes,” particularly for social cohesion and governance. “Our review finds that CDD has no impact on social cohesion. There is no heterogeneity there. The lack of effect on social cohesion is consistent across contexts. It is in building social cohesion that CDD has not worked. This is where meta-analysis is so useful, as it clearly illustrates the consistency of this finding. . . . As we say in the report, the lack of impact on social cohesion is not a new finding. Indeed, one of us was a co-author of the 2002 World Bank review of social funds – including the CDD-like Malawi and Zambia funds – which reported no impact on social capital, as did the CDD report three years later. The review confirms this finding now that we have additional evidence from high-quality impact evaluations.”30 In sum, while the anticipated results, underlying assumptions, and causal logic behind TIS+’s CPO Theory of Change follows decades of conventional thinking around CDD and is logically plausible, evidence amassed from impact evaluations conducted over the past decade indicates that the programming logic is fundamentally flawed and that CDD does not contribute to improved community cohesion or local governance. That said, findings from 3ie’s synthesis study is passionately contested and future parallel studies might in all likelihood find contrary evidence supporting the use of CDD approaches for small- 28 Howard White, “Community-Driven Development: Does It Build Social Cohesion or Infrastructure? A Mixed-Method Evidence Synthesis,” International Initiative for Impact Evaluation (March 2018). 29 Ibid. 30 Howard White and Radhika Menon, “Community Driven Development: Howard White and Radhika Menon Respond to Scott Guggenheim,” Oxfam (July 2018). “CDD programs may be using social cohesion rather than building it. Numerous factors may affect community involvement, such as the role played by the elite or prime movers in the community, intra-community divisions and the perceived benefits of participation. CDD’s impact on governance is sometimes undermined by the creation of parallel structures for the sake of the program. The function of these governance structures is not clear once the community projects end.” ~Howard White et al. “Community Driven Development: Does It Build Social Cohesion or Infrastructure? A Mixed-Method Evidence Synthesis.” 61 scale community infrastructure projects. Therefore, the team concluded that the plausibility of TIS+ CPO Theory of Change is, at present, uncertain. FIDELITY OF IMPLEMENTATION The next step in developing the contribution story involves assessing the extent to which implementation actually followed the intended design of the Theory of Change discussed above. Assessment of Fidelity: Moderate. The midline assessment found clear evidence that TIS+ took active steps to ensure that communities in both Banadir and Gedo/Bay were supported to collectively identify and prioritize their shared community needs. There was documentary evidence that TIS+ held a three-day “Bridging Plan Stakeholders Consultation” workshop with over 60 participants in Banadir. In Gedo/Bay, TIS+ held a three-day “District Consensus Building” workshop in Baardheere on May 2–4, 2016, involving 42 participants31 and a similar three-day session in Dinsoor on June 6–8, 2016, involving 43 participants of diverse backgrounds.32 Interviews further confirmed that the workshops followed the 10-step process outlined in the “Community Process Guide.” Interviews with TIS+ staff indicate that district-level POC members were identified during the last day of each workshop and the agendas from the workshops confirm that sessions were indeed planned to identify these members, although the assessment team could not independently verify that these sessions actually took place (it should be noted that the team has no reason to suspect that they did not). However, interviews with the same TIS+ staff members also revealed that very limited training was provided to the POC members during these workshops and that they mostly just discussed the anticipated role of the POCs as they looked to recruit volunteers. Based on both document review and interviews with TIS+ staff, substantive training of the POCs did not take place until nearly 24 months after the establishment and the subsequent reconstitution of the POCs. While TIS+ had originally intended to form POCs at the district level, the program decided in late 2018 that this approach might lead to a concentration of power in the hands of a limited group of people and thereby present opportunities for corruption and that it would be more prudent to establish PIKA-level POCs. Accordingly, PIKA-level POCs were established in November 2018 in Banadir and Baardheere and January 2019 in Dinsoor. This meant that all of the new POCs established in both Banadir and Gedo/Bay were established after the start of construction and even after the completion of construction, except for the POC in Baardheere, which was established just two months before the completion of construction. The evidence, therefore, suggests that the POCs had very little formal engagement in the activities in their intended capacities and did not have the opportunity to fulfill their intended role of providing community oversight and regular engagement with local authorities. While the capacity-building and function of the POCs deviated considerably from the original design, both interviews and the assessment team’s review of sign-in sheets confirm that the POC membership looks to be inclusive of both women and youth, and in most cases included the participation of government officials and representatives from the business community. Interviews also confirmed that although the community contracting was not led by the POCs as intended, community contracting did occur in each of the Wave I and II areas. 31 Participants included stakeholders from the Jubaland State of Somalia, Somali Federal Government, the Local Administration, youth, women, elders, traditional leaders, religious groups, and the local private sector. 32 Participants included representatives from the interim South West Administration, the Somali Federal Government, the Local Administration, youth, women, elders, traditional leaders, religious groups, and the local private sector. 62 In sum, TIS+ seems to have followed most of its intended CPO steps but substantially deviated from the original design when it came to the establishment, training, and use of the POCs. While it is true that POCs form only part of the overarching CPO process, it is nevertheless a crucial part to ensure regular dialogue with authorities, community monitoring and oversight, and for the building of a sense of ownership and responsibility for the sustainability of the activities within the community. EVIDENCE SUPPORTS INTENDED RESULTS Having outlined the intended Theory of Change and extent to which implementation followed that intended design, the next step in building the contribution story is to examine the extent to which key elements of the Theory of Change are confirmed by evidence. Assessment of Supporting Evidence: Moderate As could be expected given the fluid and unique contexts of individual settlements across Somalia, there was a great deal of variation in results across the areas. As demonstrated below, there were inconsistent results both within and across TIS+ clusters reviewed during Wave I and II. The evidence was supportive of some intended results in some settlements—e.g., community participation and awareness in Banadir— but then contradictory in others—e.g., participation in Dinsoor and awareness in Baardheere. Thus, while there was supportive evidence of TIS+’s CPO approach achieving intended results in a specific settlement, the results were too diverse to draw any overarching conclusions on the effectiveness of TIS’s CPO approach across multiple clusters and diverse contexts. The three TIS+ settlements in Banadir demonstrated particularly strong results on “community awareness and participation.” The midline assessment found an increase of 58 percent in respondents indicating that they were aware of community planning events and a significant increase in participation by all groups with the exception of clan elders. Likewise, TIS+ settlements demonstrate a very strong level of satisfaction with projects meeting community needs with an impressive 92 percent of respondents reporting that “some” or “all” selected projects addressed community priorities. The midline also identified a significant increase in awareness of oversight committees in both TIS+ settlements (+46 percent); however, notably, there was no change in the perceived effectiveness of those POCs. This evidence would seem to support the finding in the “Fidelity of Implementation” section that the POCs did not perform as had been originally anticipated. There were also fairly positive results in terms of the community interaction and performance of local and state government in Banadir. Encouragingly, the midline saw a strong increase in the reported roles of state and local governments in managing shared assets, particularly in the management of roads and sports arenas, which were PIKAs. There was also a substantial increase in perception of local government providing services; however, most of these services were not related to the activities supported by TIS+. On the less positive side, the perception of services provided by state government decreased across most areas and there was no change in the reported citizen satisfaction with those services. Likewise, there were no notable changes in the perception of decreased conflict over resources, and perceptions of corruption actually increased toward both the federal and local government. In terms of inclusion, the midline assessment confirmed that all TIS+ POCs included the participation of women and youth, with an average membership rate of 40 percent for women and 30 percent for youth. The results were considerably more varied, and often less positive, in TIS+ settlements in Gedo/Bay. While the participation of different stakeholder groups in community consultations generally increased in both TIS+ and non-TIS+ areas across the cluster, they decreased considerably in Dinsoor. However, despite reporting decreased participation across groups, respondents showed an increase in the 63 perception that projects were selected based on community planning events. Paradoxically, Baardheere demonstrated the exact inverse, reporting an increase in the participation rates across groups but a decrease in the perception of projects being selected based on community planning events. Troublingly, there was a decrease in both Dinsoor and Baardheere in respondents reporting that “most” or “all” of the selected projects effectively addressed community priorities, as well as a substantial decrease in the awareness of POCs and no change in their perceived effectiveness. Likewise, the results were not generally very positive related to shared asset management and quality of services. The midline showed a perceived decrease in the role of state and local government in the management of nearly all assets, including in roads and meetinghouses, which were key PIKA activities. In terms of service provision, INGOs are seen as having an increased role in the provision of nearly all types of services across the cluster, and the role of both state and local government in the provision of roads, key PIKA activities in both areas, decreased in both Dinsoor and Baardheere. Finally, while most settlements across the cluster reported an increase in citizen satisfaction levels with services, Baardheere saw a decrease and the TIS+ constructed road and airstrip were specifically identified as being of poor quality. On the positive side, Dinsoor was noted as a unique “positive deviant” and witnessed a substantial reduction in resource-based conflicts where other settlements across the cluster saw increases. Likewise, both Dinsoor and Baardheere reported reduced levels of perceived corruption by both the federal and local government. Gedo/Bay reported similar results as Banadir in terms of inclusion with women and youth being present on all TIS+ POCs and accounting for roughly 30 percent of the membership for each group. INFLUENCING FACTORS AND ALTERNATIVE EXPLANATIONS The final step in building the contribution story then is to analyze the extent to which “other influencing factors” and “alternative explanations” explain the identified results. Assessment of Influence of Alternative Explanations: High. As demonstrated by the inconsistency in the results highlighted above, a number of other influencing factors and alternative explanations account for the variety of results, including the following: ● Activities of Other Development Partners. There was a wide and diverse set of groups of actors implementing activities in each of the TIS+ settlements reviewed during Wave I and II, most of whom reported utilizing some aspect of community engagement, planning, and oversight. The assessment team identified at least seven different organizations working in Baardheere, 11 organizations working in Dinsoor, and over 20 different organizations working in Banadir. Both KIIs and FGDs conducted during the midline indicate that there has been a marked increase from these development partners to more systematically engagement with communities at the outset and involve community members in the planning and design of activities. Likewise, other major partners, such as Danish Refugee Council, Finnish Church Aid, Nordic International Support Foundation (NISF), and International Committee of the Red Cross, as well as UN agencies such as the Food and Agriculture Organization and World Health Organization report specifically using POCs or established community structures such as the Peacebuilding Committees to guide the planning and oversight of their committees. ● Carryover Effects from TIS: The strong performance of Banadir on results related to community awareness and participation is in large part likely explained by strong carryover effects from the activities of TIS. Mogadishu was one of the first areas in Somalia to pilot CPO approaches during the initial TIS program and TIS conducted numerous community consensus planning sessions and trauma healing sessions, as well as investments in infrastructure including rehabilitation of district administration buildings and sports facilities during its program life. The TIS final evaluation concluded that these activities led to notable improvements in both community awareness and participation. Specifically, the evaluation found that “TIS’s innovative essence was the consultative and participatory 64 process that brought communities (civil society and business people), local administration, and central government together to discuss and identify the community’s priority needs;” “TIS proved to be a strong catalyst in encouraging greater engagement between communities;” and that “TIS contributed significantly toward increasing collaboration between local government and stakeholders.”33 ● Unique Characteristics of Settlements. Qualitative interviews (both KII and FGD) clearly conveyed that the unique historical, socioeconomic, and geopolitical characteristics of each settlement played a large role in the results seen during the midline. For example, interviewees identified the unique characteristics of the three TIS+ settlements in Banadir to explain the high levels of both government and development partner interest in supporting activities in these areas. They explain that these districts are closer to the city center and are key revenue drivers for the local and state government, accounting for up to 70 percent of revenue generation by some estimates. Likewise, clan dynamics and the influence of powerful individuals were identified as reasons for the variations between community awareness and participation in both Dinsoor and Baardheere. ● Historical and Contextual Factors. Interviewees also explained that the building of trust between communities and respective levels of government following decades of conflict is a long-term endeavor and requires generational change. Interviewees explained that while TIS+ activities might facilitate community and government interaction on the planning and oversight of specific shared assets, there is a long list of grievances and frustrations that still need to be overcome. These include factors such as widespread perceptions of nepotism and clannism, unfulfilled election promises, corruption, security lapses coupled with a fear of victimization by AS, injustices and a weak judiciary, poor service delivery, and the outsized influence of diasporas. ASSESSMENT OF CONTRIBUTION CLAIM Summary of Assessment Criteria Criteria Assessment Plausibility: Is the theory of change plausible? Moderate. The anticipated results, underlying assumptions, and causal logic behind TIS+’s CPO Theory of Change follows decades of conventional thinking around CDD and is logically plausible; however, recent evidence from a meta-analysis of 25 impact evaluations suggests that this programming logic is fundamentally flawed and that CDD does not contribute to improved community cohesion or local governance. Implementation per plan: Has the program been implemented with high fidelity? Moderate. TIS+ followed most of its intended CPO steps but deviated substantially from its original design when it came to the establishment, training, and use of the POCs. While it is true that POCs form only part of the overarching CPO process, it is nevertheless a crucial part to ensure regular dialogue with authorities, community monitoring and oversight, and for the building of a sense of ownership and responsibility for the sustainability of the activities within the community. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? Moderate. Inconsistency in results across locations means that cross￾cluster conclusions on the effectiveness of TIS+’s approach cannot be determined. Identification and examination of other influencing factors: To what extent have other influencing factors been identified? Identification of alternative explanations: To what extent do alternative explanations explain results? High. As demonstrated by the inconsistency in the results, a number of other influencing factors and alternative explanations contributed to the results seen during the midline. These factors include activities of other development partners, carryover effects from TIS, unique characteristics of settlements, and historical and contextual factors. 33 IBTCI. “Final Evaluation of ‘Transition Initiatives for Stabilization’ (TIS)”. 65 SUMMARY ASSESSMENT OF CONTRIBUTION CLAIM: WEAK. At present, there is limited evidence that TIS+’s Community Planning and Oversight approach is consistently contributing to higher-order, outcome-level results related to improved citizen engagement and confidence in government, as well as reduced conflict over resources or support for violent extremism. While the midline assessment found evidence of TIS+ achieving notable results in terms of community awareness and participation, as well as local and state government service provision and management of shared assets, these results were usually only at the individual settlement level and were not consistent across clusters. TIS+’s contribution claim is also limited by the lack of a consistent implementation approach across the program. While the community consensus workshops look to have been consistently implemented, the form and timing of other key parts of the CPO process varied across settlements. This includes a difference in the way that community contracting took place across the settlements, and most notably, significant deviations on how POCs were formed, trained, and utilized throughout the process. TIS+’s own program design documents clearly highlight the central role that POCs play in the overarching CPO process; however, evidence indicates that TIS+ did not prioritize their role until late in the PIKA process, almost always only after the construction of the different structures was already complete. 66 CONTRIBUTION STORY 2: TECHNICAL COORDINATORS AND TIS+ TIS+ is a complex program with different interventions that allow it to adopt a flexible approach to programming in Somalia. In 2017, the TIS+ program and assessment teams developed a program-wide Theory of Change (ToC). Nested within this larger ToC were three sub-ToCs that specifically focused on key approaches utilized by the program to achieve stabilization results. One of these approaches was the use of Technical Coordinators (TCs) embedded within host-country government ministries at the federal and state level. This contribution story will focus on critically examining the role of the TCs and any influence they may have had on the achievement of higher-level stabilization, governance, and community cohesion results. In the original TIS+ documents on coordination and engagement, as TIS+-funded positions embedded within ministries, TIS+ TCs were designed to support TIS+ Objectives 1, 2, and 3. These were explicitly seen as roles at least partly designed to enhance the efficacy of TIS+ interventions, for example, adding value to the projects delivered under our “core” contribution story on community planning and oversight of infrastructure. The TCs were also intended to deliver wider work in supporting communication between levels of government more broadly. These two functions are summarized in the TIS+ Approach to Engagement document from 2016: “To support continued coordination and collaboration of TIS+ plans and activities with federal and state government plans, TIS+-funded Technical Coordinators are proposed to sit within government ministries. These Technical Coordinators will facilitate not only planning coordination but will support better communication across levels of government and with other stabilization programs in support of more effective stabilization activities.”34 In assessing the TC contribution claim, the assessment team will therefore be looking for evidence that TCs have demonstrably improved TIS+ or other interventions, and that they have improved communications between levels of government, as an early step in the results chain, and for evidence that they have contributed to results under the three TIS+ objectives below: 1. Increase confidence in governance based on equitable participation in decision-making and management of community assets. 2. Empower community and government representatives to engage with the private sector and development actors in a collaborative process for community growth. 3. Increase Somali engagement in creating a more stable future (federal-level ministry TCs only). The contribution story is structured according to the five criteria to analyze the extent of the contribution claim between the TIS+ activities and results witnessed during the midline assessment. PLAUSIBILITY OF THE TIS+ TCS THEORY OF CHANGE The practice of embedding TCs and advisors in ministries and government agencies has been long utilized in Somalia. Against the background of nascent institutions and limited capacity in the civil service, it is seen as an effective way of facilitating donor engagement with government institutions while also building the capacity of the local staff in those institutions. This approach has not been without challenges, which have forced a rethink of the interventions to make them more effective. 34 TIS+ (March 2016) “TIS+ Process of Engagement,” p. 4. 67 United Nations Development Programme (UNDP) Somalia35 identified the following obstacles: ● Radically weakened institutions with little legitimacy ● Dramatic variation in the political landscape from region to region with shifting power and allegiances over time ● Security threats ● The challenge of maintaining neutrality in a complex situation The approaches to technical assistance delivered in Somalia has varied from utilizing diaspora experts [International Organization for Migration (IOM)/United Nations Development Programme (UNDP)’s “Migration for Development in Africa” (MIDA)], project implementation units (Aid Coordination Units), Technical Advisors/Coordinators (UN, Care, Concern International, Deutsche Gesellschaft für Internationale Zusammenarbeit [GIZ], USAID) and policy development. TIS+ approach is in line with this thinking to provide embedded capacity within the government ministries at the state and federal level. TIS+ APPROACH Across TIS+’s three objectives, TCs are expected to do the following: ● Support coordination and collaboration of TIS+ activities with government activities. ● Support better communication with government at all levels and with other stabilization programs. ● Increase knowledge and gain support for locally based Objective 3 activities at all levels of government. Figure 8 on the next page shows the causal steps through which the TCs were expected to contribute to TIS+ objectives and the assumptions on which these steps are based. 35 UNDP, “Supporting capacity development in conflict and fragile contexts,” https://www.undp.org/content/dam/undp/library/crisis%20prevention/2012SupportingCapacityDevelopmentinConflictFragileSetti ngs.pdf 68 Figure 8: Theory of Change (ToC) 69 Since its inception, TIS+ has provided 27 different TCs to different government institutions36 including the following: ● Banadir Regional Administration ● Ministries of Interior in Mogadishu, Baidoa, and Kismayo ● Ministries of Gender, Human Rights, and Family Affairs in Mogadishu and Kismayo ● Ministry of Public Works and Reconstruction ● Ministry of Youth and Sports ● Ministry of Information Some of these TC positions have already been scrapped either due to funding or strategic realignment of the program. The work and output of the TCs will be highlighted in the later section of this document. Assessment of Plausibility: Moderate. The approach of utilizing embedded experts in government institutions across Somalia is common. That said, there are challenges associated with implementing this sort of assistance that constrains the overall prospects for institution and state building. For example, an approach paper commissioned by the World Bank Group and authored by the Overseas Development Institute warns that “in fragile contexts when institutional capacity has been eroded, the strategic positioning and short-term calculations of leadership figures and elite groups carry disproportionate weight in shaping policy directions and outcomes in what are frequently fluid and rapidly changing circumstances.”37 In the absence of strong institutions, policy is often dictated by strong personalities and interests. The success of TCs placed in government institutions depends more on the personality and network of the individuals than on their skills. High turnover in these ministries due to dynamic political changes further affects the influence and progress made on delivering key interventions. Interviews with TIS+ TCs confirmed the challenge of dealing with high turnover often preceded by a political impasse, which often disrupts the project timelines. “We often deal with new Ministers who stay between 6 months and a 1 year and it takes time to bring them up to speed. Sometimes, these are political appointments and focusing too much on the work without cozying up politically can threaten the buy-in of a project. As a Technical Coordinator, you’re seen as having divided loyalties between the project and the institution.”38 There is also the challenge of sustaining positive results beyond the short-term and managing expectations in order to achieve long-term capacity and effective coordination as envisioned in the TIS+ Theory of Change. In interviews, TIS+ TCs highlighted that they constantly worried about their own tenure and funding priorities, which affected their ability to commit to long-term visions by the government institutions they were supporting. “As a Technical Coordinator, my biggest challenge is the uncertainty of the work. We have temporary contracts. Mine runs out next year in February. The ministry relies on us and we are developing plans going beyond our contract term, which is a bit difficult to do if you know you may not be there to implement them.”39 36 AECOM, “FY2018 TIS+ Annual Report,” (2018). 37 Overseas Development Institute, “An Approach Paper for Capacity Development in Fragile States,” (2009). 38 Interview with TIS+ Federal TC. (2019). 39 Interview with TIS+ South West TC. (2018). 70 The challenge of divided loyalties and inherent capacity substitution effects has been noted by other partners supporting technical assistance interventions. In its “Guidelines for Technical Assistance” document, the Danish International Development Agency notes that “program coordination and supervision must be vested in the partner institution. The partner institution may be supported in the overall planning, coordination, monitoring and reporting tasks related to program management by technical assistance on the basis of the partner organization’s own efforts to build up such systems.”40 Both the TIS+ program and TCs noted the challenge of subsuming the authority of the government institutions in some instances, with TCs acting as representatives of the government, rather than externally injected capacity. What is evident, however, is that the choice of institutions receiving the TIS+ technical assistance is in line with the priorities of the government. This was confirmed in government interviews, as well as a review of government strategies. For example, the FGS National Stabilization Strategy identifies the TIS+ supported Radio Wadajir platform, designed and produced by a TIS+ TC, as “the main strategic communication initiative supporting the stabilization strategy.” The validation in the government’s plans is an indicator for the expected buy-in and political will from the government for the work of the specific TC in this instance. “The Technical Coordinators are one of the main conduits through which the government engages TIS+. They provided much-needed capacity in the ministries. At the national level, the institutions are struggling due to lack of capacity. It is even worse at the state level because, for the first time in the history of the country, we are trying to have fully functioning ministries in the states. Previously, we had local administrations managed centrally from Mogadishu. To establish the functioning institutions in Kismayo, we need to make double the effort in a short time to make this a reality.”41 FIDELITY OF IMPLEMENTATION The below summary of some of the outputs by the TCs highlights their contribution: Jubaland State: At the Ministry of Interior and Security, the two TCs listed their deliverables as follows: ● Drafting the Jubaland Stabilization Plan ● Drafting the Ministry of Interior’s Briefing Note on Jubaland Public Expenditure Management Toolkit ● Organizing a Countering Violent Extremism (CVE) Workshop ● Drafting the Jubaland State Public Expenditure Management and planning the District Public Expenditure Management Cycle ● Organizing a training on the Public Expenditure Management Toolkit ● Supporting the Sanguuni District Community Consultation organized by SSI/OTI ● Delivering a Presentation on the Jubaland Stabilization Plan at the bimonthly Stabilization Meeting in Mogadishu ● Organizing the Jubaland Quarterly Stabilization Meeting ● Developing the organogram for the ministry ● Drafting the Local Government Financial Management Manual At the Ministry of Youth and Sports in Jubaland, the TC’s activities and deliverables were listed as follows: ● Developing the ministry’s Strategic Plan ● Supporting the development of the National Sports for Development Framework ● Supporting the Kismayo Bridging Plan Consultation Workshop 40 DANIDA, “Guidelines for Technical Assistance,” (2009). 41 Interview with former Director General, Ministry of Interior, Jubaland State of Somalia. (2018). 71 ● Presenting on the Sports for Development Framework at the Coordination Meeting in Mogadishu ● Supporting the Kismayo Book Fair ● Supporting the Jubaland Youth Conference in Doolow ● Organizing tournaments for Girls’ Basketball and Boys’ Soccer The TC at the Ministry of Women, Family Affairs, and Human Rights in Jubaland listed the following deliverables: ● Developing the ministry’s three-year strategic plan ● Contributing to the drafting of the Sexual Offenses Bill ● Drafting the Standard Operating Procedures for Prevention and Response to Gender-Based Violence ● Delivering a Training on Leadership and Management for Women Leaders ● Organizing a Workshop for Civic Engagement on Women’s Constitutional Rights ● Supporting the Opening Ceremony of the Garbahaarey Women’s Center The TC at the Ministry of Public Works and Housing was relatively new but is working on a capacity￾building plan for the ministry as well as a strategic plan in addition to supervising the construction of public buildings currently undergoing rehabilitation through the support of various donors. South West State: In Baidoa, the two TCs at the Ministry of Interior and Local Governance listed the following among their deliverables: ● Developing the Stabilization Strategic Plan for South West State ● Developing the Local Government’s Employment Regulations and the ministry’s organogram ● Developing the Terms of Reference for the ministry’s departments ● Developing the ministry’s Human Resources Policy ● Developing the Finance Policy ● Drafting the Local Government Financial Manual for Trainers ● Delivering Leadership Training in Hudur and Bardale districts ● Delivering Induction Training for the new District Council Members in Hudur At the federal level in Mogadishu, TCs contributed to the following: ● Developing the Wadajir Radio Program ● Developing and launching the Wadajir Framework for Local Governance ● Hosting the Mogadishu Book Forum ● Hosting the National Women’s Basketball Tournament ● Hosting the Gedo Reconciliation Meeting ● Supporting the District Council Formation Processes ● Developing the Marka Corridor Stabilization Plan In addition to the individual documentary output and coordination roles, TCs and government partners noted their contributions beyond the individual Terms of Reference. Noting the inherent low capacity in the institutions they were embedded in, TCs often played larger-than-life roles in the day-to-day running of the ministries. While this had a positive short-term impact on the capacity of these institutions, it presents the risk of capacity substitution and sustainability challenges in the long-term. This is particularly of concern for stabilization programs that have to deal with the frequent alignment of priorities because of changing dynamics. As mentioned above, TCs regularly highlighted their concerns over their tenure in the long-term: “When I arrived, there were about four staff members in the entire ministry, so the items on my job description are minuscule compared to the work I ended up doing. My formal title is Strategic 72 Governance Advisor and Technical Program Coordinator. My real work has ended up being the de facto Chief of Staff at the ministry because I facilitated discussions between the Minister and the international partners, and I also coordinated stabilization activities with partner government agencies.”42 Despite their contributions in the institutions they were embedded in, TCs expressed frustration that their strategic role was not adequately utilized by TIS+ and USAID. Technical Contributors expressed that they could play a much more pronounced role in TIS+’s decision-making on strategic interventions as well as in improving the relationship with government stakeholders. TCs noted that in a lot of instances, they were informed of decisions after they had already been made, particularly whenever the program made realignments which affected planned activities. Many of the TCs felt that their role was not well understand, citing that they were often asked during TIS+ strategic review sessions to clarify what exactly they did in the ministries and also added that the lack of an efficient feedback loop with the TIS+ program team sometimes led to missed opportunities, especially in situations where the governments’ unmet expectations had to be managed. In such instances, TCs felt that TIS+ could have relied on their understanding of the local dynamics and individual relationships to navigate these challenges. “There is still some work to be done on improving the relationship between the Nairobi office and the technical coordinators. Because of this, sometimes we do not have visibility on how the decision-making process takes place. We are at the ministries and have managed to build a solid relationship in the last two years and sometimes we feel there is lost opportunity to take advantage of the goodwill and sort out any potential misunderstanding or managing expectations.”43 It is important to note that some of these challenges and missed opportunities have been acknowledged by TIS+. One of the main outcomes from the 2018 annual Strategic Review Session (SRS) held by TIS+ was to “define the best possible way as a best practice—to engage the TCs and Advisors in the FGS and Emerging Federal States (EFS) ministries.”44 As part of the realignment, TIS+ has implemented systematic reporting from the TCs and incorporated their participation as part of TIS+ progress on improving links to the host government. TIS+ lists the contribution and participation of the TCs as a core approach to improving the program’s linkage with the government institutions it works with. Assessment of Fidelity of Implementation. Moderate. It is evident from the review of the program documents, the output from the TCs, and interviews with government partners that the work of the TCs contributed to both improved coordination between the government and TIS+ and increased capacity by the government to engage with stabilization actors and donors and better planning by the government. Evidence Support Intended Results. Moderate. The next step is to examine the extent to which key elements of the Theory of Change are confirmed by evidence. TCs were designed to improve existing TIS+ interventions, and to improve coordination in government in order to contribute to Program Objectives 1 and 2: 1. Increase confidence in governance based on equitable participation in decision-making and management of community assets. 2. Empower community and government representatives to engage with the private sector and development actors in a collaborative process for community growth. 42 Interview with TIS+ JSS TC. (2018). 43 Interview with TIS+ JSS TC. (2018). 44 AECOM, “FY2018 TIS+ Annual Report,” (2018). 73 Ultimately, the TCs are aiming to contribute, alongside other TIS+ interventions, to increased confidence in state government. The assessment team has a mixed picture of confidence in the government from qualitative and quantitative surveys. Moreover, the results chain connecting the TCs’ inputs to these results is so long and is decisively influenced by so many other variables that it would be unfair to expect any contribution at this level in any survey data. Our results chain for the TCs’ work has therefore concentrated on intermediate steps, such as improving TIS+ interventions, improving coordination, and, crucially, injecting sustainable capacities in the relevant ministries. The TCs themselves certainly confirmed that their activities had been focused both on supporting TIS+ interventions and on promoting coordination. This is supported by the evidence of their produced documents and interviews with government officials and other partners as well as the program documents. TCs have also worked on processes of vital importance to the state government, such as district council formation processes. There was evidence that government counterparts appreciated the work of TCs and saw them both as providing a point of contact with TIS+ programming, and promoting government ownership of TIS+ interventions, as well as building capacities in their ministries but respondents emphasized the importance of larger-scale institution building. “At [the] national level, the institutions are struggling due to lack of capacity. It is even worse at the state level because, for the first time in the history of the country, we are trying to have fully functioning ministries in the states. . . . The Technical Coordinators provide their own skills and experiences as required. In addition, they also train and build the capacity of the few available civil servants in the ministries.”45 Broader CA interviews revealed that many other programs that work with government generally heard good feedback on TIS+ TCs from government partners, and some who had encountered TCs in Kismayo or been involved in events organized by TCs in Jubaland were positive about these experiences. Some of these respondents did, however, suggest that following up on these events and deliverables was important. Others suggested that TIS+ could benefit from a more explicit focus on capacity-building in the work of its TCs. This was confirmed by other respondents who said TIS+ advisors had made a positive impact to the development of stabilization strategies, but this should be matched with an assessment of the real capacities but also motivations and leverage of the political actors in the ministries to implement these. An approach more focused on improving capacities beyond the lifetime of supported TC roles would likely include such an assessment. Other donors working on capacity injection in Somalia have noted that there has often been a failure of capacity injection interventions in Somalia to be tailored to specific needs, which are different in different regions.46 TCs themselves did feel that they were building capacities in their ministries but felt that the needs in this respect were very large: “We provide a lot of support especially in terms of capacity for the ministry because the skillset is not available because the South West State and the system of administration is less than five years old and when you think of a civil service in a country, you’re talking about building up capacity over 50 years or more.”47 Assessment of Influence of Alternative Explanations: High. 45 Interview with former Director General, Ministry of Interior, Jubaland State of Somalia. (2018). 46 World Bank (2015) Somalia-Capacity Injection Project (English). Washington, D.C.: World Bank Group. 47 Interview with TIS+ South West State TC. (2018). 74 The next stage in our analysis is to ask what other influencing factors could be accounting for the changes observed. These include other externally funded advisors as well as positive and negative factors affecting coordination and capacities in the relevant ministries. Qualitative research showed that in most of the ministries where TIS+ TCs are deployed, they were often the only ones at the ministry, with the exception of the Ministries of Interior in both states, where there were a few other TCs deployed by the International Development Law Organization and the United Nations Joint Programme on Local Governance (UNJPLG). TIS+ coordinates very closely with programs such as Somalia Stability Fund (SSF), which deliberately does not duplicate advisors in the same ministries where TIS+ TCs are placed. As a result, the TCs contribution to the state-level ministries is visible but there is a mixed picture at the federal level due to multiple actors providing an assortment of Technical Assistance to the institutions in the form of short-term consultants as well as long term technical coordinators. Levels of existing capacity within the ministries where TCs are based has a clear impact on the contribution TCs will make. One of the main challenges that the FMS have to contend with is the capacity gaps in the civil service. The advent of federalism has meant that the powers and services have been devolved to the regions, but this did not come with any transfer of capacity or resources from the center. Mogadishu itself struggles with lack of capacity in the federal civil service. However, there has been heavy investment through programs such as the Capacity Injection Program by the World Bank and the deployment of short-term and long-term technical assistance by a variety of donors. Jubaland was the first of the FMS to be established, and as a result of the relative stability, it has seen a steady influx of professionals from Mogadishu and abroad moving to Kismayo. However, there has been no major donor investment in the capacity of the civil service beyond programs focused on public financial management. Short-term technical assistance (STTA) has come through IOM’s MIDA, especially in public health care, but there remains a serious lack of capacity especially at the ministerial level as a result of the limited available wage bill and the lack of donor investment when compared to Mogadishu and Puntland. The South West State administration’s expansion plans are uncertain, especially as the date of the move of the capital to Barawe remains unclear. The administration has not moved much from the vicinity of Baidoa airport, where the Presidency is located. Limited resources at federal and state level have resulted in poorly capacitated ministries. The fairly nascent state of the administration has meant that there has been little investment in institution building and technical assistance to the ministries. However, the establishment of the civil service commission in Baidoa may improve the capacity of the ministries. Traditionally, the overwhelming investment in technical assistance and capacity injection has been focused at the federal level in Mogadishu as well as in more established regions such as Somaliland and Puntland. Programs including IOM MIDA, UNDP’s Multi-Sectoral Capacity Development Program, the World Bank’s Capacity Injection Project, and SSF, as well as USAID-funded programs including Strengthening Somali Governance and Growth, Employment, Economy, and Livelihood have provided various forms of technical assistance in the aforementioned regions. However, as of 2018–19, there has been increasing investment in the FMS, which will likely improve the capacity of state-level institutions in the long-term. In addition to the contribution of other development partners, the assessment also acknowledges the carryover effect of TIS, particularly at the federal level in Mogadishu, where the program was first piloted. The evaluation of the program concluded that the program had “contributed significantly toward increasing collaboration between government and stakeholders.”48 TIS utilized the “Dan Guud” model, where it embedded professionals into the Banadir Regional Authority (BRA). These professionals were paid for by TIS but effectively served as the nascent cadre of civil servants that improved the capacity of 48 IBTCI. “Final Evaluation of ‘Transition Initiatives for Stabilization’ (TIS)”. 75 the local government’s functions. With the end of TIS, many of these professionals were absorbed into the BRA as government civil servants. The political and security environment had a strong influence on the contribution of the TCs. In particular, the frequent political crises that contributed to the high turnover in the leadership of the ministries often constrained the momentum built by the TCs. This is further compounded by an unpredictable security situation, particularly in Mogadishu, where the federal-level TCs are based. ASSESSMENT OF CONTRIBUTION CLAIM Summary of Assessment Criteria Criteria Assessment Plausibility: Is the theory of change plausible? Moderate. The approach of utilizing embedded experts in government institutions across Somalia is common. However, there are challenges associated with the approach including the potential for elite capture, frequent turnover of government leadership, and the challenge of sustaining positive results beyond the short term. Implementation per plan: Has the program been implemented with high fidelity? Moderate. It is evident from the review of the program documents, interviews, and the output of the TCs that they were implemented as per the plan. However, there was also evidence that TCs were not adequately utilized in situations where they would have greatly contributed to. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? Moderate. There is evidence that TC work contributed to improving TIS+ interventions, improved coordination, and injected capacity in their host ministries. However, the high-level expected contributions, such as increasing confidence in government, presents a mixed picture. Moreover, the results chain connecting the TCs inputs to these results is so long and is decisively influenced by so many other variables that it would be unfair to expect any contribution at this level. Identification and examination of other influencing factors: To what extent have other influencing factors been identified? Identification of alternative explanations: To what extent do alternative explanations explain results? High. A number of other influencing factors and alternative explanations contributed to the results seen. These include the activities of other actors, the carryover effects from the previous TIS program, and the highly dynamic political and security situation. SUMMARY OF ASSESSMENT OF CONTRIBUTION CLAIM: MODERATE. Despite missed opportunities and the impact of external factors, there was evidence that TIS+ TCs accomplished the following: ● Supported continued coordination and collaboration of TIS+ plans and activities with federal and state government plans ● Supported coordination with other stabilization programs ● Improved communication especially on the government’s stabilization plans The contribution claim is also limited by the fact that other actors, and in particular, stabilization programs, also embedded TCs and advisors to carry out similar functions such as supporting state- and federal-level coordination of stabilization programs. There was also limited evidence that the TIS+ TCs contributed as expected to higher-level results such as increasing confidence in government. 76 CONTRIBUTION STORY 3: SPORTS FOR DEVELOPMENT AND PEACE The use of sports is a key feature of TIS+’s youth engagement strategy. The program has made sizable investments in either building or rehabilitating football stadiums and basketball courts as a shared community asset and to provide a venue for the community, and youth in particular, to gather and interact. TIS+ has also undertaken efforts to develop and roll out a national Sport for Development and Peace (SDP) strategy through its Technical Coordinators and an outside SDP consultant. According to its 2018 Annual Report, TIS+’s investments in “sports and youth facilities reaches thousands of youth across Somalia and provides them with opportunities to meet, break down barriers of clan and geography, develop a national identity, participate in peaceful social interactions and learn to negotiate losses, compromise and non-violently resolve conflict, all of which directly counter the divisive narrative extremists propagate.”49 The following contribution story will critically examine the extent to which TIS+ has contributed to the positive engagement of youth and reduced support for violent extremism through the use of SDP. It is structured according to the five criteria identified by John Mayne, the creator of the Contribution Analysis methodology, to analyze the extent of a contribution claim (or plausible association) between TIS+ activities and the results witnessed during the midline assessment. These criteria include the following: 1. Plausibility: Is the theory of change (ToC) plausible? 2. Implementation per plan: Has the program been implemented with high fidelity? 3. Evidentiary confirmation of key elements: To what extent are the key elements of the ToC confirmed by new or existing evidence? 4. Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? 5. Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? The contribution story will then conclude with an overarching assessment of the strength of TIS’s contribution claim to the results relevant to increased youth engagement and reduced support for violent extremism identified in the midline assessment. PLAUSIBILITY OF TIS+’S CPO THEORY OF CHANGE Theory of Change and Underlying Rationale TIS+ did not have a defined ToC specifically focused on SDP at the outset of the program. Rather, the renovation of sports stadiums was covered under its rationale for supporting quick-impact, small-scale infrastructure projects to promote the use and management of shared community assets. Likewise, the hosting of tournaments and other larger-scale sports activities was also covered under TIS+ more general community engagement approach and strategic communication plans. However, in order to enable a Contribution Analysis of TIS+’s results, the assessment team worked with the TIS+ team to develop an SDP ToC in June 2018 (see Figure 9 below). The resulting ToC highlighted some of the key intended results, causal logic, and underlying assumptions guiding TIS+’s SDP-related activities. However, the ToC was rather weak and included substantial leaps in the causal logic and several unrealistic assumptions. Identifying an opportunity to be more strategic in its approach to SDP, as well as wanting to support the Ministry of Youth and Sports (MOYS) develop a national sports policy, TIS+ contracted an outside SDP expert to work with the MOYS to develop a national SDP policy. A final draft of the “Somali Ministry of Youth & Sport (MOYS) Sport for Development and Peace (SDP) Strategy & Implementation Framework” was produced in August 2018. 49 AECOM, “Transition Initiatives for Stabilization Plus: FY 2018 Annual Progress and Financial Report,” (2019). Pg. 78. 77 Figure 9: TIS+’s SDP Theory of Change 78 The SDP Strategy draws on international good practice and global experience on positive youth development through the use of sports and highlights several factors that make it particularly appropriate for the Somali context. The strategy posits that SDP can help reduce youth propensity toward violent extremism by offering an “attractive alternative” and using sports as a “hook” to attract at-risk youth.50 SDP was also assumed to bring additional benefits, such as physical and mental benefits of physical exercise; a mechanism for developing life and social skills such as teamwork, sportsmanship, and camaraderie; the possibility to link with educational and vocational opportunities to not only attract youth but also strengthen their capacity and potentially employability; and even to promote community-wide civic engagement through the use of Community Planning and Oversight approaches to the selection, implementation, and oversight of SDP projects, as well as an opportunity to promote shared-asset management between the community and local authorities. The strategy even identified opportunities to use SDP to promote other cross-sector approaches supporting gender equity, social inclusion, and trauma healing. Figure 10 below provides an overview of the strategy’s conceptual framework. The strategy also included an updated ToC behind the intended SDP activities (see text box below). Specific to Preventing and Countering Violent Extremism (P/CVE), SDP was seen as providing a “structure that young people detached from their communities and schools may need. It can also act as a ‘diversionary 50 Paraphrased from Stephen Reynard, “Somali Ministry of Youth & Sport (MOYS) Sport for Development and Peace (SDP) Strategy & Implementation Framework” ed. AECOM/TIS+ (August 2018). Figure 10: SDP Conceptual Framework Revised SDP Theory of Change “If the federal and state MOYS are able to implement the Holistic SDP Conceptual Framework, then youth will develop relevant professional, life, and social skills and feel more empowered and connected to local development issues, which will enable them (youth) to counter the negative physical, psychological, and emotional effects of their environments and will help them (youth) to contribute to improved social cohesion, social inclusion, and positive peace.” ~ “Sport for Development and Peace (SDP) Strategy and Implementation Framework” 79 activity from violent and illicit activities.’ Building resilience to radicalization can be viewed in a similar light as crime prevention in that there is ‘crossover in the potential influencing factors that may contribute to an individual becoming radicalized, or engaging in crime.’”51 Relevance with Somali Context and National Efforts Somalia has a long history of using sports as a mechanism for community-mobilization and developing a shared identity (see textbox). Sports have also been used a tool to promote peace by several Somali government ministries, such as MOYS, the Somali Olympic Committee, and the Somali Football Federation, as well as by a variety of national and international NGOs. Private companies—such as Hormuud Telecom, NationLink Telecom, Premier Bank, and Dahabshiil Money Transfer—have also hosted sports tournaments or funded renovations to community sports fields and stadiums as Corporate Social Responsibility (CSR) and community outreach activities. The use of sports is also widely regarded as an effective way to engage youth, a key demographic with over 70 percent of Somalia’s population under the age of 30 according to UNICEF Somalia.52 This is reflected in Somalia’s “National Youth Policy” which states that “Sports and recreation are very important factors for youth since this provides opportunities to socialize, build social connections, counter boredom, spend their time productively, develop character and learn new skills and teamwork. They also shape the mental and physical wellbeing of the youth.”53 The policy recommends several sports-focused interventions, which strongly align with those utilized by TIS+, including (1) “strengthening sports and restarting the interregional, regional and national tournaments for both male and female athletes;” (2) “increase the opportunities for young women and men to engage in a diverse range of sports and sporting competitions including soccer, basketball, volleyball, and athletics;” (3) “provide greater support and direction to youth and sports centers to enable them to become more friendly and conducive to young people with special needs; and” (4) “instigate media awareness programs to raise awareness amongst parents and community and religious leaders of the importance of recreational and sporting programs to the personal development of both young men and women.” Additionally, the TIS+ SDP approach is also consistent with other Somali policy frameworks, including the Somali National Development Plan and the Somalia Youth for Peace Pact. Evidence from Other SDP Projects 51 Ibid. 52 UNICEF Somalia, “Education in Somalia.” 53 Ministry of Youth and Sports of the Federal Government of Somalia, “The National Youth Policy of the Federal Government of Somalia.” History of Sport in Somalia 1969–1991: Sport was used as a “mobilization and orientation” tool by the Siad Barre regime, mostly focused in Mogadishu and in a few major cities. All sports clubs were government funded; private sports clubs were not encouraged. 1982–1990: Following the war with Ethiopia and subsequent political and economic effects, government￾funded sport deteriorated. 1991–2009: Despite civil war and destruction of sports infrastructure, the private sector initiated sports programs to promote peace. Somalia was represented at various international sports competitions. 2010–2016: Banadir Stadium and Aden Yabarow Wish Sports Center were rehabilitated by the Fédération Internationale de Football Association (FIFA) and the International Olympic Committee (IOC); Sports for Peace was included in the Somali Olympic Committee’s strategic plan. 2017–Present: Development of 1st National Sports Policy (pending Cabinet approval) and Sport for Development and Peace (SDP) Strategy and Implementation Framework. 80 There are a number of studies that have examined the use of SDP both in Somalia and across the globe. An independent review of the International Labour Organization’s “Youth for Change” project, which worked with over 6,000 youths determined to be at risk of Al-Shabaab recruitment, found that the project “facilitated healthy competition and cohesion amongst the groups” and helped “reduce aggression among participants.”54 Further, an independent evaluation of the USAID-funded “Somalis Harmonizing Inter- and Intra-Communal Relationships” (SHIIR) program implemented by Pact found that the program contributed to positive changes regarding the beliefs and attitudes of the 480 male youth participants toward other clans but that “to have a meaningful impact on youth, the soccer drills should be accompanied by an in-depth educational program with a long-term perspective.”55 The MOYS SDP Strategy and Implementation Framework also cited a number of other international studies that demonstrates SDP’s potential to building social cohesion and promote “positive peace,” mend “broken social relationships,” promote psychosocial healing, address social exclusion, and challenge negative gender stereotypes and mitigate potential gender-based violence. Assessment of Plausibility: Plausible The updated SDP ToC developed by TIS+ presents a convincing argument supporting its intended results, causal linkages, and underlying assumptions. The ToC is aligned to the Somali context and follows a deep and widespread history of using sports to promote community engagement, cohesion, and outreach. It is also aligned with host-government strategies such as the National Youth Policy and obviously also the SDP Strategy within which it is presented. Finally, the ToC is also supported by evidence from independent evaluations and secondary studies of SDP-related programs in both Somalia and across the globe. FIDELITY OF IMPLEMENTATION TIS+ has implemented a number SDP-related activities, including the construction or renovation of football and basketball stadiums, organization of tournaments and other sports events, establishment of sport facility Project Oversight Committees, and distribution of sports equipment to both community and government stakeholders. Table 9 below provides an overview of all SDP-related PIKAs implemented at the time of the midline assessment. Table 9: List of SDP-related PIKAs Location PIKA No. and Title Description Status at Midline Abdiaziz (BRA004) BRA Districts Sports Facilities and Youth Engagement 59-member District Sports Facilities & Programming Bridging Plan Task Force formed (BRA, district consensus groups, technical experts, donor/partner organizations and civil society representatives) Completed Abdiaziz (BRA012) BRA Abdiaziz District Basketball Mini Stadia Rehabilitation Rehabilitation of basketball stadium, setting up and training Infrastructure Facility Oversight Committee (IFOC) (formerly POC) Ongoing Hawlwadag (BRA013) Mogadishu Hawlwadag District Basketball Court Rehabilitation Rehabilitation of basketball stadium, setting up and training IFOC Ongoing Warta Nabada (BRA011) Mogadishu Warta Nabada District Basketball Court Rehabilitation Rehabilitation of basketball stadium, setting up and training IFOC Ongoing Kismayo (JSS013) JSS Kismayo National Youth Day Youth Engagement National Youth Day support with football, basketball, bicycle race, and peace march in Kismayo Closed 54 Lilla Schumicky-Logan, “Addressing Violent Extremism with a Different Approach: The Empirical Case of at-risk and Vulnerable Youth in Somalia,” Journal of Peacebuilding & Development (August 2017). 55 Reynard. 81 Kismayo (JSS026) JSS Ministry of Youth and Sports Technical Coordinator/Sports Technical Coordinator support Ongoing Garowe (PSS002) Puntland State of Somalia (PSS) Garowe Town Basketball Court Upgrading Construction of basketball stadium Ongoing Baidoa (SWS008) SWS Ministry of Youth and Sports Baidoa Youth Day Supported National Youth Day celebration Closed Dinsoor (SWS016) SWS Dinsoor District Football Stadium Construction Construction of football stadium, setting up and training IFOC Completed Barawe (SWS015) SWS Ministry of Youth and Sports Barawe District Football Stadium Construction Development of engineering documents Ongoing National (SOM014) Somali Women’s Basketball Support Regional Training 8 training sessions for already established women’s/ girls’ basketball teams; new equipment, uniforms, and training in preparation for their participation in a national tournament (96 girls trained) Closed National (SOM015) Somali National Women’s Basketball Tournament A national women’s basketball tournament, with 8 regional teams (96 young women) participating, and the selection of a women’s national team Closed National (SOM028) Somalia Sports for Development and Peace Framework STTA STTA for FGS MOYS Sport for Development and Peace Framework and a rollout plan for Somalia in consultation with the FMSs MOYS and Sports Federations; SDP approach for TIS+ produced Ongoing In 2016, TIS+ supported preparatory basketball training camps for 119 young women (including in Kismayo) and then supported the Somali National Olympic Committee to organize a national women’s basketball tournament, which took place in the court rehabilitated by TIS+ in Garowe. Working through federal and regional MOYS, TIS+ supported logistics, transport, and accommodation for the second national men’s soccer tournament in December 2017. Both these tournaments generated significant interest and impact on new social media. Respondents interviewed during the midline assessment confirmed that the men’s soccer tournament had been a major event that the nation had stopped to watch. However, as evident in the PIKA list above, the majority of TIS+’s SDP-related activities have focused on the construction or renovation of simple community sports facilities. Only one of these facilities has finished construction at the time of the midline and unfortunately, that facility, a sports stadium in Dinsoor, was blown up by AS militants and almost completely destroyed. As noted in the ToC section, TIS+ did not have a defined strategy to guide its SDP work until mid-2018. This has resulted in a trend for most SDP activities to be stand-alone with few follow-on or complementary activities to support the achievement of higher-level community cohesion and youth engagement results. That said, the wide range of actors currently supporting SPD activities or interested in support SDP activities in the future, combined with capacity challenges in the MOYS, suggest that the TIS+ approach of working on a strategic framework for SDP for MOYS, as well as providing technical support to MOYS, was appropriate. Assessment of Fidelity: Weak. TIS+’s implementation of the SDP strategy and ToC has to date been limited mostly due to the recent development of these strategies. Interviews with TIS+ staff as well as evidence recent periodic program reports suggest that the program is committed to its new SDP strategy and endeavor to undertake a more systemic approach to implementing activities following this strategy. EVIDENCE SUPPORTING INTENDED RESULTS According to TIS+’s 2018 Annual Report, the program has achieved the following SDP-related results: ● Facilitated the development and rollout of the Somalia Sports for Development and Peace (SDP) Framework while utilizing TIS+ investments in sports facilities and sectors. 82 ● Completed 20 infrastructure projects: Five sports stadiums. ● Held four special celebrations including Somali National Youth Day and the International Day of Sport for Development and Peace in Baidoa (SWS), Galkayo (PSS) and Kismayo (JSS), with one celebration held in Q4. ● Launched one National Sports for Development Framework in Mogadishu. ● Held one Men’s National Soccer Tournament in Mogadishu that was attended by over 30,000 people. ● TIS+ Technical Coordinator provided capacity-building and technical support to the Jubaland Ministry of Youth and Sports. However, evidence from the midline assessment indicates that these output-level results have not yet translated into clear outcome-level results. There was a consensus among key informants working on sports￾related activities that conducting or facilitating sports events alone was not enough to deliver desired results. Physical education does deliver positive health benefits by itself, but the SDP framework is not a health intervention first and foremost. There may be some inherent benefits for social cohesion of providing sports venues and events. As one business leader in Kismayo noted: “People from different clans meet and interact with each other in places such as the sports stadiums, restaurants, businesses, and mosques. These venues provide an opportunity for people with different views to exchange ideas, learn from each other, reconcile and bridge differences.” Additionally, simply having sports venues without additional supporting activities was regarded by interviewees as insufficient for achieving higher-level results. “Awareness creation is important as well as having stadiums and youth centers.” This message was reinforced by several interviewees who had worked on, or had an interest in, sports. As one specialist in youth programming put it, “It is important not just to support events—it has to be more than kicking a football around.” It was seen as very important for these interventions to be conducted in a purposive way. As much as bringing people together in events and tournaments could support a sense of shared identity, these events needed to be more intentional in their design to be fully inclusive, as well as to avoid potential perverse outcomes from interstate games that could exacerbate regional tensions. The recent SDP approach developed by TIS+ focuses on using sport to deliver or to boost the delivery of other skills and messages. These can be through side-by-side programming in which educational, civic, or vocational training is provided alongside sport, or through “pedagogical sports drills” in which sport training sessions are used as a platform to diffuse educational and life skills messages.56 This approach appears to have been used by TIS+ in its support to the men’s football tournament in December 2017: “The Sports for Development and Peace approach uses sports to create social change, to facilitate community development, to increase cohesion, and to support peace and conflict mitigation efforts. Four SDP drills were conducted to show how working together can facilitate healthy competition that will extend off the playing field. . . . Before today we were just playing football for health and personal skills, but today we have learned that sports are a great tool for mitigating conflict and fostering peace.” The main challenge to the suggested mechanism in the SDP results chain is the focus on individual youth choices and decisions. This assumption was seen as crucial by some of our expert, peer donor, and program interviewees. As one such interviewee noted, “As for sports for development and peace, I don’t know if this would work according to my understanding of how recruitment happens. If we are saying sports will keep youth distracted and keep them away from AS, perhaps this could work when we have heard some cases of individual recruitment. But these are not many. In many cases, recruitment to AS is a collective decision, for example of a sub-clan who decides to franchise with AS because they think that in their particular circumstances, they best represent their interests. When that happens, it is the whole clan militia that is 56 Interviews with experts and peer programs revealed some skepticism about the link between vocational training and even between employment and countering violent extremism. Several interviewees felt that young people were not engaged in political violence because they had nothing else to do. 83 franchised to AS. They have to undergo training, most of which is ideological indoctrination, and out of this process a good proportion become hardcore jihadi extremists.” Related to gender, midline interviews suggested that Somali society is generally accepting of men’s participation in sports events. However, the participation of women and girls is much more controversial. AS prohibits women and girls from playing sports, but disapproval of women and girls’ participation is much wider than AS. For example, before the TIS+-supported basketball tournament in Garowe in December 2016, the Somali Religious Council, a group of influential Somali clerics, issued a warning to women not to engage in sports: “We warn that the women’s basketball violates Islamic law, culture, and its values, and it is a place where women can be easily corrupted.”57 The tournament was conducted with high-security costs. From a “do no harm” perspective, this has clear implications for the protection of women and girls’ safety during sports activities and their protection from subsequent stigma or threats because of their involvement in such activities. The TIS+ 2016 basketball tournament and the training camps that preceded them are some examples of interventions targeted specifically at women and girls. TIS+ supported basketball training camps that involved 119 young women between the ages of 16 and 25 who came from seven different regions (states) in Somalia. TIS+ supported the Somali National Olympic Committee to organize a national women’s basketball tournament, the first such event in over two decades. The tournament was held on a TIS+ rehabilitated basketball court in Garowe and was watched by an estimated 900 viewers per day, many of whom were women. In addition to the direct participants and spectators, the tournament reached over one million people through social media; it was broadcast live on national and regional television, and sparked significant debate about women and sports across Somalia. Our respondents confirmed that these interventions had had a sustainable impact on women’s’ basketball in Somalia, which was progressing at a federal level, as well as at the Jubaland level, where women’s basketball teams were still playing. Other respondents confirmed the huge positive impact that access to sport could have on women and girls who previously had no opportunity to engage in physical activity. Assessment of Supporting Evidence: Weak. TIS+’s SPD-related results to date have been primarily at the output level and there is little evidence of TIS+ implementing an integrated and intentional strategy that explores and utilizes potential complementarities between activities. That said, the development of the SDP Strategy and Implementation Framework is a key recent result and is expected to help systematize TIS+’s SDP approach going forward and increase the prospects of achieving outcome-level results. INFLUENCING FACTORS AND ALTERNATIVE EXPLANATIONS Our assessment indicates that although many stabilization and development actors have supported sports activities often in an ad hoc manner, efforts to develop a more consolidated programming approach to sport interventions are nascent. TIS+’s own stakeholder mapping, conducted by the SDP consultant in 2018, revealed a range of actors working on sports in Somalia and identified a wide range of actors: ● UNSOM, UNFPA, and UN-Habitat, which use sport as a tool to engage women and youth ● The EU, which is planning to build cultural centers with sport as a component ● Somalia Olympic Committee, which is planning to develop a Sports for Peace program (2017–2020) ● The Pact SHIIR program, which piloted pedagogical sports drills in Jubaland ● IOM, which has used sports interventions in its programs for rehabilitating AS defectors 57 https://www.voanews.com/a/somali-clerics-warn-women-aginst-playing-basketball/3646679.html 84 ● CARE Somalia, Mercy Corps, and Concern Worldwide, which were also identified as working or having worked on sports58 ● Private sector companies such as Hormuud Telecom, NationLink Telecom, Premier Bank, and Dahabshiil Money Transfer, which have sponsored the Somali Olympic Committee, and Somali Football Federation, and SSF, as well as conducted a CSR program focused on education and health in the Horn of Africa ● Other donor, program, or implementing partner actors such as US OTI, GIZ, SSF, and the NISF, who have worked on sports interventions or have plans to do so. This range of actors who have worked/are working on sports interventions are certain to affect TIS+ future results and there was some evidence that these initiatives have already influenced perceptions of TIS+’s activities. The Kismayo sports stadium, mentioned by several Kismayo interviewees as a positive asset, was reconstructed by NISF in 2014 with UK funding and has more recently been upgraded in 2017 by the American Refugee Committee and the United Nations High Commission for Refugees. Sports tournaments were also mentioned by respondents as positive developments. TIS+ will certainly have contributed to these through its basketball tournament (which was in 2016) and the men’s soccer tournament in 2017.59 However, there are many other actors supporting tournaments, which may have had a larger effect on responses because they are more recent. For example, in May 2018, Save the Children Somalia, in partnership with MOYS, co-organized an interschool sports tournament for 14 public schools. The football and athletics tournament were funded by the German Federal Foreign Office.60 Results are also likely to be substantially affected by factors in the political economic and security context that affect sports events specifically. There are many supporting factors in the Somali environment that would enhance the effectiveness of sports interventions for youth. The fact that Somalis are big sports fans was mentioned by a number of expert, peer donor, and program interviewees. A number of respondents noted that there were positive Somali role models on the international sporting arena, such as Mo Farah (athletics), Abdi Bile (athletics), Maryam Nah Muse (athletics), Abdisalam Ibrahim (soccer), and Mukhtar Ali (soccer), and other Muslim sporting heroes such as Mo Salah, who were wildly popular in Somalia. A further positive factor, also made by TIS+, is the existence of several private sector sports football stadiums and investors in sports events, including banks and telecom companies. This means that sports facilities and events would take place without donor intervention in Somalia. These interventions are sustainable after the initial investment. There are also threatening and challenging factors in the Somali environment that negatively affect sporting interventions and pose serious risks to them. These are related to AS threats and attacks and to the broader acceptance of women’s involvement in sport. This is a threat is exemplified by the AS attack on the TIS+ Dinsoor stadium in 2018. It is also demonstrated by the closure of 30 football fields in Mogadishu follow threats by AS. AS also handed down terms in AS prisons for some of those defying these orders.61 These orders are an effective way for AS to demonstrate and project its symbolic power into areas not formally under its control. There have also been AS attacks on sporting events, such as the bombing of a friendly match in a Barawe football stadium in April 2018, which killed five people.62 This does not indicate that AS has an inherent opposition to football. There have been stories reporting that AS allows and encourages the playing of “halal” football between AS fighters and local teams in areas under its control.63 However, it does illustrate the security risks of public events and venues, especially those connected to the government, and the need to protect participants. 58 TIS+. (2018). Sport for Development and Peace Strategy and Implementation Framework stakeholder mapping PowerPoint presentation. 59 https://www.kawowo.com/2017/12/24/jubbaland-wins-somalia-inter-states-championship/ 60 https://somalia.savethechildren.net/news/enhancing-integration-through-inter-school-football-athletics-tournament-kismayo 61 https://www.aljazeera.com/indepth/features/somalia-al-shabab-shuts-football-pitches-mogadishu-180607045144830.html 62 https://www.aljazeera.com/news/2018/04/blast-somalia-stadium-kills-football-fans-180413070232458.html 63 https://www.aljazeera.com/blogs/africa/2014/06/99086.html 85 Assessment of Influence of Alternative Explanations: Moderate. There are several other actors who are engaged in sports activities in Somalia, including donors and development programs, but much of this work has been ad hoc in nature. TIS+ could maximize its impact in this area by playing a coordination role among donors and programs, working with the MOYS. The popularity of sports among Somalis and the proliferation of private sector venues suggest that sports would be positively regarded by Somalis with or without interventions by donors and programs, and are likely to enhance the impact of interventions. However, AS attacks pose a risk to sports venues and events, and social and pervasive resistance to the participation of women in sports threatens to undermine the objectives of sports interventions, especially those engaging women. ASSESSMENT OF CONTRIBUTION CLAIM Summary of Assessment Criteria Criteria Assessment Plausibility: Is the theory of change plausible? Plausible. The updated SDP ToC developed by TIS+ presents a convincing argument supporting its intended results, causal linkages, and underlying assumptions. It is aligned to the Somali context and government strategies such as the National Youth Policy. It is also supported by evidence from independent evaluations and secondary studies of SDP-related programs in both Somalia and across the globe. Implementation per plan: Has the program been implemented with high fidelity? Weak. TIS+’s implementation of the SDP strategy and ToC has to date been limited mostly due to the recent development of these strategies. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? Weak. TIS+’s SPD-related results to date have been primarily at the output level and there is little evidence of TIS+ implementing an integrated and intentional strategy that explores and utilizes potential complementarities between activities. That said, the development of the SDP Strategy and Implementation Framework is a key recent result and is expected to help systematize TIS+’s SDP approach going forward and increase the prospects of achieving outcome-level results. Identification and examination of other influencing factors: To what extent have other influencing factors been identified? Identification of alternative explanations: To what extent do alternative explanations explain results? Moderate. There are several other actors who are engaged in sports activities in Somalia, including donors and development programs, but much of this work has been ad hoc in nature. TIS+ could maximize its impact in this area by playing a coordination role among donors and programs, working with the MOYS. However, AS attacks pose a risk to sports venues and events, and social and pervasive resistance to the participation of women in sports threatens to undermine the objectives of sports interventions, especially those engaging women. SUMMARY ASSESSMENT OF CONTRIBUTION CLAIM: WEAK. The midline assessment found that any potential claim for TIS+’s SPD activities to have influenced higher￾order outcomes related to community cohesion or positive youth development to date are not well substantiated by current evidence. However, there is room for optimism as TIS+’s recent SDP Strategy and Implementation Framework not only demonstrates a commitment to more systematically link SDP activities to achieve higher-level results but also provides a concrete road map for how to do so. It is expected that TIS+’s contribution claim around its SDP results is likely to be strengthened in the coming years. 86 CONTRIBUTION STORY 4: GENDER EQUALITY AND SOCIAL INCLUSION The promotion of gender equality, women’s empowerment, and social inclusion has been a cross-cutting goal of TIS+ since its inception. One of its four overarching objectives (Objective 4) is specifically focused on reducing gender gaps to promote inclusive and sustainable development, and the program has taken steps to ensure that GESI considerations are mainstreamed across the other three objectives as well. The following contribution story will critically examine the extent to which TIS+ has succeeded in its GESI efforts and ensured the equal access of and benefit for women, youth, and minority groups across its activities. The story is structured according to the five criteria identified by John Mayne, the creator of the Contribution Analysis methodology, to analyze the extent of a contribution claim (or plausible association) between TIS+ activities and the results witnessed during the midline assessment. These criteria include the following: 1. Plausibility: Is the theory of change (ToC) plausible? 2. Implementation per plan: Has the program been implemented with high fidelity? 3. Evidentiary confirmation of key elements: To what extent are the key elements of the ToC confirmed by new or existing evidence? 4. Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? 5. Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? The story will then conclude with an overarching assessment of the strength of TIS’s contribution claim to the outcome-level GESI results that the program hoped to achieve. BACKGROUND AND CONTEXT Both the needs for, and challenges to, the promotion of GESI within the Somali context are extensive and well-documented. According to a 2018 USAID-commissioned GESI Analysis, “Somali women, girls, minorities, and internally displaced people (IDPs) are among the most marginalized and discriminated against groups in the world.”64 While a detailed situational analysis of each of these groups is outside of the scope of this contribution story, a quick overview of key indicators is provided below to help convey the severity of some of the challenges faced by these groups. Women According to UNDP, Somalia has one of the highest gender inequality index scores in the world (0.773), surpassed only by Yemen, Mali, Afghanistan, and Papua New Guinea. A recent USAID gender analysis found that Somalia faces systemic inequalities and social norms when it comes to issues of gender equality and social inclusion. “Somalia is organized along a deeply entrenched clan, sex, and age hierarchy that determines one’s roles, responsibilities, and rights in society. The structure is supported by patriarchal principles, justified by customary (Xeer) and religious (Shari’a) law, and perpetuated through patrilineal descent and rigid gender norms. . . . Traditional women’s roles focus on the household and family, and gender norms restrict their participation in social, political, and economic leadership and decision-making.”65 These inequalities and social norms have been exacerbated by over two decades of conflict and instability and resulted in Somalia having some of the worst gender indicators globally, particularly in terms of education, health, and civic engagement. UNICEF Somalia estimates that only 30 percent of primary-aged children are in school, among which only 40 percent of these are girls. The situation is even worse for children from rural households where less than 1 in 5 children (18 percent) are estimated to attend school.66 As a result, only 64 Pact, “Gender Equality and Social Inclusion Analysis: Expanding Access to Justice (Eaj) Program,” (2018). 65 IBTCI, “Gender Analysis Task Order: Final Report,” (May 2018). 66 UNICEF Somalia, “Education in Somalia.” Accessed on January 19, 2019. 87 25 percent of girls and women between15 to 24 are estimated to be literate. These “poor education and literacy rates coupled with low rights and opportunities around asset accumulation and responsibilities divided by gender roles leave women with few formal employment options.”67 Somali girls and women also disproportionately suffer from restrictive health and cultural practices. “To reduce the chances of unsanctioned sexual activity at any point in their lives, Somali culture imposes two extreme forms of control on young girls—female genital mutilation/cutting and child/early/forced marriage. Both are widely practiced, with as many as 98 percent of girls undergoing female genital mutilation and nearly half of all girls married by age 18, including an estimated 8 percent by age 15.”68 In addition, the poor quality of family planning and health services available have resulted in Somalia having both one of the highest fertility rates in the world and one of the highest maternal mortality rates (1,600 per 100,000 live births) with only nine percent of births attended by a midwife, according to UNICEF. Finally, the rates of gender-based violence and intimate partner violence are extensive and have grown in recent years given Somalia’s history of turmoil. Youth According to UNICEF Somalia, over 70 percent of Somalia’s population is under the age of 30, making youth a key demographic for any stabilization or governance program operating in the country. According to the World Bank, Somali youth face a myriad of challenges. It estimates that “nearly half (46 percent) of Somali youth ages 15–24 lived in households below the poverty line, and nearly one in three youth lived in households with conditions of extreme poverty.”69 Further, the International Labour Organization (ILO) estimates that nearly 75 percent of youth are unemployed, leading to an increased risk and vulnerability “to illegal migration, organized crime, and/or violent extremism.”70 This represents one of the highest joblessness rates in the world, according to the World Bank. “Moreover, real or perceived discrimination in access to economic and educational opportunities, especially among youth, has been cited as a potential driver of conflict.”71 Somali youth also face a range of threats to their safety and physical well-being. According to the Somali Ministry of Youth and Sports, “the large youthful population has been vulnerable to various national problems, including violence, immigration, and terrorism. In terms of security and the prolonged civil wars, the youth have been exploited by clan agendas and terrorist groups like Al-Shabaab. Aside from recruitment by extremist groups, youth face the additional risks of human trafficking, forced prostitution, gang membership, and drug abuse. Unfortunately, youth even face vulnerabilities from state actors such as the police, with the US State Department noting that “young men in Mogadishu are routinely picked up by police and jailed simply for being young or for possibly being an Al-Shabaab sympathizer, with or without justification other than age.”72 A key final vulnerability ironically comes from a response or coping strategy often employed by youth in response to the threats and vulnerabilities highlighted above. According to the EU Somalia Mission, there has been an increase in youths attempting “tahriib”—an Arabic word associated with human smuggling and trafficking. According to the Rift Valley Institute, this exposes youth to a variety of risks including kidnapping, sexual assault, and rape, extortion, as well as physical and psychological violence.73 Other Marginalized Groups 67 Pact. 68 IBTCI. 69 World Bank, “Youth as Agents of Peace: Somalia,” (2018). 70 International Labour Organisation, “ILO in Somalia,” (2017). 71 World Bank. 72 Bureau of Democracy US Department of State, Human Rights, and Labor, “Country Reports on Human Rights Practices for 2015,” (2015). 73 Nimo-Ilhan Ali, “Going on Tahriib: The Causes and Consequences of Somali Youth Migration to Europe,” The Rift Valley Institute (2016). 88 Majority/Minority Clans. While the subject elicits a multitude of opinions, clan affiliation is undoubtedly an important marker of sociopolitical identity and determinant of social inclusion in Somalia. According to a United Nations Office for the Coordination of Humanitarian Affairs (UNOCHA) estimate, nearly one third (2 of 7 million) of Somalia’s population come from minority clans.74 Minority clans have faced a long history of “social segregation, economic deprivation, and political manipulation” wherein “minority groups were systematically excluded from mainstream government positions and the few minorities who held positions had no power to speak on behalf of their communities.”75 This finding was echoed in USAID’s recent GESI assessment, which concluded that “majority clans enjoy greater numbers, wealth, and stronger negotiating power. Minority clans, by contrast, suffer discrimination, enjoy less protection, and have limited access to resources.”76 This includes those who have been subject to “political, social, economic, and judicial discrimination, and have experienced human rights abuses including harassment and violence, killings, torture, rape, kidnapping for ransom, and looting of land and property, which in some circumstances may amount to persecution.”77 As a result, a 2015 briefing paper by the UK Home Office warned that “in a country where there is no national Government that would be responsible for safeguarding and upholding the rights of minority groups, Somalia minorities are truly in a vulnerable position. Careful and thorough attention needs to be focused on the issues faced by vulnerable populations in order to develop concrete assistance strategies that will have a positive impact on the security and livelihoods of minority groups.”78 However, being from a minority clan does not automatically confer marginalization. In Somalia, the use of the term “minority clan” usually refers to clans with smaller populations and whose collective sub-clan networks do not form part of a larger clan constellation. Some of these minority clans such as the Somali Bantu do not even share any ethnic lineage with the major clans. In a lot of cases, these clans are often “marginalized.” However, this does not tell the whole story. Sometimes, even clans that are not a ”minority” can be often ”marginalized” based on their geographical location (away from their larger network’s traditionally inhabited areas such as the Biyo Maal) as may be the case in some areas in Lower Shabelle and Middle and Lower Jubba. ”Marginalization” can also in some instances be fluid, determined by certain security and political developments that can temporarily move a certain clan/sub-clan in or out of marginalization. Another element to keep in mind, however, is clans/sub-clans that are marginalized not just due to their numbers but due to other socioeconomic perceptions. For example, the Tumaal are marginalized and outcast because of their economic activity. Similar stereotypes/perceptions are held about the Yibir or Midgaan. Finally, it is important to understand that even though in some cases, certain groups are referred to as minority clans and lumped into a group, they often have nothing to do with each in terms of lineage. They have been lumped into this category of minority clans, but the wider nomadic clansmen to assimilate into Somalia’s clan structure, which has security and political implications. Internally Displaced Peoples (IDPs). The United Nations High Commissioner for Refugees (UNHCR) estimates that approximately 1.7 million people are displaced within Somalia, with as many as 1.1 million considered to be in a situation of “protracted displacement.”79 As explained by USAID’s GESI assessment, “IDPs face severe challenges, including widespread sexual and gender-based violence and rape. Throughout the country, over half of households are headed by women who have been widowed, abandoned, or separated from men, many of whom remain behind in drought-affected communities or conflict areas.”80 The UK Home Office briefing paper further explains that “they may face discrimination and various human rights violations including economic exploitation, extortion, forced labour, harassment and forced evictions which amount to persecution. This particularly applies to single women or a woman who is head of a household who may also be liable to gender-based violence and trafficking.81 74 UNOCHA, “A Study on Minorities in Somalia,” (2002). 75 UK Home Office, “Country Information and Guidance: South and Central Somalia - Majority Clans and Minority Groups,” (March 2015). 76 Pact. 77 UK Home Office. 78 Ibid. 79 UNHCR, “Cccm Cluster Somalia,” (2019). 80 Pact. 81 UK Home Office. 89 “Multiple Levels of Vulnerability” A consistent and central finding across the gender assessments and secondary studies reviewed is that while it can be helpful to understand the unique challenges faced by each of the groups identified above, it is critical to understand that complete social inclusion strategies must recognize that individuals can have “multiple levels of vulnerability.” “Care must be taken to avoid generalizations when crafting policy and programs. Within these groups, there are hierarchies of power and influence, as well as large pockets of multi-faceted vulnerability. Poverty, clan status, displacement, and bereavement can each operate to expose individuals or families to broadened risks and threats. It is particularly important to avoid generalizations based on gender. The power structures and social orders that regulate Somali society are complex, geographically heterogeneous and dynamic. For some women, their clan, marital status, resources or political connections allows them to exercise power over or even exploit others, including the displaced, those from minority clans, the poor and some groups of men. Reciprocally, men – especially young men, minority clan members and those who are displaced – share many of the challenges generally attributed to women and girls.”82 This means that donors and development partners interested in pursuing equity-based programming or implementing activities need to move beyond simply establishing quotas for participation to more substantively analyzing who is most marginalized in the communities where they operate and what efforts can be undertaken to ensure meaningful participation of these groups in activities. Positive Factors and Opportunities However, while the above sections have highlighted a number of grave challenges faced by women, youth, and other marginalized groups in Somalia, there are also potential signs for optimism. While Somalia’s more than two decades of conflict is responsible for exacerbating many of the negative factors above, it has also served to challenge Somalia’s traditional social structure and gender roles. “Today, large numbers of women￾headed households support families and contribute to the market economy, including in an ownership capacity. While the most lucrative occupations are still reserved for male breadwinners, women can be found in virtually every field regardless of its gendered past.”83 Additionally, USAID’s 2018 “Gender Analysis” as well as TIS+’s “Gender Equality and Women’s Empowerment Strategy” (GEWE) identify other positive factors that can be capitalized on, including the following: • Mobility: Somali women are not subject to the strict purdah requirements that restrict the mobility of women in many Muslim-majority countries. This “opportunity” has been limited by safety concerns in insecure areas, which affect both males and females. • Government policies: Explicit commitments to the equal rights and roles of girls and women in official policy documents provide a mandate on which strategic plans can be built and implemented, and also provides a starting point for discussions about gender equity institutionally. • Commemorations: Occasions such as International Women’s Day, 16 Days of Activism Against Gender-Based Violence, and Mother’s Day can be used to send messages and convey information to women and raise gender-specific issues, such as gender-based violence and intimate partner violence. PLAUSIBILITY OF THE TIS+ GESI THEORY OF CHANGE TIS+’s Overarching GESI Approach 82 Pact. 83 IBTCI. 90 TIS+ program documents and gender analyses demonstrate a clear commitment by the program to be both gender responsive and socially inclusive. “Integrating a gender perspective and addressing the unique needs of youth into the TIS+ programming mandate is critical to improving social cohesion in the communities targeted by the program.”84 Further, “TIS+ has an opportunity and responsibility to capitalize on the tremendous capacity and agency Somali women have, to meaningfully integrate them into decision-making and facilitate greater community participation.”85 TIS+’s GEWE strategy carefully explains that gender considerations are particularly important for stabilization programs as “an explicit focus on outreach to girls and women, implementation of activities that appeal to and resonate with them, and efforts to link them with district officials to help establish longer-term relationships illustrate effective strategic gender planning.”86 It also recognizes that “the systemic inequalities that form the basis of Somali culture stand in stark contrast to USAID’s goals of gender equality and women’s empowerment, and rigid Somali gender norms impede the achievement of every Transition Objective.”87 Theory of Change However, despite the strongly stated commitment to gender equality, and at times broader social inclusion, TIS+ does not have an explicit Theory of Change (ToC) highlighting its intended results, causal pathways, or underlying assumptions. While its GEWE strategy identifies some of the higher-level GESI goals that the program hopes to achieve, as well as a list of tangible steps the program could take to help implement the strategy, it does not articulate the intermediate results that would like activities and higher-level outcomes. Because the development of a clearly articulated ToC is necessary for the implementation of a Contribution Analysis approach, as well as a desire by the assessment team to provide a more nuanced assessment of what results TIS+ did or did not achieve, the assessment team reconstructed a GESI ToC based on various TIS+ program documents available, such as the initial and revised GEWE strategies, TIS+ “Process of Engagement,” and 2018 USAID “Gender Analysis” (see Figure 11 next page). 84 AECOM, “Gender Equality and Women’s Empowerment Strategy (Revised),” (December 2017, p. 1). 85 “Transition Initiatives for Stabilization Plus: FY 2018 Annual Progress and Financial Report,” (2019). 86 “Tis+ Gender Equality and Women’s Empowerment Strategy,” (September 2015). 87 “Transition Initiatives for Stabilization Plus: Fy 2018 Annual Progress and Financial Report.” TIS+’s Expressed Commitment to Gender Equality “TIS+ activities for women and girls . . . help develop a path to self-reliance, self-realization, and respect for themselves – themes that are at the cornerstone of this project. They . . . provide women with the resources and opportunities they need to become effective agents of change by gaining skills, building self￾confidence, developing self-reliance, solving problems and working within supportive communities. TIS+ is committed to providing all citizens the opportunity to articulate their needs and interests and participate in the decisions that affect their lives.” ~ TIS+ Revised GEWE Strategy (2018) 91 Figure 11: TIS+’s (Implicit) GESI Theory of Change 92 Key Principles of TIS+’s GESI Approach While neither the original nor the revised GEWE strategy may have included a defined ToC, both documents clearly conveyed a set of key gender and youth principles and steps that the program would follow. These included the following: • Recognize that women and men are equally as concerned about responsive governance institutions and expect the delivery of basic services, transparency, and opportunities for civic dialogue. • Ensure that the program would continuously advocate for expanding women’s participation in public decision-making to help promote stability, deliver peace dividends, encourage Somali ownership, and enhance government legitimacy. • Analyze the root causes and consequences of gender inequalities in Somalia and provide practical, community-specific guidance for activity design and implementation. • Provide a concrete approach with respect to building awareness of gender and holistically integrate gender into the planning, design, implementation, and evaluation of programmatic activities. • Integrate gender across the program in a holistic manner that goes beyond simply increasing women’s participation—integrating gender means bringing the experience, knowledge, and interests of women and men to bear on programmatic objectives and anticipated results. • Explore the relative roles, status, and agency of both men and women (and, when appropriate, girls and boys) in order to improve planning that contributes to an overall value system— one that promotes equitable participation and collective benefit for all. • Move away from activities that reinforce the notion that something is being “done for” women, allowed, and bestowed by men, and instead encourage women’s agency and ensure equitable access to and benefit from all TIS+ supported activities. • Provide gender-sensitive facilitation training and deploy local gender experts to ensure that both men and women do not perceive women’s participation to be token. • Identify ways to provide physical meeting spaces that allow women the space to gather so that they can develop confidence and public speaking and leadership skills. • Appreciate contextual nuances from district to district and region to region, and adapt GESI programming accordingly. • Recognize that communities and local governments cannot be effective if women and youth are unable to exercise their right to participate in making decisions that affect their lives. • Mainstream gender and youth programming beyond just Objective 4 activities. Assessment of Plausibility: Minimal. TIS+ undertook laudable efforts to analyze macro-level gender issues that affect stabilization programming across Somalia. Its original and revised GEWE strategies provide clear higher-level results that the program intended to achieve, as well as clear principles and steps that the program will undertake to try and achieve those results. However, the strategies did not include an explicit ToC and oscillate between very specific, and at times overly ambitious, steps that the program will take and higher-level results that it hopes to achieve in a relatively short timeframe. The implicit ToCs found in TIS+ documents omit several intermediate results that would link its gender-focused activities and intended results. They also do not clearly identify the causal pathways of these intended results or the underlying assumptions and conditions that would need to be met to achieve these results. Additionally, the documents only make slight reference to efforts to engage youth and make almost no substantive comments on how the program will look to engage other marginalized groups. All in all, the implicit ToC is considerably incomplete at present. 93 FIDELITY OF IMPLEMENTATION Activities Implemented The next step in analyzing TIS+’s GESI contribution claim involves reviewing the extent to which its implementation followed its TOC and principles and steps outlined in its GEWE strategies. As an initial step, the assessment team reviewed the overarching list of Pool of In-Kind Activities (PIKAs) implemented by the program at the time of the midline assessment. Figure 12 below identifies a list of 30 PIKAs activities that the assessment team identified as involving a GESI component. Figure 12: PIKA GESI-Related Activities Location PIKA No. PIKA Name Description BRA Abdiaziz BRA002 Mogadishu Book Fair 3-day Book Fair with Somali authors, poets, scholars, playwrights and professors from around the world; Public lecture by authors at local university in Mogadishu after Fair Abdiaziz BRA004 BRA Districts Sports Facilities and Youth Engagement 59-member District Sports Facilities & Programming Bridging Plan Task Force formed (BRA, district consensus groups, technical experts, donor/partner organizations and civil society representatives) Abdiaziz BRA012 BRA Abdiaziz District Basketball Mini Stadia Rehabilitation Rehabilitation of basketball stadium; setting up and training IFOC Hawlwadag BRA013 Mogadishu Hawlwadag District Basketball Court Rehabilitation Rehabilitation of basketball stadium; setting up and training IFOC Karaan BRA008 Mogadishu Karaan District Qoob Dooro Playground Construction Engineering documents (TA, design, etc.) and community contracting; construction not yet started Warta Nabada BRA011 Mogadishu Warta Nabada District Basketball Court Rehabilitation Rehabilitation of basketball stadium; setting up and training IFOC Gedo Baardheere JSS033 Youth Motivational Sessions Motivational talks with youth on CVE Lower Juba Afmadow JSS017 JSS Afmadow Capital City Increasing Youth Participation and Role in Society Motivational talks with youth on CVE Afmadow JSS016 JSS Afmadow Women’s Market Constructed market; set up and trained IFOC Kismayo JSS003 JSS Ministry of Women, Family Affairs, and Human Rights Increasing Women’s Role in Peacebuilding and Social Reconciliation 5-day Consultation Forum – produced statewide Action Plan on Women’s Empowerment and Participation championed by Ministry of Women, Family Affairs, and Human Rights Kismayo JSS006 JSS Kismayo Int’l Women’s Day 1-day Int’l Women’s Day celebration in Kismayo Kismayo JSS013 JSS Kismayo National Youth Day Youth Engagement National Youth Day support with football, basketball, bicycle race, peace march in Kismayo. Kismayo JSS018 JSS Kismayo Youth Iftar 1-day Iftar event for 100 Kismayo youth Kismayo JSS024 JSS Ministry of Women, Family Affairs, and Human Rights Technical Coordinator/Gender Technical Coordinator support; rent office space for 12 months, office equipment for 4 staff, TPC and the Minister’s office, and cost to run office (internet, office stationery, and transportation) Kismayo JSS026 JSS Ministry of Youth and Sports Technical Coordinator/Sports Technical Coordinator support Bay Baidoa SWS002 SWS Ministry of Information and Social Awareness Promoting Peace and Stability through Arts and Culture Provision of musical equipment and music training for 8 youth group members to support development and dissemination of peace and stability messages Baidoa SWS003 SWS Ministry of Women, Family Affairs, and Human Rights 5-day workshop for 50 women from 18 SWS districts; development of agreed Action Plan on Women 94 Promoting Women’s Roles in Leadership and Stability in ISWA Empowerment and Participation championed by Ministry of Women, Family Affairs, and Human Rights Baidoa SWS008 SWS Ministry of Youth and Sports Baidoa Youth Day Supported National Youth Day celebration Baidoa SWS012 SWS Barawe District Town Peace Committee Youth Iftar Youth motivational talk for 100 youth in support of peace and stability; collection of stories on acts of kindness Dinsoor SWS013 SWS Reconstruction Dinsoor Town Women’s Center Construction Construction of Dinsoor Women’s Center; setting up and training IFOC Dinsoor SWS016 SWS Dinsoor District Football Stadium Construction Construction of football stadium; setting up and training IFOC Lower Shabelle Barawe SWS015 SWS MYS Barawe District Football Stadium Construction Development of engineering documents National NAT/FED SOM003 FGS Ministry of Women, Family Affairs, and Human Rights Development Technical Coordinator Stabilization Technical Coordinator support NAT/FED SOM014 Somali Women’s Basketball Support Regional Training 8 training sessions for already established women’s/girls’ basketball teams, provided new equipment, uniforms, and training for 96 girls in preparation for their participation in a national tournament; NAT/FED SOM015 Somali National Women’s Basketball Tournament National women’s basketball tournament with 8 regional teams (96 young women) participating, and the selection of a women’s national team; NAT/FED SOM017 FGS Ministry of Women, Family Affairs, and Human Rights Development Somali Women’s Cultural Festival 2-day Women’s Cultural Festival held in PSS, SWS, JSS, and BRA in which 80 women and 20 men participated; and 3-day Women’s Cultural Festival in Mog for 145 people from across Somalia NAT/FED SOM022 Mogadishu International Book Fair 3-day book fair attended by 2,500 people, featuring acclaimed authors, poets, scholars, playwrights and professors from around world. Youth Writers Workshop Series NAT/FED SOM027 Somali National Men’s Soccer Tournament Support National men’s soccer tournament in Mogadishu with 30,000 fans NAT/FED SOM029 Mogadishu Book Fair 2018 (MBF2018) 3-day Book Fair in Mogadishu; Youth Writers Workshop Series (6 sessions) at university; Baidoa Book Forum Series (6 sessions) at university NAT/FED Pre-PIKA Gender and Social Inclusion Support None to date; Yellow Light Approval pending The majority of the GESI-related PIKAs focused on providing five main types of support: (1) constructing sport stadia and associated trainings and tournaments to engage youth; (2) constructing women’s markets and centers; (3) holding celebrations and cultural events such as book fairs, International Women’s Day, and a Women’s Cultural Festival; (4) holding youth motivational talks; and (5) delivering technical assistance to support the functioning and gender-sensitivity of ministries. Significantly, the final PIKA listed involved hiring a GESI Specialist, which the program did not complete at the time of this report (June 2019). An important GESI-related activity to highlight not covered by the PIKA list above is TIS+’s “Young Engineers Initiative” (YEI). YEI was not originally planned in TIS+’s design documents but was an idea developed by the project’s Director of Engineering, who had worked on similar initiatives in other countries. In collaboration with the Banadir Regional Administration Department for Urban and Engineering (DUPENG), “TIS+ introduced a merit-based recruitment process in order to identify and hire individuals for six Junior Engineer positions. The recruitment aimed to expand the diversity within the engineering field by ensuring that the applicant pool consisted of youth, especially females and individuals from marginalized and trade clans. . . . Six finalists—one young woman and five young men all under the age of 30—were selected and offered short- 95 term positions as Junior Engineers and placed in the TIS+ [Mogadishu Field Office].”88 These Junior Engineers were then provided on-the-job training and mentorship by TIS+’s Senior Engineers over a period of six months on a variety of topics, including construction site supervision, environmental compliance, document control, quality assurance/quality control, software for engineering designs. “During their tenure, the Junior Engineers collectively completed more than 20 infrastructure designs, which will lead to the construction of 10 facilities, including new roads, sports grounds, and government offices.”89 Significantly, despite the significant challenges of finding employment in Somalia, particularly within the civil service, all six Junior Engineers were offered full-time positions by DUPENG following their six-month tenure with TIS+. Given YEI’s success in BRA, TIS+ intends to scale-up the program with the JSS, PSS and SWS Ministries of Public Works going forward. More broadly, interviews with PIKA participants and TIS+ staff, as well as TIS+’s own periodic reporting suggests that TIS+ had a clear focus on ensuring the participation of women and youth in the discussion, selection, and oversight of the other non-GESI specific PIKAs as well. According to TIS+’s 2018 annual report, 37 percent of its total participants since inception (19,288 of 51,498) have been females, while the assessment team’s review of sign-in sheets and periodic reporting suggests roughly 40 percent of Project Oversight Committee members were female and nearly 30 percent were youth. The program’s monitoring and evaluation (M&E) data did not capture information on clan status or other measures of marginalization. Interviews conducted as part of the midline suggest that while TIS+ implicitly sought members of minority clans in their activities but that this was not a priority focus. Intended Activities Not Implemented The original TIS+ GEWE Strategy (2015) suggested 23 concrete steps and activities that the program could implement in its first year to help the program on its way to achieving its higher-level results. These included steps such as conducting detailed gender assessments and subsequent studies highlighting root causes of gender inequalities, as well as potential gender champions and spoilers; using strategic communication to promote and influence public perception of women’s leadership abilities and highlight key gender success stories; providing dedicated education, vocational, and micro-credit trainings to women’s groups; providing women leadership courses and exchange forums; identifying and encouraging progressive male leadership and “gender champions;” providing gender-sensitive training and encouraging the use of male and female co￾facilitators during TIS+ trainings; updating the GEWE strategy on an annual basis; and finally, ensuring that the program’s intended social reconciliation and trauma healing activities would be gender-sensitive and include the participation of women. Based on interviews with beneficiaries and TIS+ staff, as well as a review of TIS+’s periodic reporting, the assessment team determined that the program completed 10 of 23 of the intended steps and activities at the time of the midline assessment (mid-2018). These included four general categories of activities, including ensuring a minimum level of female and youth participation in Community Consensus groups and POCs, using a combination of male and female facilitators in trainings, mapping existing gender-focused organizations in areas where TIS+ works, and engaging with progressive male leaders. Three critical areas where the program did not make sufficient progress include: (1) hiring a dedicated GESI Specialist and ensuring that GESI activities went beyond just establishing minimum quotas for participation; (2) using strategic communications and gender-sensitive messaging to disseminate “success stories” and highlight concrete examples of effective women’s leadership; and (3) providing targeted women’s leadership training and exchange forums to help motivate and capacitate women leaders. Future Steps According to the program’s own 2018 Annual Report, TIS+ recognized that the majority of its GESI efforts during its first three years of implementation focused mostly on ensuring participation as measured in numbers and that additional effort was needed to promote the quality of participation by women, youth, and 88 Ibid. 89 Ibid. 96 other minority groups in its activities. The program has responded with a revamped effort to deepen its GESI efforts and ensure that its GESI activities have more of a “felt impact” in the lives of its women, youth, and minority group beneficiaries. As the program did not have a dedicated GESI Specialist, these efforts have been spearheaded by the program’s Deputy Chief of Party with support from the AECOM Home Office Gender Advisor. A key step in deepening TIS+’s GESI efforts came in the form of four major revisions to the program GEWE strategy in late 2017. First, the strategy outlined key quality assurance steps that TIS+ would undertake going forward to ensure that gender and social inclusion considerations were carefully analyzed at the outset of each PIKA activity development process. This included the development of a GESI questionnaire that provided a targeted list of questions for staff to answer during their development process, including gender￾sensitive design questions, such as the locations of latrines and potential users and managers of facilities. This was intended to ensure that TIS+ staff carefully considered GESI challenges in their areas and provided tailored solutions and explicit linkages to how the proposed activity would address or respond to these challenges. Second, the Strategy renewed TIS+’s commitment to including gender in all training materials and also to ensure that all deliverables were reviewed with a gender focus. Third, TIS+ committed to ensuring that reporting on activities included specific information on steps the program has and will take to support women’s leadership. The assessment team noted that this focus has been more evident in TIS+ periodic reporting being during Quarter 3 of 2018. Fourth and finally, the Strategy outlined steps that the program would take to more thoroughly consider gender issues specific to construction projects. This included steps such as offering skills-based trainings in construction to women, identifying new opportunities for involving women more in the construction process, hiring female laborers, and conducting a mapping exercise to identify whether there are any female engineers or women-owned businesses that could be engaged for construction activities. Additionally, the program has developed a “GESI in Infrastructure Strategy” and supported the development of a “Sports for Development and Peace” (SDP) with the Somali Ministry of Youth & Sport (MOYS). In short, TIS+’s “GESI in Infrastructure Strategy” seeks to expand opportunity and benefits, vertically and horizontally “by ensuring the bid documents and contracts are GESI-sensitive and encouraging women-owned businesses (construction companies, supply companies, service companies) to bid on contracts.”90 It also commits the program to encourage more women-owned engineering and architecture businesses to bid on opportunities, as well as provide targeted capacity-building to these businesses to help make them more competitive. The strategy also identified ways of piloting women-only construction crews and the inclusion of persons with disabilities in construction. The SDP Strategy is covered in depth in Contribution Story #3. In short, the SDP Strategy posits that SDP can help reduce youth propensity toward violent extremism by offering an “attractive alternative” and using sports as a “hook” to attract at-risk youth.91 SDP was also assumed to bring additional benefits, such as physical and mental benefits of physical exercise; a mechanism for developing life and social skills such as teamwork, sportsmanship, and camaraderie; the possibility to link with educational and vocational opportunities to not only attract youth but also strengthen their capacity and potentially employability; and even to promote community-wide civic engagement through the use of Community Planning and Oversight approaches to the selection, implementation, and oversight of SDP projects, as well as an opportunity to promote shared-asset management between the community and local authorities. Assessment of Fidelity: Moderate. The majority of TIS+’s GESI efforts have focused on promoting the participation of women and youth in its activities. While program documents comment on the importance of including other minority groups, few concrete activities or steps for doing so are evident in either the program’s strategy documents or periodic reporting. To its credit, the program has undertaken at least 29 GESI-related PIKAs and ensured that minimum targets for the participation of women and youth across all its PIKAs have been met. However, the 90 Ibid. 91 Paraphrased from Stephen Reynard, “Somali Ministry of Youth & Sport (MOYS) Sport for Development and Peace (SDP) Strategy & Implementation Framework,“ ed. AECOM/TIS+ (August 2018). 97 program has not yet succeeded in strengthening the quality of participation of women, youth, and minority groups in its activities. EVIDENCE SUPPORTS INTENDED RESULTS According to TIS+ M&E data, 37 percent of TIS+’s participants have been female (19,288 of 51,498). The program’s Performance Management Plan identifies two standard US Government indicators focused on the participation of women. As of the 2018 Annual Report, the program has succeeded in ensuring 38 of its 91 POC members (42 percent) are women, which slightly exceeds its intended target of 40 percent. The assessment team reviewed POC sign-in sheets and confirmed an average of 40 percent of women members across the clusters reviewed. TIS+ did not meet its intended target for the “Number of local women participating in a substantive role or position in a peacebuilding process supported with USG.” The program reported reaching 141 out of an intended 300 during FY 2018. However, the program looks to be on track to reach its cumulative target at the end of the program. To date, the program reports having supported 803 women in a substantive role/position since its inception out of a total target of 850. Additionally, TIS+ reports that 9 of 26 of its staff are female, as well as 3 of 9 of its Technical Coordinators. TIS+’s Performance Management Plan does not include disaggregated targets for youth or other minorities. However, the assessment team reviewed POC sign-in sheets across its five assessment clusters and found an average participation rate for youth at around 30 percent. Unfortunately, TIS+’s project monitoring tools do not enable the assessment team to comment on the extent to which other minorities were included in activities. However, while TIS+ may have achieved its quantitative targets for the participation of women and youth, the assessment team’s qualitative interviews (both KIIs and FGDs) suggest that the quality of participation of these groups could be improved. As explained in greater detail in Section VII: Findings in the main body of the report, women and youth consistently requested additional capacity-building to enable them to engage more substantively in activities such as community contracting, PIKA project oversight, and facility management and maintenance. Additionally, a considerable minority of youth respondents expressed a desire to have more substantive roles on shared assets other than just the management of sports facilities. In terms of wider community perceptions, the midline assessment indicates quite positive and encouraging results related to gender equality and women’s participation in community planning and contracting (see Figure 13 and Figure 14 respectively on the following page). While the trends are positive in both TIS+ and non-TIS+ areas on community perceptions that “women are equal to men,” TIS+ areas showed slightly stronger results (+7 percent for Banadir and +6 percent for Gedo/Bay). In analyzing the associated sub￾questions, respondents generally justified their responses by noting changes in the perception of women’s leadership roles and responsibilities outside of the household. The results were more mixed on women’s participation in community planning and contracting. Again, the general trends were positive across both TIS+ and non-TIS+ areas; however, TIS+ areas performed slightly better in Banadir (+2 percent) but considerably worse in Gedo/Bay (-30 percent). Analyzing the associated sub-questions, the drop in Gedo/Bay is likely best explained by prevailing social norms and clan rules, as well as by a continuing perception that women lack the leadership skills to participate in such events. 98 Figure 13: Community Perceptions of Equality If no, which of the following makes the unequal? Lower Juba Banadir TIS+ Gedo/Bay TIS+ Lower Shabelle Banadir non￾TIS+ Gedo/Bay non-TIS+ Bakool Women are not as educated as men -11% 3% -18% 25% -8% 39% -8% Women’s sole responsibility is to raise the children 3% -11% -21% 18% -10% 1% -13% Women are necessary to maintain the household 3% 8% -17% 22% 8% 8% -10% Women are not able to speak in public 0% -1% -10% 1% 4% 18% 6% Women cannot make decisions for men 4% -21% -7% -9% 1% 4% -13% Women are not allowed to make decisions without men 5% -9% -13% -12% 1% 65% -6% Women are dependent on men for their protection and survival 6% -10% -2% -16% 4% 15% -7% Women lack leadership skills -18% -5% -12% 13% -2% 31% 0% 99 Figure 14: Community Perception of Women’s Participation in Community Planning and Contracting If “no,” what prevents them from participating? Lower Juba Banadir TIS+ Gedo/Bay TIS+ Lower Shabelle Banadir non￾TIS+ Gedo/Bay non-TIS+ Bakool Clan rules will not allow the participation of women 34% 54% -4% -13% 87% 24% -7% Only male members of the clan are entitled to participate in such events 3% 67% 14% -49% 72% 13% -11% Women lack the skills necessary to take part in such events -23% 36% 4% 4% 33% 33% -16% Only male members of the society have skills to manage/lead such responsibilities 4% 42% 23% -11% 36% 31% -14% 100 Assessment of Supporting Evidence: Moderate. Evidence of TIS+’s results across both the program’s own M&E data as well as the assessment team’s baseline and midline data suggests a mixed performance by the program in terms of GESI results. TIS+ has largely succeeded in meeting its output-level targets on the participation of women and youth. However, the program does not report on the participation of other minority groups and few concrete efforts to actively encourage their engagement are evident in program reports. While there are encouraging signs of improved trends on gender equality and women’s participation in community planning and contracting activities, both the assessment team’s survey and qualitative interview data, as well as TIS+’s own M&E reporting, suggests that the program is not yet adequately promoting and building women’s leadership skills. Likewise, there was little to no evidence that the program has succeeded in achieving outcome-level results specific to either youth or other minority groups. INFLUENCING FACTORS AND ALTERNATIVE EXPLANATIONS Strong Local Advocacy A key driving force behind the civil society in Somalia has been women-led organizations such as Save Somali Women and Children, Somali Women Development Centre, and the Somali National Women’s Centre, historically a key organization that has advocated for women’s empowerment since the 1970s. The existence of these robust civil society mechanisms has influenced a change in perceptions but more importantly, contributed to a present voice for women’s advocacy. Somali Government Efforts Under the 2012 elections rules, women were allocated a minimum quota of 30 percent of seats in the Federal Parliament. While the 30 percent quota was not achieved, the 2012 Parliament still saw the highest number of women parliamentarians at 14 percent. In 2015, as renewed advocacy began for the quota, the government engaged the Somali Religious Scholars Council, who issued a religious verdict supporting the quota and stating that Islam did not prevent women from participating in politics. At the executive level, the government elevated the government agency tasked with gender and children’s affairs to a full ministry: The Ministry of Women, Family Affairs, and Human Rights Development. In March 2019, the ministry endorsed the Somali Women’s Charter at the end of a three-day convention attended by over 300 female delegates calling for “full inclusion of women across the political, economic and social spectrum in Somalia.” In addition, the charter proposes a 50 percent quota across all three levels of government, including in the security sector and in public administration. It also calls for zero tolerance for gender-based violence and for women’s rights to be enshrined in the revised Constitution, and in the finalized electoral, security, and political laws. The launch of the charter was attended by Somalia’s Prime Minister, First Lady, and key political and religious leaders. Notably, at the event the Prime Minister stated that there was a “need to integrate gender and human rights in the work of all ministries.” The government has also made similar efforts to encourage the inclusion of disenfranchised young people, disabled people, and minorities. In June, the president signed a convention on the rights of the disabled. Role of International Actors While most international donors and development partners are careful to ensure that all their programs include a gender or social inclusion component, interviews conducted during the midline suggest that few donors are trying to address GESI in infrastructure. One exception is the Somalia Stability Fund (SSF), which has a specific output dedicated to the inclusion of women and marginalized communities. According to its 2018 annual report, SSF has tried to promote GESI through infrastructure in Baardhere, Balanbale, Abudwak, and Hobyo. Interestingly, an external midterm evaluation of these projects found similar results to the experience of TIS+, namely that while the program looks to have had some limited success in 101 enhancing women’s participation, it has had “less influence on changing norms around women’s and marginalized groups participation than hoped and the extent to which participation has led to an active voice for women and marginalized groups is mixed.”92 This finding was echoed in a separate interview with a local Somali scholar and gender advocate who had experience working with several major international donors on GESI issues. She argued that while she had seen donors commit to ensuring that women, youth, and marginalized groups participated in community planning sessions, their participation was often token and participants were handpicked by the local District Commissioner or other powerful local figures and not based on a careful analysis of who was really marginalized in the community. Assessment of Influence of Alternative Explanations: Strong. While there are relatively few international donors and development partners working specifically on GESI in infrastructure, there are a considerable number of other INGO/NGO, civil society, and government actors working on gender and inclusion issues more broadly across Somalia, including in nearly all TIS+ areas reviewed. Given the strong gender norms faced, multiple actors, and fairly limited substantive engagement on GESI, the assessment will be challenged in trying to assess the contribution to TIS+. ASSESSMENT OF CONTRIBUTION CLAIM Summary of Assessment Criteria Criteria Assessment Plausibility: Is the theory of change plausible? Weak. While TIS+ has undertaken efforts to identify key gender issues specific to civic participation in Somalia, its “Gender Equality and Women’s Empowerment Strategy” does not articulate an explicit ToC and oscillates between specific (and ambitious) steps the program will take in its first year and higher-level results with little detail on important intermediate results. Even less detail is provided on specific issues and steps to engage youth and other marginalized groups are rarely covered. Implementation per plan: Has the program been implemented with high fidelity? Moderate. TIS+ has worked hard to ensure at least 1/3 women and youth membership on POCs and has taken steps to implement 10/23 steps outlined in its GEWE strategy. However, the program has focused mostly on the number of participants and not the quality of participation, which its current M&E tools do not adequately capture. There was little documentary evidence of specific steps or results to encourage minority clan or other marginalized group participation. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? Moderate. TIS+ has broadly met its targets for the participation of women and youth in POCs and there is evidence (documentary and KII/FGD) that Community Consensus sessions included the mapping of key gender issues and existing organizations supporting gender-related work in its areas. However, there is little evidence that TIS+ has contributed to higher-, outcome-, or impact-level GESI results. Identification and examination of other influencing factors: To what extent have other influencing factors been identified? Identification of alternative explanations: To what extent do alternative explanations explain results? Strong. While there are relatively few international donors and development partners working specifically on GESI in infrastructure, there are a considerable number of other INGO/NGO, civil society, and government actors working on gender and inclusion issues more broadly across Somalia, including in nearly all TIS+ areas reviewed. Given the strong gender norms faced, multiple actors, and fairly limited substantive engagement on GESI, the assessment will be challenged in trying to assess the contribution to TIS+. 92 Adam Smith International, “Annual Report 2018: Somalia Stability Fund (Ssf),” (2019). Pg. 8. 102 Summary Assessment of Contribution Claim: Weak. TIS+ has demonstrated a commitment to analyze and mainstream GESI considerations across its activities. It has undertaken considerable efforts to understand specific challenges and opportunities related to the participation of women in its activities, and to a lesser extent also youth. The program has also developed and updated a concrete GEWE Strategy that outlines a number of steps that the program can implement to achieve its intended higher level of gender results. TIS+ has also succeeded in meeting its output-level targets related to the participation of women and youth in its activities. However, while the program may informally and implicitly consider strategies for involving other minority groups, specific efforts to do so are not visible in program documents or the program’s M&E tools. Likewise, the program’s intended intermediate results for it will link its gender and youth-focused activities to its intended higher, outcome￾level results are missing from its program documents and there is little evidence that the program has succeeded in supporting the quality of participation of women, youth, and other minority groups in its activities. While the program should be commended for recommitting itself to pursuing a “deeper” GESI approach that leads to more “felt” results by its beneficiaries, its efforts to date have been largely focused on ensuring minimum quantitative targets for participation have been met. 103 Sources Cited • Adam Smith International. “Annual Report 2018: Somalia Stability Fund (SSF).” (2019). • AECOM. “Gender Equality and Women’s Empowerment Strategy (Revised).” (December 2017). • AECOM. “General Principles of the O&M Plans for TIS+ Supported Infrastructure Activities in Somalia,” (February 2018). • AECOM. “TIS+ Gender Equality and Women’s Empowerment Strategy.” (September 2015). • AECOM. “Transition Initiatives for Stabilization Plus: FY 2018 Annual Progress and Financial Report.” (2019). • AECOM, “Transition Initiatives for Stabilization Plus (TIS+) Process of Engagement,” (March 2016). • Ali, Nimo-Ilhan. “Going on Tahriib: The Causes and Consequences of Somali Youth Migration to Europe.” The Rift Valley Institute, (2016). Goldwyn, Rachel, & Chiagas, Diana. (2013). “Monitoring and Evaluating Conflict Sensitivity: Methodological Challenges and Practical Solutions.” DFID, CARE, and CDE Collaborative Learning Projects Paper; Rogers, Mark. (2012). “Selecting Evaluation Approaches: Options in Peacebuilding Evaluation.” CARE International Paper. • Federal Government of Somalia Ministry of Interior and Federal Affairs, “Consensus Process Guidebook.” • IBTCI, “Final Performance Evaluation of the “Transition Initiatives for Stabilization” (TIS) Project,” ed. Somalia Program Support Services (June 2016). • IBTCI. “Gender Analysis Task Order: Final Report.” (May 2018). • IBTCI. “Transition Initiatives for Stabilization Plus: FY 2018 Annual Progress and Financial Report,” (2019). • International Labour Organisation. “ILO in Somalia.” (2017). • Mayne, John. (2008). “Contribution Analysis: An Approach to Exploring Cause and Effect,” http://dmeforpeace.org/sites/default/files/0501_Contribution_Analysis_ILAC.pdf • Mayne, John. (2011). Addressing cause and effect in simple and complex settings through contribution analysis. In: Schwartz, R., Forss, K., and Marra, M. (Eds.) Evaluating the Complex: Attribution, Contribution and Beyond. New York: Transaction Publishers. • MOIFAR, “Community Contracting Guide,” (2018). • Pact. “Gender Equality and Social Inclusion Analysis: Expanding Access to Justice (EAJ) Program.” (2018). • Quinn Patton, Michael. (2008). Utilization-Focused Evaluation: The New Century, 4th edition. Thousand Oaks: Sage Publications. • Reynard, Stephen. “Somali Ministry of Youth & Sport (MOYS) Sport for Development and Peace (SDP) Strategy & Implementation Framework,“ edited by AECOM/TIS+ (August 2018). • UK Home Office. “Country Information and Guidance: South and Central Somalia - Majority Clans and Minority Groups.” (March 2015). • UK Stabilization Unit. “Strategic Communications in Conflict and Stabilization Interventions,” July 2016. • UNHCR. “Cccm Cluster Somalia.” (2019). • UNICEF. “Education in Somalia.” • UNOCHA. “A Study on Minorities in Somalia.” (2002). • US Department of State, Bureau of Democracy, Human Rights, and Labor. “Country Reports on Human Rights Practices for 2015.” (2015). • The World Bank Group, “Community Driven Development,” https://www.worldbank.org/en/topic/communitydrivendevelopment#2. • World Bank. “Youth as Agents of Peace: Somalia.” (2018). 104 X. INFLUENCING ACTOR TABLES – BY CLUSTER BANADIR Major Clans: The Hawiye clan dominates the city and its surroundings, but because Mogadishu is the capital of the country, all major clans are present with varying population sizes. Minority Clans: Banadiri is the major minority clan attributed as some of the first settlers of Mogadishu; however, there are people from every Somali minority clan. There are also hundreds of thousands of IDPs, most of whom fled to Mogadishu in 2011 after Somalia’s worst famine in its history. Other Stabilization Partners Influencing Factors: Activity The Somalia Stability Fund (SSF) SSF has not worked sufficiently in Banadir to exert a significant influence on TIS+ results and is no longer working there. The UN Joint Programme on Local Governance (UNJPLG) JPLG has focused on national-level coordination with MOIFAR on the formation of district councils. TIS+ and JPLG overlap in Mogadishu in their work supporting the Wadajir Task Force at MOIFAR. World Bank Somali Urban Investment Planning Project This project is quite bounded but had some overlap with TIS+. This project focuses on building the capacity of the Banadir Regional Authority to conduct road investment projects. African Development Bank (AfDB) Multi-Partner Somalia Infrastructure Fund (SIF) The SIF has 27 pipeline projects in Banadir in its current phase (2017–21). These include repairs to Mogadishu airport, the Mogadishu household energy project, urban road renewal, the repair of Afgoye well-fields, and the water pipeline to Mogadishu/Afgoye. UNHABITAT UNDP has been supporting youth programming and support to the Banadir Regional Authority Planning Department. Turkish Government The Turkish government is working on road construction with the Banadir Regional Authority. It has also supported municipal service provision such as garbage collection, drainage clearance, and sanitation services. UNHCR and other humanitarian actors A number of humanitarian actors are working in Mogadishu, working both on government coordination of the Somalia-wide drought response and with drought-displaced and other IDPs in Mogadishu. Nordic International Support Foundation (Norway Funded – RNE) NISF runs the Bilateral Labor-Intensive Stabilization program with Norwegian support in Daynile and Yaqshid districts of Mogadishu, where it has worked/will work on solar streetlights, the electrification of Daynile hospital, and a vocational training school. Influencing Factors: Positive Political Will and Women’s Representation in Government 105 At baseline, the new president appointed a new Banadir region governor/mayor. Together with the president, the new mayor also replaced all former District Commissioners and appointed young District Commissioners, five of them women, making this Somalia’s largest proportion of female DCs. In addition, for the first time since the collapse of the state, someone from Darod Majeerteen (mainly from Puntland) was appointed as a DC in Banadir. This bold appointment is seen as fostering political inclusivity of the capital and cementing Mogadishu’s status as an all-Somali owned capital. There has not been noticeable resistance from the “locals.” Relatively Stronger Government Structures and Political Stability Over the past years, the Banadir region has been relatively stable politically, despite suffering from high levels of political violence and rapid turnover of administrators. A rough equilibrium exists in the city, expressed in terms of Mogadishu’s status as the capital and informal understandings about which clans have political preeminence in each district. These efforts were expected to impact gender and inclusion efforts of TIS+ and will positively influence community consensus, planning, financing, and monitoring processes. However, the weakness of these nascent capacities (particularly with new commissioners) will weaken the extent of their influence on TIS+ results, but TIS+ is specifically seeking to work with these representatives. There is the assumption that the more TIS+ has done to build them up, the more TIS+ will have contributed to community engagement results that are more inclusive. Relatively Better, and Improving, Infrastructure and Service Delivery The rehabilitation of Mogadishu’s devastated infrastructure has progressed including public–private partnership projects for roads, hospitals, seaports, and airport rehabilitation. The baseline HHS found that a much larger percentage of respondents felt that the federal government and local administration were responsible for the maintenance of key structures and services. This was a departure from other clusters surveyed, where expectations of this were more moderate. Trends varied at midline. Wide Access to Information and Communication Infrastructure The proliferation of multiple media outlets, widespread availability of high-speed internet, and relatively better communication infrastructure compared to the rest of the country has contributed to increased citizens’ awareness of stabilization and wider development activities around them as well as higher levels of engagement with government and international actors. Influencing Factors: Negative Al-Shabaab Al-Shabaab formally retreated from Mogadishu in August 2011. However, they adopted a new strategy, exercising indirect control with an elaborate intelligence and extortion system. Al-Shabaab has maintained its attacks targeting government officials, security forces, as well as areas frequented by the civilian population. The group represents the single greatest threat to both the federal and regional government. The issue was compounded by the emergence of ISIS, which has also compounded the security threats faced by ordinary citizens. Non–Al-Shabaab Criminal Violence Mogadishu also suffers from non–Al-Shabaab violence with regular targeted killings, bombings, kidnappings, and pervasive criminal violence. Some districts are particularly affected. Heliwa and Daynile, for example, have high levels of assassinations that are attributed to both AS and other criminals. Non–Al-Shabaab violence contributes to the insecurity that is the main threat to stabilization in Mogadishu. Non–Al-Shabaab violence continues to affect stabilization with increasing news of organized crime and targeted assassination. Poor Coordination Among Security Actors 106 Poor coordination among Mogadishu’s security agencies—the police force, the National Intelligence and Security Agency, the Somali National Army (SNA) and AMISOM—has been cited as a cause of insecurity in Mogadishu. There have been historical reports of AMISOM and the SNA conducting operations without informing district authorities or the police. Land Disputes Very high real estate values in parts of the city raise the stakes over ownership to levels worth fighting for. High demand for real estate, including by Somalia’s large diaspora, increases values still further. Twenty-five years of displacement, squatting, undocumented sales, expropriation, and inheritances have rendered much of the city’s real estate very prone to contested claims. The Banadir administration has no effective land titling system that it can apply and enforce. Land disputes are still present in Mogadishu and they’re made worse by the high prices of real estate in the city, causing competition for the limited land resources. The inability of the formal court system to properly adjudicate and enforce settlements in land disputes further exacerbates the situation, creating a vacuum that has been exploited by AS, which has offered swift and enforceable verdicts on land disputes in Mogadishu. High Unemployment In spite of improvement in economic growth, Somali cities have high rates of unemployment and underemployment. The link between youth unemployment and violent extremism is nonlinear and complicated, but youth unemployment is widely cited as both a cause and result of insecurity. Unemployment amongst young people continues to make them vulnerable to organized crime and radical groups. Drought and High WASH Vulnerability and Drought-Driven Displacement All of Mogadishu is deemed to have extremely high Water, Hygiene, and Sanitation (WASH) vulnerability based on Acute Watery Diarrhea (AWD)/cholera risk, flood risk, drought risk, number of IDPs, water, and sanitation coverage. More than 2.6 million people have been displaced by drought, and most of them have settled in urban areas, including an estimated 173,230 new drought-related IDPs in Mogadishu. Although drought allows FGS to be seen as the provider/coordinator of humanitarian assistance, drought and resultant displacement can be a significant conflict driver. Response to the 2017–18 drought was well-coordinated and timely and thus, huge displacement from the countryside was averted. Gedo/Bay Region Major Clans: Major clans present in Gedo are sub-clans of Darod (Marehan and Ogaden), Rahanweyn, Hawiye, and Dir. Minority Clans: The main minority clans are Jareerweyne, who are spread throughout the region. Other Stabilization Partners Influencing Factors: Activity The Somalia Stability Fund (SSF) SSF has a moderate presence in Gedo, mainly due to its continued work in Baardheere. SSF has previously worked in El Wak through Pact, but no longer operates there. In Baardheere, SSF is working on infrastructure including a district office, solar streetlights, a sports stadium, a community center, and a market. The projects were implemented on behalf of SSF by NISF. SSF has coordinated closely with TIS+ to avoid duplication. The UN Joint Programme on Local Governance (UNJPLG) UNJPLG and TIS+ work in different parts of a stabilization continuum, with UNJPLG generally coming in at a later point than TIS+. They are overlapping only in Garbahaarey, which has been prioritized for the formation of a district council for which JPLG will take the lead in assistance. 107 The UK Early Recovery Initiative (ERI) ERI and TIS+ work in different parts of a stabilization continuum, with ERI generally coming in at an earlier point than TIS+ as one of the first responders, along with military actors. ERI is present in Garbahaarey, working on District Council formation. Local Governance Consortium (EU funded) Led by Finn Church Aid (FCA) FCA is supporting District Council formation in Doolow. It has governance components (e.g., community consultation, gender, civic education, peacebuilding, and capacity-building). USG Partners Influencing Factors: Activity The US Office of Transition Initiatives (OTI) TIS+ and OTI are designed to work together to impact stabilization needs for the USG as well as political imperatives evident in a transition context. OTI was present in Garbahaarey working on support to schools, capacity support to District Administration, empowering women’s groups, and engaging local youth as peace ambassadors. In El Wak, OTI is working on engaging youth through soccer. Somalis Harmonizing Inter￾and Intra-Communal Relationships (SHIIR) SHIIR has been operating cross-border community work in El Wak, implemented by Pact. This intervention is working on only one TIS+ area, and on a different type of intervention than TIS+, although there are elements of this program that are complements and impact TIS+ interventions. USAID and UNICEF Pastoralist Education Program Targets Gedo and Bay regions. Influencing Factors: Positive Cross-Border Trade and Economic Growth Although there have been conflicts over control of trading opportunities, Gedo’s geographical position means it benefits from trade with Kenya and Ethiopia, contributing to a thriving economy and a fairly stable economic environment in border towns. Established Peace and Safety Committees Outside the more recently liberated Baardheere, there are established peace and safety committees in Gedo that comprise local elders, religious leaders, clan leaders, and district administrations. They play roles of conflict and dispute resolution. There are also established women and youth organizations. Awareness of local oversight or planning committees was higher in this cluster than in others in the baseline HHS, showing the influence of these structures. Established District Security Institutions There are district and regional security institutions, including police units, intelligence, custodian authority, and district courts. The existence of these institutions should help the FGS provide better rule of law. However, in the baseline HHS, respondents in this cluster ranked security services very badly. Influencing Factors: Negative Al-Shabaab While AMISOM and SNA have successfully liberated many districts (most recently Baardhere in 2014), AS still controls rural parts of El Wak (and they conducted a raid on El Wak town in 2016), Luuq, Garbahaarey, and Baardhere. AS restricts trade and movement of people in the areas under 108 their control and there have been targeted killings of people perceived to oppose them. Although AS is deemed to have lost support and popularity in Gedo, their targeting of opponents has the capacity to undermine several key TIS+ mechanisms of change. AS has also primarily sought to ensure restricted access between the towns in the cluster by ambushing commercial and government convoys. Persistent Inter- and Intra-clan Conflicts These include protracted conflict between the Marehan and Garre clans over political control of El Wak, unresolved by various peace processes; political tensions among Marehan sub-clans over the sharing of parliamentary seats; and resource-based conflicts between Garre and Ali Dhere (Marehan) also in El Wak. In Garbahaarey there are intra-Marehan rivalries over territorial, trade, and administrative control. In Baardheere there has been frequent conflict over control of district administration and trade routes among Maheran sub-clans. The baseline and midline HHS as well as qualitative interviews found that long-standing clan conflicts cause severe mistrust, and AS is seen by some respondents as a “manipulator” of these sentiments. These are structural and sometimes proximate conflict drivers. Conflict Between Federal Government and State Government Political conflict between the Mogadishu-based federal government and the state government based in Kismayo has exacerbated tension and contributed to instability in towns such as Baardheere. This has had an impact on expected government buy-in, accessibility of towns and stability. The conflict between the federal and state governments impacts the TIS+ delivery schedule and the expected government contribution in terms of buy-in. This also contributes to tension in the community, which has taken on a clan dynamic in towns such as Baardheere. Lack of Infrastructure Some of the region is very remote and lacks basic infrastructure, except in border towns. Most locations in Gedo can be accessed by land and air, but some remote villages are inaccessible and many roads between districts and neighboring regions are in bad condition. Lack of infrastructure makes it harder to have an impact in delivering services and demonstrating the value of government as a service provider. Lack of Fully Functioning District and Regional Administrations The region and its six districts have not elected functioning district and regional administrations. Administrations are often understaffed and under￾resourced. There is not enough staff or resources to support most of the local and regional needs, particularly around access to safe roads and to key services. Drought and High Water, Sanitation, and Hygiene (WASH) Vulnerability Gedo is deemed to have extremely high (clean) Water, Sanitation, and Hygiene (WASH) vulnerability (except for El Wak, which has very high vulnerability) based on AWD/cholera risk, flood risk, drought risk, number of IDPs, water, and sanitation coverage. Drought has prompted migration into urban centers in Gedo since November 2016. Although drought allows FGS to be the provider/coordinator of humanitarian assistance, drought and resultant displacement can be a significant conflict driver. It has already caused depletion of livestock and agriculture-based livelihoods and increased clan conflicts over recent years. The increasing effects of climate change will contribute to the stresses on the minimal water and pasture resources and has the potential to exacerbate micro-conflicts that have often been overlooked. Influencing Factors: Unknown Regional Powers and the Ethiopia National Defense Forces (ENDF) and Kenya Defense Forces (KDF) The ENDF and KDF are present because of the borders that the region shares with Ethiopia and Kenya. Kenya and Ethiopia are not seen as impartial and AS has in the past manipulated the perception that these forces favored certain clans/sub-clans over others. The planned AMISOM withdrawal from Somalia, if it materializes, will much depend on how much land is recovered from AS and how the local communities are incorporated to the new political setup. However, Kenya and Ethiopian troops are likely to stay longer, especially in this part of the country, to 109 prevent further influx of extremism and illegal commerce to their borders. Ethiopia has treated the Gedo region as its sphere of influence since the mid-90s when it quelled the extremist Al Ittihad (Jihadist organization linked to Ogaden National Liberation Front, also on the US Terrorist List) that formed military training in border towns of Gedo (Luuq and Beled Hawo). The ENDF and KDF forces have had both positive and negative influences on the region. For example, the Kenyans have been accused of facilitating the illegal charcoal trade as well as the movement of contraband sugar and poaching. The Ethiopians have been accused of human rights abuses, especially in the towns on the border between the two countries. At the same time, both forces play a counteracting role against AS, which has influenced the security dynamics in the region. AMISOM and the Somali National Army (SNA) The implementation of the new FGS Security Architecture endorsed by Federal Member States and later approved by the FGS parliament may incorporate Jubaland forces into the SNA. Ras Kamboni/IJA forces The Ras Kamboni militia headed by Jubaland’s current president, Ahmed Madobe, helped liberate towns along the Lower Jubba culminating in the capture of the capital Kismayo in 2011. They have served as the de facto regional security agency in the Interim Jubba Administration. Even though, their influence is in the primarily Ogaden dominated areas, it remains to be seen how they will fit into the National Security Architecture. Ahlu Sunnah Waljama’a (ASWJ) The ASWJ played a crucial role in liberating towns such as Baardhere from AS with support from Ethiopia. They currently have political and military control in some of the towns in Gedo. They have been accommodating of the SNA troops in their presence, but because they do not form part of the national security architecture, their role as future stabilization actors has remained unclear. 110 LOWER JUBA Major Clans: All Somali major clans (Darod, Hawiye, Dir, and Rahanweyn) are found in this region. There are over 40 sub-clans in the region, including Darod, Digil, Mirifle (Rahanweyn), Hawiye, and Dir sub clans, making it one of the most diverse regions in Somalia. Minority Clans: Some of the minority clans are Jareerweyne, Bajun, and Boni/Reebi. Other Stabilization Partners Influencing Factors: Activity The Somalia Stability Fund (SSF) SSF has worked in Afmadow, where they focused mostly on employment-intensive works clearing an airstrip and building two youth centers, and in Kismayo, where they supported the Jubaland Public Financial Management system and the Civil Service Commission and built the Ministry of Finance building as well as a youth center. SSF and TIS+ collaborated closely to avoid duplication. However, SSF is mostly working in different spheres than TIS+ and is not focused on a community engagement process. The UN Joint Programme on Local Governance (UNJPLG) JPLG and TIS+ work in different parts of a stabilization continuum, with JPLG generally coming in at a later point than TIS+. Kismayo was targeted by TIS+ for a Bridging Plan, which is designed for areas that have progressed further from early stabilization. IOM/UNHABITAT IOM/UNHABITAT worked in Kismayo on returnees and capacity-building for local administrations. This work is in a very different area than TIS+. AfDB Multi-Partner Somalia Infrastructure Fund (SIF) The SIF has worked on technical assistance to infrastructure ministries in Jubaland as well as support to infrastructure improvement of Kismayo port, airport, key roads, and the Kismayo electric power grid and water supply system. Local Governance Consortium (EU funded) Led by Finn Church Aid (FCA) This intervention supported District Council formation in Afmadow. It has governance components (e.g., community consultation, gender, civic education, peacebuilding, and capacity-building). FCA district formation work included Afmadow community consultations. This process included activities similar to TIS+, such as community consultations, strengthening/capacity-building of district administration with gender, conflict resolution, quick-impact projects, and civic education components. American Refugee Committee The committee worked on a fish market, a meat market, a vegetable market, and a workshop for port maintenance (handing over to Jubaland Chamber of Commerce and Industry). Additional projects included a youth/sports center, training for football teams, a returnee support center, and a short-story writing competition. Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GIZ has completed a women’s center and plans a milk market in Kismayo. GIZ is also expanding into vocational training and sports for development. The UK Early Recovery Initiative (ERI) ERI has worked in the villages around Kismayo on the frontline of the campaign against AS. ERI and TIS+ work in different parts of a stabilization continuum, with ERI generally coming in at an earlier point than TIS+ as one of the first responders, along with military actors. Other USG Partners Influencing Factors: Activity 111 The US Office of Transition Initiatives (OTI) At baseline OTI was working in Kismayo and Afmadow. In Kismayo, work includes dialogue between returnees and host communities, radio drama and PSAs, 16 Days of Activism, youth engagement through soccer, and female youth and sports. In Afmadow, work includes empowering youth through sports. Somalis Harmonizing Inter￾and Intra-Communal Relationships (SHIIR) SHIIR worked only in Dhobley town in Lower Juba. Influencing Factors: Positive Established Existence and Awareness of Peace and Safety Committees and other Local Capacities At baseline, there were peace committees, women’s groups, youth groups, and business groups in government-held areas, and traditional leaders play important roles in peace building. The baseline HHS found high levels of awareness and perceptions of the effectiveness of local oversight or planning committees in this cluster, perhaps reflecting the efforts of INGOs and stabilization actors. Established Chamber of Commerce that is Willing to Collaborate Of the established capacities and nongovernmental organizations mentioned above, a particularly valuable partner for TIS+ is the Jubaland Chamber of Commerce, which has also demonstrated a willingness and capacity to collaborate including leveraging its own resources to contribute to projects. The TIS+ is explicitly designed to leverage support and resources from actors such as chambers of commerce. District Security Institutions In Place There are functioning police stations, correctional facilities, and district courts in Kismayo and Afmadow. Nascent District and Regional Authorities Functioning district and regional authorities exist in the region, but these have been appointed and not elected. The existence of these institutions, if built upon, should help contribute to greater trust in government as the authorities provide democratic accountability and service delivery. 112 Influencing Factors: Negative Al-Shabaab Although the Jubaland Army, SNA, and AMISOM control the Kismayo, Afmadow, and Badhadhe districts, AS controls some rural areas with mobile units and remains in control of the Jamame and Hagar districts, restricting the movement of people and trade in the Lower Juba region. This in turn affects the purchasing power of people in the region. Movement of government officials and aid workers remains difficult. Although AS is a serious proximate conflict driver, its presence also serves to suppress some of the clan and sub-clan conflicts described below and serves as an external enemy uniting actors that would otherwise be in conflict among themselves. Historical Grievances and Persistent Inter- And Intra-Clan Conflicts Lower Juba has been the focus of protracted conflict between different clans, sub-clan, and political actors since the collapse of the Somali state. Struggles over ownership and control of resources as well as political ideologies and control have caused significant grievances. The main players are Ogaden, Harti and Marehan (Darod), Rahanweyn, and the Hawiye. Different Darod sub-clans and Hawiye all make historical claims to the control of Kismayo. As noted above, the presence of AS Imay serve to mask or suppress these conflicts. These are structural and proximate conflict drivers. Removal of AS from Middle Juba and parts of Lower Juba by the long-awaited “Jubaland Offensive” may create a vacuum. Continued Contestation over Jubaland State Formation Because of the protracted conflicts mentioned above, there has been political contestation over the formation of Jubaland state, with new President Hassan Sheikh Mohamud declaring the process had not consulted stakeholder clans. Since the Intergovernmental Authority on Development (IGAD) brokered an agreement between the FGS and the new Jubaland administration, the latter has made progress in accommodating and reconciling with political opponents. However, many key actors still feel marginalized. Threats And Limits to Political Rights There have been reports of arrests and detention of some voicing political opposition to the Jubaland administration. If these reports continue, they can risk undermining the credibility of the new administration and further progress of democratization and good governance. Drought and High Water, Sanitation, and Hygiene (WASH) Vulnerability Lower Juba is deemed to have extremely high Water, Sanitation, and Hygiene (WASH) vulnerability (except for Badhadhe, which has very high vulnerability) based on AWD/cholera risk, flood risk, drought risk, number of IDPs, water, and sanitation coverage. Although drought allows FGS to be the provider/coordinator of humanitarian assistance, drought and resultant displacement can be a significant conflict driver. With increased climate crisis stress, this is likely to become a stabilization factor in the years to come. 113 Influencing Factors: Unknown Highly Strategic Location The strategic location of Kismayo is a key advantage for economic growth and development but also accounts for the protracted conflict over the city and the fact that it has changed hands so much during the conflict. Kismayo, a deep-water port, is the commercial hub of the Jubaland region and southern parts of the country located halfway between Mogadishu and the Kenyan border. It acts as a commercial center for products from the pastoral, fishery, and agriculturally rich hinterland. The strategic location of Kismayo is a key advantage economically but conflict over control of this resource has also acted as a structural conflict driver. Kismayo is unfortunately also at the epicenter of the charcoal trade that has devastated the environment in Lower Jubba and also contributed to millions of dollars to the AS coffers. In 2012, the UN Security Council passed a resolution banning the export and sale of Somali charcoal. This contributed to a reduction of the amount of charcoal exported through Kismayo, but the trade goes on undeterred in smaller ports such as Bur Gaabo. Regional Powers – The Ethiopia National Defense Forces (ENDF) and Kenya Defense Forces (KDF) The Kenya Defense Forces (KDF) and Ethiopian National Defense Forces (ENDF) are part of AMISOM that was part of the movement that liberated Kismayo and are also key stakeholders in Jubaland. The KDF are based in AMISOM Sector Two along the western side of Lower Juba, with key bases in Kismayo’s airport and seaport as well as along the road running from west to east between Dhobley and Afmadow towns. Many Somalis have long accused neighboring countries of having a destabilizing effect on Somalia. Kenyan and Ethiopian involvement in the Jubaland process is seen by some as a self-interested attempt to establish proxies. KDF and ENDF soldiers have retreated from several bases, and this is anticipated to give more space for AS to recapture or reoccupy liberated towns in Lower Juba. The planned AMISOM withdrawal from Somalia, if it materializes, will much depend on how much land is recovered from AS and how the local communities, (especially the clans not feeling accommodated by the current Jubaland administration) are incorporated into the new political setup. However, Kenyan and Ethiopian troops are likely to stay longer, especially in this part of the country, to prevent further influx of extremism and illegal commerce to their borders. Regional Powers – The Intergovernmental Authority on Development (IGAD) IGAD comprises several states in the region and has supported the Jubaland Initiative since mid-2012. Common interests between Kenya and Ethiopia formed the basis for the 2012 IGAD Grand Stabilization Plan for South Central Somalia. Kenya and Ethiopia are both believed to have been keen to establish a buffer zone to protect their respective borders with Somalia. AMISOM and the Somali National Army (SNA) The SNA in Lower Juba is made up of the Kenya-backed and Isiolo-trained troops who officially fall under the authority of the Jubaland administration. There have been criticisms that there was no SNA in Kismayo and merely clan militias. The implementation of the new FGS Security Architecture endorsed by Federal Member States and later approved by the FGS parliament may incorporate Jubaland forces to the SNA. Ras Kamboni Brigade (RKB) This anti–Al-Shabaab militia was part of the offensive with Kenya that secured Kismayo and the controversial appointment of Ahmed Madobe as president thereafter. This militia has evolved and has become the Jubaland force that is hugely engaged in stabilization of Kismayo and Afmadow. 114 LOWER SHABELLE Clans: As one of the most populated regions, Lower Shabelle hosts the biggest number of clans. The clans are spread throughout the seven districts of the region and there is no one clan that dominates all districts or even most districts. Three out of the four Somali major clans (based on the 4.5 power-sharing formula, major clans are Darod, Hawiye, Dir, and Rahanweyn), namely Hawiye, Dir, and Rahanweyn, are spread throughout the region. However, there are a significant number of minority clans, some of them dominating districts, for example Barawe. Other Stabilization Partners Influencing Factors: Activity The Somalia Stability Fund (SSF) SSF was present only in Barawe, where worked to build an airstrip and have provided some technical assistance to government. They have not conducted any community engagement work here. The UN Joint Programme on Local Governance (UNJPLG) JPLG and TIS+ work in different parts of a stabilization continuum, with JPLG generally coming in at a later point than TIS+. JPLG works in Afgoye and Wanlaweyn to support District Council formation. UNDP support to FGS Stabilization Plan UNDP has been supporting District Peace and Stability Committees in Barawe. AfDB Multi-Partner Somalia Infrastructure Fund (SIF) The SIF has worked on technical assistance to South West infrastructure ministries as well as supported repairs to local roads in Barawe and Merka and repairs to Barawe port. TIS+ and USAID discussed partnering on 30 km of the Mogadishu-Afgoye road project. Nordic International Support Foundation (UK and Norwegian funding) NISF has worked in Wanlaweyn, where it has supported 75 solar streetlights, market refurbishment, and rehabilitation of the District Community Center. It is not clear in the TIS+ mapping whether this is the same as the ERI work in Wanlaweyn noted below. NISF also runs the Bilateral Labor-Intensive Stabilization program with Norwegian support in Afgoye, where it has worked/will work on solar streetlights, canal excavation, and road rehabilitation. The UK Early Recovery Initiative (ERI) ERI has worked in Wanlaweyn, Barawe, Merka, Afgooye, Sabiid, Anole, and Qoryooley. ERI and TIS+ work in different parts of a stabilization continuum, with ERI generally coming in at an earlier point than TIS+ as one of the first responders, along with military actors. Other USG Partners Influencing Factors: Activity The US Office of Transition Initiatives (OTI) At Baseline, OTI is present in Wanlaweyn, where it has worked on women’s dialogue, soccer and dialogue, capacity support to District Administration, and conflict prevention dialogue. OTI was also present in Merka and Qoryooley, where it has worked on conflict prevention dialogue. It also worked in Afgoye, where it has worked on community engagement through participatory theatre, conflict prevention dialogue, and a cultural festival, and in Barawe, where it worked on conflict prevention dialogue. Harnessing Opportunities for Peaceful Engagement in Afgoye (HOPE) project (IRC) HOPE works on increasing community resilience to violence in South Central Somalia by strengthening community relations and decision-making at the local level. The project works on both community engagement and strengthening local government institutions. Influencing Factors: Positive Nascent, but Limited Peace and Safety Committees and other Local Capacities 115 At baseline, there were peace committees, women’s groups, youth groups, and business groups in government-held areas, and traditional leaders play important roles in peace building. They are limited in their capacities to fulfill their mandate as they are newly formed. The baseline HHS found that awareness of local oversight or planning committees was also extremely low in this cluster. Nascent, and Limited District and Regional Authorities There is a Regional Governor for Lower Shabelle Region, and in the liberated districts there are local administrations headed by District Commissioners (DCs) and their deputies. However, in the three districts that are still under control of AS, local administration structures are lacking. There are no structures such as DCs and community-level social groups below the Regional Governor. The existence of these institutions, if built upon, should help contribute to greater trust in government as the authorities provide democratic accountability and service delivery. These institutions are starting from a low base in this case. Influencing Factors: Negative Al-Shabaab In this region, AS has complete control over only Kurtunwaarey and Sablaale. However, AMISOM and SNA are present in only urban towns or district capitals and their suburbs (Afgoye, Qoryoley, Wanlaweyn, Marka, and Barawe) and AS rules the rest of the region including roads between those urban towns under AMISOM and SNA. AS carries out hit-and-run guerrilla attacks and ambushes against AMISOM and SNA bases. However, AS does not restrict the movement of civilians to and from districts under its control to the same extent as in other regions, perhaps because of the need to maintain the flow of agricultural goods from which it extracts significant revenues. There is less local resistance to AS than in other parts of Somalia. This is a proximate conflict driver. AS targeting of opponents has the capacity to undermine several key TIS+ mechanisms of change. In addition, if it appears to provide better governance and rule of law than FGS, this will seriously undermine FGS trust and legitimacy. Perception that AMISOM/SNA Areas Are Less Secure Reports of mismanagement insecurity and human rights violations committed by the SNA, who are from outside the region, compounded by the history of abuses by warlords and militias, also from outside the region, against locals have contributed to the perception that Al-Shabaab– controlled areas are more secure. Areas under the SNA are thought to have a lot of illegal checkpoints that each levy tax from trucks, passenger buses, and even private cars, while areas under AS have fewer checkpoints and levy tax only after issuing an “official receipt.” The baseline HHS confirms that the “government” (mainly referred to as FGS) is unpopular in this cluster and area of Somalia. AMISOM and the Somali National Army (SNA) Reports of human rights abuses by AMISOM and the SNA (who are also perceived as outsiders) have a current negative impact on stabilization aims and FGS trust and legitimacy. The baseline HHS confirms that SNA’s reputation has been damaged as they are seen locally as contributing to inter-clan tensions. Inter- and Intra-Clan Conflicts, and Conflicts Between “New Settler” and “Original Inhabitant” Communities There is a plethora of conflicts over competition for agricultural resources in the districts and villages along the long and fertile Shabelle river and over market and commerce in the coastal districts. Many grievances surround the tension between “new settler” communities, who migrated south from central regions and gained control of valuable resources such as farmland from older original communities, and “original inhabitant” communities. These dynamics are mostly unresolved, continue to represent major political fault lines, and have been manipulated by AS. There are also conflicts over political control, notably between Biyomal and Habar Gidir in Marka, but also many others between clans and sub-clans at the district levels that are latent and/or suppressed by AS. These conflicts are expected to emerge when AS is removed. These are structural and proximate conflict drivers. 116 Lack of Infrastructure and Services Social services and infrastructure are poor. Schools and hospitals exist but lack infrastructure, equipment, and qualified staff. In some districts, Marka for instance, facilities such as water are privately owned and are only accessible at a fee. Drought and High WASH Vulnerability and Drought-Driven Displacement Lower Shabelle is deemed to have extremely high Water, Hygiene, and Sanitation (WASH) vulnerability (except for Sablaale, which has very high vulnerability) based on AWD/cholera risk, flood risk, drought risk, number of IDPs, water, and sanitation coverage. Influencing Factors: Unknown Large Agricultural and Farming Resources Large farming projects, legacies of Italian investments from the early 1900s, as well as Somali Government projects after 1960, are an obvious asset from the perspective of longer-term economic growth and development. But these resources have become a curse during the conflict, as evidenced by protracted conflicts over who controls them. Warlords seized these resources as well as access to coastal towns, displacing local communities. Subsequently the Union of Islamic Courts and later AS returned some of these areas to their original owners, gaining increased legitimacy as a result. SOMALIA PROGRAM SUPPORT SERVICES TRANSITION INITIATIVES FOR STABILIZATION PLUS MIDLINE ASSESSMENT, WAVE 1 FINAL REPORT DELIVERABLE #17 IDIQ AID-623-I-14-00009 TASK ORDER AID-623-TO-16-00015 Resubmitted: January 30, 2019 This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by International Business & Technical Consultants, Inc. (IBTCI), under IDIQ AID-623-I-14-00009, Task Order AID-623-TO-16-00015. 118 SOMALIA PROGRAM SUPPORT SERVICES (SPSS) TRANSITION INITIATIVES FOR STABILIZATION PLUS (TIS+) MIDLINE ASSESSMENT, WAVE 1 FINAL REPORT DELIVERABLE #17 IDIQ AID-623-I-14-00009, TASK ORDER AID-623-TO-16-00015 Resubmitted: January 30, 2019 Contributors: Sarah Wood (Team Leader), Lucas Malla (Quantitative Specialist), Patrick Okwarah (Qualitative Specialist), Suleiman Omar Subject Matter Specialist and Contribution Analysis), and Anna Paterson (Contribution Analysis) Cover Photo: Alamy Stock Photo, Somalia 2016 International Business & Technical Consultants, Inc. (IBTCI) In US: 8618 Westwood Center Drive, Suite 400 Vienna, VA 22182 USA +1.703.749.0100 In Kenya: 8th Floor Block C Westcom Point Building Mahiga Mairu Road, off Waiyaki Way Nairobi, Kenya +254 (0) 737 544 912 IDIQ AID-632-I-14-00009 Task Order AID-623-TO-16-00015 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 119 Table of Contents ACRONYMS...................................................................................................................121 EXECUTIVE SUMMARY..............................................................................................124 Lower Juba Summary................................................................................................................................................................. 125 Lower Shabelle Summary......................................................................................................................................................... 128 I. INTRODUCTION ...............................................................................................130 Project Background..................................................................................................................................................................... 131 II. TASK ORDER PURPOSE AND OBJECTIVE...................................................131 Task Order Purpose ................................................................................................................................................................... 131 Assessment Questions................................................................................................................................................................ 132 III. METHODS AND LIMITATIONS ......................................................................133 Limitations and Operational Challenges............................................................................................................................... 136 Security and Access.................................................................................................................................................................... 137 VI. CONTRIBUTION ANALYSIS AND TIS+........................................................137 The Role of Contribution Analysis........................................................................................................................................... 137 Testing the Theory of Change for TIS+................................................................................................................................ 138 V. DATA ANALYSIS ...............................................................................................140 Influencing Factors and Alternative Explanations.............................................................................................................. 141 Developing a Stabilization Narrative: Contribution Stories............................................................................................. 141 VI. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS.......................142 VII. LOWER JUBA......................................................................................................144 Section 1: Political, Economic, and Security Analysis ............................................................. 144 Section 2: Lower Juba Findings ............................................................................................... 145 Assessment Question 1 .......................................................................................................... 147 A. Community Ownership of Development Projects and Processes......................................................................... 147 B. Effective Shared Asset Management........................................................................................................................... 150 C. Improved Delivery of Services........................................................................................................................................ 152 D. Reduced Conflict over Resources.................................................................................................................................. 155 E. Greater Citizen Confidence in Government............................................................................................................... 156 F. Reduced Support for Violent Extremism..................................................................................................................... 157 Assessment Question 2 .......................................................................................................... 158 Assessment Question 3 .......................................................................................................... 160 Assessment Question 4 .......................................................................................................... 161 Section 3: Lower Juba Conclusions ........................................................................................ 163 Section 4: Lower Juba Recommendations .............................................................................. 167 VIII. LOWER SHABELLE ...........................................................................................169 Section 1: Political, Economic, and Security Analysis ............................................................. 169 Section 2: Lower Shabelle Findings ......................................................................................... 171 Assessment Question 1 .......................................................................................................... 173 A. Community Ownership of Development Projects and Processes......................................................................... 173 B. Effective Shared Asset Management........................................................................................................................... 174 C. Improved Delivery of Services........................................................................................................................................ 176 120 D. Reduced Conflict over Resources.................................................................................................................................. 178 E. Greater Citizen Confidence in Government............................................................................................................... 179 F. Reduced Support for Violent Extremism..................................................................................................................... 180 Assessment Question 2 .......................................................................................................... 181 Assessment Question 3 .......................................................................................................... 182 Assessment Question 4 .......................................................................................................... 183 Section 3: Lower Shabelle Conclusions .................................................................................. 184 Section 4: Lower Shabelle Recommendations........................................................................ 188 121 ACRONYMS AfDB African Development Bank AMISOM African Union Mission in Somalia AS Al-Shabaab BRA Banadir Regional Administration CA Contribution Analysis CPO Community Planning and Oversight CVE Countering Violent Extremism DARS Data and Research Solutions FGD Focus Group Discussion FGS Federal Government of Somalia FMS Federal Member State GESI Gender Equality and Social Inclusion HH Household Survey IBTCI International Business and Technical Consultants, Inc. IDP Internally Displaced Persons IED Improvised Explosive Device IOM International Organization for Migration IP Implementing Partner JCCI Jubbaland Chamber of Commerce and Industry JSS Jubbaland State of Somalia KAQ Key Assessment Questions KDF Kenya Defense Forces KII Key Informant Interview LJ Lower Juba LS Lower Shabelle M&E Monitoring and Evaluation MOIFAR Ministry of Interior, Federal Affairs, and Reconciliation MOYS Ministry of Youth and Sports NDP National Development Plan NGO Nongovernmental Organization NISF Nordic International Support Foundation OTI Office of Transition Initiatives PESA Political Economy and Security Analysis POC Project Oversight Committees PPP Public–Private Partnership PIKA Pool of In-Kind Activities S4DP Sports for Development and Peace SIF Somalia Infrastructure Fund SNA Somali National Army SOO Statement of Operations SPSS Somalia Program Support Services SSF Somalia Stability Fund SSI Somalia Stabilization Initiative SWS South West State TC Technical Coordinator TIS Transition Initiative for Stabilization TIS+ Transition Initiative for Stabilization Plus 122 ToC Theory of Change UNDP United Nations Development Programme UNJPLG United Nations Joint Program for Local Governance UNFPA United Nations Population Fund UNOSOM United Nations Operations in Somalia USAID United States Agency for International Development USAID/SFO USAID/Somalia Field Office USG United States Government 123 Figure 1, TIS+ Baseline/Midline Assessment Geographical Clusters 124 EXECUTIVE SUMMARY Project Background The Transition Initiatives for Stabilization Plus (TIS+) activity is a five-year, US$66.9 million project implemented by AECOM that will run from May 2015 to April 2020. TIS+ seeks to implement community-centric stabilization activities that (1) are implemented with multiple levels of government (i.e., caretaker and interim administrations, new states, and the Federal Government of Somalia [FGS]); (2) support inclusive participation of stakeholder groups in community decision-making; and (3) encourage shared asset management between citizens and the government to demonstrate the government’s interest in an emerging capacity to contribute positively to citizens’ daily lives. TIS+ activities are designed to contribute to improvements in government service delivery, leading to strengthened governance and community cohesion. Task Order Purpose The primary purpose of the TIS+ baseline and midline assessment is to generate a better understanding of the extent to which TIS+ interventions have achieved their objectives and contributed to USAID’s goal of increased stability in Somalia through participatory processes that promote good governance and community cohesion. Specific clusters or geographic areas have been strategically chosen to be analyzed to assess the influence that the TIS+ program activities have had on mitigating the effects of violent extremism and insurgency in these high-risk areas of Somalia. During the midline, data was collected in two waves. In Wave I, data was collected in Lower Juba and Lower Shabelle, and in Wave II, data was collected in Gedo/Bay, Banadir, and Bakool. This report focuses on findings from Wave I data collected in Lower Juba and Lower Shabelle. The table below shows the questions asked at midline. No Midline Key Assessment Questions (KAQs) 1 Did TIS+ contribute to any of the following anticipated results? ➢ Community ownership of development projects and processes ➢ Effective shared asset management ➢ Improved delivery/satisfaction of services ➢ Reduced conflict over resources ➢ Greater citizen confidence in government ➢ Reduced support for violent extremism 2 Were TIS+ approaches used in the cluster (e.g., the community consensus process) instrumental in contributing to the achievement of TIS+ results described in Question 1? 3 Did government at all appropriate levels contribute sufficiently to asset management in communities in which TIS+ was implemented? How? Did government at all appropriate levels have both the capacity and the will to contribute? 4 Were TIS+ activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? Methods Both baseline and midline employed a mixed-methods approach that integrated a suite of quantitative and qualitative data collection methods. Data collection methods consisted of desk review/secondary literature review, a household survey, Key Informant Interviews (KIIs), and Focus Group Discussions (FGDs). Assessment questions were answered using a combination of analyses of data from these four collection methods and the use of contribution analysis to estimate TIS+’s unique contribution to observed midline results About Contribution Analysis Contribution analysis (CA) is an analytical approach to estimating program effects in complex contexts with many factors and actors influencing intervention outcomes. CA is a good option for a stabilization 125 program because it examines stabilization as a continuum and explores conflict dynamics. CA works within a Theory of Change (ToC) model to look at contribution from several angles. It employs an applied ToC approach to look beyond high-level outcomes (goal/objectives) to more specific activities and interventions across diverse project spaces. CA is a pragmatic and methodologically rigorous approach to situations for which the activity is one of many potential contributory factors and where it is not easy to measure attribution directly, such as is often the case in stabilization programming. Security and Access Somalia is a difficult country in which to gather data, and the team had to consider security limitations as well as the dynamic nature of districts and areas under varying stages of control by Al-Shabaab (AS). AS is a dynamic and increasingly networked group, both at the transnational and local level (cluster/district). It has continued to control many rural areas in several regions, including Lower Shabelle and Lower Juba. The Assessment Team designed the approach and management plan to mitigate anticipated effects as much as possible through means such as multiple data points for each question, confirmation of facts, customized data collection protocols that helped the team collect data in remote and insecure areas, and staggered data collection time frames. Developing Findings, Conclusions, and Recommendations TIS+ is a multilayered project, and this midline analysis focuses on two strategic cluster areas. These are not administrative regions, but clusters of districts where AS has continued influence and security is dynamic and stability critical for the United States Government (USG) and the national interests of Somalia. TIS+ focuses on community planning, contracting, and improving the oversight frameworks that strengthen citizen and government relationships at the district, regional, and national level. TIS+ efforts follow on initial interventions in newly recovered areas with a consensus-based community-driven development model that prioritizes equitable participation in decision-making and management of community assets. Lower Juba Summary Currently, the Lower Juba region is administered as a territory under the Jubbaland State of Somalia (JSS). The key towns of Kismayo, Afmadow, and Badhadhe are under the control of the administration while Jamame is held by Al-Shabaab. Lower Juba is the most diverse region in Somalia, with more than three dozen sub-clans residing in the region. Kismayo, the capital of the region, is one of the safest urban areas in the country due to a strict security architecture instituted by the administration. AS continues to rely on minority communities for support, manipulating their anxiety over marginalization and displacement, especially when the Juba Valley opens after the military campaigns. As witnessed in other regions of Somalia, sub-clans that do not feel sufficiently represented in government often ally with AS to balance the scales and gain a seat at the table. Some of the towns in Lower Juba have been under AS control for more than a decade, during which time the group has established systems of administration and a recruitment structure focused on indoctrination, training, and radicalization of young people. Summary of Key Findings 1. There is evidence of increased awareness and ownership of development projects and processes at the district/settlement level, but this has not translated to the cluster level. The reasons for this are multifaceted: (1) settlements in Lower Juba are very diverse and this impacts cluster level impact; (2) nongovernmental organizations (NGOs) are taking a significant “lead” in development planning with poor follow-through on support for implementation from government partners; and (3) greater technical skill transfer is needed at the local and state level to manage certain development assets (e.g., size and scope of project matters). 2. Somalis show a clear preference for NGO and private sector service delivery, leaving citizens with high expectations that government may not easily meet. 126 3. Public–private partnerships (PPPs) seem to work more effectively on highly localized efforts where the community can interact and provide direct oversight of these efforts. Larger projects seem to have less community engagement and citizens seem less informed as to who is responsible for such critical functions as budget management, maintenance, quality control, and access. 4. Citizens feel that government should provide more services, but there was increased awareness that the government was well staffed, resourced and skilled as some other service providers such as NGOs and the private sector. 5. Contributing factors noted by respondents that are linked to increases in confidence in government are (1) improvements in security, (2) increases in confidence in state and local administrations’ prioritization of citizen needs, and (3) improved communication and information exchange between citizens and government. 6. Where oversight committees were evident and well formed, there was greater trust and awareness of the roles of government (state and local) as well as other actors responsible for maintenance of the activity. Infrastructure activities, where these were not active or well managed, had less evident impact. 7. There was evidence of increased localized conflict, particularly around land and water access. This finding was consistent in all settlements in this cluster. 8. Security has improved, but there is still a noted need to diminish the influence of AS. Clan elders and business members are often manipulated and targeted by AS, which influences their ability to participate in development planning and oversight. 9. AS still provides some services at the cluster level that are linked to security and community relations. Access to justice is a critical area that AS seems to fill, particularly involving small clan conflicts over land and resources. Summary of Conclusions 1. Private sector service providers, NGO dominance, and poorly staffed and resourced state and local administrations are factors that make service delivery support by donors complex, contested, and an important space for coordination. Donor efforts can undermine local and state legitimacy if not carefully planned and coordinated. 2. Based on analysis of qualitative and quantitative data, it is evident that the premise of the TIS+ design could contribute to improved methods of service delivery, shared asset management, and decreased conflict over resources. However, intervention planning and oversight structures were not always put into place to support implementation of infrastructure efforts. This impacted the effect of these interventions on community inclusion and anticipated improvements in confidence in government. 3. As evidenced in qualitative interviews, there is an expectation that state and local government has a role to play in asset management. But there is evidence that the private sector and NGOs are more trusted in assisting communities in managing assets collaboratively. This presents a challenge for TIS+ in partnering with FGS, Federal Member States (FMS), and local administration staff, which are often not trusted by local communities. 4. Lower Juba improved in both good governance and community cohesion elements from baseline to midline measures. These changes reflect a complex arrangement of donor efforts, national political changes, and security improvements and challenges that impact both community access and citizen perceptions of federal, state, and local capacity. TIS+ contributed, but implementation modalities can improve to more directly impact governance and cohesion elements. 5. Community Planning and Oversight (CPO) in collaboration with government in key areas impacted by AS is an essential element of TIS+, but this is not always prioritized in implementation planning of infrastructure. These CPO processes are essential to apply the Theory of Change (ToC) for TIS+. 127 Summary of Key Recommendations 1. Ensure that all significant infrastructure interventions have inclusive Project Oversight Committees (POCs). Oversight committees play an essential role in helping strengthen and improve the accountability and performance of local administration staff and functions. This is a key element of the TIS+ stabilization design. All significant field interventions should have POCs. These POCs must be inclusive to adequately address the objectives of TIS+. It is essential that women, youth, and minority groups find a place on these committees. 2. Infrastructure projects that allow for maximum community and government interaction should be prioritized. It is critical for TIS+ staff to weigh the disadvantage and advantage of larger infrastructure needs with smaller and more localized community-based infrastructure efforts. It may be that smaller, more localized planning and development approaches may stimulate more participation from both citizens and their government. Larger infrastructure projects may act as a platform for improving governance behaviors such as improved planning and public accountability with government officials, but are less likely to be effective vehicles for increasing community engagement, unless these efforts have clearly defined co-management structures that are inclusive of citizens and government. 3. There is an evident need for greater participation of women in field activities. Improvements have been made since baseline in incorporating more gender-sensitive programming. However, TIS+ field staff should increase gender targeting, sensitization campaigns, and gender equality awareness activities. Separate planning sessions for women may be helpful in ensuring that projects that benefit women are clearly identified and prioritized. Staff need a clear understanding of Gender Equality and Social Inclusion (GESI) practices that can be employed in the field. 4. Coordination with local partners and other stabilization actors is essential in improving governance. TIS+ coordination between state and local entities is often not well harmonized with other donor efforts. Technical Coordinators have been helpful but there is still work to be done on field level coordination. Improved coordination between implementing partners will help avoid duplication and manage expectations of the communities. This field level coordination is essential for state and local governments that are often under-resourced. 5. Conflict mitigation is an important and consistent local need expressed at the settlement and cluster level. Although clan violence seems to have decreased in Lower Juba, clan conflicts are often cited as a problem in planning development efforts at the community level. Inter-clan reconciliation and conflict mitigation is still a pressing local and state-level need. TIS+ interventions could have a positive effect in mitigating the local micro-conflicts through the community consensus process by understanding the history of these conflicts and working with the communities to select priorities that are aimed at mitigating aspects of these conflicts. There appears to be a role for women in inter- and intra-clan reconciliation. This should be explored, researched, and targeted as a TIS+ intervention. 6. Youth are still susceptible to AS influence and they are craving a more substantive role in development planning. There is a continued need for youth programming that provides youth with direct access to community engagement and input into community planning. Youth are particularly interested in economic opportunities and in being more directly engaged in local decision-making, particularly development planning. TIS+ should coordinate with other donors working on youth engagement, employment and leadership and link interventions in community level planning to these youth and vice versa. This practice will also provide a more comprehensive platform for youth to engage in community building, development planning, and engagement in their localities and states. This may help to mitigate the concerns in this cluster of the susceptibility of youth to criminal employment and activity with AS. It may provide them with alternative outlets, ultimately mitigating the recruitment practices of AS. 7. Rule of Law and Justice is directly linked to stability. While this is no new revelation, it is more pronounced in this region and TIS+ should consider avenues to link with the USG’s new 128 Access to Justice Program to find opportunities for joint intervention. Access to justice is essential at the district level and there are clear indications that AS is filling this niche. People, particularly those from minority groups, are craving a greater understanding of political and legal rights and venues to address these needs. TIS+ should seek to link where possible with this new USAID project and assist local communities in accessing community based legal aid systems and services, particularly for women and minority clans. Lower Shabelle Summary The Lower Shabelle region is adjacent to Mogadishu, bordered by Banadir region to the West, Juba to the South, and Bay to the West. Known for its fertile land and abundant natural resources, the region has been one of the most heavily contested territories in the history of the Somali civil war. Lower Shabelle still faces high levels of insurgency and is an active location of armed conflict between Al￾Shabaab and the Somali National Army, allied militia groups, and African Union Mission in Somalia (AMISOM) forces. In addition, the US military conducts targeted operations alongside Somali Special Forces’ Danab Battalion. In 2014, Somali and AMISOM forces conducted a joint operation aimed at freeing the last remaining strongholds of AS in the region. Operation Indian Ocean managed to dislodge AS from towns that it had held for almost a decade, including Qoryooley, Sablaale, Kurtunwarey, and, most importantly, Barawe, the proposed new capital for this state. However, AS managed to blockade these towns’ main supply routes, turning them into islands with no access to vital supplies. This, combined with the absence of a strategy to govern the towns after their initial capture, has allowed AS to maintain considerable influence in the region and use it as a base to launch attacks on Mogadishu. Summary of Key Findings 1. TIS+ implementation was very limited in this cluster. TIS+ had established this area as viable to work in, but was not able to fully implement as planned. 2. This cluster had several donors who started to implement and plan with communities and then had to delay or interrupt their activities due to insecurity. 3. Expectations were particularly high in Barawe, which has been designated as the new state capital. Citizens feel neglected and that insecurity is preventing forward progress in this cluster. 4. Citizens felt that the federal, state, and local governments should provide more services and there was significant criticism in interviews. Local administration is under increased pressure to provide more services and the private sector supports critical gaps in this cluster. 5. AS makes it difficult to access programs that enhance service delivery or target shared assets. Local government officials, as well as local leaders, are greatly impacted by insecurity and threats from AS. Summary of Conclusions 1. TIS+ implementation was very limited in this cluster. Consensus-building sessions were held with limited follow-up with resources and community engagement activities. Expectations were raised by TIS+ and then not fully managed. This has impacted a feeling at the community level of being neglected. 2. There is a very high need for collaboration regarding shared asset management, but very few partners able to operate in this cluster safely. 3. There are very high community level expectations of local and state government in this cluster. This, combined with the low level of implementation, has impacted TIS+ programming in this cluster. 4. This cluster, when contrasted to Lower Juba, does illustrate that contribution is evident when TIS+ implements CPO processes based on the original program design. However, when the CPO process is started and not followed by implementation support, expectations are raised and frustrations expressed. 129 Summary of Key Recommendations 1. Consensus-planning sessions should not be held unless interventions of a significant scope and scale are to follow. Expectations are raised through the prioritization process and in areas such as Lower Shabelle, there is evidence that these sessions did raise community expectations. It is essential that community planning sessions be tempered where interventions are not guaranteed. 2. Develop a process to close the loop on areas in which early planning does not lead to intervention. It is essential that, in areas where consensus planning was started, there be a way to close the loop with communities and local government stakeholders. Conflict- sensitive program planning methods should be engaged to ensure that when program priorities shift, there is a systemized approach to reaching out to stakeholders and partners engaged in initial planning efforts. 3. Smaller and collaborative interventions are essential. In an area, such as Lower Shabelle, it may be appropriate to work on smaller and more influential interventions with multiple partners. Sharing the risk in areas that are volatile and difficult to operate in may yield innovative and influential opportunities to enhance citizen engagement and build local alliances. Short-term training opportunities for youth and women and/or cluster-wide events that allow settlements to work together may yield surprising results and help manage expectations. 4. Lower Shabelle is critical for stabilization in Somalia and cannot be ignored. TIS+ should consider working more closely with AMISOM and the USG to increase the development space through community-based interventions. This is a volatile place to work, but there is a USG base and it may be possible to do some small and coordinated interventions that would have a significant and strategic impact. Projects that focus on shared assets between settlements and are accessed by multiple clans should be prioritized. USG coordinated short-term interventions may be most appropriate currently in Lower Shabelle, where security and volatility still make longer-term interventions unlikely. 5. Focus on programming that addresses recurrent and systemic needs for inclusion in Lower Shabelle. The following interventions are essential to enhance inclusion and mitigate grievances in Lower Shabelle: (1) assisting youth, women, and minority clan citizens in short-term training for local positions of representation; (2) supporting access to justice interventions and legal aid services for women and marginalized groups; and (3) interventions that address equitable economic opportunities for youth, women, and minority groups. TIS+ may not be well positioned to support these interventions directly, but working with government partners to encourage this type of development planning may be something TIS+ can do in coordination with other donors or through national level activities that support planning in Lower Shabelle. 130 I. INTRODUCTION Chronic instability in the late 1980s and the collapse of the Somali state in 1991 resulted in two decades of warfare and violence in Somalia. The decades of instability, inter-communal conflict, and humanitarian crisis that characterized the second half of the 20th century in Somalia were marked by numerous attempts at peacekeeping, stabilization, and international relief. Clan conflict, violent extremism, drought, famine, out-migration, and multiple failed central state governments have typified the obstacles facing a unified, independent Somali State capable of service delivery and crisis response. In 2012 Somalia was finally able to establish a complete central government, after 20 years without a parliament, and held the first presidential election since 1967.1 Despite the continued threats of disorder and violence posed by Al-Shabaab, the Federal Government of Somalia (FGS), in partnership with the international community, has taken impressive strides toward durable peace and nation building. Most recently, Somalia, with extensive support from the international community, held an “indirect” election on February 8, 2017, during which Mohamed Abdullahi Farmajo was elected in the second round of voting, unseating incumbent Hassan Sheikh Mohamud. In November 2016, FGS launched a draft of the first National Development Plan (NDP). This plan is organized per the following objectives: poverty reduction, infrastructure repair, strengthening state capacity, and sustaining political inclusivity.2 Gender mainstreaming and a focus on youth are both features of the NDP. USAID Strategic Framework for Somalia: 2016–2019 USAID’s development hypothesis for its 2016–2019 strategy is as follows: “If Somali governance systems and processes that enable inclusive governance are strengthened, and the Somali people benefit from expanded, tangible dividends of the state-building project, then the foundation will be built for Somalia’s longer-term recovery.” This hypothesis recognizes that the process of Somalia’s recovery will extend far beyond the life of this strategy. However, it is in USAID’s manageable interest to focus on core strategic areas that are most important for Somalia’s transition and to maintain sufficient flexibility to adapt both the strategic approach and programmatic mechanisms to adjust to changing political, security, and developmental circumstances. To assist Somalia in this historic effort, the USAID Strategic Framework for Somalia 2016–2019 has identified three primary Transition Objectives (TO) to guide investments over the 2016–2019 period that support the goal of building durable foundations for a more stable, democratic, and prosperous Somalia: ● Transition Objective 1: Systems and Processes that Enable Inclusive Governance Strengthened ● Transition Objective 2: Service Delivery Improved ● Transition Objective 3: Inclusive Economic Growth Expanded The core assumption built in the strategy is that, if there is an increase in political participation and trust in the political process; effective, accountable, and representative governance institutions are strengthened, and key communities are stabilized, then the necessary systems and processes that enable more inclusive governance in Somalia will be strengthened. 1 Between 2000 and 2012, Somalia had a Transitional Federal Parliament but a full parliament was constituted in 2012. 2 Federal Government of Somalia. Ministry of Planning and International Cooperation. National Development Plan Summary. Accessed: February 15, 2017. http://mopic.gov.so/wp-content/uploads/2016/11/National-Development-Plan-Summary.pdf 131 In fragile communities, such as those formerly under the control of Al-Shabaab, USAID/Somalia, in partnership with USAID’s Office of Transition Initiatives (USAID/OTI), works to identify quick impact programming to provide immediate assistance and public assets to improve basic welfare in the months following a successful military operation. These efforts seek to increase the levels of confidence in African Union Mission in Somalia (AMISOM), the Somali National Army (SNA), and the government in each area so that communities do not feel abandoned after military interventions. Through this initial assistance, citizens may conclude that they will not be excluded from services under the FGS and that development dividends may increase. USAID follows on these initial interventions with a consensus-based community-driven development model that prioritizes equitable participation in decision-making and management of community assets. USAID works in tandem with multiple levels of government (i.e., caretaker and interim administrations, regional and state authorities, and the FGS) to help them build and strengthen these communities. Participatory, consensus-building approaches are used to build citizen confidence in the governing institutions and address critical local-level needs by facilitating the construction and/or rehabilitation of vital community infrastructure (such as roads, bridges, community centers, local administration offices, health clinics, schools, water sources, markets, and playgrounds). Project Background The Transition Initiatives for Stabilization Plus (TIS+) activity is a five-year, US$66.9 million project implemented by AECOM that will run from May 2015 to April 2020. This TIS+ activity intends to support the achievement of USAID’s goal by fostering participatory processes that promote good governance and community cohesion. Given the fragile nature of the Somali government and the existence of an insurgency in parts of South Central Somalia, USAID is maintaining a continued focus through TIS+ on quick impact stabilization activities at the district and community levels. At the same time, Somalia is aiming to move from crisis response and stabilization to medium- and longer-term development, so TIS+ is also designed to support this aim. Under Somalia’s New Deal Framework, as well as the recently completed NDP, the FGS and the donor community have committed to advancing inclusive political dialogue, improving security, and increasing economic growth—pillars that TIS+ will also support. TIS+ seeks to implement community-centric stabilization activities that (1) are implemented with multiple levels of government (i.e., caretaker and interim administrations, new states, and the FGS; (2) support inclusive participation of stakeholder groups in community decision-making; and (3) encourage shared asset management between citizens and the government to demonstrate the government’s interest in an emerging capacity to contribute positively to citizens’ daily lives. TIS+ activities are designed to contribute to improvements in government service delivery, leading to strengthened governance and community cohesion. II. TASK ORDER PURPOSE AND OBJECTIVE Task Order Purpose The primary purpose of the TIS+ baseline and midline assessment is to generate a better understanding of the extent to which TIS+ interventions have achieved their objectives and contributed to USAID’s goal of increased stability in Somalia through participatory processes that promote good governance and community cohesion. Specific clusters or geographic areas have been strategically chosen to be analyzed to assess the influence that the TIS+ program activities have had on mitigating the effects of violent extremism and insurgency in these high-risk areas of Somalia (see Figure 2). This task order is designed to assist USAID in identifying lessons learned from conducting stabilization activities in Somalia. 132 Assessment Questions The TIS+ baseline/midline assessment focuses at two levels of analysis: within cluster analysis (cluster￾level analysis) and cross-cluster analysis (comparative cluster analysis). USAID has identified five clusters for analysis: (1) Banadir (Mogadishu), (2) Lower Shabelle, (3) Bakool, (4) Lower Juba, and (5) Gedo/Bay. During the midline, data will be collected in two waves. In Wave I, data was collected in Lower Juba and Lower Shabelle, and in Wave II, data will be collected in Gedo/Bay, Banadir, and Bakool. This report focuses on findings from Wave I data collected in Lower Juba and Lower Shabelle. The assessment will answer the following assessment questions. Table 1: Key Assessment Questions: Baseline and Midline No Key Assessment Questions (KAQs) Baseline Assessment Questions Phase 1 and 2 Midline Assessment Questions Phase 3 Cluster-Level Analysis 1 Did TIS+ contribute to any of the following anticipated results? ➢ Community ownership of development projects and processes ➢ Effective shared asset management ➢ Improved delivery/satisfaction of services ➢ Reduced conflict over resources ➢ Greater citizen confidence in government ➢ Reduced support for violent extremism What is the status of the following at the cluster level? ➢ Community ownership of development projects ➢ Community ownership of development processes ➢ Shared asset management in the communities targeted by TIS+ ➢ Quality/Delivery of service targeted by TIS+ ➢ Conflict over resources ➢ Citizen access to information ➢ Citizen satisfaction in government Confidence in government ➢ Support for violent extremism Same as Assessment Question 1 2 Were TIS+ approaches used in the cluster (e.g., the community consensus process) instrumental in contributing to the achievement of TIS+ results described in Question 1? Midline only Same as Assessment Question 2 3 Did government at all appropriate levels contribute sufficiently to asset management in communities in which TIS+ was implemented? How? Did government at all appropriate levels have both the capacity and the will to contribute? What is the government (at all appropriate levels) contribution to asset management in communities targeted by TIS+? Level of communities’ inclusion in the following: ➢ Setting priorities and identifying needs ➢ Community contracting ➢ Community contribution ➢ Community advocacy with government to provide services needed for intervention and response by government Same as Baseline Assessment Question 3 133 THE STEPS OF CONTRIBUTION ANALYSIS 1. Set out the cause–effect issue to be addressed in a detailed ToC. 2. Identify the risks to the postulated ToC, including rival explanations. 3. Gather evidence (Baseline and Endline). 4. Assemble and assess the contribution claim, and challenges to it. 5. Seek out additional evidence. 6. Revise and strengthen the contribution story. Figure 2: Steps of Contribution Analysis 4 Were TIS+ activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? What is the status of inclusion of women and youth as applicable to the context of the cluster? ➢ Level of existing gender gaps ➢ Level of women, youth, and minorities inclusion in planning and implementation of activities ➢ Extent of respect for women’s and other marginalized populations’ interests among other community members Same as Baseline Assessment Question 4 Aggregate and Cross-Cluster-Level Analysis (Phase IV) 5 In the aggregate, did TIS+ contribute to the anticipated results described above? What range of results can be observed across assessed clusters? Midline only Same as Assessment Question 5 6 Which cluster-level factors contributed most significantly to achievement of results? For example: ✓ Did successful inclusion of women, youth, and minority clans amplify the effectiveness of TIS+ activities? ✓ Were results greatest in newly liberated clusters or in bridge clusters? ✓ Were some kinds of interventions more effective than others in improving citizen confidence or reducing support for violent extremism? Midline only Same as Assessment Question 6 III. METHODS AND LIMITATIONS Both baseline and midline employed a mixed￾methods approach that integrates a suite of quantitative and qualitative data collection methods implemented over two phases to answer the above key assessment questions. This methodology affords USAID a greater depth of comparative analysis across the clusters as well as aggregated analysis at the midline stages (phases III and IV). Data collection methods consisted of desk review/secondary literature review, a household survey, Key Informant Interviews (KIIs), and Focus Group Discussions (FGDs). Assessment questions were answered using a combination of analyses of data from these four collection methods and via triangulation of evidence and the use of contribution analysis to estimate TIS+’s unique contribution to observed midline results. Contribution analysis (CA) is an analytical approach to estimating program effects in complex contexts with many factors and actors influencing intervention outcomes. CA is a good option for a stabilization program because it examines stabilization as a continuum and explores conflict dynamics. CA works within a Theory of Change (ToC) model to look at contribution from several “angles”. It employs an applied ToC approach to look beyond high-level outcomes (goal/objectives) to more specific activities and interventions across diverse project spaces. CA is a pragmatic and methodologically rigorous approach to situations for which the activity is one of many potential contributory factors and where it is not easy to measure attribution directly, such as is 134 often the case in stabilization programming. In the case of this Task Order, CA will be used to assess the contribution of TIS+ activities to observed outcomes based on the totality of evidence gathered. The essential value of contribution analysis, in this case, is that it offers the approach designed evidence to reduce uncertainty about the contribution TIS+ to the observed results through an increased understanding of why the observed results have occurred (or not) and the roles played by TIS+ activities and other internal and external factors.3 A quantitative household (HH) survey was administered to residents living in communities where TIS+ activities are planned to be implemented. The qualitative data collection instruments included secondary literature review in-depth KIIs with diverse TIS+ stakeholders, and FGDs with persons living in the targeted clusters, designed to better understand within each cluster the status of community infrastructure planning. This data will be analyzed to create evidence base that will provide comprehensive answers to KAQs 1–4 within clusters and KAQs 5–6 across clusters. A description of each data collection method is provided below as well as a description of the catchment and cluster areas accessible at Midline for Wave I data collection. Secondary Literature Review The first data collection method undertaken by the team was to conduct a comprehensive review of internal and external documents of relevance to the TIS+ activity. The IBTCI team worked in coordination with USAID/Somalia Field Office (SFO) and the TIS+ management team to identify, collect, and review the following documents over the course of the evaluation: ● Activity designs, training curricula, event agendas, and other key documents for understanding the content of activities ● Annual Work Plans (initial and subsequent) ● Activity Monitoring and Evaluation Plan (initial and revised) ● Quarterly and annual reports ● Reports on TIS+ participatory monitoring and evaluation (M&E) activities ● SPSS third-party monitoring and verification reports ● Additional documents provided such as Stability Assessments and Community Consensus Reports ● External publications by other stabilization stakeholders, donors, academic researchers, and NGOs Household Survey During the design phase of the assessments, it was envisioned that a total sample size of 848 (baseline = 424 and midline = 424) per cluster would be large enough to draw statistically meaningful conclusions from the results of TIS+ activities. Under the cluster-level approach, each cluster is treated as a separate analytical population for sampling purposes. At baseline, data were collected in two settlements of the Lower Juba cluster and five settlements of the Lower Shabelle cluster. TIS+ activities were implemented in all the baseline settlements for Lower Juba while activities were just planned in only two out of five baseline settlements for Lower Shabelle. The midline assessment was conducted only in settlements where TIS+ had at least an activity including consensus building or any planning meeting. This implied reducing baseline sample size for Lower Shabelle, compared with sample size used for data collection at midline. At midline, in partnership with Data and Research Solutions (DARS), a subcontracted Somali research firm, the IBTCI team administered at least 424 face-to-face interviews with individuals randomly selected at the household level (with households also selected through random walk). The following table shows the distribution of sample sizes used for cluster level baseline and midline comparative analysis. Additional numbers, for Lower Juba baseline and midline for both clusters, reflect data quality control measures to ensure gender balance. 3 John Mayne, (2008), “Contribution Analysis: An Approach to Exploring Cause and Effect,” http://dmeforpeace.org/sites/default/files/0501_Contribution_Analysis_ILAC.pdf. 135 Table 2: Cluster Sample Size, Baseline – Midline South West State Baseline Midline Achieved Sample Male (%) Female (%) Achieved Sample Male (%) Female (%) L/Shabelle 200 90 (45%) 110 (55%) 429 217 (50%) 212 (50%) L/Juba 453 221 (49%) 232 (51%) 432 217 (50%) 215 (50%) Total 653 311 (48%) 342 (52%) 861 434 (50%) 427 (50%) The HH survey instrument was in part modeled on a past stabilization survey fielded in Somalia, which was used to generate the Transition Initiatives for Stabilization (TIS) Stabilization Report in 2013–2014, and the TIS performance evaluation done by SPSS in 2016. The survey instrument for TIS+ was tailored to ensure that it accurately reflects TIS+ activities and their expected outcomes, while also ensuring that at baseline and midline the assessment questions could be assessed. Key Informant Interviews The IBTCI team and DARS conducted 63 KIIs at midline in Wave I. This included 27 targeted KIIs for the two clusters, and 36 additional KIIs relevant to CA. A major advantage of the KII method, compared with other forms of qualitative research, was the opportunity it provided for in-depth exploration of the knowledge and experience of individuals directly exposed to TIS+ and stabilization work in Somalia. This was particularly important for employing contribution analysis. In Wave II of the midline a similar number will be achieved. The selection of the KIIs was done in collaboration with USAID/SFO and the TIS+ implementing partner, AECOM, to develop a representative list of informants well-versed in the activity design, implementation, and anticipated results. The following represents the universe of KIIs that were targeted for data collection at the midline stages: • USAID officials in the USAID/SFO • AECOM senior management, regional offices staff and sub-contractors staff • Officials from the Federal Member States and relevant government actors at the central and district levels in Somalia • Key staff from other stabilization programs that interact with TIS+ or support programming that shares its objectives and geography • Representatives from private sector, nongovernmental, or governmental donor organizations collaborating with or asked to collaborate with TIS+ communities • Participants purposively selected relevant to analyzing the effects of TIS+ program design and activities within the targeted clusters and catchment areas Focus Group Discussions The IBTCI team conducted a total of 24 FGDs during midline Wave I. A major advantage of the FGD method, compared with other forms of qualitative research, is its flexibility for in-depth exploration of the knowledge and attitudes of the group toward specific cluster level conditions relevant to TIS+ activities and processes (i.e., participation; initial levels of trust in district, regional, and FGS officials; community contracting and planning approaches; and the inclusion of women, youth, minority clans, and members of other marginal groups in decision-making). Four FGDs were conducted in each selected settlement with the following groups: (1) female elder beneficiaries, (2) male elder beneficiaries, (3) male youth beneficiaries, and (4) female youth beneficiaries. This demography of FGDs was conducted in up to 50 percent of the TIS+ communities within a cluster. Index Tool An index tool was developed to allow the team to aggregate a series of interrelated questions from the HH survey to establish benchmarks at the cluster level. This index assists the team in cross-cluster and 136 aggregate analysis as well as providing an analytical anchor to look at the relationship between these factors: key assessment questions, TIS+ interventions, and impacts on stability. These measures were used at midline to assess changes in areas of TIS+ programming and to also look at potential trends evident by comparing baseline measures to midline measures. Assessment of Preconditions for TIS+ Goal The goal for the TIS+ project is to increase stability in the clusters of implementation. For stability to be achieved, two preconditions should be satisfied: (I) increased good governance and (II) increased community cohesion. Annex II explains in more detail the development of index scores to measure these preconditions.4 Household questions were grouped into three themes underlying good governance, including confidence in government, service delivery, and stakeholders’ partnerships as the general constructs defining governance status. Questions were also grouped into three themes explaining community cohesion: network diversity, interpersonal and institutional trust, and social and political inclusiveness. These themes for community cohesion were informed by the impact evaluation of Peace through Radio Programming conducted in Chad and Niger.5 In addition, this report provided further guidance on what would constitute community cohesion—in which relevant questions (and in other cases proxies) were identified and classified into the relevant groups documented in the chosen and Niger evaluation report.6 This allowed the SPSS team to compare other country settings where community cohesion measures had been developed to explore the relationship between these measures and reductions in other factors relevant to countering violent extremism (CVE) environments. Limitations and Operational Challenges Complex conflict contexts are challenging. The IBTCI team designed the approach and management plan to mitigate anticipated effects as much as possible through means such as multiple data points for each question, confirmation of facts, customized data collection protocols that helped the team collect data in remote and insecure areas, and staggered data collection time frames. Conducting complex multidimensional data collection requires tight discipline, efficient use of time and resources, close communication with the team and the client, the ability to adapt and solve data collection problems on the spot, and the ability to be an evaluation diplomat when dealing with informants at all levels, especially in senior positions. While considerable, these challenges were not unfamiliar or insurmountable. Anticipating the Somali context, the team identified several important challenges that the IBTCI approach addressed: ● Identifying contribution to higher-level outcomes derived from relatively low-intensity, short-term, and issue-specific interventions. ● Ruling out alternative explanatory variables along the ToC based on multiple service providers in a similar geographic space. ● Ensuring that HH-level data collection was applied consistently across diverse geographic spaces and diverse populations. ● Ensuring that triangulation conducted in non-TIS+ activity areas is incorporated into CA analysis and compared with findings in TIS+ activity areas reasonably and based on consistent methods across clusters. ● Collecting sufficient data to inform the extent of the assessment questions and planning for baseline and midline tool needs. ● Balancing the validity of data and outcome analysis with a reasonable and well-evidenced causal analysis (i.e., applying CA) and balancing this with quantitative data evidence. 4 Index score is a single measure derived from a set of variables. 5 https://pdf.usaid.gov/pdf_docs/pa00ktf3.pdf 6 http://www.d3systems.com/wp-content/uploads/2015/10/MISTI-Wave-5-Analytical-Report-FINAL_USAID.pdf 137 ● Ensuring that data analysis is well articulated within cluster and across cluster groups to show variations and distinctions that will contribute effectively to the design and implementation of future stabilization programs in Somalia. ● Tool consistency is essential, but adaptation to the tools was needed at midline to capture effects of community contracting, bridging plan development, consensus process, etc. ● Reaching the perspective of women and marginalized groups is difficult and imperative for sound data analysis. Women enumerators and field researchers are essential as are local staff knowledgeable about the local clan dynamics and impacts on cluster conditions. ● Data is hard to move around. Al-Shabaab is a serious impediment to safety at the local level, so moving data is not easy. Data movement requires time, patience, and creative techniques. Security and Access Somalia is a difficult country in which to gather data, and the team had to consider security limitations as well as the dynamic nature of districts and areas under varying stages of control by Al-Shabaab (AS). AS is a dynamic and increasingly networked group, both at the transnational and local level (cluster/district). AS has continued to control many rural areas in several regions, including Lower Shabelle and Lower Juba. The following table shows the clusters and settlements that were accessible during the baseline and midline Wave I stages and the ones removed due to insecurity or access issues. Table 3: Clusters Surveyed at Midline, Wave I Data Collection Cluster Originally Planned Settlements (7) Baseline Settlements (5) Midline Settlements (2) Lower Shabelle Wanleweyne, Barawe, Afgoye, Qoryooley, Merca, Kurtunwarey, and Awdeghle Wanleweyne, Barawe, Afgoye, Qoryooley, and Merca Wanleweyne and Barawe Settlement Selection Process Chosen by USAID, in collaboration with TIS+ Kurtunwarey and Awdeghle removed prior to baseline data collection due to insecurity and AS operations in these settlements Afgooye,* Merka,* and Qoryooley* removed because they did not have sufficient TIS+ implementation activity Cluster Originally Planned Settlements (4) Baseline Settlements (2) Midline Settlements (2) Lower Juba Afmadow, Kismayo, Jamame and Jilib Afmadow and Kismayo Afmadow Kismayo Settlement Selection Process Chosen by USAID, in collaboration with TIS+ Jamame and Jilib removed prior to baseline data collection due to insecurity and AS operations in these settlements VI. CONTRIBUTION ANALYSIS AND TIS+ The Role of Contribution Analysis TIS+ operates within a wider system in which it is only one of several factors that potentially affect activity outcomes. The main purpose of contribution analysis in this assessment is to identify the effect that TIS+ programming had in an environment with other actors and influences in the same geographic area at baseline and in the midline stages. Contribution analysis uses a stepwise process of data collection and analysis containing six steps. During this phase of the midline, the TIS+ Assessment team focused on steps 3, 4, 5, and 6 of the overall CA process to ensure that CA is applied to the 138 development of sound and reasonable conclusions for USAID and TIS+ stakeholders at both the baseline and midline stages. At Midline (Phase III and IV) CA contributed the following to the overall analysis: ● Step 3: Gather evidence. This step involves the collection of data using whatever data collection tools have been selected for the assessment. In our case, it involved the implementation of our main survey and qualitative instruments and the review of findings from these tools. We also began to assemble relevant studies and reports from other sources. ● Step 4: Assemble and assess the contribution claim and challenges to it. This step constructs the contribution story while assessing the strengths and weaknesses of the cause-and￾effect logic considering the evidence collected during the previous step, the relevance of other contributory factors, and the evidence for rival explanations. We included all relevant results from our available data in step 3 above in CA performance sheets. This allowed us to identify gaps in our understanding of what contribution TIS+ had made and how the causal chain had worked in practice. ● Step 5: Seek out additional evidence. This step determines what kind of additional evidence is needed to enhance the credibility of the contribution claim and gather the appropriate evidence, as feasible. We conducted 46 supplementary interviews to better our understanding of how the CA stories’ causal claims had operated alongside other factors. ● Step 6: Revise and strengthen the contribution story. Having identified and addressed any challenges to our credibility, such as the strengths of certain assumptions and/or the roles of other influencing factors, this step builds a more credible contribution story. Because this process involves both bottom-up and top-down measurement of change, it provides triangulation of evidence for change up and down the project ToC. It also provides a reasonably robust approach to measure contribution and guard against “self-importance bias,” which occurs when the relative importance of other contextual factors is underestimated, and the importance of project-specific factors is overestimated. We used our CA story performance sheets to finalize narrative contribution stories, broken down by cluster where appropriate. Testing the Theory of Change for TIS+ USAID/Somalia has a ToC that summarizes at a high level how change is expected to happen in its focal areas of intervention in Somalia, and the TIS+ ToC feeds into this higher-level theory. TIS+ had previously worked on articulating a “development hypothesis” at the overarching program level, including a summary of the intervention logic, mechanisms, and assumptions. Contribution analysis requires a ToC that is much closer to the level of the interventions, or what the program is doing, and that describes the sequence of resultant behavior changes that are expected to lead or catalyze outcomes.7 It was necessary to ask at midline whether the program was implemented as originally planned and to assemble evidence that may have influenced these changes. In response to this need, and as required in the task order, we built on TIS+ existing work to develop a fuller ToC for these purposes. The development of this ToC was based on a one-day workshop on June 14, 2017, in Nairobi with TIS+ field and headquarters staff, as well as the USAID/Somalia lead for the TIS+ program. Figure 3 below shows the overarching ToC workshopped with TIS+ and shows how the CA Contribution Stories map onto it. 7 Davies, R., 2012, “Criteria for assessing the evaluability of a Theory of Change,” Rick on the Road Blog. http://dmeforpeace.org/learn/criteria-assessing-evaluability-theory-change 139 Figure 3: Applied Theory of Change for TIS+, showing CA Stories 140 The ToC development workshop, as well as TIS+ documents, revealed that “The TIS+ model is strongly based on a community-government programming approach.”8 This model is the basis of TIS+ Objective One work, which accounts for the program’s largest expenditure, and of the Objective Two work, which is a modification designed for areas that have achieved a higher level of stability and had a stronger semblance of district government. TIS+ Objective Three, on supporting Somali engagement, is a cross-cutting objective that feeds into the work under Objectives One and Two. Objective Three allows for activities supporting social reconciliation or cohesion in communities that may be too volatile for Objective One activities, where Objective One is planning to roll out, or where it is already implementing. It follows a similar process of using community consensus groups but also allows for some activities to be supported without conducting the full community engagement process. Objective Four integrates gender across all TIS+ interventions. This objective encompasses both the focus on women’s participation in the community consensus process and coordination within and between the district, state, and federal levels of government on gender-sensitive planning and implementation. We have included TIS+ work on capacity development in ministries at the state and district levels on the right-hand side of the ToC diagram. Although this work is not covered by a separate objective and is only partly intended to support Objective One and Two activities, it is also intended to strengthen communication across levels of government on stabilization activities. TIS+ is a complex program that includes several different types of interventions, allowing it to use an adaptive approach to programming in Somalia. In consultation with TIS+, we broke this overarching ToC down into its core elements for our “Contribution Stories.” These stories will collectively ensure that our analysis provides good coverage of the overarching TIS+ ToC, TIS+ work plan activities, and expenditure. Further detail on the Contribution Stories is included in the findings and recommendations section and in Annex III. In summary, we selected the following three contribution stories: 1. Community Planning and Oversight Story: This story encompasses the “core offer” of the TIS+ model. This reflects the fact that TIS+ is, at its core, a program delivering infrastructure, but one where the process of planning and delivery engaging community and government is crucial to delivering the expected results. 2. Sports for Peace and Development Story: TIS+ has supported several sports-related activities across all its objectives, from constructing and rehabilitating sports infrastructure to setting up or maintaining and managing committees, organizing sports events, distributing sports equipment, and supporting Ministries of Youth and Sport (MOYS) with technical assistance at federal and state levels. To consolidate and further develop this work, TIS+ enlisted a technical consultant specialist on Sport for Development and Peace in March 2018 to work on a Sport for Peace and Development and Peace Strategy and Implementation Framework for the MOYS. 3. Technical Coordinators Story: TIS+ has several Technical Coordinators (TCs) embedded within ministries, designed to support TIS+ objectives one, two, and three. These are roles at least partly designed to enhance the efficacy of TIS+ interventions, and to deliver wider work in supporting communication between levels of government more broadly. V. DATA ANALYSIS At midline, it is evident that there are several factors that will influence changes in these environments, and that TIS+, although the largest program in these clusters, is not the only program working on improving governance, community cohesion, and stabilization in Somalia. At midline IBTCI has mapped carefully the partners, programs, and factors that are most likely to impact and influence the work of TIS+. Because TIS+ is currently not working in all the clusters and settlements, it was important to 8 Transition Initiatives for Stabilization Plus (TIS+) Process of Engagement document (March 2016). 141 understand clearly the current conditions in these clusters prior to interventions. At midline, the baseline tools were refined to look at specific project activities and processes in these clusters and settlements and compare TIS+ planning to methods used by other donors. Interviews were targeted to direct beneficiary groups. IBTCI looked more deeply at cross-cluster differences, and identified where these differences impacted governance, social cohesion, and stabilization and in what ways TIS+ approaches contributed to this. IBTCI considered all objective areas of TIS+ and looked at more technical interventions, such as capacity development approaches used within the FGS and at the state￾level embedding staff in ministries. The interviews conducted at midline and the analysis of midline data reinforce that CA methods are an excellent match for USAID and TIS+. CA is particularly appropriate in situations where “multiple initiatives are working towards the same end” and where complex environments mean that linear attribution of results to any one intervention is “neither meaningful nor accurate.”9 Stabilization programs fit these criteria and are good candidates for the use of CA methods. The application of CA methods is an innovation for USAID/Somalia and TIS+ that can add to improving methods of assessing programming in the future. Influencing Factors and Alternative Explanations It is important to outline the difference between influencing factors and alternative explanations in CA. Influencing factors are used either to support contribution stories or to support alternative explanations for results observed. Influencing factors for stabilization include both contextual factors, and other interventions. Contextual factors will be strongly linked to conflict analyses at the intervention level produced by TIS+, other development partners, and academics. Other interventions include donor and partner contributions where relevant in the targeted areas. Other donor programs working on similar activities and overlapping geographically are also potential data sources. TIS+ maps and actively collaborates with stabilization actors in TIS+ areas and this will be analyzed for contribution, impact, and relevance. In addition, there are other initiatives to map stabilization such as host country, development, and humanitarian actors in Somalia, with which the baseline had begun to engage and others that will require more research, and targeted KIIs and FGDs with direct beneficiary groups that occurred at midline. Developing a Stabilization Narrative: Contribution Stories Since CA is an analytical approach and not a method of data collection, it relies on the types of data that are being generated to feed the analytical process. TIS+ itself, and the baseline and midline assessments, operate and collect data at the cluster level. Therefore, observed results and contribution stories are generated at this level. Major results observed in each cluster will be turned into contribution stories that will be finalized during Phase IV. This cross-cluster analysis will reach an assessment of the overall contribution to outcomes of TIS+. To reach robust conclusions, CA contribution stories must meet certain criteria for establishing a plausible link between the intervention and the observed results.10 The narrative sections of the contribution stories at midline will be structured per these criteria, with the addition of a final section explaining the overall assessment: ✓ Plausibility: Is the theory of change plausible? ✓ Implementation per plan: Has the program been implemented with high fidelity? ✓ Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? 9 Quinn Patton, Michael. (2008). Utilization-Focused Evaluation: The New Century. 4th edition. Thousand Oaks: Sage Publications. 10 Mayne, J. (2011). Addressing cause and effect in simple and complex settings through contribution analysis. In: Schwartz, R., Forss, K., and Marra, M. (Eds.) Evaluating the Complex: Attribution, Contribution and Beyond. New York: Transaction Publishers. 142 ✓ Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? ✓ Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? ✓ Overall assessment of contribution (including a performance table). Initial contribution stories that will be finalized at Phase IV are presented in a separate annex to this report. Findings from this analytical process informed findings, conclusions, and recommendations of the next section of this report. VI. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Prior to the inception of TIS+, USAID/Somalia had invested over the years a considerable amount of resources in specific settlement areas in the strategic clusters. Where there has been overlap with TIS, the USAID-funded predecessor project to TIS+, and where TIS+ was active prior to baseline data collection, there was evidence that the impacts of these efforts have shifted the rubric of participation in some of these settlement areas. KII and FGD interviews with youth leaders, women, elders, minority group members, and district and regional administration staff provided initial indicators of impact from more targeted stabilization assistance by the USG and partners. It is clear from findings that these clusters are strategic, critical to state-level and regional stability, and the most heavily impacted by the presence of AS. At midline, it was important to identify where TIS+ staff and interventions have carried over from former TIS activities. This factor has made clear the contribution USAID/Somalia has made in linking citizens to more participatory and inclusive forms of programming. It is also important to be mindful that there are a significant number of partners working in these clusters and that their contributions and impact are catalysts and complements of TIS+. Developing Findings, Conclusions, and Recommendations: Midline (Phases III and IV) TIS+ is a multilayered project and this midline analysis focuses on two strategic cluster areas. These are not administrative regions, but rather clusters of districts where AS has continued influence, security is dynamic, and stability is critical. TIS+ focuses on community planning, contracting, and improving the oversight frameworks that strengthen citizen and government relationships at the district, regional, and national level. Activities are structured to foster constructive interaction between citizens and newly established governing authorities, such as basic training of authorities and community leaders, supporting visible small-scale infrastructure, and creating forums for regular dialogue such as town halls and cultural events. TIS+ follows on initial interventions in newly recovered areas with a consensus-based community-driven development model that prioritizes equitable participation in decision-making and management of community assets. Participatory, consensus-building approaches are used to build citizen confidence in the governing institutions and address critical local-level needs by facilitating the construction and/or rehabilitation of vital community infrastructure (such as roads, bridges, community centers, local administration offices, health clinics, schools, water sources, and markets). The findings sections of this report provide analysis of the HH survey data for Lower Juba and Lower Shabelle, and triangulation of this information with the KIIs and FGDs conducted in the settlements in which data was collected. Contribution analysis was the primary approach applied to analyze the relationship between TIS+ planning and coordination activities with other donors and contextual factors. Quantitative analyses of baseline and midline differences were triangulated with qualitative data and supplemental CA interviews with key donors working on stabilization to ascertain these findings. Findings are primarily aggregated at the cluster level, but where relevant individual settlement information is provided to illustrate to USAID and TIS+ the diversity of these settlements and the anticipated impact on stabilization planning and program implementation. 143 Our research during this wave of the assessment considered that TIS+ is working in an environment where stabilization is a collaborative effort between multiple actors who are contributing to shared objectives. Moreover, the program is operating in a fluid and unpredictable environment, which sometimes offers opportunities to support stabilization objectives as well as throwing serious obstacles in the way of those objectives. The use of CA is particularly appropriate in situations where “multiple initiatives are working towards the same end” and where complex environments mean that linear attribution of results to any one intervention is “neither meaningful nor accurate.”11 Stabilization programs fit these criteria remarkably well. To answer the key assessment questions, we have integrated CA into the presentation of findings and recommendations for assessment questions 1, 2, 3, and 4. The suitability of CA for programming in conflict-affected contexts has been recognized in several sources, but a literature search has revealed that few (if any) CA approaches have been used in stabilization programs for USAID.12 Using CA methods in this task order is an opportunity to capture TIS+ effects using the most appropriate methods, and to add to the global evidence base on stabilization programming. To ensure that we covered all assessment questions fully we added sub-questions to assessment questions 3 and 4 to be sure that we could provide evidence of the contribution of TIS+ interventions to these issues which are more cross-cutting. Key Assessment Question Sub Questions for CA 3 Did government at all appropriate levels contribute sufficiently to asset management in communities in which TIS+ was implemented? How? Did government at all appropriate levels have both the capacity and the will to contribute? What is the government (at all appropriate levels) contribution to asset management in communities targeted by TIS+? Level of communities’ inclusion in the following: ➢ Setting priorities and identifying needs ➢ Community contracting ➢ Community contribution ➢ Community advocacy with government to provide services needed for intervention and response by government 4 Were TIS+ activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? What is the status of inclusion of women and youth as applicable to the context of the cluster? ➢ Level of existing gender gaps ➢ Level of women, youth and minorities inclusion in planning and implementation of activities ➢ The extent of respect for women’s and other marginalized populations’ interests among other community members To fully answer the assessment question we have developed CA stories to better understand the contribution of TIS+ at the cluster level. Two further CA stories will be refined in Phase IV during the synthesis phase of the midline assessment. We hope these tools can be used to also illustrate the value of CA as a planning tool for improved stabilization programming. See Annex III: Further Contribution Story Tools. The following sections are organized by cluster, but due to the diversity of the settlements within these geographic areas, it is in some cases necessary to differentiate approaches and context. To assist the reader in understanding some of these cluster level dynamics, a political economy and security analysis (PESA) section has been added to explicate some of the historical, political, and conflict aspects of these 11 Quinn Patton, Michael. (2008). Utilization-Focused Evaluation: The New Century, 4th edition. Thousand Oaks: Sage Publications. 12 Goldwyn, Rachel & Chiagas, Diana. (2013). “Monitoring and Evaluating Conflict Sensitivity: Methodological Challenges and Practical Solutions.” DFID, CARE, and CDE Collaborative Learning Projects Paper; Rogers, Mark. (2012). “Selecting Evaluation Approaches: Options in Peacebuilding Evaluation.” CARE International Paper. 144 areas. We hope this will help frame the findings, conclusions, and recommendations presented in this report. VII. LOWER JUBA Section 1: Political, Economic, and Security Analysis Currently, the Lower Juba region is administered as a territory under the Jubbaland State of Somalia (JSS). The key towns of Kismayo, Afmadow, and Badhadhe are under the control of the administration while Jamame is held by Al-Shabaab. Each district is managed by a district administration appointed by the Jubbaland President and is led by a District Commissioner. The region also has a governor based in Kismayo. Because political representation is based on the 4.5 power sharing formula, sub-clans in the area are represented both in the state parliament and the federal parliament by Members of Parliament selected by elders. All administrations are considered interim, including the Presidents and state parliamentarians, with selection for the State President set to take place in 2019. Lower Juba is the most diverse region in Somalia, with more than three dozen sub-clans residing in the region. Like Lower Shabelle, the complex clan composition and diversity of cultures, dialects, and social organization has caused a protracted conflict that is ongoing. Lower Juba remains an active security theater with military operations conducted by Jubbaland forces and the Kenya Defense Forces (KDF). Kenyan Defense Forces entered Somalia on a unilateral campaign and were eventually reassigned to AMISOM and are now the official peacekeeping forces in the territory. The Somali National Army (SNA) is also present in the region, though at times it is sidelined by the administration due to its perceived status as being representative of the government in Mogadishu. Kenyan forces under AMISOM as well as US forces present in Kismayo are supported by drone strikes. Kismayo, the capital of the region, is one of the safest urban areas in the country due to a strict security architecture instituted by the administration. Jubbaland forces, National Intelligence and Security Agency officers, Daraawish police, and AMISOM forces patrol the town day and night with heavy security around the main economic hubs of the airport and port. However, little military progress has been made to extend this security beyond Kismayo. Critics observe that the government of President Madobe and international partners have only moved “five miles outside Kismayo in five years.” A long-planned Juba Valley offensive to dislodge AS from key towns in Lower and Middle Juba is yet to take place. The internal political rifts between the clans that support the government also make it difficult to create a uniform campaign against AS. There are pockets of stability in the region, with small towns under the control of Ethiopian, Kenyan, or Somali forces, but they are cut off from the other towns. The road from Kismayo to the Kenyan border remains impassable. Lower Juba’s thick forests and abundant supplies make it an ideal ground for AS fighters to train, plan attacks, and operate. Implications for TIS+ Programming Due to the complex clan networks, events in Lower Juba are influenced not only by actors physically present in the region but also by outside forces with clan-based, ideological, or geopolitical motivations. The Ras Kamboni Brigade, the military force that captured the capital Kismayo along with Kenyan forces, is the primary political and security organization in the region, and its members form most of the JSS administration. Because of the fractious process through which the JSS administration was formed, clan networks still maintain considerable influence. Residents identify with their sub-clans and rely on them to seek political or economic settlements. The biggest rivalry is between the Marehan and Ogaden sub-clans as well as within the Ogaden sub-clan itself. Other groups include the Sheikhal, Gaaljecel, Awramale, and smaller minority groups. 145 Al-Shabaab continues to rely on minority communities for support, manipulating their anxiety over marginalization and displacement, especially when the Juba Valley opens after the military campaigns. As witnessed in other regions of Somalia, sub-clans that do not feel sufficiently represented in government, inevitably ally with AS to balance the scales and gain a seat at the table. Some of the towns in Lower Juba have been under AS control for more than a decade, during which time the group has established systems of administration and a recruitment structure focused on indoctrination, training, and radicalization of young people. If the anticipated Juba military campaign is poorly implemented without taking consideration the dynamics in these towns, extremism will remain a problem even after AS is physically gone from the area. This will mean that even after towns are liberated, AS will maintain influence and the ability to plan and conduct attacks. With the presidential transition coming up, the approach to these issues will determine the stability and dynamics in the region. The following three issues are particularly important to consider for future TIS+ programming: 1. Access and Security Considerations The planned Juba offensive as well as ongoing security dynamics will continue to impact TIS+ programming in areas where interventions have been planned. For example, Baardheere, where TIS+ implemented activities were constantly cut off by AS with limited air access. The challenges witnessed in this area will most likely be replicated in other areas, particularly due to the upcoming Juba offensive aimed at opening AS strongholds and transferring them to government control. It is imperative that expectations from TIS+ remain tempered and careful analysis and mitigation plans are in place before activities are planned in such areas. 2. Equity and Representation Because of historic marginalization in this area, compounded by intense inter-clan conflict over access to resources, activities planned by TIS+ will be confronted by this reality. How and where these activities are planned and implemented will be met with different reception by the different stakeholders. It is important that programming does not flare up old tensions and is carefully considerate of the history and dynamics before any interventions take place. 3. Al-Shabaab as a Very Recent Memory The Lower Juba area remains one of the last areas held by Al-Shabaab. It is also the longest area held by AS in Somalia. For the last decade, residents in these areas have known AS to be the authority in this cluster. As legitimate government authority is extended into this area, it is important to ensure that an organic interaction takes place between the residents and the government. The government and by extension TIS+ will be compared to AS in parallel as has been witnessed in other areas of Somalia. The challenge will be to prove that the legitimate government supported by TIS+ provides a better alternative for the community. Section 2: Lower Juba Findings The sections below will provide general frequency analysis of the Lower Juba (Afmadow and Kismayo) data and triangulation of this information with the KIIs and FGDs conducted in the settlements in which data was collected. Contribution analysis was also applied to analyze the relationship between TIS+ planning and coordination activities with other donors and contextual factors. Findings are primarily aggregated at the cluster level, but where relevant, individual settlement information is provided to illustrate to USAID and TIS+ the diversity of these settlements and the anticipated impact on stabilization planning. 146 Limitations Specific to Lower Juba • There is variation in the findings between Afmadow and Kismayo, and, therefore, generalizability at the cluster level may be limited. • Because TIS+ intervention modalities are different in each settlement, it was important to establish what TIS+ has implemented in each of the settlements and how these activities have impacted the results we are observing at the cluster level. Summary of TIS+ Interventions in Lower Juba The following interventions were the primary activities by which TIS+ sought to engage government and stakeholder partners to ensure that community cohesion and governance improvements were observed in Lower Juba. We have broken these interventions down by settlement to illustrate to the reader the primary activities conducted in this cluster by TIS+. Afmadow: The Afmadow Community Consensus Building Workshop on March 1–3, 2016, included 30 participants. Of these 12 were members of women’s groups, five were members of youth groups, five were elders, three were businesspeople, and five were members of local administration. Other stakeholders included representatives of the Jubbaland State of Somalia Ministry of Interior and Security. Participants prioritized the following projects: • Construction of a building for the local administration • Drilling of a water borehole and water storage tank (50,000 liters) with pipelines • Installation of solar-powered street lights in the town • Vocational skills training for 200 young people • Construction of a standard market • Construction of a standard slaughterhouse13 The consensus-building report indicates that facilitators introduced the concept of Project Oversight Committees (POCs) but the forming of POCs was inconsistent and not all activities had formed POCs. TIS+ implementation in this settlement included a hydrological survey, and constructing a women’s market, the district headquarters, and community hall. The district headquarters and women’s market has one combined POC composed of seven members. Kismayo: In recognition that Kismayo is further along the continuum from stabilization to development, there was no community consensus planning conducted. Instead, a bridging plan workshop was held November 15–17, 2016.14 This workshop included 190 participants (of whom 42 were women), including representatives from the Kismayo Administration, international and local agencies, the private sector, traditional elders, members from women’s and youth groups, and representatives of marginalized groups such as Internally Displaced Persons (IDPs). The workshop focused on two priority issues: Kismayo infrastructure, and engagement and empowerment of youth and women. During this workshop, different agencies shared information about which priorities they were working on. The priority noted for TIS+ was the Freedom Square construction. Project Oversight Committees were discussed, with an important focus on securing the proper roles for government in overseeing projects and supporting ministries to monitor projects alongside stakeholders. TIS+ has implemented several activities and projects in Kismayo. These include: (1) design work for the Kismayo Market Road and for the Airport Road (the implementation of the latter is on hold), (2) 13 TIS+ (March 2016) Final Afmadow Community Consensus-Building Workshop Report. 14 TIS+ (November 2016). Final Kismayo Bridging Plan Workshop Report, November 15–17, 2016. 147 rehabilitation and maintenance of 115 solar street light poles (originally built by Nordic International Support Foundation (NISF) with Somalia Stability Fund (SSF) funds), and (3) rehabilitation of Kismayo Freedom Square. During construction, the Freedom Square project did have a POC, but because this is a public–private partnership (PPP), its management has been handed over to the Jubbaland Chamber of Commerce and Industry (JCCI) and the municipality who have handed its management to a private investor. The solar lighting reconstruction was also a public-private partnership, which was taken over by the municipality and had no POC. Assessment Question 1 Did TIS+ Contribute to Any of the Following Anticipated Results? A. Community Ownership of Development Projects and Processes Key Findings 1. In isolated areas where TIS+ implemented community consensus planning was followed through with POCs, the impact is more visible. In these areas, there is evidence of increased awareness and ownership of development projects and processes, but these settlement level activities did not translate to the cluster level. 2. The reasons for this are multifaceted. (1) Settlements in Lower Juba are very diverse and this impacts the effect at the cluster level. (2) In some areas, there was poor follow-through on support for implementation. (3) Greater technical skill transfer is needed at the local level to manage sustainably project assets (i.e., size and scope of project matters). 3. Ownership requires confidence in the process of development planning, oversight, and management. TIS+ development support efforts often lacked the resources to support the sustainable transfer of skills needed to manage some development assets. 4. In TIS+ interventions where POCs were formed, there was evidence of more impact. The community was better informed of the management and maintenance as well as the process of prioritizing the intervention and its value to the community. Evidence To examine changes in community ownership of processes and development projects, the assessment focused on community consensus-building, consultation/participation at decision-making, and overall management of shared assets. • At the cluster level, people had increased access to planning events: adult women: +30%, adult men: +38%, female youth: +18%, male youth: +21%, religious leaders: +17%, state officials: +22%, federal government officials: +29%, and NGOs +21%. • Significant associations between awareness and access by these groups were also evident, which implies that the more people were aware of planning events the more they were likely to access them. • Awareness of oversight committees established to manage services/assets increased significantly by 20% on average, although the difference by settlements was significant (Afmadow: +57% and Kismayo: +16%). • Two factors may be contributing to this: (1) The governance model utilizing oversight committees is new to Afmadow and the transparent and voluntary process through which the POCs are established gains goodwill; and (2) In Kismayo, no POCs have been formed through the consensus process. Instead we have had private institutions such as the Chamber of Commerce take on this function. 148 • There was an 8% (p – value = 0.002) decrease in levels of satisfaction with decisions of community members participating in planning and contracting meetings in Kismayo, compared with a significant increase in satisfaction in Afmadow of 30%. • Our HH survey tool in Afmadow showed increases in the percentage of respondents reporting that projects were selected through community planning (up 62%, p – value = 0.000). Further, there was an increase (up 25%, p – value = 0.000) in those who thought all or most projects selected through community planning addressed community priorities. • Between 2017 and 2018, at the cluster level, the awareness of community planning events increased significantly by an average of 28% (p-value < 0.0001) with the increase being approximately six times in Afmadow (+62%, p-value < 0.0001) compared with Kismayo (+10%, p-value = 0.041) — see Figure 4a. • Key informants mentioned a much broader range of organizations delivering projects in Kismayo, such as SSI under USAID, SSF, UNHABITAT, United Nations Development Programme (UNDP), United Nations Industrial Development Organization (UNIDO), GIZ, NISF, Norwegian Refugee Council, and Save the Children. There were far fewer organizations working in Afmadow. • Government officials working with TIS+ reported that community planning did take place, even in Kismayo where community consensus was not carried out: “In the TIS+ project, which I am working on with the government, we have made a community building, where we consult with the community about projects and priority goes to projects proposed by the community. In the framework, we have developed, in all projects by various organizations, community consultation is a must” (37: K_K_GO1). Figure 4a: Do you know whether local projects are selected through community planning events? • Overarching KIIs conducted at the cluster level suggested that it is very difficult not to build community expectations when conducting community consensus processes because needs were often so high, but that this was a vital part of stabilization. Remaining engaged in communities was cited as important when there were delays (CA KII#3:10). 149 • The HH survey respondents indicating “most” to “all” of the selected projects effectively addressing community priorities increased significantly (p-value < 0.0001) by 25% in Afmadow, while in Kismayo the increase was 1%, which was not statistically significant (p-value = 0.704) but does demonstrate the variation of these settlements (see Figure 4b). Figure 4b: How many of the community planning meetings have been effective in selecting local projects/activities that address community priorities? • Community expectations, and the perception of unfair distribution of projects, were highlighted by a KII from Kismayo: “It may happen that many projects supposed to be implemented in Jubbaland state were not implemented because organizations are interested in easy project implementation, which does not benefit the community well. The organizations are not ready to establish a Memorandum of Association, and when our Jubbaland State asks them to have such things, they may divert projects to other regional states for implementation, which enables organizations to mostly move to the South West Regional State. As the youth, we appreciate and welcome how our state administration is doing. If photos are taken from a community and nothing is implemented, it will bring no development” (42: K_K_YL). • The Kismayo Freedom Square had a POC during the construction but because it’s now a PPP it has been taken over by JCCI and the municipality; they’ve given it to a private investor and they do maintenance. • Solar street lighting was a PPP; the municipality suggested support for six months and took over the initial maintenance. There was no POC for this activity, although now there is a suggestion that all projects should have a POC. TIS+ is also working on improving guidance on POC formation. • The District HQ and women’s market has one POC composed of seven members. TIS+ is now revisiting them so they can get separate POCs. • Respondents were aware of the construction of the TIS+ women’s market and the District HQ: “Yes, the community members are the ones who use the projects and they are also the ones who present the ideas” (9: C2_A_ME_FGD). “The district administration and TIS+ called the people for consultations and the community participated by sharing their views with them. Most of the suggestions brought forth by the community were considered” (10: C2_A_WE_FGD). • In Kismayo, FGD respondents reported that community members had a common understanding of community needs and suggested an existing high level of collaboration. There were reports of differences of opinion, and some suggestion that NGOs, elders and committees, and district commissioners could resolve any conflicting interests: “Mostly, we have [the] same needs sometimes but there can be someone [who] thinks he needs another thing but the clan elders and the committees solve the 150 differentiation” (14: C2_K_FE_2_FGD). “At times, some of the issues cannot be agreed upon such as implementing projects in the area. Although most things are agreed upon by the people in the community like the development and sharing a common interest, it is also possible that some projects are not implemented equally” (24: C2_K_YG1_FGD). • TIS+ practices in Afmadow were distinctly different from those in Kismayo, but even in Kismayo, the TIS+ approach was valued. As one youth male stated, “I was invited to take part in the decision-making process as well. The district administration called me in and they even requested me to bring other youth members who could contribute both ideas and physical strength in the interventions. The way the district administration and TIS+ respected our participation was very good and this made us feel that we belong to this country. I wish the other organizations could think and implement their projects in a similar way. Some of the organizations are hostile towards us; they simply do what they want to do and then just leave without even asking us how the things they have put up will be of benefit to us. I wish that one day the government will highlight these organizations and take legal action” (42: K_K_YL). • Based on quantitative and qualitative data, the Afmadow experience with community consensus planning was novel and showed significant community-level impact. Based on interviews, no similar process had been held before in this settlement that respondents were aware of. • In contrast, Kismayo had more development efforts but less satisfaction, expressed both in the HH survey and on qualitative data pointing to changes in expectations, and in approaches utilized to engage communities in planning and decision-making. However, based on interviews with TIS+ beneficiaries there was a noted difference in the TIS+ approach in Kismayo on specific interventions. B. Effective Shared Asset Management Key Findings 1. In interviews with beneficiaries, there is an expectation that state and local government have a role to play in asset management, but there is evidence that the private sector and NGOs are more trusted in assisting communities in managing assets collaboratively. 2. PPPs work more effectively on highly localized efforts where the community can interact and provide direct oversight of these efforts. Larger projects seem to have less community engagement and citizens seem less informed about who is responsible for such critical functions as budget auditing, quality control, and access. 3. Where oversight committees were evident and well formed, there was greater trust and awareness of the roles of government (state and local) as well as other actors responsible for maintenance of the activity. Evidence • Respondents in FGDs reported several services and shared assets in the community, including public or government offices, public and private schools, public and private hospitals, roads, and the airport and port of Kismayo. Many FGD respondents expressed dissatisfaction with government services: “The government should play a great role in the management of the shared assets because currently the management of the private assets is much better than the public ones” (13: C2_K_FE_1_FGD). • FGD respondents widely reported that shared assets generally were managed by the district administration and the community, and that this included the TIS+ projects. “The district administration educated people and the relevant or assigned people [manage shared resources]. For example, 12 people were assigned to manage the women’s market, which was built by TIS+ in the last year” (9: C2_A_ME_FGD). • FGD respondents reported that the involvement of the community improved the quality of services: “The quality has increased because the communities are active these days and they always respond to the requests of the other service providers and the government” (9: C2_A_ME_FGD). Thus, TIS+ projects were more effective: “In general, the quality is good because the district administration and the other service providers give out good efforts. For example, the committee that was selected by TIS+” (11: C2_A_YB_FGD). 151 • One FGD respondent from Afmadow cited, “Yes, the people in this settlement work well together when it comes discussing their needs; for example, they didn’t clash when TIS+ offered them the construction of the district commissioner’s office and the women’s market.” • Key informants confirmed that the business community was involved in projects, often contributed money toward projects, and that the Chamber of Commerce played a role. • There was some suggestion from government key informants that government was not sufficiently aware of or involved with projects delivered by other international organizations: “The services provided by the other international organizations is not known to us mostly as they do it on their own . . . while we hear from the media millions of dollars that the humanitarian organizations invested in the country and we don’t know how that money is utilized” (38: K_K_W_MIN). Apart from the role of oversight committees, we examined increased or decreased role in the management of services/assets by other players. • In Afmadow, government at local and state levels had a predominantly increased role in the management of schools, health centers, meetinghouses, market facilities, sports arenas, and roads, compared with other players (with all changes being statistically significant: p-values < 0.05), followed by NGOs in the management of health centers (see Table 5). • In Kismayo, the state government had an increased role in the management of schools (+8%, p-value < 0.0001), health centers (+8%, p-value = 0.001), meetinghouses (+5%, p-value = 0.007), market facilities (+8%, p-value < 0.0001), and roads (+15%, p-value < 0.0001) compared with other players. • This was followed by NGOs and business traders in the management of some of these services/assets (see Table 5). It is, however, important to note that the local administration had a significantly reduced role in the management of all assets in this settlement. • The local administration had a significantly reduced role in the management of all assets as seen in Kismayo, although it played a significantly increased role in the management of shared assets in Afmadow. • The HH assessment also examined the maintenance quality by other players (not oversight committees) as one of the parameters for checking management effectiveness. • In Afmadow, maintenance quality for all the services significantly improved in schools: +31% (p-value < 0.001), health centers: +46% (p-value < 0.001), meetinghouses: +44% (p-value < 0.001), market facilities: +42% (p-value < 0.001), sports arena: +29% (p-value < 0.001), and roads: +24% (p-value = 0.001). • However, maintenance quality for all the services in Kismayo reduced in schools: -1% (p-value = 0.384), health centers: -1% (p-value = 0.262), meetinghouses: -5% (p-value = 0.312), market facilities: - 5% (p￾value = 0.024), sports arena: -2% (p-value = 0.047), and roads: -1% (p-value = 0.322). • In Afmadow, the role of maintenance has increasingly been undertaken by the local and state government, followed by oversight committees and NGOs. While the maintenance role in Kismayo has increasingly been dominated by peace committees, the state government, oversight committees, and NGOs, with sports arena increasingly maintained by the business community. • In Kismayo, the oversight committees have reduced effectiveness in the management (though not statistically significant) of the following services/assets: schools: -1%, (p-value = 0.633), health centers: - 1%, (p-value = 0.707), meetinghouses: -5%, (p-value = 0.057), sports arenas: -4%, (p-value = 0.759), and roads: -2%, (p-value = 0.641). A significant reduction in management effectiveness was seen in market facilities: -2% (p-value = 0.035). 152 Table 5: Who else is responsible for the management of the following services/assets? (Changes between 2017 and 2018.) C. Improved Delivery of Services Key Findings 1. TIS+ did not significantly contribute in this area in this cluster. Although there were changes in the HH survey, TIS+ did not track or contribute to these measures. 2. Private sector service providers, NGO dominance, and poorly staffed and resourced state and local administrations make service delivery support by donors complex, contested, and an important space for coordination. TIS+ contribution was primarily in this area where staff sought to improve coordination between the government and communities. 3. There is a clear preference for NGO and private sector service delivery, leaving citizens with high expectations for government service delivery. Evidence: Relating to Service Provision • In Afmadow, the respondents perceived government at state level to have significantly increased the provision of almost all the services (all p-values < 0.013) including health (+31%), education (+39%), water (+22%), roads (+15%), sanitation (+14%), security (+35%), and financial services (+35%), with a nonsignificant increase of 1% (p-value = 0.764) in electricity provision (see Table 6). Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Clan Elders -8% -7% -7% -9% -3% -5% -8% -2% -3% -1% 0% -1% Militia groups 0% 0% 0% 0% 0% 0% 0% 1% 0% 0% 0% 0% State Government 15% 32% 26% 27% 25% 21% 8% 8% 5% 8% -2% 15% Religious leaders 11% -2% -8% -4% -3% -3% -9% 0% 0% 0% 0% 1% Local administration 6% 19% 23% 23% 23% 24% -34% -35% -39% -38% -52% -37% Ugaas/Sultan 0% -2% 0% -4% -2% -4% 0% 1% 2% 0% 0% 1% Diaspora -3% -2% -5% -3% -4% -3% 0% -1% 0% 1% 0% 0% Businesses/Traders 0% 0% 0% 3% 0% -1% 1% 0% 0% 2% 20% 0% Local community -6% -1% 0% 1% 1% -1% -2% -2% 0% 0% 0% 0% NGOs 6% 14% 2% 1% 0% 1% 9% 11% 5% 4% 2% 0% International organisations 0% -1% 0% 0% 0% 0% -2% -5% -1% -1% -3% -4% Youth 0% 0% 0% 0% 2% 0% 0% 0% 0% 0% 1% 0% Afmadow Kismayo 153 Table 6: Service provision (changes between 2017 and 2018). • In Kismayo, the provision of some of these services by the local people, local NGOs, and the business community increased significantly. However, the provision of almost all the services in Table 6 by the state and local government reduced significantly. • In both Afmadow and Kismayo, provision of almost all the services by the local structures (clan elders, religious leaders, local courts, and Ugaas/Sultan) either reduced slightly or had no significant change. • This may indicate a positive trend that points to the fact that the formal institutions, though at a nascent stage, are being able to deliver services. The expectation of service delivery from public institutions is a sign of confidence. Evidence: Relating to Service Delivery Satisfaction • Changes in satisfaction levels varied between Afmadow and Kismayo for all the services. • In Afmadow, the survey respondents were increasingly satisfied with the following services: health: +45% (p-value < 0.0001), education: +28% (p-value <0.0001), water for other purposes: +51% (p-value < 0.0001), sanitation: +13% (p-value <0.0001), electricity: +47% (p-value < 0.0001), security: +11% (p-value =0.001), and police: +7% (p-value <0.045), apart from roads, which saw satisfaction significantly reduced by 2% (p-value =0.002). • In Kismayo, satisfaction levels reduced for all services: health: -7% (p-value = 0.580), education: -5% (p￾value = 0.428), drinking water: -18% (p-value =0.400), water for other purposes: -8% (p-value = 0.617), roads: -8% (p-value = 0.385), sanitation: -9% (p-value = 0.135), electricity: -14% (p-value = 0.894), security: -6% (p-value = 0.393), and police: -9% (p-value = 0.932), apart from financial services, which had increased satisfaction levels by 5%. All these changes in Kismayo were not statistically significant at 5%. Health Education Water Roads Electricity Sanitation Security Financial services Health Education Water Roads Electricity Sanitation Security Financial services Clan Elders -15% -14% -1% -4% 0% -5% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% State Government 31% 39% 22% 15% 1% 14% 35% 38% -10% -11% -46% 0% -41% -9% -4% -16% Local people -22% -15% -17% -13% -3% 11% -7% 7% 25% 25% -3% 23% -8% 38% 20% 22% Local NGO’s 35% 32% 1% 17% 0% 33% 2% 31% 24% 36% 2% 4% -1% 9% 4% 11% International NGO’s 51% 44% 6% 26% 1% 31% 2% 38% 4% 11% -6% -5% -1% -8% 1% -14% Religious leaders -1% -1% -1% -1% -1% -4% 1% 0% 0% -1% 0% 0% 0% 0% 0% 0% Local administration -2% 3% -8% 16% -18% 10% 23% 19% -36% -39% -54% -33% -55% -37% -31% -50% Police -2% 0% 0% -4% 0% -6% 5% 1% 1% 0% 0% 14% 0% 0% 0% 1% Local Courts -2% -2% -1% 0% 0% -3% -11% -1% 1% 0% -1% 0% 0% 0% -1% 0% Business community -2% -5% 14% -5% 17% -2% -3% 14% 16% 15% 55% -2% 49% 18% 0% 22% Ugaas/Sultan -1% 0% 0% 1% 0% -1% -1% 0% 0% 0% 0% 0% 0% 0% 0% 0% Diaspora -4% 1% -1% 0% 0% -1% -1% 3% 0% 0% 1% 0% 0% 0% 0% 0% Foreign Governments 0% 0% 0% 0% 0% 0% -1% 0% -1% -1% 0% 0% 0% 0% 0% 0% Women support groups 0% 0% 0% -1% 0% -10% -2% 0% 0% 0% 0% 0% 0% 1% 0% 0% Afmadow Kismayo 154 • Because there was no notable impact on service delivery satisfaction in Kismayo during this period, this may be attributed to the fact that the increasing stability in Kismayo and the increase in capacity of the government agencies means that citizens expect better standards of service delivery. • Service delivery satisfaction was important to respondents and TIS+ respondents did note reasons for this. As one woman from Kismayo stated, “People have confidence in the government and the reason this confidence is increasing is the people in Kismayo are getting more services that are implemented by the government particularly or services that the government is facilitating. Apart from the services that the government is providing to the community there are also many youth, women, and elders who are working for the government.” Evidence: General Security as a "Service" • About 4% more respondents in Afmadow felt there was an improvement in the general security. However, the security situation decreased by 2% in Kismayo. • This can be attributed to the political uncertainty surrounding Jubbaland ahead of the elections, the fact that Kismayo has remained an island for so long, and AS is on the outer boundaries of the city (see Figure 5). • The household assessment also specifically examined the security situation between settlements. Calmness increased by 3% (p-value 0.585) in Afmadow indicating nonsignificant change, although there was a significant reduction by 8% (<0.0001) in calmness in Kismayo. • Citizens interviewed were clear about the link between security and increased confidence in government. Security as a service was a critical government function. As one clan elder stated, “Confidence in government has increased because there is law and regulation as well as courts, which have dealt with the chaos and disorder that was present when the government was not involved. Government forces hand over any thieves to the court. People now have no fear of insecurity and can transact without fear; they respect and obey the government.” • Community members also stated that the ability to report crimes had become easier because more local services were being offered. As one government official shared, “The government has instituted back law and order. Citizens can now report problems to our functioning police department. There is an OB (known as a ‘Police Occurrence Book’) where citizens can record complaints and cases. We have functioning courts that adjudicate these cases, [and] all this has definitely returned citizens confidence in the government.” Figure 5: Do you feel that security has improved in your settlement over the past year? 155 D. Reduced Conflict over Resources Key Findings 1. There was evidence of decreased conflict between clan groups. However, there was increased localized conflict, particularly around land and water access. 2. People indicated greater trust in local resources for conflict mitigation, but poor access to justice for contentious land and resource issues. Evidence • Respondents in both Afmadow and Kismayo believed conflicts over resources had significantly reduced between 2017 and 2018 (see Figure 6). This was also evident in KIIs and FGDs. • There was evidence that inter-clan conflicts were still significant, although there were references to community events (e.g., sports) improving the interaction between clans and groups that before had experienced significant conflict. • Land conflict was noted as a continued concern in several FGDs and KIIs, particularly among elder women and men. • Access to justice issues were raised in KIIs and FGDs in both settlements. A significant gap in access to justice services for larger inter-clan issues was noted, particularly land access and water rights issues. Figure 6: Compared to last year, have conflicts over shared resources increased or decreased among various groups? • Even though resource related conflicts had significantly reduced in Kismayo (-2%), the survey respondents had increased distrust in conflicts resolved by: clan elders: -55%, national army: -57%, peace committees: -52%, oversight committees: -56%, state government: -49%, religious leaders: - 43%, local administration: -57%, police: -57%, local courts: -39%, mediation: -44%, and Ugaas/Sultan: - 36%. There was a significant increase in trust in conflicts resolved by militia groups (+9%). This illustrates the complexity of clan dynamics and its importance in Kismayo. • Many FGD respondents in Kismayo reported being involved in peace messaging and other peacebuilding activities. Respondents reported a strong level of existing inter-clan interaction, reporting that clans increasingly intermarry and live alongside each other. This was seen as contributing to peacebuilding: 156 “Yes, in the past there has been conflicts among clans, but today people eat together, sleep together, and it is part of the peace we have.” (24: C2_K_YG1_FGD). • Kismayo may be seen as a peaceful city, but conflicts are still evident to citizens. Many respondents reported continued frequent conflicts over land and property. “Farm conflicts and land disputes are the most constant that do exist in this area and the government and the clan elders are always busy but still the conflicts go on” (14: C2_K_FE_2_FGD). “Yes, there are conflicts relating to land disputes, which has caused many casualties” (19: C2_K_YB_1_FGD). These disputes were frequently resolved by elders, many respondents said, with some indicating that the courts and judiciary should take over. • FGD respondents mentioned that the land commission (on which SSF is working) had helped conflict resolution: “Kismayo is developing and the main new things we are seeing nowadays include the land commission, which helped a lot in the resolution of the land and farm conflicts” (C2_K_ME_2_FGD). • In Afmadow, respondents did feel that the interactions between clans improved security and reduced conflict: “Yes, I think these interactions contribute to the peacebuilding and reconciliation.” (9: C2_A_ME_FGD). E. Greater Citizen Confidence in Government Key Findings 1. Responses were quite mixed and although there were noted variations across settlements, there was a decrease in confidence in state and local government across several areas of service delivery and citizen support. 2. Citizens felt that government should provide more services, but there was an increased awareness and perception that the government was not as well staffed and skilled as some other service providers. 3. Noted improvements that seem to be linked to citizen increases in confidence are (1) improvements in security, (2) increases in confidence in state and local administrations’ prioritization of citizen needs, and (3) improved communication and information exchange between citizens and government. Evidence • Confidence in state and local administration (concerning the ability to work for the benefit of local development and community needs) increased by 11% (p-value < 0.001) and 14% (p-value < 0.001) respectively in Afmadow. While in Kismayo, the confidence in state and local administration was reduced by 3% and 2%. • Other aspects of confidence in government were examined, including public trust (local administration prioritizing community needs and the trustworthiness of the local officials to utilize funds for development), whether Federal officials are directed by clan interests, sharing of information by the local administration, corruption levels, and representation. • At cluster level, the percentage of respondents believing the following increased significantly: the local administration prioritized community needs (52%), the local/district officials were trusted in utilizing funds for development (44%), Federal officials were not guided by clan interests (43%), local administration shared information (53%), government encouraged people to participate in planning (52%). These results had positive direction in both Afmadow and Kismayo districts. • Changes in corruption levels at local and Federal levels differed between respondents in the two districts. At local levels, corruption significantly reduced by 16% in Afmadow, while there was no change in Kismayo. However, at Federal levels, respondents in Afmadow thought corruption had significantly increased by 16% and those in Kismayo thought it had significantly decreased by 13%. • A greater majority of FGD and KII respondents from both Kismayo and Afmadow reported communities’ increased level of confidence in government largely due to security improvements albeit with some misgivings. But there was still evidence of frustration with security and the role of AS. 157 • When asked directly about confidence in government, respondents largely said they had confidence in government and that it had increased, with some citing improved services and security, including the ability to move at night. • At the cluster level, confidence in state government to work for local development and community needs decreased from 99 to 96% (p – value = 0.006) and for local government from 99 to 97% (p – value = 0.027) and confidence in federal government remained stable at 96%. • Perceptions of corruption at the local level affecting the settlement remained constant at 26%, whereas the perception that corruption at the federal level impacted development in the settlement decreased from 39 to 23% (p – value = 0.000). • In 2018, 75% said local/district officials were trustworthy for utilization of funds for the development of the settlement, up 45% (p – value = 0.000) since 2017; 72% said federal officials are not directed by clan interests in making decisions for the development of Somalia (up 48%, p – value = 0.000). • These are very high levels from the baseline. A reflection of the increased perception of positive change based on the FGS presidential campaign and recent political changes in this cluster have likely impacted these shifts in measures. • In FGDs and KIIs, when asked directly about confidence in government, respondents largely said they had confidence in government and that it had increased, with some citing improved services and security, including the ability to move at night in Kismayo. But a smaller number also said it was not at optimal levels, or even that it was decreasing, that officials were corrupt. • Confidence was frequently connected to the government’s provision of security and combatting terrorism. Some expressions of confidence were qualified, indicating that government could be better if it delivered more services. F. Reduced Support for Violent Extremism Key Findings 1. There were significant settlement variations in this cluster. Afmadow seems to be at greater risk of manipulation from AS. 2. Security has improved, but there is still a noted need to diminish the influence of AS. As noted in KIIs and FGDs, clan elders and business members are often targeted by AS. 3. There are key services still provided by AS that are linked to security and community relations. Access to justice is a critical area that AS seems to fill, particularly involving small clan conflicts over land and resources. Evidence In terms of reducing factors and conditions relating to violent extremism, the following data were collected. These measures are difficult to attribute to TIS+, but are important to consider for programming. Respondents were asked if young people were prone to violent extremism, and changes in perceptions differed between Afmadow and Kismayo. • The respondents were asked if young people were prone to violent extremism, and changes in perceptions differed between Afmadow and Kismayo. In Afmadow, there was a significant reduction by 11%, while there was an increase by 26% (p-value < 0.0001) in Kismayo (see Figure 7). 158 Between 2017 and 2018, about 4% more respondents in Afmadow indicated violent extremist groups were not welcomed in their settlements, while there was a reduction by 8% in Kismayo. • At the cluster level, those reporting improved security in their own settlement over the past year reduced from 100% to 98% (change not significant). Those describing the security situation between communities as calm decreased by 8% (p – value = 0.000). • Violent AS conflict was widely reported but many respondents said such conflicts tended to happen outside Kismayo. “There were not many conflicts in the area, but this year there [were] some conflicts in some parts of Jubbaland state particularly in the lower Juba region, which has very heavy fighting between Al￾Shabaab and Jubbaland state” (24: C2_K_YG1_FGD). • Many respondents said they were not afraid of AS, but there was evidence from FGDs that elders felt targeted by AS: “We, as clan elders, facilitate the interactions between the government and the community, and that is why the Al-Shabaab militia wants to kill us” (17: C2_K_ME_2_FGD). • In Afmadow there were noted differences from Kismayo. AS-related violence was commonly reported in Afmadow FGDs: “Fighting between the Al-Shabaab and the government is something that is always happening in this area” (9: C2_A_ME_FGD). “The only conflict that even the small kids know of is the one between the government and the Al-Shabaab” (11: C2_A_YB_FGD). “People are still dying in the hands of Al-Shabaab and the government does not take action against them” (11: C2_A_YB_FGD). Assessment Question 2 Were TIS+ Approaches Used in the Cluster, e.g. the Community Consensus Process Itself, Instrumental in Contributing to the Achievement of TIS+ Results Described in Question I? Key Findings • The TIS+ community consensus process is a system that is relatively unique in this area. Donors working in this cluster and in stabilization in Somalia feel that the approach used by TIS+ for planning and oversight of small-scale infrastructure has influenced the landscape in this cluster. Figure 7: Do you think that young people in your settlement are highly prone to violent extremist group? 159 • The impact was more evident in Afmadow than Kismayo and the processes were quite different. Where applied as a “package” of services that allow POCs to be formed and maintenance to be planned, the approach is effective. • There is, however, a need to refine this process clearly with staff and prioritize activities that allow for its full implementation. Monitoring the process is essential, per staff interviewed in the field. Evidence In both settlements under discussion, TIS+ did implement the community engagement process from the consensus and planning stage through to project oversight and delivery of projects. The approach was different in Kismayo and Afmadow, but both settlements promoted participatory processes. Our data certainly suggests that the community consensus aspect of this work is appreciated and that communities themselves think this adds value by mitigating tensions, making community members feel more included, and linking community needs and priorities with governance processes and decision￾making. However, responses also reflected the differences between these settlements. TIS+ did not conduct community consensus planning in its traditional format in Kismayo, but the Bridging Plan Workshop conducted here was itself quite consultative, and promoted participatory and consultative approaches for government. • In Kismayo, there was a significant 8% decrease (p-value = 0.002) in levels of satisfaction with decisions of community members participating in planning and contracting meetings. • Community consensus was clearly valued, but some respondents also said that community consensus planning carried the risk of raising community expectations beyond the capacity to deliver, particularly for newly formed government units. • In Afmadow, by contrast, there was more improvement related to community planning. Those reporting that projects were selected through community planning significantly increased by 62% (p￾value = 0.000). • There was a significant increase (+25%, p-value = 0.000) in those who thought all or most projects selected through community planning addressed community priorities. And there was a substantial 30% increase (p-value = 0.000) in levels of satisfaction with decisions of community members participating in planning and contracting meetings, compared with a relatively smaller though statistically significant decrease of 8% (p-value = 0.002) in Kismayo. • Many interview respondents in Afmadow reported that they had been personally involved in planning processes, and youth and women were reported as playing key roles in decision-making. As stated by one female elder from the Afmadow FGD, “The community participated actively because TIS+ invited them to come on board. Some of the community members competed to win the contracts while others attended the public meetings where all the community members discussed the benefits and disadvantages of these interventions.” • The community consensus process clearly had an impact at the community level as well as with government officials, but expectations raised without effective community engagement and management also presented some challenges. As one government staff from Kismayo stated, “The main challenge is with raising expectations, especially when you go into a town and you ask them to select projects. People become hopeful, but when the wish list becomes long it is difficult to explain how all the projects cannot be implemented. There needs to be more upfront explanation of this.” • In Afmadow, there was a significant 57% increase (p-value = 0.000) from 25% to 82% in awareness of oversight committees established to manage certain services at the settlement level. Oversight committees were seen as having an increased role in maintenance of services (up 1–12%). • Afmadow respondents suggested that community consultation and oversight might be increasing interaction at the community level. This was less evident in Kismayo in the HH survey, but both settlements had respondents that viewed the consultative process of TIS+ positively. 160 • In Kismayo, there was a 16% significant increase (from to 69% to 85%, p-value = 0.022) in awareness of oversight committees, but the percentage of respondents reporting that POCs were responsible for maintaining certain services was very low (a maximum 3%). • This may reflect the fact that in Kismayo, where capacities are higher, government and other actors may be the appropriate actors to maintain services. • Indeed, the TIS+ solar lighting rehabilitation and Freedom Square projects were both public-private partnerships and were handed over to the municipality and the chamber of commerce, respectively. Almost all our interviews with donors, peer programs, and Somalia experts felt that the TIS+ model of engaging communities in planning and oversight was an important and appropriate programming approach in Somalia. The fact that TIS+ brings government and communities together in this process was particularly important to respondents. TIS+ is seen by many as one of the main large-scale actors promoting this approach. Assessment Question 3 Did Government at all Appropriate Levels Contribute to Asset Management in Communities in Which TIS+ Was Implemented? How? Did Government at All Appropriate Levels Have Both the Capacity and the Will to Contribute? Key Findings 1. Weak government capacity and minimal resources make government contribution to shared asset management challenging. 2. A history of privatized services make collaboration with the private sector essential, but effective models for community oversight are still essential to achieve project objectives. 3. Local government is easier to collaborate with, but has far fewer resources; however, community relationships seem to be stronger, allowing for localized planning and management efforts to be more collaborative. Evidence Interviews noted that governance structures in Lower Juba are still quite weak at the local and state level. Both State and District administrations are understaffed and under-resourced. TIS+ sought to strategically address the need for partnership with government by embedding TCs in the State Ministries, assisting in the formation of public private partnerships, and supporting community consensus and participatory development planning processes in collaboration with local government staff and units. These TIS+ systems of government support and coordination were met with varying levels of success. This was because of complex inter- and intra-governmental relationships and a lack of coordination between government units that was evident from interviews with State and local government as well as other donors working in this area. There was robust participation by the government at the district and state level in the community planning sessions as evidenced by attendance and participation in the community planning sessions in Afmadow and Kismayo. • As evidenced in qualitative interviews, there is an “expectation” that state and local government has a role to play in asset management. Numerous FGDs mentioned this in discussing the role of local and state government in asset management. • Per HH survey data as well as interviews, people trust the private sector and NGOs more than the government to assist communities in managing assets collaboratively. • Afmadow had a community consensus process and Kismayo did not. Qualitative interviews showed higher levels of satisfaction with government (local) in Afmadow, and there was strong evidence that the process was inclusive and also engaged state and local administration elements. 161 • There were far fewer development efforts in Afmadow. Therefore, TIS+ was noted in several interviews and details about the impact and contribution of their interventions was quite clear. Oversight committees were evident and participation of government (state and local) as well as other actors responsible for maintenance of activities well noted by the community. • Although TIS+ is not the only actor in Afmadow, TIS+ projects did figure prominently in our qualitative data. Our research did not find many actors who were working on the same type of small infrastructure as TIS+ with the same approach. The impact in Afmadow was much easier to attribute to TIS+ than in Kismayo. • Two factors contribute to this: (1) the governance model utilizing oversight committees is new to Afmadow and the transparent and voluntary process through which the POCs are established generates goodwill, and (2) whereas in Kismayo, no POCs have been formed through the consensus process. Instead activities were managed by membership-based institutions such as the Chamber of Commerce. • Larger projects seem to have less community engagement and citizens seem less informed as to who is responsible for such critical functions as budget auditing, quality control, and access. • In Lower Juba, services are also often privatized with multiple service providers. Donor support for service delivery and shared asset management is complex as management and planning of services varies across districts and requires significant staff time to support government partnership. Governments are often unable to manage these systems and require significant private sector and donor support. Assessment Question 4 Were TIS+ Activities Inclusive of Women, Youth, and Minority Clans, as Applicable to the Context of the Cluster Being Evaluated? Key Findings 1. Effective participation of women is challenging because of cultural constraints and lack of exposure to essential skills such as planning, public speaking, and financial management. 2. TIS+ has significantly included women and improvements have been made since the baseline, but there is still not enough targeted support for women on these POCs. 3. Sports and cultural events are consistently cited as contributing to improved clan interactions. 4. Minority clans may require more targeted interventions because there is evidence that they are harder to reach unless a specific conflict or minority issue is prioritized as an intervention. Evidence Not dissimilar from the baseline, both male and FDG participants in Afmadow and Kismayo believe that men have more access to opportunities compared to women. Some respondents noted the positive influence of TIS+ and methods used to engage youth constructively in consensus and planning sessions held. Participants stated that there were religious and cultural aspects that limit women’s opportunities and access. Youth stated that, although this may change, elders and clan leadership still enforce these limitations. There are impediments to increasing the participation of women in public life as security and access makes traveling for women complicated in Somalia. Solar streetlights, which was a TIS+ intervention, were noted as particularly important for security and ease of movement by women. • Access to planning and contracting events in the cluster now significantly include more stakeholders (adult women, adult men, female youth, male youth, religious leaders, state officials, federal government officials, and NGOs/CBOs) rather than the dominant local leaders. • FGD and KII respondents have seen improvements in the recent past on women’s role in development. The role of women in taking part in asset management and oversight functions is beginning to be recognized. 162 • However, participation by women is still a challenge, with many TIS+ activities falling well below the required standards of at least 30% involvement. • Although most FGD and KII respondents expressed the view that women and men have equal access to public assets and services in Kismayo and Afmadow, based on the HH survey there were distinct differences in access for women and men in both settlements. • Based on data collected, sports and cultural events were important in improving peaceful coexistence and inter-clan dynamics in this cluster. This was evident from all data sets. • Women have been significantly engaged in some of the planning processes of TIS+ and several interviewees in this cluster noted the potential that this provided in allowing women more public roles for leadership. • It was noted in interviews that culturally women are often seen as peacemakers for intra- and inter￾clan issues because women often intermarry across clans. Several respondents noted that they felt there was a greater role for women to play in resolving clan issues but that there were limited public forums elevating this asset or making these efforts publicly known. Assessments within each of these districts asked further questions on why women were considered not equal to men. • There was a reduction of 6% (p-value = 0.723) in Afmadow of those who believed women were considered equal to men. However, in Kismayo, there was an increase of 13% (p-value < 0.001). • In FGDs with young men in Kismayo, there was evidence that deeply held cultural beliefs still limit the opportunities and potential of women and girls in this cluster. • Respondents in Afmadow increasingly supported reasons most which are closely related with cultural beliefs at the clan level. Positive attitude toward women seem to increase in Kismayo. Achieving participation of women and is challenging and requires specific staff and planning systems that allow for participation and in some cases, additional training. Based on staff interviews there is clear evidence that this is a focus for TIS+ field staff, despite cultural constraints. TIS+ works very to address inclusion issues and staff have had training and this is a topic discussed at annual staff development events. Partners working with TIS+ were also aware of these constraints. As stated by one business leader from Kismayo who partnered with TIS+, “Women are involved in leadership roles but not in sufficient numbers. They have not even reached the promised 30% quota for women. You can see this at many levels, from the parliament to ministries, committees, and other government institutions. Women’s participation in leadership can be said to be about 10%. The role of women on oversight matters is completely missing. They are not represented at all.” But this is only one perception, as TIS+ has managed to create space even under trying circumstances. As shared by one female elder in Kismayo, “I participated. I was among the females assigned to monitor the construction of the Liberty Square. I was managing about 30 people who made up the taskforce constructing the Liberty Square.” Minority clan issues are not as effectively focused on by TIS+. This may improve with a more focused approach on assisting minority clans in prioritizing their needs within the consensus building process. Although minority clan members were included it is not evident that prioritization processes during consensus planning focused on these issues. There is evidence that field staff would benefit from applying more political economy analysis in understanding minority clan dynamics and perhaps even holding separate planning sessions with minority clan groups critical in key areas of intervention. 163 Section 3: Lower Juba Conclusions The next stage in our analysis is to ask what other influencing factors could be accounting for the changes observed relevant to the key assessment questions. These include other stabilization and development programs as well as positive and negative factors in the context around the intervention that might have important roles in determining outcomes. Key Conclusions 1. Based on analysis of qualitative and quantitative data, it is evident that the premise of the TIS+ design could contribute to improved methods of service delivery, shared asset management, and decreased conflict over resources. However, there are significant gaps in the planning and oversight process, compromising support for implementation and management of these efforts. 2. Some fissures in the field are created not by the design of the TIS+ intervention modality, but by the support for intervention and effective field monitoring. 3. As evidenced in qualitative interviews, there is an “expectation” that state and local government has a role to play in asset management, but there is “evidence” that the private sector and NGOs are more trusted in assisting communities in managing assets collaboratively. This will present a challenge for TIS+ in partnering with FGS, FMS, and local administration staff. 4. Lower Juba improved in both good governance and community cohesion elements from baseline to midline measures. These changes reflect a complex arrangement of donor efforts, national political changes, and security improvements and challenges that impact both community access and citizen perceptions of federal, state, and local capacity (see Table 8). Table 7: Index Scores for Lower Juba (Summary) Good Governance Index Community Cohesion Index 2017 2018 Change 2017 2018 Change Lower Juba Afmadow 43% 64% +21%* 54% 71% +17%* Kismayo 61% 67% +6%* 62% 73% +11%* Total (cluster level) 55% 66% +11%* 59% 72% +13%* Good Governance Index The themes contributing to the good governance index included confidence in government, service delivery, and stakeholder partnerships. Variables under each of these have been triangulated with qualitative data to establish cluster level findings. ● The good governance index score increased by 21% (p-value < 0.0001) and 6% (p-value < 0.0001) in Afmadow and Kismayo respectively. These positive trends were observed despite negative trends in some theme specific variables, particularly in Kismayo. ● There has been a significant increase in donors’ efforts to work in partnership with state and local governments in this cluster, and it is likely that these shared efforts have influenced changes in this measure. ● Donors are seeking to work more closely with local administrations, and this is putting increased pressure on government staff. TIS+ has worked to improve coordination and cooperation with state and local authorities. ● Citizen perceptions of capacity of government staff still vary greatly. There is an expressed desire in both settlements to have the government manage more service delivery, but a recognition that currently these services are better provided by NGOs and the private sector. ● Confidence in government has improved at all levels in this cluster, but the reasons for this are largely based on perceived changes in security and access to services. 164 Community Cohesion Index The themes contributing to the community cohesion index included interpersonal and institutional trust, social and political inclusiveness, and network diversity. ● Scores for the community cohesion index significantly increased for Afmadow (+17%) and Kismayo (+11%). ● In Afmadow, there was increase in trust in formal institutions. The opposite was observed in Kismayo, where people increasingly trust informal institutions (militia groups) to solve their resource related conflicts. This is largely due to complex clan dynamics. ● At an interpersonal level, the following perceptions were noted: o Increase in the number of people working collectively toward problem solving for the benefit of the settlement (Afmadow: +2% (p-value = 0.039) and Kismayo: +1% (p-value = 0.218). o Significant decrease in the number of people who do not trust one another across clans and within settlement areas (Afmadow: -23% and Kismayo: -30%). o Increase in the number of people who would cooperate on development initiatives and efforts in Afmadow: +9% (p-value = 0.003) and Kismayo: +1% (p-value = 0.928). ● The general involvement (inclusion) of people in decision-making has significantly increased (Afmadow: +27% and Kismayo: +27%). Based on qualitative data, most projects have some measure of participation built into their programming and many programs in Kismayo and several in Afmadow target marginalized groups. ● Concerning network diversity: o Between clan peaceful coexistence also increased (Afmadow: + 8% (p-value < 0.0001) and Kismayo: +2% (p-value 0.061). o There was a reduction in people believing that clans only cooperated when necessary (Afmadow: -14% and Kismayo: -17%). o This is a positive sign that work in mitigating clan tensions and activities that encourage inter-clan interaction are having an impact in this cluster. What else could be accounting for these changes? There were distinct differences in the number of actors in the various settlements that may have impacted the outcomes. There were far fewer other development partners in Afmadow than there were in Kismayo, and this may contribute to this variation. Kismayo is also one of the safest urban areas in Somalia due to the strict security infrastructure instituted by the state administration. National Intelligence and Security Agency officers, Daraawish police, and AMISOM forces patrol with heavy security around the main economic hubs of the airport and the port. There has been little military progress to extend this security beyond Kismayo, but within the city security has improved dramatically. The improved security of the town was repeatedly mentioned in our qualitative interviews, and was a significant factor linked by many respondents to their confidence in government. There were other stabilization, development, and humanitarian actors in Afmadow, but far fewer than in Kismayo. In Kismayo, these organizations included IOM’s SSI project (funded by USAID/OTI and the Somalia Stability Fund (SSF), which focused mostly on employment-intensive works through Enhancing Youth Employment (EYE). The SSF work also included the rehabilitation of the Afmadow airstrip in 2016, before our assessment period. Finn Church Aid works in Afmadow but has moved from its conflict resolution focus to more of a state-building and governance approach, and is currently involved in preparatory work for District Council Formation. Although TIS+ is not the only actor in Afmadow, TIS+ projects did figure prominently in our qualitative data, and our research did not find many actors who were working on the same type of small infrastructure as TIS+ with the same approach. TIS+ interventions have made a clear impression on 165 community members, as revealed in several of our qualitative interviews. It was also evident that TIS+ has leveraged the work of other programs to add value to its own contributions. For example, in Afmadow, TIS+ used a Project Oversight Committee already established by NISF for a previous project for TIS+ infrastructure projects, since this existing committee was regarded as a “respected and representative of the community.” Of all the settlements researched in this phase, Kismayo contains the largest number of other stabilization, governance, and development actors. Our qualitative data suggested there had been many development projects in Kismayo in the past year that were not TIS+ interventions, but which had affected optimism considerably. The team is aware of the following donors and programs in Kismayo: • SSI has worked in Kismayo on dialogue between returnees and host communities, and on boys’ and girls’ youth engagement through sport, among other activities. ● Adam Smith and the Somalia Stability Fund (SSF) has worked in Kismayo where they supported the Jubbaland Public Financial Management system and the Civil Service Commission, now implemented by a different implementing partner, and built the Ministry of Finance building as well as a youth center. ● SSF is currently supporting youth and women’s inclusive economic empowerment in the city. At the Jubbaland state level, SSF also supports Gender Equality and Social Inclusion (GESI) in Jubbaland, which helps women and marginalized youth increase their participation in political processes. It also supports a dialogue program between Jubbaland and Gedo and supports the State Assembly in Jubbaland State. ● The UN Joint Program on Local Governance (UNJPLG) works in Kismayo on the legal, regulatory, and operational frameworks for district administrations, after they are formed. ● The African Development Bank (AfDB) Multi-Partner Somalia Infrastructure Fund (SIF) has worked on technical assistance to infrastructure ministries in Jubbaland as well as support to a wide range of infrastructure improvements. Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) has completed a Women’s Center and plans a milk market in Kismayo. ● UNDP is known to have worked on youth projects and a market. ● The American Refugee Council has worked on IDP camp shelters. ● The Norwegian Refugee Council has worked on community centers. It is perhaps understandable that, given this range of actors, the TIS+ contribution in Kismayo was far less prominent in our interview data than in Afmadow. However, this does not mean there was no contribution. Maximizing impacts in Kismayo clearly requires coordination between multiple actors, and TIS+ is to be credited with trying to support coordination. The TIS+ Bridging Plan workshop in Kismayo brought multiple agencies together, and the resulting list of priorities included projects led by SSF, IOM, GIZ, ARC, NISF and others. Such coordination can be difficult to achieve in practice. There was significant evidence that where possible staff do try to coordinate with other programs, and there was an evident improvement from baseline interviews with field staff working on coordination issues. Interestingly, having multiple actors might affect results observed in our data in other ways. Confidence in government was already very high at baseline and remained high. However, there was some dissatisfaction with services expressed, and it is possible that community expectations have changed as communities begin to learn what services they should expect from government, and that they are therefore (rightly) more critical of delivery and maintenance. The results observed in Kismayo will also have been substantially affected by wider factors in the political economic and security context in and around the settlement. The most obvious influencing factor in Kismayo is the impact of the consolidation of power of State President Madobe. This is a factor 166 that has strong positive influences but also causes some concerns for the future. The consolidation of Madobe’s power has caused some concerns over reported crackdowns on human rights and over a perceived failure to be sufficiently inclusive of groups excluded from government, including groups within Kismayo, which has a diverse range of clans and sub-clans. The increasing antagonism of JSS in the federal system, and the upcoming Jubbaland State presidential selection and transition process in 2019, have the potential to become negative influencing factors. Positive factors include the existence of an established Jubbaland Chamber of Commerce and Industry (JCCI), which has demonstrated a willingness and capacity to collaborate, including leveraging its own resources to contribute to projects. The TIS+ approach is explicitly designed to leverage support and resources from actors such as chambers of commerce, and it has successfully done so in Kismayo. In Kismayo, this is evidenced by the fact that the management of the Freedom Square project was handed over to JCCI. Another positive factor is the increasing number of people returning home from the diaspora. Due to its relative stability, many professionals and investors have come back to Kismayo after years living in the West. Their resulting contribution to the economy, civil society, and public service was well noted in our qualitative interviews. Negative influencing factors that have affected Kismayo include the ongoing battle with Al-Shabaab around Kismayo itself. Critics observe that the government of President Madobe and international partners have moved only “five miles outside Kismayo in five years.” The conflict with Al-Shabaab was widely reported in our interviews but many respondents said such conflicts tend to happen outside Kismayo. Security was regarded by some as unpredictable, because the departure of AMISOM from towns in Lower Juba has the capacity to confound progress made there. TIS+ used two distinct approaches in this cluster, and there is clear evidence that the community consensus process does impact communities and that few other donors have a similar manual or approach. TIS+ coordination and work on project oversight groups in close cooperation with local government also has had a positive influence on youth and women and has influenced government staff. There are aspects of the TIS+ consensus process that are clearly affecting communities. There is a danger, however, that government counterparts cannot always meet expectations. This will continue to be a challenge for many years, particularly in this cluster with a long history of AS influence and well￾established administration and recruitment systems. 167 Section 4: Lower Juba Recommendations 1. Ensure that all significant infrastructure interventions have inclusive Project Oversight Committees (POCs). Oversight committees play an essential role in helping strengthen and improve the accountability and performance of local administration staff and functions. This is a key element of the TIS+ stabilization design. All significant field interventions should have POCs. These POCs must be inclusive to adequately address the objectives of TIS+. It is essential that women, youth, and minority groups find a place on these committees. 2. Infrastructure projects that allow for maximum community and government interaction should be prioritized. It is critical for TIS+ staff to weigh the disadvantage and advantage of larger infrastructure needs with smaller and more localized community-based infrastructure efforts. It may be that smaller, more localized planning and development approaches may stimulate more participation with both citizens and their government. Larger infrastructure projects may act as a platform for improving governance behaviors, such as improved planning and public accountability with government officials, but are less likely to be effective vehicles for increasing community engagement, unless these efforts have clearly defined co-management structures that are inclusive of citizens and government. 3. There is an evident need for greater participation of women in field activities. Improvements have been made since baseline in incorporating more gender-sensitive programming. However, TIS+ field staff should increase gender targeting, sensitization campaigns, and gender equality awareness activities. Separate planning sessions for women may be helpful in ensuring that projects that benefit women are clearly identified and prioritized. Staff need a clear understanding of gender equality and social inclusion (GESI) practices that can be employed in the field. 4. Coordination with local partners and other stabilization actors is essential in improving governance. TIS+ coordination between state and local entities is often not well harmonized with other donor efforts. Technical Coordinators have been helpful but there is still work to be done on field level coordination. Improved coordination between implementing partners will help avoid duplication and manage expectations of the communities. This field level coordination is essential for state and local governments that are often under-resourced. 5. Conflict mitigation is an important and consistent local need expressed at the settlement and cluster level. Although clan violence seems to have decreased in Lower Juba, clan conflicts are often cited as a problem in planning development efforts at the community level. Inter-clan reconciliation and conflict mitigation is still a pressing local and state-level need. TIS+ interventions could have a positive effect in mitigating the local micro-conflicts through the community consensus process by understanding the history of these conflicts and working with the communities to select priorities that are aimed at mitigating aspects of these conflicts. There appears to be a role for women in inter- and intra-clan reconciliation. This should be explored, researched, and targeted as a TIS+ intervention. 6. Youth are still susceptible to AS influence and they are craving a more substantive role in development planning. There is a continued need for youth programming that provides youth with direct access to community engagement and input into community planning. Youth are particularly interested in economic opportunities and in being more directly engaged in local decision-making, particularly development planning. TIS+ should coordinate with other donors working on youth engagement, employment and leadership and link interventions in community level planning to these youth and vice versa. This practice will also provide a more comprehensive platform for youth to engage in community building, development planning, and engagement in their localities and states. This may help to mitigate the concerns in this cluster of the susceptibility of youth to criminal employment and activity with AS and provide them with alternative outlets, ultimately mitigating recruitment practices of AS. 168 7. Rule of Law and Justice is directly linked to stability. While this is no new revelation, it is more pronounced in this region and TIS+ should consider avenues to link with the USG’s new Access to Justice Program to find opportunities for joint intervention. Access to justice is essential at the district level and there are clear indications that AS is filling this niche and that people, particularly those from minority groups, are craving a greater understanding of political and legal rights and venues to address these needs. TIS+ should seek to link where possible with this new USAID project and to assist local communities in accessing community based legal aid systems and services, particularly for women and minority clans. 8. TIS+ can work to influence and support policy work for youth and minorities. Donor interventions are limited in scope and scale, and working on youth and gender issues need to be finely tuned at the district and state level. Local administration youth policies at the district and state level and gender- sensitive planning practices for government officials at the state and local level were particularly noted as places for potential engagement for TIS+. 169 VIII. LOWER SHABELLE Section 1: Political, Economic, and Security Analysis The Lower Shabelle region is adjacent to Mogadishu, bordered by Banadir region to the West, Juba to the South, and Bay to the West. Known for its fertile land and abundant natural resources, the region has been one of the most heavily contested territories in the history of the Somali civil war. The situation today remains highly fluid and dynamic with armed opposition groups, including Al-Shabaab, contesting territory with Somali government and African Union forces. This volatility is felt in Mogadishu, giving credence to the statement that when Lower Shabelle sneezes, the capital catches a cold. Currently, Lower Shabelle is one of the three regions that make up the South West State along with Bay and Bakool in the federal makeup of Somalia’s new governance system. Barawe, one of the towns in the region, is the capital of the FMS and is currently witnessing an uptick in infrastructure development in anticipation of its new status. Each of the major districts including Barawe, Merca, Afgoye, Kurtunwarey, Qoryooley, Sablaale, and Wanleweyne have District Commissioners appointed by the President of the South West State. Some of these towns remain under AS control and thus, do not have the administrations physically present. It is important to note that all these administrations are considered interim until a district formation process is completed. This process began with the formation of the Hudur District Council in Bakool with other districts awaiting commencement of the process. The main stakeholder is the South West Administration, which appoints the interim local administrations. In July 2018, the South West President appointed a new governor for the region. The region is very cosmopolitan with multiple sub-clans of the Dir, Hawiye, and Digil Mirifle clans calling it home. In addition, there are minority clans including Bantus, Barawe (Brava), and Gibil’ad. This intricate makeup is one of the key factors in the region’s conflict dynamics. Each of the towns in the region is inhabited by an assortment of sub-clans, which at various points have clashed over access to land and natural resources. The conflict in Lower Shabelle has historically been due to the tension between the Asal (indigenous) and Farac (newcomers). The smaller minority clans consider themselves the indigenous clans while sub-clans such as the Habar Gidir and other Hawiye groups are considered newcomers due to their more recent migration from Mogadishu and other parts of South Central Somalia. AS utilized its trademark harsh administration of justice to address conflicts between sub-clans with heavy taxation, death sentences, and forced recruitment of all males of fighting age into the group’s ranks. When Marka was captured by troops from the Somali National Army and AMISOM in August 2012, the old rivalry between the Biyo Maal and Habar Gidir was reignited. The Somali Army’s 3rd Brigade was predominantly made up of Habar Gidir members, and this caused resentment among the Biyo Maal. A promise by Minister of Interior and National Security Abdikarim Hussein Guled (also Habar Gidir) to establish an integrated national force comprising all sub-clans in the region did not materialize. During this conflict, the well-intentioned initiatives of the international community were stymied by the intricate and complex history of the region. Additionally, sub-clans that fell out of favor with the government quickly switched sides and aligned with AS. This interest-based allegiance was a shift from the ideological-based support the group relied on during its heyday. The increased pragmatism by AS to manipulate the local fissures also frustrated the knee-jerk interventions of the government and the international community. 170 Lower Shabelle still faces high levels of insurgency and is an active location of armed conflict between Al￾Shabaab and the Somali National Army, allied militia groups, and AMISOM forces. In addition, the US military conducts targeted operations alongside Somali Special Forces Danab. In 2014, Somali and AMISOM forces conducted a joint operation aimed at freeing the last remaining AS strongholds in the region. Operation Indian Ocean managed to dislodge AS from towns that it had held for almost a decade, including Qoryooley, Sablaale, Kurtunwarey, and most importantly, Barawe. However, AS managed to blockade these towns’ main supply routes, turning them into islands with no access to vital supplies. This, combined with the absence of a strategy to govern the towns after their initial capture, has allowed AS to maintain considerable influence in the region and use it as a base to launch attacks on Mogadishu. AS operates a key base in Toro Torow village and runs a court there that adjudicates on issues ranging from land disputes to business disputes and coordinates tax collection. In 2017, attacks attributed to ISIS were reported in Afgoye near Mogadishu. The attacks, which targeted low-level government officials, further degraded the public’s confidence in the nascent administration in Lower Shabelle. The presence of ISIS and AS in the same territory is bound to cause further violence, as witnessed in Bakool, with both groups seeking to rout the other in its area of influence. The unsustainable strategy by AMISOM and the SNA to occupy and then vacate towns in the region further compounds the fragile situation. In 2017, government and AMISOM forces occupied, vacated, and re￾occupied towns including Marka, Kurtunwarey, and Sablaale. Tension in the region has been exacerbated by drone strikes as well as targeted operations by US Special Forces. In 2017, during a joint operation in the town of Bariire, locals claimed that at least 10 innocent civilians were killed by US forces. AS continues to manipulate such narratives to extract support. This is made easier by the clan fissures already present in the region. For example, this attack allowed AS to gain sympathy from the mostly Hawiye sub-clans whose clansmen were killed in the operation. The increased military activities, if leveraged properly, can yield results for the region’s stability. The drone strikes have suppressed and degraded AS’s ability to plan and organize attacks. However, this military activity needs to be followed up with swift ground operations and the establishment of local administrations to fill in the vacuum. The planned move by the South West administration to Barawe may create an impetus for more efforts to stabilize the region. Throughout the conflict, traditional elders have maintained great influence due to their role in the traditional conflict resolution process. Currently, AS engages elders more than the government. A shifting attitude by the public, which views the elders’ role in the conflict negatively, has driven the elders into a closer alliance with AS. The upcoming presidential elections in South West State, as well as the administration’s planned move to the new capital Barawe, will be a new chapter for the region. For the local communities in Barawe, a new migration by another nonindigenous community to the capital may exacerbate historical grievances. However, having the political leadership of the state closer may create a new impetus to find a permanent solution to the longstanding problems in the region. Implications for TIS+ Programming Despite the seemingly intricate and dynamic nature of issues in Lower Shabelle, three distinct issues continue to remain recurrent and will be pertinent for any future programming. 1. Fast-changing security dynamics and new emerging threats Due its proximity to Mogadishu, a primary target area for Al Shabaab, Lower Shabelle witnesses rapid changes in the security situation. In some areas, towns have changed hands multiple times as happened 171 in Awdeghle and Qoryooley in 2016. In addition, the uncertain transition plan aimed at transferring responsibility for security from AMISOM to Somali forces makes it difficult to carry out implementation even in key areas such as Barawe where in 2017, a hasty retreat by AMISOM made the strategic town immediately vulnerable to AS. As of 2017, the security situation has been made further vulnerable by the emergence of ISIS, which is contesting for control with AS. This has led to frequent clashes between ISIS and Al Shabaab as well as a three-way fight between government and AMISOM forces on one end and the two terrorist groups on the other hand. These dynamics will affect access for TIS+ staff and make it difficult to oversee and monitor ongoing activity in this area. While there are no foolproof mitigation avenues for this challenge, it points to a greater need to determine whether TIS+ should work in this area after all and an imperative to develop criteria for where TIS+ engages and implements activities in. 2. Competing for space with Al-Shabaab as a service provider Across South-Central Somalia, one of AS’s key strategies has been to draw a clear distinction between its service offering and services from the nascent state and local administrations. Lower Shabelle has significance because it hosts the AS judicial infrastructure where it adjudicates over disputes involving land, inheritance, and other civil cases. It is common knowledge that residents in Mogadishu frequently have their cases determined in these courts, effectively discrediting the legitimate government in Mogadishu. Combined with the culture of illegal checkpoints manned by a variety of security forces, AS has continued to present itself as an alternative service provider. For TIS+, this presents a challenge to support the legitimate government authority to provide services and infrastructure that the communities in this area can depend on. When this is hampered by security and access issues and unmanaged expectations, this lends further credence to the AS position. 3. Navigating a complex political economy Lower Shabelle’s strategic importance not only as a food basket, but as a region with other economic opportunities including fisheries, extractives, access to a seaport has drawn the attention of both local and international actors, creating an enclave economy that has tightly intertwined development and humanitarian and commercial activities, with often the same players in all three spaces. This is further compounded by the fact that stakeholders must interact with an effective tax system run by AS. TIS+ interventions will run into this reality, which affects the entire engagement process from prioritization and community engagement to implementation of activities. The challenges faced by the Somalia Stability Fund (SSF) in construction of the Barawe airport are an example of what could happen if all factors are not carefully assessed before intervention takes place. As above, this can only be mitigated to an extent, but it comes back to the question of what criteria inform the choice of a settlement for TIS+ to carry out activities in. Section 2: Lower Shabelle Findings The sections below will provide general frequency analysis of the Lower Juba (Afmadow and Kismayo) data and triangulation of this information with the KIIs and FGDs conducted in the settlements in which data was collected. Contribution analysis was also applied to analyze the relationship between TIS+ planning and coordination activities with other donors and contextual factors. Findings are primarily aggregated at the cluster level, but where relevant individual settlement information is provided to illustrate to USAID and TIS+ the diversity of these settlements and the anticipated impact on stabilization planning. 172 Limitations Specific to Lower Shabelle • There is variation in the findings between settlements, therefore generalizability at the cluster level may be limited. • To assess TIS+ contributions to observed results, it was important to establish what activities TIS+ has implemented in each of the settlements and how these activities have impacted the results we are observing at the cluster level. Summary of TIS+ Interventions in Lower Shabelle The following interventions were the primary activities by which TIS+ sought to engage government and community planning, and oversight lies at the heart of TIS+ interventions and is the primary programmatic approach used to achieve the four project objective goals. To assess TIS+ contributions to observed results it is important to establish what TIS+ has implemented. The following interventions were the primary tools by which TIS+ sought to engage government and stakeholder partners to ensure that community cohesion and governance improvements were measured and observed in Lower Shabelle. We have broken these interventions down by settlement to illustrate to the reader the primary activities conducted in this cluster by TIS+. Barawe: The Barawe consensus building was held in March 2016 and included participants from different communities in Barawe. Out of the total of 39 participants, 13 (33%) were female and 26 (77%) were men. Participants were drawn from a list provided by the Ministry of Interior (MOI) and the Barawe Administration, with some validation being conducted by facilitators to ensure fair representation of groups from local government, civil society organizations, private sector/traders, women, youth, and minority groups. The consensus planning identified eight prioritized activities: ● Road construction ● Construction of two Maternal and Child Health Clinics ● Rehabilitation of schools and construction of a football field ● Construction of offices for women and youth groups ● Regional government buildings ● Capacity building training sessions There were some details on the formation of a POC, and it appears that one POC was established because of the workshop. TIS+ implementation of projects in Barawe has been very limited. The program implemented an event titled “Youth Engagement for Stability” and organized the Barawe Youth Iftar. It financed an upgrading survey for Dinsoor Road and airstrip. A planned football stadium reconstruction was cancelled and an idea for Barawe roads construction is currently being discussed. Wanleweyne: The Wanleweyne consensus building in August 2016 included 30 participants, of whom eight (27%) were members of women’s groups, five were from youth groups, eight were community elders, four were businesspeople, and five were local administration representatives. The following nine projects were prioritized in the workshop: ● Road and bridges construction ● Garbage collection equipment ● Vocational training and capacity building ● Sporting activities including sports pitch constructions/reconstruction ● Support to farmers ● Offices for women and youth ● Hospital construction ● Solar lighting 173 ● Construction of a market adjacent to one already constructed by NISF TIS+ implementation of projects in Wanleweyne has been more limited. The program provided capacity building for the Wanleweyne Council of Elders, and conducted an infrastructure and design works survey. There is a road construction planned and more road construction ideas being discussed. Assessment Question 1 Did TIS+ Contribute to Any of the Following Anticipated Results? A. Community Ownership of Development Projects and Processes Key Findings 1. TIS+ implementation was very limited in this cluster and contribution was minimal. Consensus￾building sessions were held with limited follow-up with resources and community engagement activities. 2. This is an extremely volatile cluster to work in, but is critical for security in Somalia, particularly Mogadishu. TIS+ had established this area as viable to work in, but was not able to fully implement as planned. 3. Expectations were raised by TIS+ and this was evident in qualitative data and field interviews. This was not fully managed, and has impacted a feeling at the community level of being neglected. Evidence To examine changes in community ownership of processes and development projects, the assessment focused on community consensus building, consultation /participation at decision making, and overall management of shared assets. Between 2017 and 2018, awareness of community planning events increased significantly by an average of 40% (Wanleweyne: +39% (p-value < 0.0001) and Barawe: +41% (p-value < 0.0001) – see Figure 9a. Figure 9 a: Do you know whether local projects are selected through community planning events? • Notably, at the cluster level, the following people had increased access to planning events – adult women: +3% (p-value = 0.354), adult men: +8% (p-value = 0.017), female youth: +1% (p-value = 0.783), male youth: +2%(p-value = 0.395), religious leaders: +3% (p-value = 0.410), state officials: +17% (p-value < 0.0001), federal government officials: +22% (p-value < 0.0001), and NGOs/CBOs: +21%(p-value <0.0001). 174 • Significant access, as the p-values indicate, was dominated by government officials (state and Federal), NGO/CBOs and adult men. However, access by youth and adult women remained limited (nonsignificant). Clan elders had reduced access to these events (-9%, p-value = 0.057). • The survey respondents indicating “most” to “all” of the selected projects effectively addressing community priorities increased non – significantly by 7% (p-value = 0.312) in Wanleweyne but decreased significantly by 24% (p-value = 0.039) in Barawe (see Figure 9 b). • Based on qualitative and quantitative evidence, this may indicate expectations that were raised after planning events though not followed by intervention, particularly in Barawe. • Awareness of oversight committees established to manage services/assets significantly increased by 38%, p-value < 0.0001 (Wanleweyne: +37% and Barawe: +40%). Apart from the role of oversight committees, increased or decreased role in the management of services/assets was examined to establish ownership and community participation in maintenance. • The local structures (clan elders and Ugaas/Sultan) had a significantly reduced role in management of all services/assets considered with changes ranging from -8% to -24% (all p-values <0.014). • In both Wanleweyne and Barawe, the NGOs, business community, and international organizations had no change or reduced roles, significant and non–significant, in managing most of the services/assets. • It is also important to note that youth had a significantly reduced role in managing the sport arenas. This is likely due to increases in roles of the private sector in managing these assets in this cluster. • TIS+ had very limited implementation in this cluster. As two female participants from the Barawe FGD mentioned, “Trainings were provided to the youths living here by TIS+. The trainings were especially focusing on how to help this community. TIS+ is willing to start interventions may be in this year because they have done assessments on the roads, football stadium and the women center” (Youth Girl1_ B_FGD). “Barawe is among the cities that have been ignored by the service providers especially the NGOs. Very few NGOs are working here and the reason behind this is the lack of an airport” (Youth Girl 2_B_FGD). Figure 9b: How many of the community planning meetings have been effective in selecting local projects/activities that address community priorities? 175 B. Effective Shared Asset Management Key Findings 1. There was little implementation by TIS+ in this settlement and this means that POCs, if established at all, had little efficacy because there were no assets to manage. 2. This cluster had several donors who started to implement and plan with communities and then had to delay or interrupt their activities due to insecurity. Evidence • In Wanleweyne, oversight committees had significantly increased effectiveness in the management of: meetinghouses (+30%, p-value = 0.049) and market facilities (+3%, p-value = 0.024). Other moderate increases, though not statistically significant, were observed for: schools (+16%, p-value = 0.897), health centers (+14%, p-value = 0.721), sports arenas (+19%, p-value = 0.508), and roads (+22%, p-value = 0.359). • In Barawe, the oversight committees had significantly reduced effectiveness in the management of the following assets: schools (-26%, p-value <0.0001), meetinghouses (-28%, p-value = 0.003), market facilities (-33%, p-value = 0.006), sports (-59%, p-value < 0.0001), and roads (-11%, p-value = 0.007). • In Wanleweyne, maintenance quality for the following assets improved significantly (schools: +6% (p-value = 0.035) and meetinghouses: +42% (p-value = 0.011). • Other noted changes included – sports arena: +18% (p-value = 0.062), health centers: +4% (p-value = 0.117), and roads: +5% (p-value = 0.131) – though not statistically significant. • However, maintenance quality for most of the services in Barawe reduced significantly (schools: - 25% (p-value < 0.0001), meetinghouses: -30% (p-value <0.0001), market facilities: -40% (p-value <0.0001), sports arena: -72% (p-value < 0.0001), and roads: -13% (p-value = 0.035). Table 8: Who else is responsible for the management of the following services/assets? (Changes between 2017 and 2018.) Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Clan Elders -4% -3% 6% 6% 8% 8% -24% -21% -17% -18% -14% -11% Militia groups 4% 5% 5% 5% 4% 5% 1% 0% 0% 0% 0% -1% Government 47% 43% 47% 43% 34% 6% 15% -2% -6% 19% 6% -13% Religious leaders -37% 7% 1% 8% 6% 3% -37% 0% 0% 2% 0% 1% Local administration 11% 23% 18% -5% -6% 3% 56% 56% 48% 27% -17% 55% Ugaas/Sultan 2% 0% 4% 1% 1% 4% -17% -12% -8% -14% -12% -9% Diaspora 5% 6% 5% 5% 9% 7% 1% 26% 0% 1% 1% 0% Businesses/Traders -6% -58% 0% -34% 14% 0% 1% -22% 0% -20% 1% 0% Local community 0% 0% 0% 2% 2% 0% 2% 0% 0% 0% 2% 1% NGOs 0% 2% 0% 0% 0% 0% 0% 1% 0% 0% 0% 0% International organisations 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% Youth 0% 0% 0% 0% -22% 0% 0% 0% 0% 0% -20% 0% Wanleweyne Barawe 176 • There is evidence from qualitative and quantitative data that this negative trend can be attributed to negative bias associated with unmet expectations resulting from initial community planning processes in Barawe, which had little follow-up post-event. • It is also important to note that ongoing investments such as the construction of the airstrip have stopped due to security concerns, further lending credence to this trend. • Al Shabaab is quite active in this cluster and creates a significant amount of fear with community members as well as NGOs. As stated by one male elder from a FGD in Barawe, “I don’t think many activities have been done for us here because the NGOs fear Al-Shabaab but I would like to tell them to increase the services they offer us.” • Government officials are aware of the pressure and the limitations. As stated by one official working in Barawe: “Public servants should at all times strive to properly manage public resources and assets. The government needs to assign specific managers, with specific responsibilities to properly safeguard public assets. In addition, government administrators require training and support to build their skills and capacity to better manage public assets more than they do now.” C. Improved Delivery of Services Key Findings 1. Local administration is under increased pressure to provide more services and the private sector supports critical gaps in this cluster. 2. Service delivery at the state and district level has increased significantly from baseline. These perceptions may be influenced by the lack of NGO and donor presence in this cluster. 3. Al-Shabaab makes it difficult to access programs that enhance service delivery. Local government officials as well as local leaders are greatly impacted by insecurity and threats from AS. Evidence: Relating to Service Provision • In Wanleweyne and Barawe, the local people and local administration significantly increased the provision of almost all the services – including health, education, water, roads, security and financial services (see Table 9). The increases ranged from +11% to 72% with all p-values < 0.02. • The local and international NGOs had more increased role in service provision in Barawe than in Wanleweyne. • Wanleweyne has had little engagement with development actors despite its strategic role in the stabilization of Lower Shabelle, unlike Barawe, which is witnessing increased attention due to its anticipated new status as the capital of South West State. • In both settlements, provision of almost all the services by the local structures (mainly clan elders and Ugaas/Sultan) was reduced. • Member state government also had a significantly increased role in the provision of health (+31%, p￾value <0.0001), education (+24%, p-value <0.0001), water (+14%, p-value <0.0001), electricity (17%, p￾value = 0.015), sanitation (+8%, p-value = 0.029), and financial services (+10%, p-value = 0.001) in Wanleweyne. • While the state government significantly improved the provision of only water (+11%, p-value = 0.0004) and financial services (+50%, p-value < 0.0001) in Barawe. 177 Table 9: Service provision (changes between 2017 and 2018). Evidence: Relating to Satisfaction • In Wanleweyne, respondents had significantly increased satisfaction with the following services – roads: +11% (p-value < 0.0001), sanitation: +7% (p-value = 0.002), electricity: +5% (p-value < 0.0001), security: +17% (p-value < 0.0001), financial services: +7% (p-value < 0.0001). • However, respondents were significantly dissatisfied with health services: -14% (p-value < 0.0001), education: -16% (p-value < 0.0001), and drinking water: - 6% (p-value < 0.0001). • In Barawe, respondents were increasingly satisfied with all services, with changes being at least +11%. • The significant changes were observed in satisfaction with health, drinking water, water for other purposes, roads, security and financial services with corresponding p-values being less than 0.0006. Evidence: General Security as a “Service” • Wanleweyne (+0%, p-value = 0.175) and Barawe (+3%, 0.078) – see Figure 10. • The household assessment also specifically examined the security situation between settlements; in which calmness increased significantly by 57% (p-value < 0.0001) and 70% (p-value < 0.0001) in Wanleweyne and Barawe respectively. Health Education Water Roads Electricity Sanitation Security Financial services Health Education Water Roads Electricity Sanitation Security Financial services Clan Elders -4% -5% -3% -1% 0% -1% -12% 1% -16% -16% -9% -8% -3% -11% -11% -13% State Government 31% 24% 14% -4% 7% 8% -12% 10% 5% -18% 11% -8% 2% 0% -3% 50% Local people 32% 30% 16% 27% 9% -8% 21% 21% 61% 52% 27% 68% 16% -23% 68% 26% Local NGOs 17% 11% -1% 11% -3% 6% 9% 5% 54% 21% 16% 25% 10% 10% 17% 11% International NGOs 18% 8% -5% 10% -2% 2% 9% 1% 44% 17% 12% 30% 10% 10% 22% 15% Religious leaders 3% 4% 5% 3% 5% 3% 3% 5% -1% 6% 0% 1% 1% 1% 1% 1% Local administration 21% 19% 11% 18% 6% 24% 34% 15% 40% 26% 13% 72% 17% -1% 47% 26% Police 4% 4% 4% 16% 4% 3% 15% 2% -3% -1% -3% 5% 0% -3% 5% -4% Local Courts 2% 1% -1% 1% 2% 0% 5% 1% 0% 1% 0% 2% 1% 1% 2% 0% Business community -46% -38% -25% -8% -15% 9% 1% -14% -19% 20% 3% 3% 0% 9% -1% -51% Ugaas/Sultan -3% -3% -3% -8% 0% -2% -12% 1% 0% 0% 0% 1% 0% 1% 1% 0% Diaspora 0% 1% 0% 1% -2% -1% 1% 3% 4% 0% 0% 1% 0% 0% 0% 0% Foreign Governments 1% 0% 0% 1% 0% 0% 5% 0% 1% 0% 0% 4% 0% 0% 42% 0% Women support groups 0% 0% 0% 0% 0% 3% 1% 0% 0% 0% 1% 0% 1% 4% -1% 0% Wanleweyne Barawe 178 Figure 10: Do you feel that security has improved in your settlement over the past year? D. Reduced Conflict over Resources Key Findings 1. There is very little evidence that TIS+ contributed to reduced conflicts over resources. Improvements in this cluster were significant and are largely a reflection of increases in reliance on local systems for conflict resolution. 2. Despite positive increases in reductions of conflict, localized conflict was cited in FGDs and KIIs. Conflicts were primarily land related or based on inter-clan conflict over access to resources such as water. 3. There were significant changes in the identification of local resources for conflict mitigation, and access to justice for contentious issues seem to still be dominated by AS systems. Evidence • Respondents in both settlements believed conflicts over resources had reduced significantly between 2017 and 2018 (Figure 11). Figure 11: Compared to last year, have conflicts over shared resources increased or decreased among various groups? 179 • In Wanleweyne, respondents had increased trust in conflicts resolved only by clan elders (+7%, p-value = 0.199). And even though resource related conflicts reduced in this settlement, there was a significant reduction in trust in conflicts resolved by the following: national army (-28%, p-value < 0.0001), militia groups (-55%, p-value < 0.0001), oversight committees (-24%, p-value < 0.0001), state government (- 13%, p-value = 0.014), police (-36%, p-value < 0.0001), local courts (-19%, p-value < 0.0001), and the diaspora (-24%, p-value < 0.0001). • Wanleweyne is diverse in terms of clans and sub-clans in the settlement but the level of cohesion amongst these groups, which is enhanced by enforced peace settlements and inter-clan pacts means that the residents are more likely to resort to this model, which is already effectively working. • In Barawe, respondents had significantly increased trust in conflicts resolved by the following: clan elders: +28% (p-value < 0.0001), national army: +9% (p-value = 0.087), peace committees: +28% (p-value < 0.0001), oversight committees: +37% (p-value < 0.0001), state government: +15% (p-value = 0.011), religious leaders: +31% (p-value < 0.0001), local administration: +39% (p-value < 0.0001), local courts: +29% (p-value < 0.0001), mediation: +26% (p-value < 0.0001), and Ugaas/Sultan: +23% (p-value < 0.0001). There was reduced trust in conflicts resolved by: militia groups (-18%, p-value < 0.0001). E. Greater Citizen Confidence in Government Key Findings 1. Responses were quite mixed in the HH survey and in interviews. Citizens often felt disappointed in the lack of government services and stated that although they relied on these services, they were not often equally distributed. 2. Citizens felt that the federal, state, and local government should provide more services, and there was significant criticism in interviews. 3. Noted improvements that seem to be linked to citizen increases in confidence are improvements in security and improved communication and information exchange between citizens and government. 4. There was noted mistrust in government forces in this cluster, as clearly stated both in FGDs and in KIIs. Evidence • Confidence in federal, state, and local administration concerning the ability to work for the benefit of local development and community needs increased significantly by 6% (p-value = 0.020), 20% (p-value < 0.0001) and 11% (p-value = 0.003) respectively in Wanleweyne. • While in Barawe, the confidence in federal, state, and local administration reduced by, 24% (p-value < 0.0001), 6% (p-value = 0.281) and 11% (p-value = 0.015). • This lack of confidence may be attributed to recent political events where a district administration that lacked the support of the various groups in the settlement was appointed without consensus from stakeholders. • Access to justice was often cited as a primary reason why there is lack of trust in government. As stated in an interview with a business leader from Wanleweyne, “The factors that led people to having little trust in the government is because the people working in the justice sector are not trained in that area. These staffs do not receive enough salaries and they are open to take money from the people and bribery. So, lack of justice is the primary reason why the level of confidence has decreased or has remained low.” • As stated by one government official, “People are not pleased with the current Government because it has failed to meet their expectations. The community has not had any support from the current President unlike the support from former presidents. However, the current President is the worst among all the other presidents in terms of achievements and supporting the community.” 180 • Further compounding the situation is the fact that despite the highly anticipated elevation of Barawe’s status to a regional capital, it remains marooned because AS operates along the main supply route from Mogadishu and there is limited access by air to this area. • As a government official from Barawe shared, “If I talk about the level of confidence in the government, I can say it is 50% because the services provided by the government are not yet complete. In terms of security, the government forces in Barawe have not received their salaries for months and AMISOM in the area do not contribute to peace building they remain in their camps. I think all these matters influence the confidence of the government.” Other aspects of governance examined included public trust (local administration prioritizing community needs and the trustworthiness of the local officials to utilize funds for development), sharing of information by the local administration, corruption levels, and representation. The findings for these aspects varied between Wanleweyne and Barawe. • In Wanleweyne, the percentage of respondents believed: (i) the local administration prioritized community needs, (ii) the local/district officials were trusted in utilizing funds for development, (iii) local administration shared information, (iv) government encouraged people to participate in planning – significantly reduced by 18% (p-value < 0.0001), 35% (p-value < 0.0001), 29% (p-value < 0.0001), and 23% (p-value < 0.0001) respectively. • While in Barawe, the percentage of respondents believed: (i) the local administration prioritized community needs significantly increased by 26% (p-value < 0.0001); (ii) the local/district officials were trusted in utilizing funds for development significantly increased by 8% (p-value = 0.014); (iii) local administration shared information increased by 9% (p-value = 0.051); and (iv) government encouraged people to participate in planning significantly increased by 25% (p-value < 0.0001). • At the cluster level, corruption at the local and Federal government levels reduced by a nonsignificant 9%, p-value = 0.122 (Wanleweyne: -11%, p-value = 0.097 and Barawe: -6%, p-value = 0.654) and a significant 22%, p-value < 0.0001 (Wanleweyne: -23%, p-value < 0.0001 and Barawe: -19%, p-value = 0.016) respectively. • There is still significant mistrust in the government. As stated by a female youth from the Wanleweyne FGD, “The confidence in the government is very low. For example, you may see a mother who has brought milk from the village being forced to pay money or tax by the government forces. If she does not pay, they pour her milk on the ground, and so many other things that the community was not expecting. Similarly, recently there was a hospital which was supposed to be built in the district by the people living abroad, but the government authorities refused to implement it. In addition, the government forces went ahead to beat and torture those who oversaw the program.” F. Reduced Support for Violent Extremism Key Findings 1. Security is extremely volatile in this cluster and there was noted influence of AS. Noted in KIIs and FGDs, clan elders, business members, and local NGO staff are often targeted by AS. 2. There are key services still provided by AS that are linked to security and community relations, such as justice. Access to justice is a critical area that AS seems to fill, particularly involving small clan conflicts over land and resources. 3. Forced AS recruitment was evident at midline, as it was in some interviews at baseline. Evidence In terms of reducing factors and conditions relating to violent extremism, we collected the following data. These measures are difficult to attribute to TIS+, but are important to consider for programming. 181 Respondents were asked if young people were prone to violent extremism, and changes in perceptions differed between Wanleweyne and Barawe. • In Wanleweyne, there was a nonsignificant increase by 6% (p-value = 0.970), while there was a significant reduction by 2% (p-value = 0.016) in Barawe (Figure 12). • Between 2017 and 2018, about 37% and 66% more respondents in Wanleweyne and Barawe indicated violent extremist groups were not welcomed in their settlements. Figure 12: Do you think that young people in your settlement are highly prone to violent extremist groups? Assessment Question 2 Were TIS+ Approaches Used in the Cluster, e.g. the Community Consensus Process Itself, Instrumental in Contributing to the Achievement of TIS+ Results Described in Question 1? Key Findings 1. In both settlements under discussion, TIS+ was unable to implement community consensus planning and oversight to the degree hoped for because no TIS+ infrastructure projects have yet begun. 2. Respondents interviewed who had taken part in initial consensus planning still expected TIS+ to return with implementation support for prioritized development projects. 3. A key finding that has emerged from Barawe and Wanleweyne is that the extent to which the entire process of community selection of projects to delivery of some of those projects is fully implemented in a reasonable time frame is very important. This is because of the vital role community expectations play in determining satisfaction with, and perceptions of, government and services. Evidence • Both settlements saw an increase in the proportion of respondents reporting that projects in their communities were selected by the community. But there was less certainty over whether the resulting projects met community needs. • Several qualitative interviewees reported they had been involved in decision-making on projects for community development or that they knew people who had been involved. There was some criticism of certain actors for failing to take consultation with the community seriously: “I take part 182 in the consultations but I don’t think the NGOs consider our points. They simply ask about what could be done but they later use their own formula.” • But even those criticisms suggest that community involvement in planning is valued by communities. Respondents in both settlements reported that youth and women had been granted access to consultations, and appeared to value this. • Respondents also highlighted the important role of district administrations in ensuring that communities were consulted and that projects met community needs. Almost all our interviews with donors, peer programs, and Somalia experts felt that the TIS+ model of engaging communities in planning and oversight was an important and appropriate programming approach in Somalia. The fact that TIS+ brings government and communities together in this process was particularly important, given that the gap between government and communities was very large in many parts of Somalia. TIS is seen by many as one of main large scale actors promoting this approach. • There was an increase in awareness of Project Oversight Committees in both settlements, suggesting that such committees, or similar structures, are being used by other actors. • Respondents reinforced some of the theoretical ways in which community planning and oversight are meant to contribute to reducing conflict in resources. In Wanleweyne, for example, many focus group participants reported that they thought more interaction between community members contributed to peacebuilding and that the organization of community members in various committees did help them to work well together. • A key finding that has emerged from Barawe and Wanleweyne is that the extent to which the entire process of community selection of projects to delivery of some of those projects is fully implemented in a reasonable time frame is very important. This is because of the vital role community expectations play in determining satisfaction with, and perceptions of, government and services. • Community expectations could multiply the effects of interventions, but involved the risk of disappointment and unintended outcomes. • As a technical coordinator in a state-level ministry reported: “I think that [Wanleweyne and Barawe] is where the effect is most because the baseline is very low there. There is basically no infrastructure in these towns and there has been no presence of government. Therefore, when you conduct a community consensus-planning session, it is the first opportunity that the people in these towns have had to interact with the government.” Assessment Question 3 Did Government at all Appropriate Levels Contribute to Asset Management in Communities in which TIS+ Was Implemented? How? Did Government at All Appropriate Levels Have Both the Capacity and the Will to Contribute? Key Findings 1. There were no asset management activities done in this cluster but there was close cooperation with local and state government during initial consensus-planning sessions. 2. This is a highly volatile area and government officials and their work at the state and local level is impacted by AS and security. Evidence There was little implementation by TIS+ in this cluster. Initial community consensus planning was collaborative and included close planning and coordination with local and state government, but minimal follow-up after these events has led to some raised expectations. This is a highly volatile area and in multiple interviews with government officials AS influence and security impacts their ability to work at 183 the local level. There were no asset management activities done in this cluster, but findings from interviews and the HH survey note some overall challenges relevant for USG programming. • In interviews a variety of perspectives illustrated the importance of this cluster to stability in the region. They also highlighted the importance of expectations that have been formed, influencing the relationship between citizens, the government and the management of shared assets. A youth from a FGD expressed, “We all know that Barawe is the official presidential palace of the South West government but our town doesn’t have a good functional airport like the one that Baidoa people have.” • Government officials also expressed their own challenges in responding to the needs of citizens. These quotes are from government officials working in Barawe. One official from Barawe expressed his frustration with capacity issues: “Public servants should at all times strive to properly manage public resources and assets. The government needs to assign specific managers, with specific responsibilities to properly safeguard public assets. In addition, government administrators require training and support to build their skills and capacity to better manage public assets more than they do now.” • Another official stated, “There should be collaboration among the federal government, the South West State, the local administration and the organizations who specialize in creating awareness towards the care needed for the public assets or services. There have been misconceptions about public assets or services. Some believe the public assets or services belong to the government only, while other people believed they are owned by particular clans. To remove such misconceptions, more awareness and a strong collaboration among all relevant stakeholders is needed". Assessment Question 4 Were TIS+ Activities Inclusive of Women, Youth, and Minority Clans, as Applicable to the Context of the Cluster Being Evaluated? Key Findings 1. There was minimal work done in this cluster so cluster level inclusion findings could not be attributed to TIS+ efforts. 2. However, during initial community consensus events held, collaboration and inclusion was well addressed by TIS+ staff and partners working on these efforts. 3. Participants interviewed who worked on initial consensus planning efforts both at the community and with the local administration stated strong initial collaboration. However, weak follow-up and communication had impacted participants interviewed. Evidence During initial consensus planning in Barawe, participants were drawn from a list provided by the Ministry of Interior and Barawe Administration, with validation being conducted by facilitators to ensure fair representation of groups from local government, civil society organizations, the private sector, traders, women, youth, and minority groups. At the cluster level, qualitative and quantitative data showed that youth and women had more limited access to public consultations. Threats from AS seem to prevent participation in this cluster by women, youth, and minority clans. • Several female respondents said they had participated in community planning events. But participation does not always mean that their perspectives are influential. FGD respondents in Wanleweyne reported that community members had been involved in projects in various ways. “Religious leaders, women groups, youth groups, mothers, school teachers, students, and district administration officials were all engaged in these projects.” • Although initial consensus planning was inclusive of government and the community, there was little follow-up on prioritization and support for implementation in Barawe and Wanleweyne. 184 • There was very limited support for post consensus planning. Expectations may have been raised with women’s, youth, and minority groups during consensus planning, and qualitative interviews indicated discontent with post-planning. • The community prioritized projects in collaboration with government, but resources did not follow. TIS+ was not able to plan interventions in this cluster post-consensus building, and this did raise some expectations both with citizens and government officials interviewed. • There was a nonsignificant reduction by 9% (p-value = 0.150) in Wanleweyne for those who believed women were considered equal to men. However, in Barawe, there was a significant increase by 6% (p-value = 0.028). • As discussed above, access to planning and contracting events now include more stakeholders (adult women, adult men, female youth, male, religious leaders, state officials, federal government officials, and NGOs/CBOs) rather than the dominant local leaders. • FGD respondents in Barawe reported increased access to community planning and contracting events for men women and youth, but these findings could not be attributed to TIS+ activities. • FGD respondents from Wanleweyne said that community members, including youth and women, had been engaged in the projects implemented, having attended public consultations and other meetings. Some respondents indicated that some community members were more interested in engagement than others, and that this is likely based on security issues. • One male elder respondent also noted the threat from AS: “Al-Shabaab threatens those who work with the NGOs and the government and for this reason the people are afraid.” • Several respondents said they had participated in decision-making processes, but several female respondents noted that they did not feel that their input was utilized for planning purposes and were still waiting for interventions that they hoped would come soon. • Youth and women who took part in the initial consensus building in Barawe stated they were still waiting for interventions from TIS+. Section 3: Lower Shabelle Conclusions The next stage in our analysis is to ask what other influencing factors could account for the changes observed relevant to the key assessment questions. These include other stabilization and development programs as well as positive and negative factors in the context around the intervention that might have important roles in determining outcomes. Because community engagement in the process has emerged as such a valuable part of the model, it is also important to ensure that delays or instances of non-delivery do not raise community expectations only to disappoint them. Barawe and Wanleweyne findings suggest this risk may be even greater in communities that have been relatively poorly served. This is particularly true where there are fewer stabilization and development actors, or where these actors struggle to implement projects because of security or other factors. Key Conclusions 1. TIS+ implementation was very limited in this cluster. Consensus-building sessions were held with limited follow-up with resources and community engagement activities. Expectations were raised by TIS+, and this was evident in qualitative data and field interviews. This was not fully managed, and has impacted a feeling at the community level of being neglected. 2. This cluster had several donors which started to implement and plan with communities and then had to delay or interrupt their activities due to insecurity. 3. Expectations were particularly high in Barawe, which has been designated as the new state capital. Citizens are beginning to feel neglected, and that insecurity is preventing forward progress in this cluster. 185 4. There is a very high need for collaboration regarding asset management, but very few partners able to operate in this cluster safely. 5. Despite challenges, there have been improvements in Lower Shabelle in both good governance and community cohesion elements from baseline to midline measures. These changes reflect a complex arrangement of donor efforts, national political changes, and security improvements and challenges that impact both community access and citizen perceptions of federal, state, and local capacity (see Table 10). Table 10: Index Scores for Lower Shabelle (Summary) Good Governance The themes contributing to good governance index included confidence in government, service delivery, and stakeholders’ partnerships. Variables under each of these have been triangulated with qualitative data to establish cluster level findings. • The good governance index significantly increased by 15% and 14% in Wanleweyne and Barawe, respectively. • Although there are positive trends evident from baseline showing improvements, there is significant evidence of unmet community expectations and increased dissatisfaction with the government at all levels. • Lower Shabelle has seen improvements in security, but this is still a highly volatile region and expectations of the FGS, state, and local authorities are very high. • Qualitative data triangulated with the HH survey gives a much more nuanced view of these numbers. Community Cohesion Index ● Findings on institutional trust to resolve resource-related conflicts have been presented above with variations noted. ● At an interpersonal level, the following changes have been perceived: o Increase by 2% (p – value = 0.463) in Wanleweyne and no changes in Barawe in people working collectively toward problem solving for the benefit of the settlement. o Significant decrease in people who do not trust one another across clans and within settlement areas (Wanleweyne: -23%, p-value = 0.0001 and Barawe: -30%, p-value < 0.0001). o Decrease in people who would cooperate on development initiatives and efforts (Wanleweyne: -18%, p-value = 0.734 and Barawe: -49%, p-value < 0.0001). ● The general involvement (inclusion) of people in decision-making has significantly increased in Wanleweyne (+48%, p-value < 0.0001) and Barawe (+60%, p-value < 0.0001). Network Diversity ● There was an increase in people believing that clans only cooperated only when necessary in Wanleweyne (19%, p-value < 0.0001) and Barawe (10%, p-value = 0.966). ● The community cohesion index significantly increased for Wanleweyne (+4%) and Barawe (+17%). Good Governance Index Community Cohesion Index 2017 2018 Change 2017 2018 Change Lower Shabelle Wanleweyne 37% 52% +15% 53% 57% +4%* Barawe 47% 61% +14%* 50% 67% +17%* Total (cluster level) 41% 55% +14%* 52% 61% +9%* 186 What else could be accounting for these changes? The next stage in our analysis is to ask what other influencing factors could be accounting for the changes observed. These include other stabilization and development programs as well as positive and negative factors in the context around the intervention that might have important roles in determining outcomes. In Lower Shabelle the two settlements were quite distinct and the findings contributing to the outcomes noted were specific to the settlements. As noted at baseline, there are several stabilization, development, and humanitarian actors in Lower Shabelle, but their reach is limited by insecurity. There was a broad consensus among donor, peer program, and expert interviewees that Lower Shabelle remained intensely volatile. Some respondents felt there had been little or no resolution of the grievances and conflicts that were exploited by AS. It is important to note that the impending move of the capital of South West State from Baidoa to Barawe is already causing some tension and apprehension, and has the potential to cause increased conflict over resources, for example land. The results observed in Barawe have been substantially affected by wider factors in the political economic and security context in and around the settlement. Many of these are negative influences. Lower Shabelle remains one of the areas worst affected by the 2016–2018 drought. Large numbers of rural people have been displaced by drought and conflict, relocating to Barawe, the Afgoye-Mogadishu corridor, and to Baidoa.15 Barawe is surrounded by AS, which also retains significant influence within the town. Some respondents in our qualitative research said that community members were afraid of AS: “We have good interactions although sometimes we fear the anti-government regime called Al-Shabaab because they don’t want to see the interactions between the community and the government.” There have also been positive influencing factors in Barawe over the past year. In this complex and fluid context, political, economic, and social developments have the potential to exert both negative and positive influences. For example, respondents who noted the challenge of the drought also noted that the humanitarian response had brought resources into the settlement: “The major change we have experienced is the drought that has affected the region, thus we have received many drought-displaced people from the rural areas, some people have moved to Mogadishu. FAO has responded massively to the droughts with its food for work program and support program for farmers.” The main development that will bring more resources to Barawe is the anticipated move of the capital of SWS to the town, a situation which has also increased tensions. One of our interviewees suggested that the additional resources brought because of the move of the capital to Barawe might have had a disproportionate effect on confidence because the resources were building on a low base and because they brought the expectation of more resources to come. There were different interpretations of the security context in and around Barawe. Some respondents in our qualitative research reported a reduction of AS attacks inside Barawe over the past two years. Some interviewees were also optimistic about the current mediation being undertaken in Mogadishu between clans involved in conflict in neighboring Marka, and suggested that this had even reduced incidents of violent extremism. Other respondents were less optimistic about this process. The household survey revealed a mixed but largely positive picture of security inside the settlement and an increase in those reporting that the security situation between communities was calm. • The Barawe consensus-building report shows participants were aware of other actors in the settlement, including NISF and other USG partners, such as IOM and USAID/OTI’s SSI activity, also 15 UN OCHA. (2018). State-by-State Drought Analysis February 2018. 187 implemented by IOM. These actors have conducted community dialogue in Barawe, but not over the past year. • Qualitative research showed community members were aware of several projects delivered to the settlement over the past year, and respondents named several projects and organizations. These included the construction of the presidential palace by the multi-donor Somalia Stability Fund (SSF), ahead of the move of the capital of SWS to Barawe; as well as nutrition programs and solar lighting. • The SSF is present in Barawe, where it has been working during the past year on strengthening the fishery sector and, importantly, on the construction of Barawe airstrip. • Despite the increased attention from stabilization and other programs, there was a sense that not enough organizations had been providing services to meet needs. Some respondents said that Barawe had been relatively neglected in terms of projects and services: “Barawe is among the cities that have been ignored by the service providers, especially the NGOs. Very few NGOs are working here and the reason behind this is the lack of an airport. There were interventions implemented last year and some of them are still ongoing. However, these interventions are unimportant to this community.” Clearly, there are great hopes vested in SSF’s work on airstrip construction, which has been temporarily paused after being hit by an IED attack, but is due to resume. The airstrip was viewed as particularly significant by several respondents since it has the potential to open Barawe to more development and business activity. In sum, especially given that TIS+ has not implemented as many projects as other actors, other actors are likely contributing more to the results observed above than TIS+. Expectations that the airport will be completed (by SSF) may be having a positive effect on community hopes for more resources and services, especially since road access to Barawe from key areas such as Mogadishu remains constrained. Although there are major tensions and fears for the future in Barawe, the very trends that have caused harm and anxiety, such as the town’s impending status as capital of South West State, have also brought more resources into the settlement. These factors have likely exerted a strong influence on the positive and hopeful results observed on services and the capacity of local administration to meet community needs. The ability of stabilization, development, and humanitarian actors to operate in Wanleweyne is limited by insecurity. IOM has undertaken “soft activities” in Wanleweyne, including women’s dialogue, football, capacity support to the district administration, and conflict prevention dialogue. SSI is currently discussing collaboration with TIS+ on stabilizing the area between Wanleweyne and Lego. The SSF is not currently active in Wanleweyne, but plans to work there as a stabilization focus district. The main actor, whose interventions figured in our qualitative interviews, is the NISF which has worked on solar streetlights, market refurbishment, and the rehabilitation of the District Community Center. Respondents in our qualitative interviews were aware of interventions in the community, especially a cholera treatment program. A child feeding program, school and hospital rehabilitation, a Maternal and Child Health Centre constructed close to Wanleweyne, solar lights, wells, rehabilitation of government offices, and the construction of a slaughterhouse were also mentioned. Respondents said that community groups and local NGOs were active in the settlement and reports suggest that local NGOs have been active in scaling up humanitarian response in the settlement. All in all, our sources suggested that needs are great in Wanleweyne, the ability of NGOs and stabilization actors to meet these needs is limited by insecurity, and respondents wanted to encourage more actors to come into the community. This likely contributes to the mixed results noted in this cluster. • As in Barawe, results observed in Wanleweyne will also have been substantially affected by wider factors in the political economic and security context in and around the settlement. 188 • As with Barawe, Wanleweyne has been affected by the 2016–2018 drought. As one respondent mentioned, “We have been affected by drought severely over the last year and I think that has been a major change that has happened to the community where many people living in this community lost livestock and crops.” • The food security situation in Wanleweyne was described as critical in the summer of 2017. Flooding of the Lower Shabelle River also led to an outbreak of cholera in December 2017, which was widely mentioned in our qualitative research. • Wanleweyne is surrounded by AS and continues to suffer from attacks inside the town. When AS seized the nearby settlement of Lego in August 2017, cutting off the Mogadishu–Baidoa supply route, local authorities reported that more than 2,000 people were displaced into Wanleweyne. • Respondents in focus group discussions reported widespread fear for their safety and that of their families, especially because of AS. Other sources of insecurity such as “bandits and roadblocks” were also mentioned, as well as insecurity caused by government forces. As one respondent stated, “Currently there are government forces in the district which are tribal based. They settle in two edges of the city and sometimes open fire where there are many people.” Fewer countervailing positive factors emerged from our data compared to Barawe. However, several respondents in qualitative interviews reported tentative improvements, many of them stemming from the projects and initiatives that have been undertaken in Wanleweyne. “Now we have educational competitions held for students, which has created community interactions, and sports tournaments for the youth, which is also aimed at peacebuilding and reconciliation. These are the positive changes we have now.” Some respondents did report improved security within the settlement. The household survey also showed perceived improvements in security inside the settlement and an increase in those reporting reduced conflict between communities. In conclusion, Wanleweyne faces several challenges that are outside the control of an intervention like TIS+ to influence and that are likely influencing the patchier and less positive results observed above. However, in this relatively underserved community, interventions can have a large effect in raising community hopes and perceptions of progress. Section 4: Lower Shabelle Recommendations 1. Consensus-planning sessions should not be held unless interventions of a significant scope and scale are to follow. Expectations are raised through the prioritization process. In areas, such as Lower Shabelle, there is evidence that these sessions did raise community expectations. It is essential that community planning sessions be tempered where interventions are not guaranteed. 2. Develop a process to close the loop on areas in which early planning does not lead to intervention. It is essential that, in areas where consensus planning was started, there be a way to close the loop with communities and local government stakeholders. Conflict- sensitive program planning methods should be engaged to ensure that, when program priorities shift, there is a systemized approach to reaching out to stakeholders and partners engaged in initial planning efforts. 3. Smaller and collaborative interventions are essential. In an area, such as Lower Shabelle, it may be appropriate to work on smaller and more influential interventions with multiple partners. Sharing the risk in areas that are volatile and difficult to operate in may yield innovative and influential opportunities to enhance citizen engagement and build local alliances. Short-term training opportunities for youth and women and/or cluster-wide events that allow settlements to work together may yield surprising results and help manage expectations. 4. Lower Shabelle is critical for stabilization in Somalia and cannot be ignored. TIS+ should consider working more closely with AMISOM and the USG to increase the development space 189 through community-based interventions. This is a volatile place to work, but there is a USG base and it may be possible to do some small and coordinated interventions that would have a significant and strategic impact. Projects that focus on shared assets between settlements and are accessed by multiple clans should be prioritized. USG coordinated short-term interventions may be most appropriate currently in Lower Shabelle, where security and volatility still make longer-term interventions unlikely. 5. Focus on programming that addresses recurrent and systemic needs for inclusion in Lower Shabelle. The following interventions are essential to enhance inclusion and mitigate grievances in Lower Shabelle: (1) assisting youth, women, and minority clan citizens in short-term training for local positions of representation; (2) supporting access to justice interventions and legal aid services for women and marginalized groups; and (3) interventions that address equitable economic opportunities for youth, women, and minority groups. TIS+ may not be well positioned to support these interventions directly, but working with government partners to encourage this type of development planning may be something TIS+ can do in coordination with other donors or through national level activities that support planning in Lower Shabelle. 6. Robust, regular, and in-depth analysis of the political economy dynamics is critical for the success of interventions. The dynamic situation in Lower Shabelle and the impact on security seemed to hamper TIS+ operations. While this is inevitable, it could be mitigated by regular analysis of the situation. It is also essential for TIS+ to closely follow the events at a micro level, with an emphasis on understanding local grievances, political positioning, and the ever-shifting engagement between AS and local clans. This will mitigate, if not indemnify, future interventions from stalling or failing, as witnessed with other partner programs. 7. Rule of Law and Justice is directly linked to stability. While this is no new revelation, it is more pronounced in this region and TIS+ should consider avenues to link with the USG’s new Access to Justice Program to find opportunities for joint intervention. Access to justice is essential at the district level. There are clear indications that AS is filling this niche and that people, particularly those from minority groups, are craving a greater understanding of political and legal rights and venues to address these needs. TIS+ should seek to link where possible with this new USAID project and assist local communities in accessing community based legal aid systems and services, particularly for women and minority clans. 1 SOMALIA PROGRAM SUPPORT SERVICES TRANSITION INITIATIVES FOR STABILIZATION PLUS MIDLINE ASSESSMENT FINAL REPORT - ANNEXES IDIQ AID-623-I-14-00009 TASK ORDER AID-623-TO-16-00015 January 30, 2019 This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by International Business & Technical Consultants, Inc. (IBTCI), under IDIQ AID-623-I-14-00009, Task Order AID-623-TO-16-00015. 191 SOMALIA PROGRAM SUPPORT SERVICES (SPSS) TRANSITION INITIATIVES FOR STABILIZATION PLUS MIDLINE ASSESSMENT FINAL REPORT – ANNEXES DELIVERABLE #17 IDIQ AID-623-I-14-00009, TASK ORDER AID-623-TO-16-00015 Submitted: January 30, 2019 Authors: Sarah Wood, Lucas Malla, Patrick Okwarah, Suleiman Omar, and Anna Patterson Cover Photo: Alamy Stock Photo, Somalia 2016 International Business & Technical Consultants, Inc. (IBTCI) In US: 8618 Westwood Center Drive, Suite 400 Vienna, VA 22182 USA +1.703.749.0100 In Kenya: 8th Floor Block C Westcom Point Building Mahiga Mairu Road, off Waiyaki Way Nairobi, Kenya +254 (0) 737 544 912 IDIQ AID-632-I-14-00009 Task Order AID-623-TO-16-00015 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 192 Table of Contents ANNEX I – Statement of Work................................................................................................ 193 ANNEX II – Index Details ............................................................................................................ 213 Good Governance Index ..........................................................................................................................213 Community Cohesion Index...................................................................................................................213 Attachment A.............................................................................................................................................................. 216 Attachment B .............................................................................................................................................................. 217 Attachment C.............................................................................................................................................................. 218 Attachment D ............................................................................................................................................................. 219 ANNEX III – Further Contribution Stories .......................................................................... 220 CA Story Tool: Technical Coordinators Contribution Story....................................................220 CA Story Tool: Sports for Development and Peace Contribution Story ............................226 ANNEX IV – CA Tables, Links to Excel Files ...................................................................... 233 193 ANNEX I – Statement of Work STATEMENT OF WORK C.1 PURPOSE OF THE TIS+ EVALUATION The primary purpose of the TIS+ evaluation is to generate a better understanding of the extent to which TIS+ interventions have achieved their objectives and contributed to TIS+’s (and USAID’s) goal of increased stability in Somalia, both within and across activity clusters, through participatory processes that promote good governance and community cohesion. Another purpose of this evaluation is to learn lessons about conducting stabilization activities in Somalia to improve future programming in this area. The evaluation will provide comprehensive answers to six key evaluation questions (see Section C.3) in the process of assessing the extent to which the activity goal and objectives were achieved and identifying lessons learned. C.2 AUDIENCE AND INTENDED USE The audience for the TIS+ evaluation is USAID Somalia, the FGS (including the Ministry of Interior and Federal Affairs, a key TIS+ partner), Emerging Federal Members States (e.g., Jubbaland State of Somalia, South West State of Somalia), AECOM, and other USAID offices, donors, or development organizations supporting Somalia. The TIS+ evaluation will help guide USAID in improving the design of its future stabilization interventions in Somalia. TIS+ evaluation reports will be uploaded to the Development Experience Clearinghouse and shared with the above stakeholders. In addition, USAID Somalia will develop an action plan to track any changes made to programming based on this evaluation. C.3 KEY EVALUATION QUESTIONS The TIS+ evaluation focuses at two levels of analysis, each with its own set of evaluation questions: within cluster analysis (cluster-level analysis) and cross-cluster analysis (comparative cluster analysis). The Statement of Operations (SOO) identified five clusters where TIS+ activities have either commenced or will soon commence, which USAID changed to five in an email on July 14, 2016. Within each of these five clusters, the evaluation will answer the following four cluster-level evaluation questions: 1. Did TIS+ contribute to any of the following anticipated results: a. Community ownership of development projects and processes? b. Effective shared asset management in the communities in which they were implemented? c. Improved delivery of services targeted by TIS+? Improved satisfaction with delivery of services? d. Reduced conflict over resources? e. Greater citizen confidence in government? f. Reduced support for violent extremism? 2. Were TIS+ approaches used in the cluster (e.g., the community consensus process itself) instrumental in contributing to the achievement of TIS+ results described in Question 1? 3. Did government at all appropriate levels contribute sufficiently to asset management in communities in which TIS+ was implemented? How? Did government at all appropriate levels have both the capacity and the will to contribute? 4. Were TIS+ activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? Besides the above cluster-level evaluation questions, the evaluation will answer two additional cross￾cluster evaluation questions as part of the comparative analysis scheduled for the first half of 2019: 194 5. In the aggregate, did TIS+ contribute to the anticipated results described above? What range of results can be observed across assessed clusters? a. Which cluster-level factors contributed most significantly to achievement of results? For example, did successful inclusion of women, youth and minority clans amplify the effectiveness of TIS+ activities? b. Were results greatest in newly liberated clusters or in bridge clusters? 6. Were some kinds of interventions more effective than others in improving citizen confidence or reducing support for violent extremism? Note that the two cross-cluster evaluation questions are, in essence, restatements of the cluster-level evaluation questions applied at the aggregate cluster level. However, for the purpose of making a clear distinction between the cluster-level and cross-cluster components of the TIS+ evaluation, we have elected the simplifying expedient of referring to them as distinct evaluation questions. Annex A contains a Data Collection Matrix that links each of the above six evaluation questions to a set of more detailed sub-questions, the type of evidence that will be used to answer the questions, and the data sources, data collection tool, data collection method, sampling approach, and data analysis method that will be used to answer each question. The Data Collection Matrix thus provides a concise summary of the evaluation methodology presented here. The SOO further holds out the possibility of conducting a final TIS+ evaluation. Although the SOW for the TIS+ baseline/endline evaluation does not include the final evaluation, the evaluation methods described in this technical proposal have also been designed to provide sufficient comparative potential for the final evaluation to answer the following two final evaluation questions: • Were results to which TIS+ activities contributed sustainable over time? In clusters implemented early in TIS+ (2016–2017), were the results that were observed at the conclusion of activities in those clusters still observed at TIS+ activity end? • Did the cluster-based approach itself contribute to the effectiveness of TIS+ interventions? Gender considerations will be taken into account in answering each of the above key evaluation questions. C.4 METHODOLOGY The TIS+ evaluation will use a mixed-methods approach that integrates a suite of quantitative and qualitative data collection methods implemented over four implementation phases to answer the above key evaluation questions. As described in Section C.4.3, data collection methods will consist of a household survey (HH), secondary research, key informant interviews (KIIs), focus group discussions (FGDs), and community checklists. Evaluation questions will be answered using a combination of these five data collection methods via the triangulation of evidence and the use of contribution analysis to estimate TIS+’s unique contribution to observed results. Contribution analysis is a pragmatic and methodologically rigorous approach to situations in which the activity is one of many potential contributory causes and where it is not possible to measure attribution directly via creation of a scientifically-valid counterfactual (as is the case here). Instead, contribution analysis assesses attribution in terms of the contribution that TIS+ activities are making to observed outcomes based on the totality of evidence. The essential value of contribution analysis in this case is that it offers an approach designed to reduce uncertainty about the contribution TIS+ is making to the observed results through an increased understanding of why the observed results have occurred (or not) and the 195 roles played by TIS+ activities and other internal and external factors.1 (For more on contribution analysis, see Section C.4.5.) C.4.1 DESCRIPTION OF EVALUATION APPROACH As noted above in Section C.1, the purpose of the TIS+ evaluation is to generate a better understanding of the extent to which TIS+ activities have achieved their objectives and contributed to TIS+’s goal of increased stability in Somalia, both within and across the five TIS+ clusters. The unit of analysis for the evaluation is the specific communities within these five clusters where TIS+ activities will be implemented. Table 1 shows IBTCI’s current understanding of the clusters and communities in which TIS+ activities will be implemented. As seen in Table 1, the evaluation will cover 38 communities in which implementation will be undertaken in the early stages of the process.2 Within the different clusters and communities, TIS+ is expected to pursue a variety of objectives and implement an even wider variety of activities falling under those objectives. Thus it is imperative that the evaluation achieves a sufficiently representative coverage of clusters and communities so as to capture this expected variety. One option to do this is to conduct data collection activities in a representative sample of communities, along with objectives and activities. Sampling is appropriate in such cases when the number of communities is too large to allow full coverage of communities within existing resource constraints. Another option is to conduct data collection activities in a census of communities, which will provide a fully comprehensive coverage of communities, objectives, and activities. A census approach is appropriate when the number of communities is not so large that comprehensive coverage is possible within existing resource constraints. The latter appears to be the case here; 38 communities is not so large that it falls outside the scope of available resources. Thus, we propose to conduct both quantitative and qualitative data collection activities in all 38 TIS+ communities during both baseline and endline evaluation rounds because this will allow us to develop a fully comprehensive picture of TIS+ results, which, we believe, will ultimately prove more useful in informing USAID programming decisions. Table 1. Number of TIS+ Communities/Districts per Cluster Cluster Region District Settlement Estimated No. of Households Estimated Population 1 Lower Shabelle Wanleweyne Wanleweyne 5,308 36,625 Lower Shabelle Afgooye Afgooye 12,037 83,055 Lower Shabelle Qoryooley Qoryooley 10,692 73,775 Lower Shabelle Barawe Barawe 3,007 20,748 Lower Shabelle Marka Marka 7,960 54,924 Lower Shabelle Kurtunwaarey Kurtunwarey * 1,890 13,041 Lower Shabelle Afgooye Awdeghle * N/A N/A 2 Lower Juba Afmadow Afmadow 6,880 39,216 Lower Juba Kismayo Kismayo 21,824 124,397 Lower Juba Jamame Jamame* 1,832 10,442 1 John Mayne. (2008). “Contribution Analysis: An Approach to Exploring Cause and Effect,” http://dmeforpeace.org/sites/default/files/0501_Contribution_Analysis_ILAC.pdf. 2 TIS+ activities will be implemented in other clusters beyond the five specified while this evaluation is still ongoing. The evaluation will not cover all activities during the timeline, and it is not anticipated that additional clusters will be added to the evaluation TO. 196 Middle Juba Jilib Jilib* N/A N/A 3 Gedo Baardheere Baardhere 4,610 29,965 Gedo Garbahaarey Garbaharey 3,852 25,038 Bay Dinsoor Dinsoor N/A N/A Gedo Ceel Waaq El Wak* 2,102 13,663 Gedo Baardheere Geriley* N/A N/A 4 Banadir Abdiaziz Abdiaziz 1,775 12,070 Banadir Bondhere Bondhere 4,447 30,240 Banadir Daynile Daynile 10,033 68,224 Banadir Dharkenley Dharkenley 13,744 93,459 Banadir Hamar Jajab Hamar Jajab 8,682 59,038 Banadir Hamar Weyne Hamar Weyne 2,331 15,851 Banadir Hodan Hodan 13,027 88,584 Banadir Howl Wadag Howl Wadag 7,327 49,824 Banadir Huriwa Huriwa 5,239 35,625 Banadir Kaxda Kaxda N/A N/A Banadir Karan Karan 12,073 82,096 Banadir Shangani Shangani 1,134 7,711 Banadir Shibis Shibis 4,034 27,431 Banadir Waberi Waberi 8,568 58,262 Banadir Wadajir Wadajir 18,929 128,717 Banadir Warta Nabada Warta Nabada 11,166 75,929 Banadir Yaqshid Yaqshid 14,083 95,764 5 Bakool Xudur Xudur* 3,075 20,295 Bakool Tayeeglow Tayeglow* 7,271 47,989 Bakool Ceel Barde El Barde* 1,780 11,748 Bakool Rab Dhuure Rabdurre* N/A N/A Bakool Waajid Wajiid* 9,958 65,723 Total 5 36 38 240,670 1,599,469 * To be determined Region Multipliers (Average Household Sizes) Lower Shabelle 6.9 Lower Juba 5.7 Gedo 6.5 197 Banadir 6.8 Bakool 6.6 Bay 4.8 Middle Juba 4.0 Note: Numbers of Districts (36) and Settlements (38) are preliminary. Catchment area of Bakool has to be determined. Note that in Somalia the last population census was conducted in 1987 due to the collapse of the Federal Government of Somalia in 1989 and the subsequent conflicts that followed. The population data shown in Table 1 represents the United Nations Population Fund’s (UNFPA)3 best estimates. The population figures for communities shown in Table 1 will be used to guide sample size distribution. C.4.2 EVALUATION PHASES This section of the SOW describes each of the four evaluation phases to be implemented under the proposed evaluation design. C.4.2.1 Phase 1 Evaluation Phase 1 will involve a thorough desk review of activity data and documents generated by TIS+, in addition to in-depth interviews with USAID officials and key members of the TIS+ and implementing partner (IP) management teams. A critical objective of Phase 1 activities will be to reach an agreement with TIS+ on the final list of communities in the five clusters selected by USAID for implementation of baseline and endline assessments. With the knowledge gained from the desk review and interviews, we will create an in-depth and fully specified activity ToC (necessary for the contribution analysis, see Section C.4.5) and finalize all details related to the evaluation design including the creation of a set of cluster-level toolkits, which include the quantitative and qualitative data collection instruments, a data analysis plan, and a final work plan, which includes a projected timeline, logistical arrangements, sampling plan, and data collection methods. C.4.2.2 Phase 2 During evaluation Phase 2, we will implement the baseline evaluation within each of the five clusters and 38 communities following the design and using the data collection methods and instruments developed during Phase 1. As specified in the evaluation SOO, the Phase 2 baseline data collection will involve a single data collection wave to be launched soon after the final work plan is approved. At the conclusion of the baseline data collection, we will analyze the baseline data, with a focus on describing and explaining the baseline conditions in the cluster using the methods described in Section C.4.3, and prepare the baseline evaluation report. The analytic focus in the baseline evaluation will be on describing the initial conditions in each of the clusters with regards to the key outcome variables related to EQs 1–4. Within clusters, all data will be analyzed in the aggregate and disaggregated by gender, age, and membership in minority clans. Once all baseline data collection has been completed, we will prepare a baseline evaluation report incorporating the findings from all five clusters. C.4.2.3 Phase 3 Phase 3 of the evaluation involves the endline data collection in the same five TIS+ clusters and 38 communities using the same data collection methods as in the Phase 2 baseline evaluation. Unlike the baseline evaluation, the endline evaluation will occur on a rolling basis as TIS+ phases out its activities within each of the clusters over time. The SOO assumes that endline data collection will occur over three separate data collection waves “based on the assumption that at least two clusters will complete activities 3 http://countryoffice.unfpa.org/somalia/drive/Population-Estimation-Survey-of-Somalia-PESS-2013-2014.pdf 198 in close enough proximity that they can be evaluated by the same team.”4 Because the number of cluster has changed from seven to five, for the purpose of cost containment, SPSS anticipates two waves: one wave will cover three clusters, the other will cover two clusters. One possible sequencing of the endline data collection waves is suggested on pages 16–17 of the SOO and is as follows: (1) Mogadishu (Banadir) in mid-2017; (2) Lower Juba 2017–2018; and (3) Lower Shabelle, Gedo, and Bakool in 2018–2019. This is, however, only one of several possible sequencing possibilities, depending on the timing of activities within each cluster and community. In any case, it is not possible at this point to propose with any precision when activities in the different clusters and communities will conclude and thus the number, timing, and composition of the endline data collection waves. For convenience sake, our proposed evaluation design accepts utilizing the same data collection team. This assumption will be thoroughly vetted during the initial discussions with TIS+ and its IPs during Phase 1 and the evaluation design and data collection work plan adjusted accordingly depending on the results of these discussions. At the conclusion of each endline data collection wave, we will analyze the data and prepare an endline evaluation report for each cluster covered by that wave, resulting in five separate cluster-level endline evaluation reports. The analytic focus in the endline evaluation will be on (1) measuring changes in key outcome variables, (2) assessing TIS+’s contribution, and that of other factors, to the changes in the key outcome variables, and (3) based on the findings/conclusions to 1 and 2, answering EQs 1–4. C.4.2.4 Phase 4 In Phase 4, we move from the within-cluster evaluation to the cross-cluster evaluation to produce a synthetic comparative analysis of results aggregated across the five clusters. Phase 4 of the evaluation will occur in close succession after the completion of Phase 3 and will begin with creating an updated data analysis plan, after which we will aggregate all of the data collected during Phases 2 and 3, analyze it, and prepare a final, comparative evaluation report, along with a set of actionable recommendations to improve the design of future stabilization interventions in Somalia. The analytic focus during Phase 4 will be similar to that in Phase 3, except that the focus will now be on measuring changes in key outcome variables and assessing TIS+’s contribution to them across all five clusters to answer evaluation questions 5–6. Cross￾cluster data will be analyzed in the aggregate and disaggregated by gender, age, membership in minority clans, and cluster. C.4.3 Data Collection Methods Under our proposed mixed-methods design, we will implement a combination of quantitative and qualitative data collection methods to answer the above key evaluation questions. The sole quantitative method is the household survey, which will be administered to residents living in communities where TIS+ activities are implemented. The qualitative data collection instruments include secondary research, in￾depth KIIs with diverse TIS+ stakeholders, FGDs with persons participating in and benefiting from the TIS+ activities, and community checklists. This combination of quantitative and qualitative data collection methods will allow us to construct a robust evidence base that will be analyzed to provide comprehensive answers to EQs 1–4 within clusters and EQs 5–6 across clusters. A description of each data collection method is provided below. C.4.3.1 Household Survey We will administer a repeated cross-sectional household survey in the baseline and endline to a randomly selected group of 424 residents living in communities participating in TIS+ activities within each of the five 4 There could be a months-long lag in the endline evaluation of some communities as it’s very likely that TIS+ work in communities will end at different dates based on circumstances on the ground. In other words, the endline of some communities will have to wait for TIS+ to finish in other communities. 199 clusters covered by the TIS+ evaluation for a total sample size of 2,200.5Under the cluster-level evaluation approach specified in the evaluation SOO, each cluster is to be treated as a separate analytical population for sampling purposes. This means that for each cluster examined, the baseline and endline survey samples must be large enough to draw statistically meaningful conclusions about the results of TIS+ activities within that cluster. This will further allow full disaggregation by cluster when the survey data from each cluster are consolidated for the comparative analysis in Phase 4. To calculate the required sample size in each cluster, we are using the sample size calculation formula for determining the appropriate sample size for detecting a difference between two proportions.6 The first step in calculating the sample size using this approach is to determine the key outcome variables and the minimum detectable effect (MDE) to be measured.7 The SOO identifies a number of outcome variables of interest (e.g., improved satisfaction with delivery of services, greater citizen confidence in government, reduced support for violent extremism), which can be used for this purpose. The question then becomes: What amount of change (effect size) does USAID expect to observe over time in the relevant outcome variable as a result of TIS+ activities? For example, if in the baseline 50% of survey respondents are satisfied with the delivery of local government services, by how much does USAID expect this to increase by the endline as a result of TIS+ activities? TIS evaluation findings have been used to estimate expected effect size for any of the key outcome variables. The assumptions include the following: Effect Size: 10 percentage points (Note that the assumed effect size is the smallest effect size that we consider to be relevant for informing USAID programming decisions that is feasible within reasonable budget parameters.) Confidence Level: 95%8 Statistical Power: 80%9 The above assumptions produce a required sample size of 385 households per cluster, which multiplied by the five clusters covered by the TIS+ evaluation yields a total baseline and endline sample size of 2,200.10 To this total we next add a 10% oversampling factor, which is done to ensure a sufficient number of responses for each survey question given an expected 10% non-response rate that is typical for household surveys in Somalia. Adding this 10% oversampling factor pushes the sample size to 424 per cluster and 2,120 across the five clusters. The survey instrument will be modeled in part on a past stabilization survey fielded in Somalia, which was used to generate the TIS Stabilization Report in 2013–2014, and the TIS performance evaluation done by SPSS in 2016. In addition to these two surveys, the survey instrument will draw on other surveys done to evaluate USAID local governance projects in other countries, many of which use methodologies similar 5 A repeated cross-sectional survey interviews a different random sample of respondents in each evaluation round. In contrast, a longitudinal or panel survey interviews the same cohort of randomly selected respondents in each evaluation round. The relative strengths and weaknesses of these two approaches are discussed in Section C.4.8. 6 See, for example, https://select-statistics.co.uk/calculators/sample-size-calculator-two-proportions/. 7 MDE is the smallest true effect or change for a given outcome variable, which is detectable for a given level of power and statistical significance. 8 Statistical power is the probability of detecting a significant change when one exists. With a power of 80%, this means that there is a 20% probability of failing to detect a significant change when one exists such as a false negative result (otherwise known as Type II error). For example, with an expected effect size of 10 percentage points, a statistical power of 80% means that we have an 80% probability of finding a 10 percentage point change, if one exists. 9 The confidence level is confidence with which we would like to detect a significant difference between the two proportions. If the confidence level is 95%, then this means there is a 5% probability of incorrectly detecting a significant difference when one does not exist, such as a false positive result (otherwise known as Type I error). 10 Including a pilot test of questionnaire with total sample size of 80 respondents. 200 to TIS+ to develop the capacity of local government institutions to deliver local services and to strengthen civil society structures within those countries. The survey instrument for TIS+ will be tailored to ensure that it accurately reflects TIS+ activities and their expected outcomes, while also answering the evaluation questions, within the constraints of the method. The following points summarize the key characteristics of the household survey: • Sample Universe: The adult population in communities where TIS+ activities are implemented • Sampling Frame: The 38 communities across the five clusters in which TIS+ activities are being implemented • Target Population: Adult members of TIS+ communities 16 years and older • Statistical Power: 80% • Confidence Level: 95% • Sample Size: N = 424 per cluster, 2,120 total • Effect Size: 10 percentage point increase (50%-60%) • Stratification: 50/50 male/female, female and male interviewers will conduct interviews with gender-matched respondents • Sample Distribution: Within each cluster, the assignment of the 424 sample households among the selected TIS+ communities will be allocated using the probability proportionate to size (PPS) approach. 11 During the cluster-level analysis, the sample will be weighted to be representative of the entire cluster population using weights developed with United Nations Population Fund (UNFPA) community-specific population data. During the comparative cross-cluster analysis, the consolidated data set will be re-weighted, again using the derived population data, to be representative of the entire study population. • Unit of Observation: One adult 16 years and older selected randomly from the members of each household who is systematically selected at each sampling point. • Household Selection Method: Random walk systematic selection. The sampling plan will specify a randomly selected starting location, walking direction, and the first household to be contacted for an interview in each community. After the first household is selected, the male interviewer selects every fifth household on the right-hand side and the female interviewer selects every third household on the left-hand side along the walk in rural areas, and every tenth household in urban areas. • Respondent Selection Method: Random selection using the two-step Kish Grid procedure. First all eligible members of the household (over 16 years old and either male or female depending on the sex of the interviewer) are listed. Then the random numbers in the grid cells are used to randomly select the respondent from the list. C.4.3.2 Secondary Research The first data collection method undertaken by the team will be to conduct a comprehensive review of internal and external documents of relevance to the TIS+ activity. We will work in coordination with the TIS+ management team to identify, collect, and review documents over the course of the evaluation that include the following: • Statements of work • Activity designs, training curricula, event agendas, and other key documents for understanding the content of activities 11 The universe will be the population size of the TIS+ communities. Table 1 presents population estimates by community as best estimated by UNFPA PESS 2014. IBTCI will estimate boundaries of these communities and also follow a similar approach used by UNFPA to estimate population data for four communities whose population figures were not presented in UNFPA reports. 201 • Annual Work Plans (initial and subsequent) • Activity Monitoring and Evaluation Plan (initial and revised) • Quarterly and annual reports • Results from TIS+ participatory M&E activities • SPSS monitoring and verification reports • Additional documents provided with the evaluation SOO • External publications by donors, academic researchers, NGOs, etc. The results from TIS+’s participatory M&E activities will further comprise a primary source of performance data for the cluster-level evaluations. TIS+ M&E data will be integrated with the data generated from the other data collection methods and will be used as a source for triangulation to answer the key evaluation questions. The evaluation team will incorporate information from the secondary research in the enumerator training to ensure that enumerators understand the background and context of TIS+ and are attentive to the need for data to answer the primary evaluation questions. Of particular importance is the geography of TIS+, its M&E framework, progress over time in implementing activities, key challenges reported by the project for achieving its objectives, and other key information for understanding the activity and how best to evaluate its performance using the other methods described below. C.4.3.3 Key Informant Interviews (N = 85, plus 20 [four in each cluster] for validation including the contribution analysis = 105) KIIs are in-depth, semi-structured interviews conducted with individual activity stakeholders. The evaluation team will use the knowledge gained from the desk review and in-depth interviews conducted during evaluation Phase 1 to inform the design of the KII discussion guides and selection of key informant interviewees. Interviews will be conducted in both the baseline and endline with key informants from the following stakeholder groups, with other stakeholder groups and key informants added in during both evaluation rounds if certain key information can be gained from others not listed below: • USAID officials in the USAID Somalia Office, N = 4 • AECOM senior management, regional offices staff, and subcontractor staff, N = 10 • Officials from Interim Regional Government Authorities and relevant government actors at the central and district levels in Somalia, N = 51 interviews (5 Ministry of Interior representatives, 10 FGS officials, and 1 district senior representative from each of the 36 sampled communities) • Key staff from other stabilization programs that interact with TIS+ or support programming that shares its objectives and geography (for example the Office of Transition Initiatives’ Somalia Stabilization Initiative, World Bank, DfID), N = 10 • Representatives from private sector, nongovernmental, or governmental donor organizations collaborating with or asked to collaborate with TIS+ communities, N=10 Participants will be selected purposively for the additional four KIIs for each cluster, which will take place after initial data has been collected and analyzed and preliminary findings produced and presented. The purpose of this second round of KIIs is to validate the preliminary findings and gather additional data required to complete the contribution analysis for the cluster. The key informants from each of the groups listed above and their locations and their numbers will be finalized in coordination with USAID and TIS+ during evaluation Phase 1 and again during Evaluation Phase 3 at the beginning of the endline data collection. It should be noted that the list of stakeholder groups and number of KIIs per group found above are estimates based on information currently available and are subject to revision based on discussions with USAID and TIS+. 202 The KII interview guides will follow an open-ended format that will give the interviewees the space to describe their experiences in their own words and allow the interviewer to probe for information that will uncover new insights into key aspects of TIS+’s performance. The interview guides will be modular so that certain question modules will be asked of all respondents while other question modules will be tailored to specific groups of respondents. C.4.3.4 Focus Group Discussions (FGDs) (N = 68 + 10 [two in each cluster] for validation including the contribution analysis = 78) FGDs are in-depth, moderated discussions with small groups of TIS+ community representatives. FGDs are a key method for measuring program performance as perceived by beneficiaries because in-depth information can be collected on the effects of participation in activity interventions. A major advantage of the FGD method, as opposed to other forms of qualitative research, is its flexibility for in-depth exploration of the knowledge and attitudes of the group toward TIS+, processes used during TIS+ activity implementation (e.g., representativeness, transparency, tendering, procurement, and participation by women, youths, minority clans, and members of other marginal groups) and the outcomes of the activities in which they participated. Four FGDs will be conducted in each selected community with the following groups: • Female beneficiaries • Male beneficiaries • Male youth beneficiaries • Female youth beneficiaries We propose to conduct four FGDs in up to 50% of the TIS+ communities within a particular cluster. This produces the total number of FGDs in each cluster shown in Table 2. The communities participating in the FGDs will be selected purposively guided by criteria to be established by the evaluation team in conjunction with TIS+ management and field staff. FGD participants within communities will be selected based on the criteria that they are permanent community residents and have participated in one or more of the activities implemented by TIS+ in their community. FGD participants will be drawn, for example, from participants in TIS+ consensus groups, monitoring committees, and local tendering committees, as well as other well-informed community members. To the extent feasible, the FGDs will consist of homogenous groups. To get a fully representative picture of activity and implementation, the composition of FGD participants will be determined on a rolling basis in each successive community depending on the nature of activities implemented within those communities, previous issues identified within those communities (e.g., via TIS+ monitoring activities), and the coverage of TIS+ participants in prior FGDs. Participants will be selected purposively for the additional two FGDs for each cluster, which will take place after initial data has been collected and analyzed and preliminary findings produced and presented. The purpose of this second round of FGDs is to validate the preliminary findings and gather additional data required to complete the contribution analysis for the cluster. Table 2. Number of FGDs per Cluster Cluster Number of TIS+ Communities Number of Communities for FGDs Total Number of FGDs 1 7 3 12 2 4 2 8 3 5 2 8 4 17 8 32 203 5 5 2 8 Total 38 17 68 The FGD guide will give the discussion moderator a set of key questions and topics for guiding the discussions to uncover key information for answering the evaluation questions and determining whether TIS+ met its objectives, how TIS+ activities unfolded within the community, and the contribution of TIS+ activities to observed changes within the community. To keep respondents focused throughout the session, FGDs will be organized at the venues where participants will be comfortable to be recorded using digital audio recorders. The FGDs will include from 6 to 10 participants for a two-hour discussion. During each session, participants will be provided with refreshments. As with the KII interview guides, the FGD discussion guides will follow an open-ended format that will give the participants the space to describe their experiences in their own words and allow the interviewer to probe for information that will uncover new insights into key aspects of TIS+’s performance. The discussion guides will be modular so that certain question modules will be asked of all respondents, while other question modules will be tailored to specific groups of respondents, clusters, and objectives. C.4.3.4 Community Checklists (N = 380) Qualitative researchers will make direct observations about sample communities using a detailed checklist, which will be used to evaluate the extent to which physical structures were rehabilitated. C.4.4 Data Analysis Methods The following presents a summary of our proposed quantitative and qualitative data analysis methods. We will disaggregate all data analysis by gender, age, membership in minority clans, or other population characteristics, as appropriate. C.4.4.1 Quantitative Data Analysis Methods The primary quantitative results of interest are the changes in key outcome variables related primarily to EQ1, which will be measured as the change in two proportions; for example, a change in the proportion of survey respondents who express confidence in government in the baseline compared with the proportion who express confidence in government in the endline. To analyze changes in key outcome variables and to understand the relationships between other variables in the survey data set, we will employ a range of statistical tests depending on the nature of the data. These tests include the following: • Tests of independent means for numerical and binary data. • The independent samples t-test will be used with numerical and binary data to determine whether the difference between two means is statistically significant. • Analysis of variance will be used with numerical and binary data to determine whether the difference between three or more means is statistically significant. • Ordinary least squares regression analysis, in which continuous outcome variables are dependent variables and survey data (for example, demographic variables for both within and across cluster analysis and location data for across-cluster analysis) are independent variables that will be used to analyze the factors that are associated observed outcomes; disaggregate the analysis by sex, age, clan, location, and other sample sub-population and control for possible intervening factors. • Logistic regression analysis (Logit), in which dichotomous outcome variables are dependent variables, and survey data are independent variables, will be used to analyze the factors that are associated observed outcomes; disaggregate the analysis by sex, age, clan, location, and other sample sub-populations and control for possible intervening factors. • Non-parametric tests for ordinal data: A number of the household survey questions will be Likert scales, which produce ordinal data. 204 • The Mann-Whitney U test is a non-parametric test that will be used to compare differences between two independent groups when the dependent variable is ordinal. • The Somers’ D test is a non-parametric measure of the strength and direction of association that exists between an ordinal dependent variable and an ordinal independent variable. • Ordinal logistic regression, in which ordinal outcome variables are dependent variables and survey data are independent variables, will be used to analyze the factors that are associated observed outcomes; disaggregate the analysis by sex, age, clan, location, and other sample sub-populations and control for possible intervening factors. • Tests of categorical data: The Pearson’s chi-square test will be used to test whether there is a difference in observed and expected frequencies and, therefore, whether two samples being tested are from the same population. For the purposes of this evaluation, p-values of greater than or equal to 0.10 will considered to indicate a statistically significant difference. C.4.4.2 Qualitative Data Analysis Methods Data from the KIIs and FGDs will be analyzed using an inductive thematic approach, which reduces text to codes and identifies patterns among the codes. Analysis will be performed using the NVivo qualitative analysis software program. Using a standardized iterative process, a codebook will be developed and codes will be systematically applied to the data. A matrix will be designed to identify themes to better understand (1) the ways in which TIS+ activities have affected outcomes related to the evaluation questions; and (2) other factors, both internal and external to TIS+, that may have affected these outcomes. At the same time, codes will be developed to facilitate the integration of findings across the qualitative and quantitative data. We will design the data analysis strategy for both quantitative and qualitative data before the baseline data collection begins to encourage team members to begin thinking about trends in findings from different data sets to determine whether findings are similar or divergent. Also, the team will consider which data relates to the contribution analysis at the settlement and cluster level to be part of answering EQs 1–4, and in turn, how these data will be analyzed across sectors to answer EQs 5 and 6. Data analysis will employ a parallel, mixed-data approach in which quantitative data are independently analyzed from qualitative data, but both use coding related to the evaluation questions. This approach takes the findings and analysis from the qualitative data and uses it to inform and explain findings from the quantitative data, and vice versa. We will then seek to expand on the findings that emerge from the quantitative data by identifying qualitative trends to provide explanations and depth for those results. At the same time, we will take the themes and trends emerging from the qualitative data and look for existing quantitative results or undertake additional quantitative analysis to determine whether there is evidence to generalize the qualitative findings to the target populations. Finally, we will utilize different types of triangulation to validate findings, analyses, and conclusions including data triangulation using a variety of data sources12 and investigator triangulation involving the use of different evaluators who bring diverse perspectives and cultural and analytical skills. C.4.5 Contribution Analysis TIS+ operates within a wider system in which it is only one of a number of factors that potentially affect activity outcomes. TIS+’s interventions will, therefore, be at best a contributory cause to any observed 12 Analysis of background documents, FGDs, KIIs, and household survey. 205 outcomes. In other words, TIS+ activities are a necessary part of a “package” of causal factors that are together sufficient to produce the observed outcomes. It is important, therefore, to place project activities within the wider range of other potential causal factors and to understand that these other factors have a potentially important influence on the occurrence, scale, and nature of change. According to Jonathan Mayne, contribution analysis infers causality from the following evidence: • The project is based on a well-reasoned ToC, while the assumptions underlying the ToC are plausible, sound, informed by evidence, and agreed-on by key stakeholders. • The project implements activities as outlined in the ToC. • The evidence verifies both the occurrence of expected outcomes in the ToC and its underlying assumptions. • External factors, including context and rival explanations, are assessed and are shown not to have made a significant contribution to observed results, or, if they did, their relative contribution is recognized. One purpose of contribution analysis is to work through a set of contribution stories that assist in answering the relevant evaluation questions. In working through the contribution stories, the analysis must be guided by the following three considerations related to the nature of causality/contribution. First, project interventions do not operate in isolation but are part of a causal package that includes other interventions, events, or conditions external to the project. Thus, our interest in doing contribution analysis focuses less on whether and how TIS+ has contributed to a particular outcome but instead on whether and how the causal package, which includes TIS+ interventions, has contributed to the outcome. Second, any particular causal package comprises the intervention outputs plus the assumptions, which together are sufficient to bring about the intended impacts. The ToC will identify the support factors (assumptions) and confounding factors (risks). It also explains how and why intended impacts are expected to occur. The causal questions will address the following: (1) Has the intervention made a difference; in what way was it a contributory cause? (2) Why has the impact occurred? How did the causal factors bring about the result? What was the context and the mechanisms? What role did the intervention play (i.e., as a trigger) compared to contributions played by other influencing factors? Therefore, this approach should shed light on the questions about what the relative contribution of TIS+ approaches was to anticipated results, which is the first EQ, which leads to the second EQ regarding the extent to which TIS+ approaches were instrumental in achieving results. Third, assessing contribution must take into account the probabilistic nature of causality and the lack of certainty that often attends our attempts to measure it. This requires that we talk more in terms of likelihoods instead of certainties with regard to our conclusions about contribution. Thus, the standard of evidence to determine contribution that we will apply through the contribution analysis is that the causal package is likely a sufficient cause of the observed change, while TIS+ activities are in turn a “likely necessary” part of the causal package. Thus, in lieu of creating a statistically valid counterfactual, contribution analysis reduces uncertainty about the contribution TIS+ is making to observed changes through an increased understanding of why the observed changes have occurred (or not occurred) and the role TIS+ played in contributing to these changes relative to other internal and external factors. As specified by Mayne, and as to be implemented in the TIS+ evaluation, contribution analysis involves the following six-step approach to arrive at evidence￾based conclusions about the factors likely contributing to observed changes. 206 CONTRIBUTION ANALYSIS PROCESS TIS+ EVALUATION PHASE PRODUCTS Step 1: Set out the cause–effect issue to be addressed in a detailed ToC. This step maps out the cause-and-effect logic leading from project activities to expected outcomes and carefully traces out the expected outcomes, possible unexpected outcomes, and causal linkages, along with key assumptions and causal mechanisms underpinning the ToC. PH 1: PREPARATION Attribution problem statement: What needs to be addressed Articulate the cause-effect question Explore contribution expected Identify other influencing factors Identify plausibility of the contribution Articulate causal questions related to the EQs Step 2: Identify the risks to the postulated ToC, including rival explanations. This step assesses the strengths and weaknesses of the cause-and-effect logic underlying the ToC and assesses the nature and extent of TIS+’s contribution to each expected result, along with factors that imperil the hypothesized causal linkages and other possible contributory factors and rival explanations. PH 2: BASELINE PHASE Draft of the TOC with assumptions and risks and roles played by other factors. How are they contested in the TOC? Review results. Questions for inclusion in tools and analysis plan Step 3: Gather evidence. This step involves the implementation of the data collection methods described above to gather evidence for the contribution analysis, particularly (1) whether the expected results did or did not occur, (2) why expected results did or did not occur, and (3) the causal process and contextual and other contributory factors explaining why expected results did or did not occur. PH 3: ENDLINE Comparative analysis section for each cluster report following analysis plan Preliminary Findings and guides for follow up KIIs and FGDs Step 4: Assemble and assess the contribution claim and challenges to it. This step constructs the contribution story while assessing the strengths and weaknesses of the cause-and-effect logic in light of the evidence collected during the previous step, the relevance of other contributory factors, and the evidence for rival explanations. This step involves three analytical tasks: (1) Bottom-up measurement: This task assesses how TIS+ activities have facilitated changes at the outcome level by analyzing the effects of interventions and their interaction with the context, including other actors, the local political economy, and other external factors. (2) Top-down measurement: This task assesses the key changes in outcomes and then investigates how activities and other factors have driven these changes. (3) Compare and triangulate findings: This task investigates the causal relationships between activities and observed outcomes by assessing the extent to which observed outcomes and contributory causal factors are consistent across data collection methods and contexts. Step 5: Seek out additional evidence. This step determines what kind of additional evidence is needed to enhance the credibility of the contribution claim and gather the appropriate evidence, as feasible. PH 3: ENDLINE Final sections with data second round of FGDs and KIIs 207 Step 6: Revise and strengthen the contribution story. Having identified and addressed any challenges to our credibility, such as the strengths of certain assumptions and/or the roles of other influencing factors, this step builds a more credible contribution story. Because this process involves both bottom-up and top-down measurement of change, it provides triangulation of evidence for change up and down the project ToC. It also provides a reasonably robust approach to measure contribution and guard against “self-importance bias,” which occurs when the relative importance of other contextual factors is underestimated and the importance of project-specific factors is overestimated. PH 4: SYNTHETIC COMPARATIVE ANALYSIS Result of analysis with added data C.4.6 Quality Assurance To reduce sources of bias and obtain valid and accurate responses from participants, we will implement a series of rigorous quality assurance (QA) methods for both quantitative and qualitative data. These methods are described below. Interviewer training. Central to overall QA will be providing training for our local research partner and its interviewers. Each member of the quantitative and qualitative field teams will participate in a two- to three-day training workshop before entering the field during which time they will receive question-by￾question instruction in research instruments, their purpose, and their implementation along with instruction and practice in the research protocols related to, for example, skip patterns, coding and use of handheld devices (quantitative training), and sound interview/moderation, probing, and participatory research techniques (qualitative training). Cognitive consistency. Instrument calibration will also be critical to ensure accurate results. Instruments will be tested on location with key informants and/or representatives of the target population. The purpose of this step is to fine-tune the translation and check the cognitive consistency and precision of the questions asked. This might result in re-phrasing of some questions, adjustment of vocabulary, or response options to better fit the local context. Use of technology. The evaluation will use both Mobile Data Collection (MDC) and traditional paper-and￾pencil data interview (PAPI) methods. MDC will be used with quantitative data collection instruments in communities that are considered very accessible. It will also be used to create digital audio recordings of KIIs and FGDs proceedings. PAPI will be used with quantitative data collection instruments in communities that are considered moderately accessible and where the use of MDC would draw attention (thus risking the lives of interviewers). It will also be used for KIIs and FGDs as a backup record of KII and FGD proceedings. The MDC platform provides real-time data functionality and back-end verification capabilities to support data collection activities, while also enabling geo-tagging and time-stamping of interviews and quick analysis of data on an ongoing basis for quality control. Programming the devices for data collection and data processing will include range, logic, and consistency checks customized for the item types and expected responses. Range checks will ensure that continuous data will be entered within predefined boundaries or that interviewers only select categorical data from a predefined list of responses. Skip logic will be scripted to ensure that respondents receive the appropriate questions based on previous responses or data derivations. Internal consistency checks will be built into the computerized script to allow interviewers to make necessary corrections to data while interviewing the respondent. 208 We are familiar with the challenges posed by both MDC and PAPI tools and as such will develop a set of rigorous data management and quality assurance protocols for each (complementing the QA methods described above) and will train all field researchers in their use. Pilot testing. We will conduct a pilot test of all quantitative instruments with the intended informants and target population and in circumstances similar to that of the study itself. The purpose of the pilot is threefold: (1) It tests the data collection protocol. (2) It further trains the survey team by giving its members the opportunity to get better acquainted with the content and the target population. (3) It permits further adjustments of the language in the instruments for better flow and to ensure internal consistency of the responses. Based on the findings of the pilot, changes will be made to the survey instrument and protocols. Household survey data checks. For the household survey, we will conduct routine quality and completion checks in the field during the household survey implementation to ensure enumerators follow protocols and ensure data accuracy. Conducting regular spot-checks will allow problems to be identified and resolved while the data collection team is on location. Each enumeration team will include one Field Supervisor to check that each record is completed before assigning other sample cases to the enumerator. A Data Collection Coordinator will oversee all enumeration teams and will check the data of each Field Supervisor at the end of each day. Based on this information, they will assign callbacks to Field Supervisors or re-conduct surveys themselves. Callbacks make sure data are collected correctly but also point to problems that are more systemic and may require refinements to the data collection protocols. Qualitative controls. Qualitative quality control methods will include standardizing introductory comments to create a common set of expectations for respondents, holding discussions in venues where participants feel comfortable and secure, excluding activity staff or local authority figures from FGDs, and monitoring a random sample of KIIs and FGDs to ensure that team members are following the moderation protocols. After completion of each FGD session, each evaluation team member will review the transcripts from the transcribers individually. Next, they will meet to discuss and review the transcriptions. If unanticipated information is uncovered, team members will design appropriate probing questions for the ensuing KIIs. Additional information collected through this approach will help the evaluation team in the final analysis during triangulation of data (findings). Finally, prior to processing the data, the qualitative evaluation team will review collected data to ensure all findings are properly recorded and documented. Data cleaning. At the end of each day, survey data will be uploaded to a local server as well as a server hosted by researchers. Back-office staff, both locally and remotely, will run analytic programs that perform data cleaning, check coding, and produce descriptive statistics. These analytic checks will be performed daily or at an adequate frequency so problems may be addressed immediately. Debriefings. During the fieldwork, both quantitative and qualitative data collection teams will hold a debriefing at the conclusion of each field day to share notes, check data quality, ensure that instruments have been completed correctly, and troubleshoot problems. Using the real-time, back-end verification capabilities embedded into the MDC platform, the Field Supervisors and Data Collection Coordinator will verify the data collected and uploaded for that day. Check-ins. We will institute a regular check-in process consisting of weekly teleconferences between the Data Collection Coordinator and the IBTCI Evaluation Manager and weekly status reports from the Field Supervisors. Challenges arising in the field will be resolved on the spot by the Field Supervisors when appropriate. When this is not possible, the Field Supervisor will refer the problem to the Data Collection 209 Coordinator for resolution. As a last resort, and for the most serious issues, the Data Collection Coordinator will refer the problem to the Evaluation Manager for resolution. C.4.5 Data Format and Sharing The evaluation data and information will be converted to the appropriate formats and shared with USAID per ADS 579, “USAID Development Data” or “USAID Open Data” requirements and guidelines. To ensure transparency and replicability, quantitative data will be submitted as annotated data sets clearly defined with codebooks and annotated analysis files. Unstructured qualitative data, such as interview transcripts, will be stored in text-based data entry templates, while structured quantitative data will be stored in Excel and SPSS files. To ensure respondent anonymity, survey data will be entered into two separate databases, one with identifying information and one without identifying information. Metadata will be generated in the form of codebooks and data summaries. The final data set will be compiled and submitted to USAID in accordance with relevant Agency guidelines. The evaluation team will also work with USAID Somalia’s Open Data Point of Contact to ensure that it submits the correct files and data in the appropriate format. C.4.6 Dissemination of Results To ensure that key findings and recommendations are harmonized into a realistic and meaningful body of analytical work, the evaluation team will vet findings with key stakeholders to get input and validate recommendations. At the conclusion of the evaluation, we will host a workshop for key project stakeholders to present our findings and receive feedback. The workshops and presentations will be an important tool in achieving stakeholder buy-in and ensuring that results are acceptable. We propose a second round of FGDs for each cluster to take place after the two waves of data collection. These will happen after analysis, preparation of preliminary findings, including each cluster contribution story and presentation to USAID and stakeholders in Nairobi. Additional questions may come out of those presentations; we will also review the topics that will be covered in the second round of FGDs and KIIs for their inputs. The second round of FGDs and KIIs will then take place, where the preliminary findings will be presented and final contribution analysis information collected. These data will be used for the final reports for each cluster. A workshop is planned in Mogadishu in the final synthetic comparative analysis phase to present the evaluation report preliminary findings before finalizing the report. This will be similar to the preliminary findings workshop for this phase with stakeholders in Nairobi, but will allow more FGS and civil society representatives to attend. Final planning on these second round FGDs, KIIs, and the final workshop will be refined during the phase one preparation stage. C.4.7 Ethical Considerations All data collection will be conducted according to the professional standards of the American Evaluation Association. Signed informed consent will be obtained prior to the start of any data collection activity, while interviewees will be given the option not to respond to any questions or to decline any interview. To protect the interviewee’s anonymity, we will implement a number of safeguards. To begin with, all evaluation team members will be required to sign a certificate of confidentiality. Any information that could be directly linked to an individual will not be used, including quotes provided in evaluation reports. Further, only general information (e.g., location, sex, age, etc.) will be utilized for analysis or reporting, while only members of the evaluation team will have access to the transcripts and raw data. Finally, data sets provided to USAID or other stakeholders will be cleaned with identifying information removed. 210 C.4.8 METHODOLOGY STRENGTHS AND LIMITATIONS USAID’s Evaluation Policy states that any methodological strengths and limitations are to be communicated explicitly in SOWs. Some of the methodological strengths and limitations of the TIS+ evaluation are described below. C.4.8.1 Methodological Strengths The sampling design provides 95% confidence level and 80% statistical power, both of which adhere to established best practice in evaluation research, thus providing reliable data for decision-making purposes. Contribution analysis provides a credible alternative approach for dealing with the issue of attribution in the absence of a counterfactual. While contribution constitutes a lower standard of causality than attribution, contribution analysis is a second-best alternative that can still provide rigorous evidence of an activity’s specific contribution to observed outcomes. SPSS is mindful of the political polarization in Somalia, and will discuss and coordinate with USAID Somalia Office and TIS+ about selection of respondents and other issues that may arise. While bias can never be entirely eliminated from an assessment, there are a number of checks the evaluation team will engage in to mitigate and minimize bias. The KIIs will be implemented using question guides rather than a detailed interview guide that might “force” respondents to provide answers to questions about aspects of the activity for which they may have inadequate knowledge. Conveying independence of the evaluation team from TIS+ implementation and decision-making and the fact that the evaluation will have no direct influence on future project activities will strengthen the independence of the TIS+ evaluation. Evaluator conflicts of interests are another potential source of bias. SPSS certifies that no individual included in its evaluation team, including contractor staff employed to collect data, will possess a material or perceived conflict of interest that might influence the evaluation process, the quality of the data collected, or the findings of the evaluation. Data collection using mobile devices will allow for quality control and analysis in near real time, enabling fast decision making by the evaluation team for quality control and quick course corrections on fieldwork plans and data collection instruments based on clearly identified need and response to exigencies that may arise during fieldwork. Emotional attachment and professional reputation may enter as a different source of bias in the data that will be collected from IP management and staff. Reputational considerations and strong attachments to the Stabilization Project and the areas where it worked may create a tendency to answer questions in ways that cast a positive light on some of the more challenging aspects of TIS+. Triangulation of data from different sources is the key analytical technique for identifying and controlling this bias. That is, bias will be controlled by cross-referencing information from interviews with USAID officials, with information from IPs, with data collected from beneficiaries in the TIS+ communities, plus data from secondary sources including independent monitoring, verification, and reporting. The triangulation of data from these different sources will ensure that the answers to the evaluation questions reflect a distillation of the full range of dynamics surrounding TIS+ programming and the various interests of its stakeholders. Insecurity is a persistent limitation that affects all types of work in Somalia, including evaluations. Insecurity may affect the mobility of evaluators and access to research subjects and activity sites. Insecurity may create bias in evaluation findings if it results in the systematic exclusion of certain types of activity sites and/or groups of respondents. If during the course of fieldwork insecurity compromises the evaluation 211 team’s ability to fulfill the evaluation plan, then SPSS will immediately identify the best means of revising the evaluation plan such that inaccessible locations and/or respondents are replaced with comparable ones that are accessible. Thereby the evaluation team will control the potential source of sampling bias that might arise from insecurity. C.4.8.2 Methodological Weaknesses The evaluation SOO specifies that data collection will be limited to TIS+ clusters and communities. This requirement forecloses the possibility of creating a counterfactual, which would allow us to attribute findings with regard to EQ1 to TIS+ activities. In lieu of a counterfactual, we have proposed to use contribution analysis (see above). Notwithstanding best efforts to avoid or mitigate potential biases (see above), they remain a risk, particularly with regards to qualitative methods, which rely heavily on respondent perceptions and interviewer interpretation. The desire to attract follow-on programming among certain types of respondents is one such source of bias. This source of bias is often a feature of the community context where project beneficiaries and stakeholders attempt to provide the information that they think will prompt the delivery of additional benefits from a new project. More generally, KII and FGD respondents have their own biases that must be recognized and considered when analyzing their responses. If not moderated correctly, biases can also emerge during FGDs that can taint their usefulness. Finally, qualitative data collection methods are also subject to some important interviewer effects because the quality of the research is heavily dependent on the skill of the researcher and can be easily influenced by their personal idiosyncrasies and biases. Interviewer bias will be mitigated by convening daily team debriefs, rolling data analysis, and presenting transcripts with 24 hours of each qualitative data collection. Key informants from important stakeholder groups (e.g., government agencies) are likely to rotate in and out over time meaning that valued key informants during the baseline evaluation may not be available during the endline evaluation and, thus, key informants during the endline round may lack the institutional/historical knowledge or context to answer questions. In addition, the selection of key informants for KIIs may be challenging and findings can be biased if the informants are not carefully selected. When only a few people are involved in a particular activity, it may be difficult to demonstrate the validity of the findings. Within the clusters and communities covered by the evaluation, TIS+ will be implementing a range of activities tailored to the needs in each community falling under Objectives 1–4. At the time of the baseline data collection, the specific needs and priorities of communities—and therefore the specific content of the TIS+ interventions—may not yet be known, although they will be by the endline. Moreover, while the qualitative methods allow significant flexibility to adjust the discussions to issues of more specific relevance to individual communities, this does not mean that questions in the household survey must be written at a higher level of generality that may not allow a fine granularity of analysis. The repeated cross-sectional survey design is a cost-effective means for tracking changes in general population characteristics over time; however, it does less well than a longitudinal (panel) survey in terms of measuring the variations in the patterns of change at the household (or unit of observation) level or in explaining why the change is occurring. On the upside, a repeated cross-sectional survey does not suffer a loss in explanatory (statistical) power that is characteristic of longitudinal designs due to panel attrition (respondents who drop out of the panel from the baseline to the endline). 212 C.4.9 IMPLEMENTATION RISKS The use of consultants for assignments that are intermittent with possible schedule changes for reasons outside of anyone’s control (e.g., political changes in Somalia, IP implementation delays, security concerns), can lead to changes due to availability issues with consultants. SPSS mitigates this by staying in regular contact with the Evaluation Team, by using some team members who are known quantities, and by exercising strong technical oversight at the field and Home Office levels to ensure continuity. 213 ANNEX II – Index Details Assessment of Preconditions for TIS+ Goal The goal for the TIS+ project is to increase stability in the clusters of implementation. For stability to be achieved, two preconditions should be satisfied: (I) increased good governance and (II) increased community cohesion. This subsection, therefore, explains the development of index scores13 to measure these preconditions. Proposed Models Two models have been formulated for good governance and community cohesion. Model 1 is primary and is validated through sensitivity analysis. The sensitivity analysis process involves the formulation of Model 2 to check how estimated summary statistics and distributions would vary under a different scenario. This is to examine whether having an unequal number of questions within themes for the two preconditions would result into different results. Good Governance Index Model 1: Household questions have been grouped into three themes explaining good governance (confidence in government, service delivery, and stakeholders’ partnerships). See Model 1 (good governance) Excel file in Table 1 for specific questions under each of these themes. Each of the three themes is assumed to contribute equally to the overall good governance index score (each contributes 33.33%). Model 2: All questions in the three themes are assumed to contribute equally to the overall good governance index score. There are 15 questions across the three themes and each contributes 6.67% (100% ÷ 15) regardless of the theme. Community Cohesion Index Model 1: Household questions have been grouped into the three themes explaining community cohesion (network diversity, interpersonal and institutional trust, and social and political inclusiveness). These themes for community cohesion were informed by the impact evaluation of Peace through Radio Programming conducted in Chad and Niger.14 In addition, the MISTI report15 provided further guidance on what would constitute community cohesion; relevant questions (and in other cases proxies) were identified and classified into the relevant groups documented in the Chad and Niger evaluation report. In other words, this index draws from both the MISTI and Chad/Niger evaluation reports. Please see Table 1 (Community Cohesion Excel file – Model 1) for all the survey questions grouped by network diversity, interpersonal and institutional trust, and social and political inclusiveness themes. Attachment A shows questions in the Niger/Chad report that are related to those in the TIS+ questionnaire. Attachment B shows questions in MISTI closely related to those in the TIS+ assessment questionnaire. 13 Index score is a single measure derived from a set of variables. 14 https://pdf.usaid.gov/pdf_docs/pa00ktf3.pdf 15 http://www.d3systems.com/wp-content/uploads/2015/10/MISTI-Wave-5-Analytical-Report-FINAL_USAID.pdf 214 Each of the three themes is assumed to contribute equally to the overall community cohesion index score (each contributes 33.33%). Model 2: All the questions in the three themes are assumed to contribute equally to the overall community cohesion index score. There are eight questions across the three themes and each contributes 12.5% (100% ÷ 8) regardless of the theme. Table 1, Specific Contribution of Survey Questions in Models 1 and 2 Model 1 Model 2 Good governance index GG model 1.xlsx GG model 2.xlsx Community cohesion index CC model 1.xlsx CC model 2.xlsx Sensitivity Analysis This involves comparing the distribution of derived scores under models 1 and Model 3.16 All models for good governance and community cohesion seem plausible because the distribution and mean of index scores are approximately similar between the two models (see Figures 1 and 2). 16 This sensitivity analysis process used the TIS+ baseline assessment data set. Figure 1: Good Governance Frequency and Sensitivity 215 Figure 2: Community Cohesion Frequency and Sensitivity Analysis 216 Attachment A Linking Questions in the Niger/Chad Report to Those in TIS+ Questionnaire Niger/Chad Report TIS+ Questionnaire Theme a) Talk to people from other communities b) Talk to people from other religions c) Talk to people from another ethnicity 34. Which of the following is true at your personal level? Family members are married to another clan. Family members are not allowed to marry with members of another clan(s). I attend meetings with other clan(s) in the same area. Clans in this settlement coexist peacefully. Clans in this settlement do not get along. Clans cooperate only when necessary and a third party (development or government actor) intervenes. Network Diversity a) Interpersonal trust b) Trust in local government c) Trust in central government d) Trust in religious leaders e) Trust in NGOs f) Trust in police 28. On a scale from 1 to 10, how much trust do you have in each of the following to resolve conflicts over shared resources and disputes among various groups in this settlement? 1 being “I do not trust them at all” and 10 being “I trust them a lot”. (clan elders, National Army, militia groups, Peace Committee, Oversight Committee, Central government, religious leaders, local administration, police, local courts, ugaas/sultan, diaspora, District Council members) Interpersonal and institutional trust a) Who makes decisions: Ordinary people, youth, women, my ethnic group, other ethnic groups b) Satisfaction with decisions and political participation 9. In your opinion, has people’s participation in decision￾making increased, decreased, or continued to remain the same, compared to the previous year? 10. Over the past year, who was most responsible for decisions concerning community development in this settlement? Who did not participate? Please check all that apply. (local community members, local administration, local community members participating in planning and contracting meetings, district council members, clan elders, religious leaders, ugaas/sultan, women, youth) 11. To what extent were you satisfied with the decisions made during the past year by the following people/ groups? (local community members, local administration, local community members participating in planning and contracting meetings, district council members, clan elders, religious leaders, ugaas/sultan, women, youth) Social and political inclusiveness 217 Attachment B Additional Questions Shows relationship between those questions in MISTI report and TIS+ assessment questionnaire MISTI TIS+ Theme 2.1.1.1 Ability to solve external problems (q34c) 2.1.1.2 Ability to solve internal problems (q35c) 2.1.1.3 How often villages work together (q36) 43. How far do you agree or disagree with the following? a) People work collectively towards problem solving when they know it will benefit the community. b) People do not trust one another across clans and within settlement areas. c) People will cooperate on development initiatives and efforts. 44. How far do you agree or disagree with the following? a) People work collectively towards problem solving which leave beneficial effect on the settlement. b) People are untrustworthy; therefore, development projects cannot be implemented. c) People are very cooperative. Interpersonal and institutional trust 2.1.2.1 Local leaders consider citizen interests (q37a) 2.1.2.3 Local leaders secure funds (q38)17 21. On a scale from 1 to 10, how would you rate the following statements? 1 being “I do not agree at all” and 10 being “strongly agree”. a) The local administration prioritizes community needs. b) The local/district officials are trustworthy for utilization of funds for the development of the settlement. Interpersonal and institutional trust 2.1.2.2 Local leaders consider women's interests (q37b) Participation of women in decision-making at the community level. 18 (This is already captured in questions 10 and 11 of ANNEX A – with “women” as a sub- question in both.) Inclusiveness 17 There is no direct survey question on local leaders securing funds in the TIS+ assessment though the constructed cohesion index assumes (as a proxy) that local officials would be careful spenders if they were the people securing the funds. 18 Also, this is used as a proxy for women’s interests because their involvement in decision-making provides them with a platform to represent women’s interests. 218 Attachment C Household survey questions that were in the baseline community cohesion index and are now excluded in the new CC index. 35. Do you think that young people in your settlement are highly prone to violent extremist groups? (Yes, No, DK) 36. To what extent are extremist groups welcomed in this settlement? (less welcomed, moderately welcomed, not at all welcomed, DK) 37. How often are cultural events organized in this settlement? (> 10 per year, 5–10 per year, 1–4 per year, Never, DK) 38. How often are sports activities/ events organized in this settlement? (> 10 per year, 5–10 per year, 1–4 per year, Never, DK) 39. To what extent do these cultural events contribute to the following: (peaceful coexistence, social integration, tolerance) 40. To what extent do these sports events contribute to the following: (peaceful coexistence, social integration, tolerance) 41. What is your preferred identity? (only Somali, Somali then member of a clan, equally Somali and member of this clan, only member of this clan) 42. Which of the following did you participate in the past year? (settlement meeting, volunteered to do settlement work) 45. To what extent do you agree or disagree with the following? (We are one nation and not a group of clans, we are proud of our country, life in Somalia is getting better for all people, I am proud to be a Somali, I fear that Somalia will never be a peaceful place for Somali people) 219 Attachment D Household survey questions that were in the baseline good governance index and are now part of the new CC index. 9. In your opinion, has people’s participation in decision-making increased, decreased, or continued to remain the same, compared to the previous year? 10. Over the past year, who was most responsible for decisions concerning community development in this settlement? Who did not participate? Please check all that apply. (local community members, local administration, local community members participating in planning and contracting meetings, district council members, clan elders, religious leaders, ugaas/sultan, women, youth) 11. To what extent were you satisfied with the decisions made during the past year by the following people/groups? (local community members, local administration, local community members participating in planning and contracting meetings, district council members, clan elders, religious leaders, ugaas/sultan, women, youth) 21. On a scale from 1 to 10, how would you rate the following statements? 1 being “I do not agree at all” and 10 being “strongly agree”. a) The local administration prioritizes community needs. b) The local/district officials are trustworthy for utilization of funds for the development of the settlement. 28. On a scale from 1 to 10, how much trust do you have in each of the following to resolve conflicts over shared resources and disputes among various groups in this settlement? 1 being “I do not trust them at all” and 10 being “I trust them a lot”. (clan elders, National Army, militia groups, Peace Committee, Oversight Committee, Central government, religious leaders, local administration, police, local courts, ugaas/sultan, diaspora, district council members) 220 ANNEX III – Further Contribution Stories The following story tools will be used during Phase III and IV as most of the technical coordination work and Sports for Peace and Development activities done by TIS+ are in Wave II clusters. These stories are only an initial view of what will be developed during Wave II data collection and at Phase IV, when completing the synthesis report. CA Story Tool: Technical Coordinators Contribution Story To assess TIS+ contributions to observed results, it is important to establish what TIS+ has implemented against the anticipated steps shown in in the diagram below, Technical Coordinators Contribution Story Flow Graphic. This story and tool helps to address key assessment questions #1c, 1e, 2, 3, and 4. In Kismayo, the capital of Jubbaland, TIS+ has five Technical Coordinators (TCs) deployed to the following ministries: ● Two Technical Coordinators at the Ministry of Interior and Security ● One Technical Coordinator at the Ministry of Youth and Sports ● One Technical Coordinator at the Ministry of Public Works and Housing ● One Technical Coordinator at the Ministry of Gender, Family Affairs, and Human Rights In Baidoa, the interim capital of the South West Administration, TIS+ has two Technical Coordinators located at the Ministry of Interior and Local Governance. There has been good reporting at the activity level for much of 2018. TIS+ TCs have provided lengthy weekly updates of activities. We also asked TCs we interviewed to list their main activities over the past year, cross-referenced with the weekly reports. Jubbaland State: At the Ministry of Interior and Security, the two Technical Coordinators listed their deliverables as follows: 221 ● Drafting the Jubbaland Stabilization Plan ● Drafting the Ministry of Interior’s Briefing Note on Jubbaland Public Expenditure Management Toolkit ● Organizing a Counter Violent Extremism Workshop ● Drafting the Jubbaland State Public Expenditure Management and planning the District PEM Cycle ● Organizing a training on the Public Expenditure Management Toolkit ● Supporting the Sungooni District Community Consultation organized by SSI/OTI ● Delivering a presentation on the Jubbaland Stabilization Plan at the bimonthly Stabilization Meeting in Mogadishu ● Organizing the Jubbaland Quarterly Stabilization Meeting ● Developing the Organogram for the Ministry ● Drafting the Local Government Financial Management Manual At the Ministry of Youth and Sports, the Technical Coordinator’s activities and deliverables were listed as follows: ● Developing the Ministry’s Strategic Plan ● Supporting the development of the National Sports for Development Framework ● Supporting the Kismayo Bridging Plan Consultation Workshop ● Presenting on the Sports for Development Framework at the Coordination Meeting in Mogadishu ● Supporting the Kismayo Book Fair ● Supporting the Jubbaland Youth Conference in Doolow ● Organizing tournaments for Girls’ Basketball and Boys’ Soccer The Technical Coordinator at the Ministry of Women, Family Affairs and Human Rights listed the following deliverables: ● Developing the ministry’s three-year strategic plan ● Contributing to the drafting of the Sexual Offenses Bill ● Drafting Standard Operating Procedures for Prevention and Response to Gender-Based Violence ● Delivering a training on Leadership and Management for Women Leaders ● Organizing a workshop for civic engagement on Women’s Constitutional Rights ● Supporting the opening ceremony of the Garbahaarey Women’s Center The Technical Coordinator at the Ministry of Public Works and Housing is relatively new. However, in addition to his supervision of donor-supported construction of public buildings currently undergoing rehabilitation, he is also working on a capacity building plan and a strategic plan for the ministry. South West State: In Baidoa, the two Technical Coordinators at the Ministry of Interior and Local Governance listed the following among their deliverables: ● Developing the Stabilization Strategic Plan for South West State ● Developing the Local Government’s Employment Regulations and the Ministry’s Organogram ● Developing TORs for the ministry’s departments ● Developing the ministry’s Human Resources Policy ● Developing the Finance Policy ● Drafting the Local Government Financial Manual for Trainers ● Delivering leadership training in Hudur and Berdale Districts ● Delivering induction training for the new District Council members in Hudur 222 Assessment Question #1: Did TIS+ contribute to any of the following anticipated results? a) Community ownership of development projects and processes? b) Effective shared asset management in the communities in which they were implemented? c) Improved delivery of services targeted by TIS+? Improved satisfaction with delivery of services? d) Reduced conflict over resources? e) Greater citizen confidence in government? f) Reduced support for Violent Extremism? Are we observing higher level results/is there evidence that higher level results can be achieved? It is worth reflecting on the results to which TCs were designed to contribute, based on TIS+ documentation. TCs were designed to improve existing TIS+ interventions and to improve coordination in government to contribute to program objectives one and two: 1. Increase confidence in governance based on equitable participation in decision-making and management of community assets. 2. Empower community and government representatives to engage with private sector and development actors in collaborative process for community growth. Ultimately, the state-level TCs alongside other TIS+ interventions, are aiming to contribute to increased confidence in state government. There is a mixed picture of confidence in state government from our household survey, but these data do not entirely cover South West State and Jubbaland. Moreover, the results chain connecting the TC inputs to these results is so long and is decisively influenced by so many other variables that it would be unfair to expect any contribution at this level in any survey data. Our results chain for the TC work has, therefore, concentrated on intermediate steps, such as improving TIS+ interventions, improving coordination, and, crucially, injecting sustainable capacities in the relevant ministries. TIS+ TCs confirmed that their activities had been focused both on supporting TIS+ interventions and on promoting coordination. For example, the Technical Coordinators were utilized in supporting community consensus planning and the bridging plan in Kismayo. As one states, "I contributed to and participated in several community consensus sessions especially in fragile towns such as Baardhere and also in Afmadow."19 TCs have also worked on processes of vital importance to state government, such as district council formation: "I worked on and finalized the Local Government Act, which is the legal framework for how these administrations will work and what their mandate is. In addition to this, we have developed a range of policies to support their work, including a finance manual and a development priorities framework that the administrations can use to engage their public and determine how to incorporate feedback into what is important for the district to work on."20 There was evidence that government counterparts appreciated the work of TCs and saw them as providing a point of contact with TIS+ programing and promoting government ownership of TIS+ interventions. They also were seen as building capacities in their ministries, but the clear emphasis was on the need for, and importance of, capacity building: "At the national level, the institutions are struggling due to lack of capacity. It is even worse at the state level because for the first time in the history of the country, we are 19 CA KII #19. 20 CA KII #23. 223 trying to have fully functioning ministries in the states. . . . The Technical Coordinators provide their own skills and experiences as required. In addition, they also train and build the capacity of the few available civil servants in the ministries."21 Our broader CA interviews revealed that many other programs that work with government generally received positive feedback on TIS+ TCs from government partners. Some who had encountered TCs in Kismayo, or been involved in events organized by TCs in Jubbaland, were positive about these experiences. Some respondents, however, suggested that following up on these events and deliverables was important.22 Others suggested that TIS+ could benefit from a more explicit focus on capacity building in the work of its TCs.23 This was confirmed by other respondents who said TIS+ advisers had made a positive impact on the development of stabilization strategies. But this should be matched with an assessment of the real capacities, motivations, and leverage of the political actors in the ministries to implement them.24 An approach more focused on improving capacities beyond the lifetime of supported TC roles should include such an assessment. Other donors working on capacity injection in Somalia have noted a failure of capacity interventions in Somalia tailored to specific needs, which are different in different regions.25 TCs themselves did feel that they were building capacities in their ministries, but felt that the needs in this respect were very large: "We provide a lot of support, especially in terms of capacity for the ministry, because the skillset is not available. South West State and the system of administration is less than five years old and when you think of a civil service in a country, you’re talking about building up capacity over 50 years or more."26 Interviews with TCs also revealed some uncertainty about what their role was meant to be in the program, suggesting they had limited “visibility” on how TIS+ decision-making takes place. They sometimes felt “there is lost opportunity to take advantage of the goodwill and sort out any potential misunderstanding or managing expectations” within ministries."27 What else could account for the change/will affect results? The next stage in our analysis is to ask what other influencing factors could account for the changes observed. These include other externally funded advisers as well as positive and negative factors affecting coordination and capacities in the relevant ministries. Compared to other areas such as Mogadishu, Puntland, and Somaliland, there are fewer Technical Coordinators or Advisors deployed in ministries in both Jubbaland and South West State. Qualitative research showed that in most of the ministries where TIS+ TCs are deployed, they were often the only ones at the ministry except for the Ministries of Interior in both states, where there were a few other TCs deployed by International Development Law Organization and UNJPLG. TIS+ coordinates very closely with programs such as SSF, which deliberately does not duplicate advisers in the same ministries where TIS+ TCs are placed. As a result, the TCs contribution to the ministries is more visible. Levels of existing capacity within the ministries where TCs are based has a clear impact on the contribution TCs will make. One of the main challenges that the Federal Member States must contend with is the capacity gaps in the civil service. The advent of federalism has meant that the powers and services have 21 CA KII #43. 22 CA KII #3, 11, 16. 23 CA KII #11. 24 CA KII #6. 25 World Bank. (2015). Somalia: Capacity Injection Project (English). Washington, DC: World Bank Group. 26 CA KII #24. 27 CA KII #19. 224 been devolved to the regions. But this did not come with any transfer of capacity or resources from the center. Mogadishu itself struggles with lack of capacity in the federal civil service. However, there has been heavy investment through programs such as the Capacity Injection Program by the World Bank and the deployment of short-term and long-term technical assistance by a variety of donors. Jubbaland was the first of the Federal Member States to be established because of its relative stability. It has seen a steady influx of professionals from Mogadishu and abroad moving to Kismayo. However, there has been no major donor investment in the capacity of the civil service beyond programs focused on public financial management. Short-term technical assistance has come through IOM's Migration for Development in Africa, especially in public health care, but there remains a serious lack of capacity especially at the ministerial level because of the limited available wage bill and the lack of donor investment compared with Mogadishu and Puntland. The South West administration remains in a condition of some uncertainty, especially with plans to move its capital to Barawe. The government, including the Presidency, has limited authority beyond the Baidoa airport. Because of this, the capacity of the ministries remains limited and is further compounded by the lack of resources from the federal government or international community to develop the capacity of the local governments. It is, however, encouraging to note that, as part of the Public Financial Management reforms, a Civil Service Commission has been established. This may improve investment in the civil service. Assessment Question 2: Were TIS+ approaches used in the cluster instrumental in contributing to the achievement of TIS+ results described in Question 1? It is important to consider elements of the process for delivering the support TCs offer because the way technical advice is delivered is important in ensuring that the support does not create unsustainable transaction costs for ministries. Best practice in delivering technical assistance has long recommended against “setting up parallel management and delivery systems” because "this can undermine local systems, diffuse consultant accountability (which should unambiguously be to the partner) and distort public sector salaries."28 This has also been identified as an issue in Somalia, where a 2017 Federal Government of Somalia, World Bank, and UN workshop Sharing Lessons on Salaries and Stipend Payments found that there continues to be a significant presence of externally financed staff embedded within the civil service operating with different pay scales, contractual arrangements, and lines of accountability.29 Some of our donor, peer program, and expert interviewees confirmed that the establishment of parallel systems, significantly in paying embedded consultants far outside civil servant pay scales, was a problem in Somalia. Many donor and program-funded embedded advisers were not being recruited through the civil service commission, or being paid per, or at least with some attention to, the harmonized salary scale. TIS+ TCs are paid per, or in reasonable proportion to, civil service pay scales. TCs and TAs supported by TIS+ report to their respective Director Generals in the ministries in which they work and report to the TIS+ office regarding their cooperation with the ministries. Evaluations and reviews of evaluations of technical assistance and capacity building interventions emphasize the following: "TA should resist the temptation to simply fill gaps or ‘get the job done.’ Even when TA has the 28 K4D. (2018). Lessons from Donor Support to Technical Assistance Programs’ Knowledge, Evidence, and Learning for Development: (K4D) Helpdesk Review Report for DFID. 29 Federal Government of Somalia. (2017). Use of Country Systems: Roadmap for Somalia, p. 20. 225 short-term objective of supporting partners to complete particular tasks, it should ensure that it delivers skills, systems, and structures that outlast the intervention."30 It is not clear from TIS+ reporting to what extent TIS+ TCs are filling gaps without leaving sustainable systems and capacities. But the reporting of TC work is disproportionately activity-based. Some interviewees were concerned that advisers in ministries might be personally dominating work that should be led by civil servants, and that it was therefore unclear whether mechanisms and systems existed to allow the capacity to outlast the coordinator’s positions.31 It is therefore difficult for us to gain a full picture of the extent to which TIS+ TCs are building sustainable systems and capacities based on data from these two clusters alone combined with TIS+ reporting, and we hope to gain a fuller understanding of the process of the TC’s work in the next phase of data collection, when we will be focusing on the substantial TC effort at the federal level in Mogadishu. 30 K4D. (2018). Lessons from Donor Support to Technical Assistance Programs’ Knowledge, Evidence and Learning for Development: (K4D) Helpdesk Review Report for DFID. 31 CA KII #1, 6, 10. 226 CA Story Tool: Sports for Development and Peace Contribution Story To assess TIS+ contributions to observed results it is important to establish what TIS+ has implemented against the anticipated steps shown in in the diagram below, Sports for Development and Peace Contribution Story Flow Graphic. This story and tool helps to address key assessment questions #1f, 2, and 4. TIS+ has implemented several sports-related activities in the period under assessment. These have included rehabilitation of the women’s basketball court in Garowe and the rehabilitation of basketball courts in three strategic districts in Banadir. In 2016, TIS+ supported preparatory basketball training camps for 119 young women (including in Kismayo) and then supported the Somali National Olympic Committee to organize a national women’s basketball tournament, which took place in the court rehabilitated by TIS+ in Garowe. Working through federal and regional Ministries of Youth and Sports, TIS+ supported logistics, transport, and accommodation for the second national men’s soccer tournament in December 2017. Both these tournaments generated significant interest and impact on new social media. Our respondents confirmed that the men’s soccer tournament had been a major event that the nation had stopped to watch.32 In 2018, TIS+ invested in a technical expert to work on a Sport for Peace and Development and Peace Strategy and Implementation Framework for the Ministry of Youth and Sports (MOYS). This framework was available in draft to the assessment team, but had not been formally finalized at the time of writing. Since most of these activities have taken place outside the focal clusters (Lower Juba and Lower Shabelle) for this wave of data collection, we have used this wave to build an understanding of how sports are viewed in these clusters, as well as conducting interviews with other donors and specialists involved in sports at the Somalia-wide level. We have looked at TIS+ sports related work in Jubbaland, where TIS+ supported the preparatory work for the 2016 basketball tournament and has a technical coordinator in the Jubbaland MOYS. We have also looked at other sources of data supporting the Sports for Development and Peace (S4DP) results chain in Somalia. We will build up this evidence in the next wave of data collection. 32 CAKII #22. 227 Assessment Question 1: Did TIS+ contribute to any of the following anticipated results? a) Community ownership of development projects and processes? b) Effective shared asset management in the communities in which they were implemented? c) Improved delivery of services targeted by TIS+? Improved satisfaction with delivery of services? d) Reduced conflict over resources? e) Greater citizen confidence in government? f) Reduced support for violent extremism? Are we observing higher level results/is there evidence that higher level results can be achieved? The first consideration in asking this question in Contribution Analysis is whether we observed the types of result we would expect if the TIS+ program was working as planned and the higher-level objectives to which it aims to contribute were being realized. The traditional approach would be to look at all the data to relevant higher-level objectives. In this case, we have not yet collected data in all our clusters, and of the clusters where we have data, TIS+ has only implemented sports interventions in Kismayo. Therefore, we will consider some results from Kismayo. Very high proportions of respondents in our survey in Kismayo reported that security had improved (98%). Respondents reported that young people in the settlement prone to violent extremism increased from 5% to 31% (p – value = 0.000). There was an 8% decrease in the percentage of respondents reporting that violent religious extremists were not at all welcomed. In both settlements for this wave of data collection and at baseline, survey respondents overwhelmingly (92–100%) reported they thought that sports events contributed to peaceful coexistence, social integration, and tolerance. Our qualitative interviews also included many positive comments about the value of sports and sports facilities, including the availability of the stadium in Kismayo and the potential of sports to facilitate reconciliation. At the overarching level, there is some evidence supporting the capacity of sport to counter radicalization in contexts other than Somalia. A study cited in the TIS+ draft S4DP strategy and implementation framework reviews literature on sports interventions in the UK and internationally (but not in Somalia or in conflict-affected settings) found that “participating in sport represents a structured, positive form of social engagement, thereby promoting an investment in communities and wider society.”33 The paper also acknowledged that, while there was a consensus in the literature that sport was a useful addition to counter-radicalization programs, it was very difficult to “determine the impact of the sport participation itself.” This was because sports interventions were often combined with other aspects, such as counseling and cognitive behavioral therapy.34 What else could account for the change/will affect results? The next stage in our analysis is to ask what other influencing factors could account for the changes observed. These include other stabilization and development programs as well as positive and negative factors in the context around the intervention that might have important roles in determining outcomes. In the case of Sport for Peace and Development, it is important to consider the other actors working on sport interventions in Kismayo and nationally. Our assessment indicated that although many stabilization and development actors have supported sports activities in an often ad hoc manner, efforts to develop more consolidated programming approach to sport interventions are nascent. 33 Cara Richardson, Paul A. Cameron, and Katherine M. Berlouis. (2017). The Role of Sport in Deradicalisation and Crime Diversion. http://journals.sfu.ca/jd/index.php/jd/article/view/123/101 p42. 34 Ibid. 228 TIS+’s own stakeholder mapping, conducted by the S4DP specialist in 2018, revealed a range of actors working on sports in Somalia and identified a wide range of actors: ● UNSOM, UNFPA, and UN Habitat, which use sport as a tool to engage women and youth ● The EU, which is planning to build cultural centers with sport as a component ● Somalia Olympic Committee, which is planning to develop a Sports for Peace program (2017– 2020) ● The Pact SHIIR program, which piloted pedagogical sport drills in Jubbaland ● IoM, which has used sports interventions in its programs for rehabilitating Al-Shabaab (AS) defectors ● CARE Somalia, Mercy Corps, and Concern Worldwide, which were also identified as working or having worked on sports35 ● Other donor, program, or implementing partner actors such as US OTI, GIZ, The Somalia Stability Fund, and the Nordic International Support Foundation who have worked on sports interventions or have plans to do so This wide range of actors, combined with capacity challenges in the Ministries of Youth and Sport, suggest that the TIS+ approach of working on a strategic framework for S4DP for MOYS, as well as providing technical support to MOYS, is appropriate. The TIS+ approach is also consistent with Somali policy frameworks, including the Somali National Development Plan, the Somalia Youth for Peace Pact, and the UN Youth Strategy for Somalia. UNESCO is supporting the development of a national sports strategy. However, coordination amongst actors engaged in youth interventions is a challenge, especially given the capacity challenges in MOYS. The TIS+ mapping and our own assessment revealed several new programs and initiatives working on developing more comprehensive frameworks and approaches for sports interventions in Somalia, notably UN Habitat. Because sport is an area that cuts across units and activity areas for donor coordination, setting up a less formal networks for donor and program information sharing and collaboration may be sensible. At the Jubbaland level, the range of actors who have worked/are working on sports interventions will certainly have cumulatively affected our results, which show that respondents are aware of sports venues and events and believe that they contribute to interaction and social cohesion. The Kismayo sport stadium, mentioned by several Kismayo interviewees as a positive asset,36 was reconstructed by NIS in 2014 with UK funding and has more recently been upgraded in 2017 by the American Refugee Committee and UNHCR. Sports tournaments were also mentioned by respondents as positive developments. TIS+ will certainly have contributed to these through its basketball tournament (which was in 2016) and the men’s soccer tournament in 2017, which was won by Jubbaland.37 However, there are many other actors supporting tournaments in Jubbaland, which may have had a larger effect on responses because they are more recent. For example, in May 2018, Save the Children Somalia, in partnership with MOYS and the Ministry of Education of Jubbaland State, co-organized an inter-school sports tournament for 14 public schools in Kismayo. The football and athletics tournament was funded by the German Federal Foreign Office.38 Any results observed will also have been substantially affected by factors in the political economic and security context that affect sport events specifically. There are many supporting factors in the Somali environment that would enhance the effectiveness of sports interventions for youth. The fact that Somalis 35 TIS+. (2018). Sport for Development and Peace Strategy and Implementation Framework stakeholder mapping PowerPoint presentation. 36 39: K_K_WGL. 37 https://www.kawowo.com/2017/12/24/jubbaland-wins-somalia-inter-states-championship/ 38 https://somalia.savethechildren.net/news/enhancing-integration-through-inter-school-football-athletics-tournament-kismayo 229 are big sports fans was mentioned by a number of our expert, peer donor, and program interviewees.39 A number of respondents noted that there were positive Somali role models on the international sporting arena, such as Mo Farah (athletics), Abdi Bile (athletics), Maryam Nah Muse (athletics), Abdisalam Ibrahim (soccer) and Mukhtar Ali (soccer) and other Muslim sporting heroes such as Mo Salah who were wildly popular in Somalia.40 This point was also made in the TIS+ S4DP strategy development. A further positive factor, also made by TIS+, is the existence of several private sector sports football stadiums and investors in sports events, including banks and telecom companies. This means that sports facilities and events would take place without donor intervention in Somalia. These interventions are sustainable after the initial investment. There are also threatening and challenging factors in the Somali environment that negatively affect sporting interventions and pose serious risks to them. These are related to AS threats and attacks and to the broader acceptance of women’s involvement in sport. In 2018, AS ordered the closure of football pitches, and over 30 did close in Mogadishu. AS also handed down terms in AS prisons for some of those defying these orders.41 These orders are an effective way for AS to demonstrate and project its symbolic power into areas not formally under its control. There have also been AS attacks on sporting events, such as the bombing of a friendly match in a Barawe football stadium in April 2018, which killed five people.42 This does not indicate that AS has an inherent opposition to football. There have been stories reporting that AS allows and encourages the playing of "halal" football between AS fighters and local teams in areas under its control.43 However, it does illustrate the security risks of public events and venues, especially those connected to government, and the need to protect participants. Our interviews suggested that Somali society is generally accepting of men's participation in sport events. However, the participation of women and girls is much more controversial.44 AS prohibits women and girls from playing sports, but disapproval of women and girls’ participation is much wider than AS. For example, before the TIS+-supported basketball tournament in Garowe in December 2016, the Somali Religious Council, a group of influential Somali clerics, issued a warning to women not to engage in sports: “We warn that the women basketball violates the Islamic law, culture and its values, and it is a place where women can be easily corrupted."45 The tournament was conducted with high security costs. From a “do no harm” perspective, this has clear implications for the protection of women and girls’ safety during sports activities and their protection from subsequent stigma or threats because of their involvement in such activities. This reconfirms the importance of having a framework to guide sports interventions and for such a framework to include an emphasis on protection of participants. The TIS+ draft framework points to the International Safeguards for Children in Sport.46 These may need to be reworked for conflict-affected settings. In sum, there are several other actors who are engaged in sports activities in Somalia, including donors and development programs, but much of this work has been ad hoc in nature. TIS+ could certainly maximize its impact in this area by playing a coordination role among donors and programs, working with the MOYS. The popularity of sport among Somalis and the proliferation of private sector venues suggest that sport would be positively regarded by Somalis with or without interventions by donors and programs, and are likely to enhance the impact of interventions. However, AS attacks pose a risk to sports venues 39 CA KII # 7, 13, 19. 40 CA KII # 17, 26. 41 https://www.aljazeera.com/indepth/features/somalia-al-shabab-shuts-football-pitches-mogadishu-180607045144830.html 42 https://www.aljazeera.com/news/2018/04/blast-somalia-stadium-kills-football-fans-180413070232458.html 43 https://www.aljazeera.com/blogs/africa/2014/06/99086.html 44 CA KIIs #22. 45 https://www.voanews.com/a/somali-clerics-warn-women-aginst-playing-basketball/3646679.html 46 https://www.unicef.org.uk/wp-content/uploads/2014/10/International-Safeguards-for-Children-in-Sport-version-to-view￾online.pdf 230 and events, and social and pervasive resistance to the participation of women in sports threatens to undermine the objectives of sports interventions, especially those engaging women. Assessment Question 2 Were TIS+ approaches used in the cluster instrumental in contributing to the achievement of TIS+ results described in Question 1? This is an important question where sports interventions are concerned because there was a consensus among our informants working on sport that conducting or facilitating sports events alone was not enough to deliver desired results. Physical education does deliver positive health benefits by itself, but the S4DP framework is not a health intervention first and foremost. There may be some inherent benefits for social cohesion of providing sports venues and events. As one business leader in Kismayo noted: "People from different clans meet and interact with each other in places such as the sports stadiums, restaurants, businesses, and mosques. These venues provide an opportunity for people with different views to exchange ideas, learn from each other, reconcile and bridge differences."47 However, as one respondent in Kismayo noted, simply having sports venues is not enough: “Awareness creation is important as well as having stadiums and youth centers.”48 This message was reinforced by several interviewees who had worked on, or had an interest in, sports. As one specialist in youth programing put it, “It is important not just to support events—it has to be more than kicking a football around.”49 It was seen as very important for these interventions to be conducted in a purposive way. As much as bringing people together in events and tournaments could support a sense of shared identity, these events needed to be more intentional in their design to be fully inclusive, as well as to avoid potential perverse outcomes from interstate games that could exacerbate regional tensions.50 The S4DP approach, which is being developed by TIS+, focuses on using sport to deliver or to boost the delivery of other skills and messages. These can be through side-by-side programming in which educational, civic, or vocational training is provided alongside sport, or through “pedagogical sport drills” in which sport training sessions are used as a platform to diffuse educational and life skills messages. This approach appears to have been used by TIS+ in its support to the men’s football tournament in December 2017: "The S4PD approach uses sports to create social change, to facilitate community development, to increase cohesion, and to support peace and conflict mitigation efforts. . . Four S4DP drills were conducted to show how working together can facilitate healthy competition that will extend off the playing field. . . Before today we were just playing football for health and personal skills, but today we have learned that sports are a great tool for mitigating conflict and fostering peace,” said Mohamed Abdullahi, a young player from Banadir region".51 The full suggested S4DP approach being developed by TIS+ proposes that sports activities conducted alongside, or hand in glove with, education or vocational training and civic engagement are the best approach for stabilization outcomes.52 It is worth considering the evidence base is for those other approaches in Somalia. There is some evidence that purposive tailoring of the format of sport activities to make them more inclusive and to introduce conflict mediation skills could enhance social cohesion among youth, at least at the level of perceptions. The PACT SHIIR program engaged 480 male youth in sports activities aimed at 47 K_K_BL. 48 41: K_K_YA. 49 CA KII # 22. 50 CA KII #22, 26. 51 TIS+ (2018). Quarter 1 Progress and Financial Report. 52 TIS+ (June 2018) S4DP with an education lens power point presentation. 231 social reconciliation across the four districts, three in Gedo and one in Juba, in August 2017. It conducted KIIs and FGDs five months later with 48 youth to assess changes in interactions, attitudes, behaviors, and levels of trust. This study found that, before the sporting drills, sports were characterized by fights and violence between clans, as teams were often village- and clan-based with all players coming from the same group. There were no structures specifically for mediating conflicts among soccer teams and injuries were often escalated to elders’ courts. After the intervention, most youth said they had changed their perceptions towards other clans, but only 29% said they had also changed their behavior. After the intervention, the clan-based teams were disbanded and mixed clan teams were constituted, including marginalized clans that previously had no teams. Structures and rules for mediating conflict were created. Youth were given the opportunity to discuss drivers of conflict in their communities, and they reported they had developed a culture of collaboration. Social support had developed among the teams (who help themselves out with money when needed) and the teams had also now attracted private sector support.53 Linking vocational training to countering violent extremism also depends upon the availability of economic opportunities for those trained. Our interviews with experts and peer programs revealed some skepticism about the link between vocational training and even between employment and countering violent extremism. Several interviewees felt that young people were not engaged in political violence because they had nothing else to do. Even when they are provided with vocational training leading to jobs and income they may still be angry at the injustices and lack of services they and their communities have faced.54 Some sports interventions include civic awareness and engagement objectives and activities. Where civic engagement is concerned, there is some evidence from Somalia that giving youth in schools access to civic engagement activities can lead to a reduction in support for and participation in political violence. A rigorous Mercy Corps evaluation of the Somali Youth Leaders Initiative in Somaliland found that, while improving access to secondary education alone decreased participation in political violence by 16%, it increased support for political violence by 11%. However, when access to education was combined with civic engagement activities that allowed youth to conduct community action campaigns, both participation and support for political violence dropped by 14% and 20%, respectively.55 This reinforced previous Mercy Corps research among youth in Kenya that suggested that youth who were civically engaged and took action to address local governance issues were less likely to engage in or be disposed towards political violence.56 However, it contradicts previous Mercy Corps survey data from Somalia in 2013, which suggested that youth who had been more civically engaged were more likely to have engaged in political violence, probably because they had a better sense of their political goals and expectations from government.57 Therefore, although the Mercy Corps evaluation shows the potential positive effects of civic engagement, when combined with formal education for youth it also highlights “the risk of increasing youth’s expectations – specifically around government response and participation in political processes,” which could end up exacerbating rather than reducing grievances.58 The report also highlights that other pieces of the puzzle need to be in place for the positive effects of civic engagement to be felt, notably that 53 CMM/SHIIR Program USAID Somalia. (2018). Contribution of Sports in Social Cohesion (PACT paper). 54 CA KII# 7, 16, 22. 55 Mercy Corps. (2016). Assessing the Effects of Education and Civic Engagement on Somali Youth’s Propensity Towards Violence. 56 Jon Kurtz, “Understanding Political Violence among Youth: Evidence from Kenya on the Links Between Youth Economic Independence, Social Integration, and Stability,” Mercy Corps, June 2011, https://d2zyf8ayvg1369.cloudfront.net/sites/default/files/full_report_-_kenya_youth_and_conflict_study.pdf. 57 Rebecca Wolfe and Jon Kurtz, “Examining the Links between Youth Economic Opportunity, Civic Engagement, and Conflict: Evidence from Mercy Corps’ Somali Youth Leaders Initiative,” Mercy Corps, January 2013, https://d2zyf8ayvg1369.cloudfront.net/sites/default/files/somaliabrief_2_13_13.pdf. 58 Mercy Corps. (2016). Assessing the Effects of Education and Civic Engagement on Somali Youth’s Propensity Towards Violence, p. 8. 232 governments need their citizens to view them as being on the forefront of improvements in basic services, suggesting that civic engagement may be most effective alongside development projects that are visibly government branded. The main challenge to the suggested mechanism in the S4DP results chain is the focus on individual youth choices and decisions. This assumption was seen as crucial by some of our expert, peer donor, and program interviewees. As one such interviewee noted: "As for sports for development and peace, I don’t know if this would work according to my understanding of how recruitment happens. If we are saying sports will keep youth distracted and keep them away from AS – perhaps this could work when we have heard some cases of individual recruitment. But these are not many. In many cases recruitment to AS is a collective decision, for example of a sub-clan who decides to franchise with Al-Shabaab because they think that in their particular circumstances they best represent their interests. When that happens, it is the whole clan militia that is franchised to Al-Shabaab. They have to undergo training, most of which is ideological indoctrination, and out of this process a good proportion become hard core jihadi extremists."59 Assessment Question #4 Were TIS+ activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? The TIS+ 2016 basketball tournament and the training camps that preceded them are some examples of interventions targeted specifically at women and girls. TIS+ supported basketball training camps that involved 119 young women between the ages of 16 and 25 who came from seven different regions (states) in Somalia. TIS+ supported the Somali National Olympic Committee to organize a national women’s basketball tournament, the first such event in over two decades. The tournament was held on a TIS+ rehabilitated basketball court in Garowe and was watched by an estimated 900 viewers per day, many of whom were women. In addition to the direct participants and spectators, the tournament reached over one million people through social media, was broadcast live on national and regional television, and sparked a significant debate about women and sports across Somalia.60 Our respondents confirmed that these interventions had had a sustainable impact on women’s’ basketball in Somalia, which was progressing at a federal level, as well as at the Jubbaland level, where women’s basketball teams were still playing.61 Other respondents confirmed the huge positive impact that access to sport could have on women and girls who previously had no opportunity to engage in physical activity.62 We have already discussed the security and stigma risks to girls and women of participation in sports, which affected the Garowe tournament. The following two overriding priorities were identified by respondents for protecting women in sports: 1) Training women coaches so that families and communities were reassured that girls were participating in women-only spaces. Male coaches were not acceptable to girls or their communities. 2) Building covered basketball courts so that women can play in protected and private spaces. Our interviewees knew of only one covered basketball court in the whole of Somalia, in Hargeisa.63 59 CA KII# 14. 60 TIS+. (2017). Quarter 1 Quarterly Progress and Financial Report. 61 CA KII # 22, 27. 62 CA KII # 26. 63 CA KII # 26, 27. 233 ANNEX IV – CA Tables, Links to Excel Files CA Story Link File Notes Lower Juba CA Table, Community Planning and Oversight Lower Juba, CA Story Tool completed during Wave I Lower Shabelle CA Table, Community Planning and Oversight Lower Shabelle, CA Story Tool completed during Wave I Initial Sports for Development and Peace (S4DP) CA Story Tool Table S4DP CA Story Tool Table. To be completed during Wave II and submitted during Phase IV in the synthesis report. Initial Technical Coordinator (TC) CA Story Tool Table TC CA Story Tool Table. To be completed during Wave II and submitted during Phase IV in the synthesis report. SOMALIA PROGRAM SUPPORT SERVICES TRANSITION INITIATIVES FOR STABILIZATION PLUS MIDLINE ASSESSMENT, WAVE 1I REPORT DELIVERABLE #21 IDIQ AID-623-I-14-00009 TASK ORDER AID-623-TO-16-00015 May 31, 2019 235 SOMALIA PROGRAM SUPPORT SERVICES (SPSS) TRANSITION INITIATIVES FOR STABILIZATION PLUS (TIS+) MIDLINE ASSESSMENT, WAVE 1I REPORT DELIVERABLE #20 IDIQ AID-623-I-14-00009, TASK ORDER AID-623-TO-16-00015 Submitted: May 31, 2019 Contributors: Sarah Wood (Team Leader), Lucas Malla (Quantitative Specialist), Patrick Okwarah (Qualitative Specialist), Suleiman Omar (Subject Matter Specialist), and Mathias Kjaer (Contribution Analyst) International Business & Technical Consultants, Inc. (IBTCI) In US: 8618 Westwood Center Drive, Suite 400 Vienna, VA 22182 USA +1.703.749.0100 In Kenya: 8th Floor Block C Westcom Point Building Mahiga Mairu Road, off Waiyaki Way Nairobi, Kenya +254 (0) 737 544 912 IDIQ AID-632-I-14-00009 Task Order AID-623-TO-16-00015 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development (USAID) or the United States Government. 236 Table of Contents ACRONYMS ............................................................................................................................ 238 I. INTRODUCTION .......................................................................................................... 241 Project Background ................................................................................................................................. 242 II. TASK ORDER PURPOSE AND OBJECTIVE .......................................................... 242 Task Order Purpose ................................................................................................................................242 Assessment Questions .............................................................................................................................242 III. METHODS ................................................................................................................... 244 III. CONTRIBUTION ANALYSIS AND TIS+ ........................................................... 248 The Role of Contribution Analysis.............................................................................................................248 Testing the Theory of Change for TIS+.....................................................................................................248 Contribution Analysis Stories: Tests for Contribution in Complex Contexts..................................................250 IV. DATA ANALYSIS ...................................................................................................... 251 Influencing Factors and Alternative Explanations.......................................................................................251 V. LIMITATIONS ................................................................................................................ 252 Limitations and Operational Challenges....................................................................................................252 VI. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ........................ 253 Developing Findings, Conclusions, and Recommendations: Midline.............................................................253 Analyzing Stabilization Interventions Using Contribution Analysis...............................................................254 VII. BANADIR CLUSTER ................................................................................................ 255 Background and Context of Cluster ........................................................................................................255 TIS+ Interventions...................................................................................................................................255 Methodological Considerations.................................................................................................................256 FINDINGS.................................................................................................................................................258 Q1: TIS+’s Contribution towards Intended Results ....................................................................................258 Q2: Contribution of TIS+ Approaches ......................................................................................................265 Q3: Government Involvement in Shared-Asset Management .....................................................................271 Q4. Efforts to Promote Inclusion ..............................................................................................................273 CONCLUSIONS.......................................................................................................................................275 Q1: TIS+’s Contribution towards Intended Results ....................................................................................275 Q2: Contribution of TIS+ Approaches ......................................................................................................278 Q3: Government Involvement in Shared-Asset Management .....................................................................279 Q4: Efforts to Promote Inclusion ..............................................................................................................280 RECOMMENDATIONS ...........................................................................................................................282 Strengthening TIS+ and/or USAID’s Contribution to Intended Results........................................................282 Strengthening TIS+ and USAID Approaches.............................................................................................282 Strengthening Government Involvement In Shared-Asset Management ......................................................283 Strengthening Efforts to Promote Inclusion ...............................................................................................284 VIII. GEDO/ BAY CLUSTER ............................................................................................... 285 Background and Context of Cluster ........................................................................................................285 TIS+ Interventions...................................................................................................................................285 Methodological Considerations.................................................................................................................286 237 FINDINGS.................................................................................................................................................288 Q1: TIS+’s Contribution Towards Intended Results ...................................................................................288 Q2: Contribution of TIS+ Approaches ......................................................................................................297 Q3. Government Involvement in Shared-Asset Management .....................................................................302 Q4: Efforts to Promote Inclusion ..............................................................................................................304 CONCLUSIONS.......................................................................................................................................307 Q1: TIS+ Contribution Towards Intended Results......................................................................................307 Q2: Contribution of TIS+ Approaches ......................................................................................................310 Q3: Government Involvement in Shared-Asset Management .....................................................................311 Q4: Efforts to Promote Inclusion ..............................................................................................................312 RECOMMENDATIONS ...........................................................................................................................314 Strengthening TIS+ and/or USAID’s Contribution to Intended Results........................................................314 Strengthening Specific TIS+ and/or USAID Approaches.............................................................................315 Strengthening Government Involvement In Shared-Asset Management ......................................................315 Strengthening Efforts to Promote Inclusion ...............................................................................................315 IX. BAKOOL CLUSTER .................................................................................................. 317 Background and Context of Cluster ........................................................................................................317 Summary of Interventions in Bakool.........................................................................................................317 Methodological Considerations.................................................................................................................318 FINDINGS.................................................................................................................................................320 Q1: Changes from Baseline in Intended Result Areas................................................................................320 Q2: Government Involvement in Shared Asset Management .....................................................................327 Q3: Efforts to Promote Inclusion and Community Cohesion.......................................................................329 CONCLUSIONS.......................................................................................................................................332 Q1: Contribution Towards Intended Results..............................................................................................332 Q2: Government Involvement In Shared-Asset Management.....................................................................333 Q3: Efforts to Promote Inclusion and Community Cohesion.......................................................................334 RECOMMENDATIONS ...........................................................................................................................336 USAID Programs that Contribute to Intended Results................................................................................336 Strengthening USAID Approaches............................................................................................................336 Strengthening Government Involvement In Shared-Asset Management ......................................................336 Strengthening Efforts to Promote Inclusion ...............................................................................................336 X. CONTRIBUTION CLAIMS AND TIS+ ................................................................ 338 Contribution Story 1: Community Planning and Oversight.....................................................................338 Contribution Story 2: Technical Coordinators and TIS+ ........................................................................349 Contribution Story 3: Sports for Development and Peace (SDP) ...........................................................359 238 ACRONYMS AMISOM African Union Mission in Somalia AS Al-Shabaab BRA Banadir Regional Administration CA Contribution Analysis CDD Community-Driven Development CPO Community Planning and Oversight CSR Corporate Social Responsibility CVE Countering Violent Extremism DARS Data and Research Solutions FGD Focus Group Discussion FGS Federal Government of Somalia FMS Federal Member States GEEL Growth, Employment, Economy and Livelihood GESI Gender Equality and Social Inclusion HH Household Survey IBTCI International Business and Technical Consultants, Inc. IDP Internally Displaced Persons INGO International Nongovernmental Organization IOM International Organization for Migration IP Implementing Partner JCCI Jubaland Chamber of Commerce and Industry JSS Jubaland State of Somalia KAQ Key Assessment Questions KDF Kenya Defense Forces KII Key Informant Interview LJ Lower Juba LS Lower Shabelle MIDA Migration for Development in Africa MV&R Monitoring, Verification, and Reporting MOIFAR Ministry of Interior, Federal Affairs, and Reconciliation MOYS Ministry of Youth and Sports NDP National Development Plan NGO Nongovernmental Organization NISF Nordic International Support Foundation O&M Operations and Maintenance OTI Office of Transition Initiatives POC Project Oversight Committees PPP Public–Private Partnership PIKA Pool of In-Kind Activities SDP Sports for Development and Peace SFO Somalia Field Office SIF Somalia Infrastructure Fund SNA Somali National Army SOO Statement of Operations SPSS Somalia Program Support Services SSF Somalia Stability Fund SSG Strengthening Somali Governance SSI Somalia Stabilization Initiative 239 STTA Short-Term Technical Assistance SWS South West State TC Technical Coordinator TIS Transition Initiative for Stabilization TIS+ Transition Initiative for Stabilization Plus ToC Theory of Change UNDP United Nations Development Programme UNJPLG United Nations Joint Program for Local Governance UNOSOM United Nations Operations in Somalia USAID United States Agency for International Development USAID/SFO USAID/Somalia Field Office USG United States Government 240 Figure 1 WASH: TIS+ Baseline/Midline Assessment Geographical Clusters Water, Sanitation, and Hy 241 I. INTRODUCTION Chronic instability in the late 1980s and the collapse of the Somali state in 1991 resulted in two decades of warfare and violence in Somalia. The decades of instability, intercommunal conflict, and humanitarian crisis that characterized the second half of the 20th century in Somalia were marked by numerous attempts at peacekeeping, stabilization, and international relief. Clan conflict, violent extremism, drought, famine, out￾migration, and multiple failed central state governments have typified the obstacles facing a unified, independent Somali state capable of service delivery and crisis response. In 2012, Somalia was finally able to establish a complete central government, after 20 years without a parliament, and held the first presidential election since 1967.1 Despite the continued threats of disorder and violence posed by Al-Shabaab (AS), the Federal Government of Somalia (FGS), in partnership with the international community, has taken impressive strides toward durable peace and nation building. Most recently, Somalia, with extensive support from the international community, held an “indirect” election on February 8, 2017, during which Mohamed Abdullahi Farmajo was elected in the second round of voting, unseating incumbent Hassan Sheikh Mohamud. In November 2016, FGS launched a draft of the first National Development Plan (NDP). This plan is organized per the following objectives: poverty reduction, infrastructure repair, strengthening state capacity, and sustaining political inclusivity. Gender mainstreaming and a focus on youth are both features of the NDP. USAID Strategic Framework for Somalia: 2016–2019 USAID’s development hypothesis for its 2016–2019 strategy is as follows: “If Somali governance systems and processes that enable inclusive governance are strengthened, and the Somali people benefit from expanded, tangible dividends of the state-building project, then the foundation will be built for Somalia’s longer-term recovery.” This hypothesis recognizes that the process of Somalia’s recovery will extend far beyond the life of this strategy. However, it is in USAID’s manageable interest to focus on core strategic areas that are most important for Somalia’s transition and to maintain sufficient flexibility to adapt both the strategic approach and programmatic mechanisms to adjust to changing political, security, and developmental circumstances. To assist Somalia in this historic effort, the USAID Strategic Framework for Somalia 2016–2019 has identified three primary Transition Objectives to guide investments over the 2016–2019 period that support the goal of building durable foundations for a more stable, democratic, and prosperous Somalia: ● Transition Objective 1: Systems and Processes that Enable Inclusive Governance Strengthened ● Transition Objective 2: Service Delivery Improved ● Transition Objective 3: Inclusive Economic Growth Expanded The core assumption built in the strategy is that; if there is an increase in political participation and trust in the political process; effective, accountable, and representative governance institutions are strengthened; and key communities are stabilized; then the necessary systems and processes that enable more inclusive governance in Somalia will be strengthened. In fragile communities, such as those formerly under the control of Al-Shabaab, USAID/Somalia, in partnership with USAID’s Office of Transition Initiatives (USAID/OTI), works to identify quick-impact programming to provide immediate assistance and public assets to improve basic welfare in the months following a successful military operation. These efforts seek to increase the levels of confidence in African Union Mission in Somalia (AMISOM), the Somali National Army (SNA), and the government in each area so that communities do not feel abandoned after military interventions. Through this initial assistance, citizens may conclude that they will not be excluded from services under the FGS and that development dividends may increase. 1 Between 2000 and 2012, Somalia had a Transitional Federal Parliament. A full parliament was constituted in 2012. 242 USAID follows on these initial interventions with a consensus-based community-driven development model that prioritizes equitable participation in decision-making and management of community assets. USAID works in tandem with multiple levels of government (i.e., caretaker and interim administrations, regional and state authorities, and the FGS) to help them build and strengthen these communities. Participatory, consensus-building approaches are used to build citizen confidence in the governing institutions and address critical local-level needs by facilitating the construction and/or rehabilitation of vital community infrastructure (such as roads, bridges, community centers, local administration offices, health clinics, schools, water sources, markets, and playgrounds). Project Background The Transition Initiatives for Stabilization Plus (TIS+) activity is a five-year, US$66.9 million project implemented by AECOM that will run from May 2015 to April 2020. The TIS+ activity intends to support the achievement of USAID’s goal by fostering participatory processes that promote good governance and community cohesion. Given the fragile nature of the Somali government and the existence of an insurgency in parts of South-Central Somalia, USAID is maintaining a continued focus through TIS+ on quick impact stabilization activities at the district and community levels. At the same time, Somalia is aiming to move from crisis response and stabilization to medium- and longer-term development, so TIS+ is also designed to support this aim. Under Somalia’s New Deal Framework, as well as the recently completed NDP, the FGS and the donor community have committed to advancing inclusive political dialogue, improving security, and increasing economic growth—pillars that TIS+ will also support. TIS+ seeks to implement community-centric stabilization activities that (1) are implemented with multiple levels of government (i.e., caretaker and interim administrations, new states, and the FGS); (2) support inclusive participation of stakeholder groups in community decision-making; and (3) encourage shared asset management between citizens and the government to demonstrate the government’s interest in an emerging capacity to contribute positively to citizens’ daily lives. TIS+ activities are designed to contribute to improvements in government service delivery, leading to strengthened governance and community cohesion. II. TASK ORDER PURPOSE AND OBJECTIVE Task Order Purpose The primary purpose of the TIS+ baseline and midline assessments are to generate a better understanding of the extent to which TIS+ interventions have achieved their objectives and contributed to USAID’s goal of increased stability in Somalia through participatory processes that promote good governance and community cohesion. Specific clusters or geographic areas have been strategically chosen to be analyzed to assess the influence that the TIS+ program activities have had on mitigating the effects of violent extremism and insurgency in these high-risk areas of Somalia. This task order is designed to assist USAID in identifying lessons learned from conducting stabilization activities in Somalia. Assessment Questions The TIS+ baseline/midline assessment focuses on two levels of analysis: within cluster analysis (cluster-level analysis) and cross-cluster analysis (comparative cluster analysis). USAID has identified five clusters for analysis: (1) Banadir (Mogadishu), (2) Lower Shabelle, (3) Bakool, (4) Lower Juba, and (5) Gedo/Bay. During the midline, data was collected in two waves. In Wave I, data was collected in Lower Juba and Lower Shabelle, and in Wave II, data was collected in Gedo/Bay, Banadir, and Bakool. This report focuses on findings from Wave II data collected in Gedo/Bay, Banadir, and Bakool. The assessment will answer the following assessment questions (see Table 1). The table below shows the task order questions and how they were adapted at baseline and midline to adjust to the contextual and operational issues of USAID/Somalia and the TIS+ project. 243 Table 1: Key Assessment Questions: Baseline and Midline Cluster-Level Analysis, (Phases I–III) No Key Assessment Question (KAQ) Baseline Midline 1 Did TIS+ contribute to any of the following anticipated results? a. Community ownership of development projects and processes b. Effective shared asset management c. Improved delivery/satisfaction of services d. Reduced conflict over resources e. Greater citizen confidence in government f. Reduced support for violent extremism What is the status of the following at the cluster level? a. Community ownership of development projects b. Shared asset management c. Quality and delivery of service targeted d. Conflict over resources e. Confidence in government f. Support for violent extremism Same as KAQ 1, except for Bakool 2 Were TIS+ approaches used in the cluster (e.g., the community consensus process) instrumental in contributing to the achievement of TIS+ results described in Question 1? Midline only Same as KAQ 2, except for Bakool 3 Did government at all appropriate levels contribute sufficiently to asset management in communities in which TIS+ was implemented? How? Did government at all appropriate levels have both the capacity and the will to contribute? What is the government (at all appropriate levels) contribution to asset management in communities targeted? Level of communities’ inclusion in the following: • Setting priorities and identifying needs • Community contracting • Community contribution • Community advocacy with government to provide services Same as KAQ 3, except for Bakool 4 Were TIS+ activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? What is the status of inclusion of women and youth as applicable to the context of the cluster? • Level of existing gender gaps • Level of women, youth, and minority clan inclusion in planning and implementation of activities Same as KAQ 4, except for Bakool Aggregate and Cross-Cluster-Level Analysis (Phase IV) 5 In the aggregate, did TIS+ contribute to the anticipated results described above? What range of results can be observed across assessed clusters? Midline only Note: Bakool will not be a part of the synthesis report as KAQ is not relevant. 6 Which cluster-level factors contributed most significantly to achievement of results? For example: • Did successful inclusion of women, youth, and minority clans amplify the effectiveness of TIS+ activities? • Were results greatest in newly liberated clusters or in bridge clusters? • Were some kinds of interventions more effective than others in improving citizen confidence or reducing support for violent extremism? Midline only Note: Bakool will not be a part of the synthesis report as KAQ is not relevant. 244 III. METHODS Both baseline and midline employed a mixed-methods approach that integrated a suite of quantitative and qualitative data collection methods implemented over two phases to answer the above assessment questions. This methodology affords USAID a greater depth of comparative analysis across the clusters as well as aggregated analysis at the midline stages (phases III and IV). Data collection methods consisted of desk review/ secondary literature review, a household survey (HH) of 2,151, 79 Key Informant Interviews (KIIs), and 64 Focus Group Discussions (FGDs). Assessment questions were answered using a combination of analyses of data from these four collection methods, via triangulation of evidence and the use of contribution analysis to estimate TIS+’s unique contribution to observed midline results. The team also used a qualitative data analysis tool, known as NVivo, to generate a framework for correlating similar information from FGD and KII respondents within and across clusters. Contribution analysis (CA) is an analytical approach to estimating program effects in complex contexts with many factors and actors influencing intervention outcomes. CA is a good option for a stabilization program because it examines stabilization as a continuum and explores conflict dynamics. CA works within a Theory of Change (ToC) model to look at contribution from several “angles.” It employs an applied ToC approach to look beyond high-level outcomes (goal/objectives) to more specific changes due effected by activities and interventions across diverse project spaces. CA is a pragmatic and methodologically rigorous approach to situations for which the activity is one of many potential contributory factors and where it is not easy to measure attribution directly, as is often the case in stabilization programming. In the case of this Task Order, CA was used to assess the contribution of TIS+ activities to observed outcomes based on the totality of evidence gathered. The essential value of contribution analysis, in this case, is that it offers evidence to reduce uncertainty about the contribution of TIS+ to the observed results through an increased understanding of why the observed results have occurred (or not) and the roles played by TIS+ activities and other internal and external factors.2 A quantitative household survey (HH) survey was administered to residents living in communities where TIS+ activities were planned to be implemented. The qualitative data collection instruments included secondary literature review, in-depth KIIs with diverse TIS+ stakeholders, and FGDs with persons living in the targeted clusters, designed to better understand within each cluster the status of community infrastructure planning. This data was analyzed to create an evidence base that provided comprehensive answers to KAQs 1–4 within clusters and KAQs 5–6 across clusters. A description of each data collection method is provided below as well as a description of the catchment and cluster areas accessible at Midline for Wave II data collection. Secondary Literature Review The first data collection method undertaken by the team was to conduct a comprehensive review of internal 2 John Mayne. (2008). “Contribution Analysis: An Approach to Exploring Cause and Effect,” http://dmeforpeace.org/sites/default/files/0501_Contribution_Analysis_ILAC.pdf. Cluster FGDs KIIs CA KIIs Banadir 36 20 10 Gedo/Bay 20 21 10 Bakool 8 8 10 Total 64 49 30 THE STEPS OF CONTRIBUTION ANALYSIS 1. Set out the cause–effect issue to be addressed in a detailed ToC. 2. Identify the risks to the postulated ToC, including rival explanations. 3. Gather evidence (Baseline and Endline). 4. Assemble and assess the contribution claim, and challenges to it. 5. Seek out additional evidence. 6. Revise and strengthen the contribution story. Figure 2: Steps of Contribution Analysis Table 2: Qualitative Data Coverage, Wave II 245 and external documents of relevance to the TIS+ activity. The IBTCI team worked in coordination with USAID/Somalia Field Office (SFO) and the TIS+ management team to identify, collect, and review the following documents over the course of the evaluation: • Activity designs, training curricula, event agendas, and other key documents for understanding the content of activities • Annual Work Plans (initial and subsequent) • Activity Monitoring and Evaluation Plan (initial and revised) • Quarterly and annual reports • Reports on TIS+ participatory monitoring and evaluation (M&E) activities • Monitoring, Verification, and Reporting (MV&R) Documents • SPSS third-party monitoring and verification reports • Additional documents provided such as Stability Assessments and Community Consensus Reports • External publications by other stabilization stakeholders, donors, academic researchers, and nongovernmental organizations (NGOs) Household Survey During the design phase of the assessments, it was envisioned that a total sample size of 848 (baseline = 424 and midline = 424) per cluster would be large enough to draw statistically meaningful conclusions from the results of TIS+ activities. This sample size was estimated to be sufficient to detect a minimum change of 10% at a power of 80% and significance level of 5%. However, due to limited implementation by TIS+, it was reasonable to limit assessment of TIS+’s contribution only to settlements where there were interventions3. This had the implication of reducing the effective cluster level baseline sample sizes that would be used together with the midline data for comparative analysis4. There were also significant variations in findings between settlements in all the clusters other than Banadir – which implied limiting generalizability to settlement level. As a result, we recalculated significance levels, at cluster and settlement levels, which widely varied between 14% and 58% (see ANNEX D). Significance level may be defined, in simple terms, as the chance to make error in distinguishing between significant and non – significant findings. What has been used commonly in practice, and may be understood as the standard, is 5%. Therefore, 58% would imply a huge chance to make error in assessing significance of findings. Recognizing this limitation, we used penalized likelihood regression approaches5 to limit bias due to data sparsity and to emphasize findings that were significant at 5%. We did not reject non-significant findings at 5% since the estimated p – values were less than the recalculated significance levels. We interpreted them where it was necessary. If the threshold was 5%, a number of true findings would wrongly be rejected and hence missed. The survey instrument for TIS+ was tailored to ensure that it accurately reflected TIS+ activities and their expected outcomes, while also ensuring that the assessment questions could be analyzed at baseline and midline. At baseline, data were collected in 12 settlements of the Banadir cluster, four settlements of the Gedo/Bay cluster, and three settlements of the Bakool cluster. TIS+ activities, however, were only implemented in a small fraction of the baseline settlements for these clusters. The midline assessment was conducted in settlements where TIS+ had at least some activity, including consensus building or planning meetings. We also added non-TIS+ settlements for comparison purposes. Tables 3a-c, on the following page, show the distribution of sample sizes used for cluster level baseline and midline comparative analysis. Table 3(a): Banadir Cluster Sample Size for Wave II, Baseline–Midline 3 For better utilization of data, we still assessed changes in settlements which did not have any TIS+ implementation. They were used for comparison purposes and not assessment of attribution. 4 For instance, to assess TIS+’s contribution in Banadir, we used a baseline sample size of only 77 while maintaining the midline sample size of 424. 5 Cole SR, Chu H, Greenland S. Maximum Likelihood, Profile Likelihood, and Penalized Likelihood: A Primer. Am J Epidemiol. 2013 09/16;179(2):252-60. 246 Banadir Cluster TIS + Settlements Baseline Midline Hawlwadag 18 99 Wadajir 34 177 Warta Nabada 25 148 Total 77 424 Banadir is assumed to be homogeneous due to its size and characteristics. Settlements were classified into two groups: TIS+ and non-TIS+. Non-TIS+ settlements were used as a comparison group rather than a control. HH analysis did not assess attribution but examined changes in all the settlements that were assessed during the baseline. Table 3(b): Gedo/Bay Cluster Sample Size for Wave II, Baseline–Midline Gedo/Bay Cluster TIS + Settlements Baseline Midline Dinsoor 82 116 Baardhere 233 309 Total 315 425 There was limited implementation at the time of data collection of TIS+ interventions in Gedo/Bay cluster as well. Non-TIS+ settlements were used as a comparison group similar to the baseline. Table 3(c): Bakool Cluster Sample Size for Wave II, Baseline–Midline Bakool Cluster All Non-TIS+ Settlements Baseline Midline Xudur 175 148 El Barde 37 36 Wajid 240 240 Total 452 424 There was no implementation at the time of data collection in the Bakool cluster, so instead we examined changes in all the settlements that were assessed during the baseline. The reported changes will be compared to the baseline measures. Key Informant Interviews The IBTCI team and Data and Research Solutions (DARS), a contracted Somali firm, conducted 79 KIIs at midline in Wave II. This included 49 targeted KIIs for the three clusters, and 30 additional KIIs relevant to CA. A major advantage of the KII method, compared with other forms of qualitative research, was the opportunity it provided for in-depth exploration of the knowledge and experience of individuals directly exposed to TIS+ and stabilization work in Somalia. This was particularly important for employing contribution analysis. In Wave II of the midline a similar number was achieved. The selection of the KIIs was done in collaboration with USAID/SFO and the TIS+ implementing partner, AECOM, to develop a representative list of informants well-versed in activity design, implementation, and anticipated results. The following represent the universe of KIIs that were targeted for data collection at the midline stage: Non-TIS + Settlements Baseline Midline Bondhere 13 15 Dharkenley 15 18 Hodan 22 26 Huriwa 99 122 Karan 125 153 Kahda 21 25 Shibis 16 22 Waberi 16 17 Yaqshid 29 34 Total 356 424 Non-TIS + Settlements Baseline Midline Garbahaarey 99 246 El Wak 86 200 Total 185 446 247 • USAID officials in the USAID/SFO • AECOM senior management, regional offices staff and sub-contractors’ staff • Officials from the Federal Member States (FMS) and relevant government actors at the central and district levels in Somalia • Key staff from other stabilization programs that interact with TIS+ or support programming that shares its objectives and geography • Representatives from private sector, nongovernmental, or governmental donor organizations collaborating with or asked to collaborate with TIS+ communities • Participants purposively selected relevant to analyzing the effects of TIS+ program design and activities within the targeted clusters and catchment areas Focus Group Discussions The IBTCI team conducted a total of 64 FGDs during midline Wave II. A major advantage of the FGD method, compared with other forms of qualitative research, is its flexibility for in-depth exploration of the knowledge and attitudes of the group toward specific cluster level conditions relevant to TIS+ activities and processes. This includes participation; initial levels of trust in district, regional, and FGS officials; community contracting and planning approaches; and the inclusion of women, youth, minority clans, and members of other marginal groups in decision-making. Four FGDs were conducted in each selected settlement with the following groups: (1) female elder beneficiaries, (2) male elder beneficiaries, (3) male youth beneficiaries, and (4) female youth beneficiaries. This demography of FGDs was conducted in up to 50% of the TIS+ communities within a cluster. Index Tool An index tool was developed to allow the team to aggregate a series of interrelated questions from the HH survey to establish benchmarks at the cluster level. This index assisted the team in cross-cluster and aggregate analysis as well as providing an analytical anchor to look at the relationship between these factors: key assessment questions, TIS+ interventions, and impacts on stability. These measures were used at midline to assess changes in areas of TIS+ programming and to also look at potential trends evident by comparing baseline measures to midline measures. Assessment of Preconditions for TIS+ Goal The goal for the TIS+ project is to increase stability in the clusters of implementation. For stability to be achieved, two preconditions should be satisfied: (1) increased good governance and (2) increased community cohesion. Annex C explains in more detail the development of index scores to measure these preconditions.6 Household questions were grouped into three themes underlying good governance, including confidence in government, service delivery, and stakeholders’ partnerships as the general constructs defining governance status. Questions were also grouped into three themes explaining community cohesion: network diversity, interpersonal and institutional trust, and social and political inclusiveness. 6 Index score is a single measure derived from a set of variables. 248 III. CONTRIBUTION ANALYSIS AND TIS+ The Role of Contribution Analysis TIS+ operates within a wider system in which it is only one of several factors that potentially affect activity outcomes. The main purpose of contribution analysis in this assessment is to identify the effect that TIS+ programming had in an environment with other actors and influences in the same geographic area at baseline and in the midline stages. Testing the Theory of Change for TIS+ USAID/Somalia has a Theory of Change (ToC) that summarizes at a high level how change is expected to happen in its focal areas of intervention in Somalia. The TIS+ ToC feeds into this higher-level theory. TIS+ had previously worked on articulating a “development hypothesis” at the overarching program level, including a summary of the intervention logic, mechanisms, and assumptions. Contribution analysis requires a ToC that is much closer to the level of the interventions, or what the program is doing, and that describes the sequence of resultant behavior changes that are expected to lead or catalyze outcomes.7 It was necessary to ask at midline whether the program was implemented as originally planned and to assemble evidence that may have influenced these changes. In response to this need, and as required in the task order, we built on TIS+ existing work to develop a fuller ToC for these purposes. The development of this ToC was based on a one-day workshop on June 14, 2017, in Nairobi with TIS+ field and headquarters staff, as well as the USAID/Somalia lead for the TIS+ program. Figure 3 shows the overarching ToC workshopped with TIS+ and shows how the CA Contribution Stories map onto it. The ToC development workshop, as well as TIS+ documents, revealed that “The TIS+ model is strongly based on a community-government programming approach.”8 This model is the basis of TIS+ Objective 1 work, which accounts for the program’s largest expenditure, and of the Objective 2 work, which is a modification designed for areas that have achieved a higher level of stability and had a stronger semblance of district government. TIS+ Objective 3, on supporting Somali engagement, is a cross-cutting objective that feeds into the work under Objectives 1 and 2. Objective 3 allows for activities supporting social reconciliation or cohesion in communities that may be too volatile for Objective 1 activities, where Objective 1 is planning to roll out, or where it is already implementing. It follows a similar process of using community consensus groups but also allows for some activities to be supported without conducting the full community engagement process. Objective 4 integrates gender across all TIS+ interventions. This objective encompasses both the focus on women’s participation in the community consensus process and coordination within and between the district, state, and federal levels of government on gender-sensitive planning and implementation. We have included TIS+ work on capacity development in ministries at the state and district levels on the right-hand side of the ToC diagram. Although this work is not covered by a separate objective and is only partly intended to support Objective 1 and 2 activities, it is also intended to strengthen communication across levels of government on stabilization activities. 7 Davies, R. (2012). “Criteria for Assessing the Evaluability of a Theory of Change,” Rick on the Road Blog. http://dmeforpeace.org/learn/criteria-assessing-evaluability-theory-change 8 Transition Initiatives for Stabilization Plus (TIS+) Process of Engagement document (March 2016). 249 Figure 3: Applied Theory of Change for TIS+, showing CA Stories 250 Contribution Analysis Stories: Tests for Contribution in Complex Contexts TIS+ is a complex program that includes several different types of interventions, allowing it to use an adaptive approach to programming in Somalia. In consultation with TIS+, we broke this overarching ToC down into its core elements for our “Contribution Stories.” These stories will collectively ensure that our analysis provides good coverage of the overarching TIS+ ToC, TIS+ work plan activities, and expenditure. Further detail on the Contribution Stories is included in the findings and recommendations section and in Section X and Annex D. In summary, we selected the following three contribution stories: 1. Community Planning and Oversight Story: This story encompasses the “core offer” of the TIS+ model. This reflects the fact that TIS+ is, at its core, a program delivering infrastructure, but one where the process of planning and delivery engaging community and government is crucial to delivering the expected results. 2. Sports for Peace and Development Story: TIS+ has supported several sports-related activities across all its objectives, from constructing and rehabilitating sports infrastructure to setting up or maintaining and managing committees, organizing sports events, distributing sports equipment, and supporting Ministries of Youth and Sport (MOYS) with technical assistance at federal and state levels. To consolidate and further develop this work, TIS+ enlisted a technical consultant specialist on Sport for Development and Peace in March 2018 to work on a Sport for Peace and Development and Peace Strategy and Implementation Framework for the MOYS. 3. Technical Coordinators Story: TIS+ has several Technical Coordinators (TCs) embedded within ministries, designed to support TIS+ objectives one, two, and three. These are roles at least partly designed to enhance the efficacy of TIS+ interventions, and to deliver wider work in supporting communication between levels of government more broadly. 251 IV. DATA ANALYSIS At midline, it is evident that there are several factors that will influence changes in these environments, and while TIS+ is the largest program in these clusters, it is not the only program working on improving governance, community cohesion, and stabilization in Somalia. At midline IBTCI has mapped carefully the partners, programs, and factors that are most likely to impact and influence the work of TIS+. Because TIS+ is currently not working in all the clusters and settlements, it was important to understand clearly the current conditions in these clusters prior to interventions. At midline, the baseline tools were refined to look at specific project activities and processes in these clusters and settlements and compare TIS+ planning to methods used by other donors. Interviews were targeted to direct beneficiary groups. IBTCI considered all objective areas of TIS+ and looked at more technical interventions, such as capacity development approaches used within the FGS and at the state-level embedding staff in ministries. The interviews conducted at midline and the analysis of midline data reinforce that CA methods are an excellent match for USAID and TIS+. CA is particularly appropriate in situations where “multiple initiatives are working towards the same end” and where complex environments mean that linear attribution of results to any one intervention is “neither meaningful nor accurate.”9 Stabilization programs fit these criteria and are good candidates for the use of CA methods. The application of CA methods is an innovation for USAID/Somalia and TIS+ that can add to improving methods of assessing programming in the future. Influencing Factors and Alternative Explanations It is important to outline the difference between influencing factors and alternative explanations in CA. Influencing factors are used either to support contribution stories or to support alternative explanations for results observed. Influencing factors for stabilization include both contextual factors and other interventions. Contextual factors will be strongly linked to conflict analyses at the intervention level produced by TIS+, other development partners, and academics. Other interventions include donor and partner contributions where relevant in the targeted TIS+ settlements. Other donor programs working on similar activities and overlapping geographically are also potential data sources. TIS+ maps and actively collaborates with stabilization actors in TIS+ areas and this will be analyzed for contribution, impact, and relevance. In addition, there are other initiatives to map stabilization such as host country, development, and humanitarian actors in Somalia, with which the baseline had begun to engage and others that will require more research, and targeted KIIs and FGDs with direct beneficiary groups that occurred at midline. 9 Quinn Patton, Michael. (2008). Utilization-Focused Evaluation: The New Century. 4th edition. Thousand Oaks: Sage Publications. 252 V. LIMITATIONS Limitations and Operational Challenges Complex conflict contexts are challenging. The IBTCI team designed the approach and management plan to mitigate anticipated effects as much as possible through means such as multiple data points for each question, confirmation of facts, customized data collection protocols that helped the team collect data in remote and insecure areas, and staggered data collection time frames. Conducting complex multidimensional data collection requires tight discipline, efficient use of time and resources, close communication with the team and the client, the ability to adapt and solve data collection problems on the spot, and the ability to be an evaluation diplomat when dealing with informants at all levels, especially in senior positions. While considerable, these challenges were not unfamiliar or insurmountable. Anticipating the Somali context, the team identified several important challenges that the IBTCI approach addressed: • The volatility of indicators on Corruption, Governance, and Community Cohesion. These aspects are not directly measured with the data collected but are measured through the perception of stakeholders interviewed. Linked to the security context and other dimensions, perceptions are volatile and may change over time depending on what happened just before the interviews take place. • Difficulty in ensuring rigorous analysis at cluster level due to the low level of implementation of TIS+ and various partners working in each cluster. Some conclusions from data analysis are more valid at the settlement level because there is variation in conditions and political and cultural composition of these districts. This has an impact on the analysis and conclusions able to be drawn at cluster level that the Scope of Work (SOW) requested. 253 VI. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Prior to the inception of TIS+, USAID/Somalia had invested over the years a considerable amount of resources in specific settlement areas in the strategic clusters. Where there has been overlap with TIS, the USAID-funded predecessor project to TIS+, and where TIS+ was active prior to baseline data collection, there was evidence that the impacts of these efforts have shifted the rubric of participation in some settlement areas. KII and FGD interviews with youth leaders, women, elders, minority group members, and district and regional administration staff provided initial indicators of impact from more targeted stabilization assistance by the US Government (USG) and partners. It is clear from findings that these clusters are strategic, critical to state-level and regional stability, and the most heavily impacted by the presence of AS. Based on security at the national and state levels, and delays in planned military campaigns, these clusters did not work effectively for programming purposes, presenting a challenge for USAID as well the implementer. At midline, it was important to identify where TIS+ staff and interventions have carried over from former TIS activities. This factor has made clear the contribution USAID/Somalia has made in linking citizens to more participatory and inclusive forms of programming. It is also important to be mindful that there are a significant number of partners working in these clusters and that their contributions and impact are catalysts and complements of TIS+. Developing Findings, Conclusions, and Recommendations: Midline TIS+ is a multilayered project and this midline analysis focuses on two strategic cluster areas. These are not administrative regions, but rather clusters of districts where AS has continued influence, security is dynamic, and stability is critical. TIS+ focuses on community planning, contracting, and improving the oversight frameworks that strengthen citizen and government relationships at the district, regional, and national level. Activities are structured to foster constructive interaction between citizens and newly established governing authorities, such as basic training of authorities and community leaders, supporting visible small-scale infrastructure, and creating forums for regular dialogue such as town halls and cultural events. TIS+ follows on initial interventions in newly recovered areas with a consensus-based community-driven development model that prioritizes equitable participation in decision-making and management of community assets. Participatory, consensus-building approaches are used to build citizen confidence in the governing institutions and address critical local-level needs by facilitating the construction and/or rehabilitation of vital community infrastructure (such as roads, bridges, community centers, local administration offices, health clinics, schools, water sources, and markets). The findings sections of this report provide analysis of the HH survey data for Gedo/Bay, Banadir, and Bakool, and triangulation of this information with the KIIs and FGDs conducted in the settlements in which data was collected. Contribution analysis was the primary approach applied to analyze the relationship between TIS+ planning and coordination activities with other donors and contextual factors. Quantitative analyses of baseline and midline differences were triangulated with qualitative data and supplemental CA interviews with key donors working on stabilization to ascertain these findings. Findings are primarily aggregated at the cluster level, but where relevant individual settlement information is provided to illustrate to USAID and TIS+ the diversity of these settlements and the anticipated impact on stabilization planning and program implementation. Our research during this wave of the assessment considered that TIS+ is working in an environment where stabilization is a collaborative effort between multiple actors who are contributing to shared objectives. Moreover, the program is operating in a fluid and unpredictable environment, which sometimes offers opportunities to support stabilization objectives as well as throwing serious obstacles in the way of those objectives. The use of CA is particularly appropriate in situations where “multiple initiatives are working towards the same end” and where complex environments mean that linear attribution of results to any one intervention is “neither meaningful nor accurate.”10 Stabilization programs fit these criteria remarkably well. 10 Quinn Patton, Michael. (2008). Utilization-Focused Evaluation: The New Century, 4th edition. Thousand Oaks: Sage Publications. 254 To answer the key assessment questions, we have integrated CA into the presentation of findings and recommendations for assessment questions 1, 2, 3, and 4. The suitability of CA for programming in conflict￾affected contexts has been recognized in several sources, but a literature search has revealed that few (if any) CA approaches have been used in stabilization programs for USAID.11 Using CA methods in this task order is an opportunity to capture TIS+ effects using the most appropriate methods, and to add to the evidence and knowledge base on stabilization programming. Analyzing Stabilization Interventions Using Contribution Analysis Since CA is an analytical approach and not a method of data collection, it relies on the types of data that are being generated to feed the analytical process. TIS+ itself, and the baseline and midline assessments, operates and collects data at the cluster level. Therefore, observed results and contribution stories are generated at this level. Major results observed in each cluster were turned into contribution stories that will also be finalized during Phase IV. This cross-cluster analysis will reach an assessment of the overall contribution to outcomes of TIS+ and help inform USG future programming. To reach robust conclusions, CA contribution stories must meet certain criteria for establishing a plausible link between the intervention and the observed results.12 Each story is structured according to the five criteria identified by John Mayne, the originator of the Contribution Analysis methodology, to analyze the extent of a “contribution claim” (or “plausible association”) between TIS+ activities and the results witnessed during the midline assessment. These criteria include the following: 1. Plausibility: Is the theory of change plausible? 2. Implementation per plan: Has the program been implemented with high fidelity? 3. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? 4. Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? 5. Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? The contribution story will then conclude with an overarching assessment of the strength of TIS’s “contribution claim” to the results relevant to the identified results chain area in the midline assessment. The following “contribution stories” critically examine the extent to which TIS+ has contributed to improvements in anticipated higher-level effects (see Section X). 11 Goldwyn, Rachel & Chiagas, Diana. (2013). “Monitoring and Evaluating Conflict Sensitivity: Methodological Challenges and Practical Solutions.” DFID, CARE, and CDE Collaborative Learning Projects Paper; Rogers, Mark. (2012). “Selecting Evaluation Approaches: Options in Peacebuilding Evaluation.” CARE International Paper. 12 Mayne, J. (2011). Addressing cause and effect in simple and complex settings through contribution analysis. In: Schwartz, R., Forss, K., and Marra, M. (Eds.) Evaluating the Complex: Attribution, Contribution and Beyond. New York: Transaction Publishers. 255 VII. BANADIR CLUSTER Background and Context of Cluster Banadir is the region that covers the city of Mogadishu, the capital of Somalia. Banadir hosts the two main income earners for the Somali government (the Mogadishu Airport and Port) and is the largest business hub. A key political and constitutional issue that remains unsolved is the status of Banadir in Somalia’s federal structure. This lack of clarity has upset the clans that call the region home and there has been intense conflict over autonomy in this region. Political wrangles take up most of the space for discourse in Mogadishu, leaving little room for discussions on security. The emergence of ISIS and the subsequent contest with Al-Shabaab (AS), especially on the imposition of taxes on local businesses, has led to a deterioration of security and threatens progress made thus far by many development actors. The region’s instability is complicated by an intense power play between the private sector, the government, AS, and clan-affiliated military groups. Security in this cluster has deteriorated significantly between the baseline and midline data collection periods. Uncoordinated security efforts between the multiple actors as well as infiltration of AS “Amniyat“ operatives has contributed to the deterioration of perception of security among the city’s residents. The complex security nature and the evolving dynamics continue to impact interventions made by programs such as TIS+. Insecurity in Banadir has increased since the baseline period going by the number of attacks as well as respondents’ perception of security. Facilities rehabilitated with support from TIS+ have been damaged or destroyed by bombings. The shift from targeting government installations to civilian populated areas means that residents are more afraid to engage government and other development actors. This further reduces the opportunity for community buy-in and ownership. TIS+ Interventions The following interventions were the primary activities by which TIS+ sought to engage government and stakeholder partners to ensure that community cohesion and governance improvements were observed in Banadir. In Table 4, we have broken these interventions down by settlement to illustrate to the reader the primary activities conducted in this cluster by TIS+. 256 Table 4: Banadir Pool of In-Kind Activities (PIKA) Activities Cluster Settlement TIS+ Activity/Intervention Banadir Regional Administration (BRA) Abdiaziz* BRA Bridging Plan Stakeholders Consultation Mogadishu Book Fair BRA Bridging Plans Task Force – Roads and District Stabilization BRA Districts Sport Facilities and Youth Engagement BRA Bridging Plans Task Force Capacity Building BRA Bridging Plans Engineering Technical Assessment Study – Roads BRA Abdiaziz District Basketball Mini Stadia Rehabilitation BRA Hawlwadag Mogadishu Hawlwadag District Sayidka Road Gap Filling and Upgrading Mogadishu Hawlwadag District Hawlwadag Capital City Basketball Court Rehabilitation BRA Shangani BRA Department of Urban Planning and Engineering Capacity Building BRA Wadajir BRA Wadajir District Administration Complex Reconstruction Mogadishu Wadajir District Dan-Wadagaha Road Upgrading BRA Warta Nabada Mogadishu Warta Nabada District Warta Nabada Capital City Basketball Court Rehabilitation * Not a Baseline settlement Other development activities and partners have also been active in this cluster. Prior USG investments are evident, and this has influenced data analysis of this cluster. Figure 4 shows the actors and activities that were evident during the WAVE II data collection period: Figure 4: Banadir, Summary of Other Actors and Interventions Actors Activities International Organization for Migration (IOM) International Community of the Red Cross (ICRC) Nordic International Support Foundation (NISF) UNHABITAT Danish Refugee Council/Danish Demining Group (DRC/DDG) World Food Programme (WFP) UNICEF Danguud, ARC Norwegian Refugee Council (NRC) CARE Save the Children International (SCI) Turkish and Egyptian governments School and MCH construction Rehabilitation of government offices Vocational training Vaccination campaigns Private sector CSR campaigns Water, sanitation, and hygiene (WASH) activities Methodological Considerations As mentioned in the methods section of this report, generalizability at the cluster level is possible given the homogeneity of the settlements in Banadir. TIS+ and non-TIS+ settlements were analyzed for comparison purposes only. Table 5 lists the areas that were assessed, and where data was collected for the HH survey. 257 Table 5: Banadir HH Survey Samples, TIS+ and non-TIS+ Settlements Findings compared baseline and midline percentages and the team weighted data to adjust for population differences within the settlements. The team also used regression analysis to identify distinctions between “significant“ and “nonsignificant“ results at midline. In addition to the HH survey, 36 FGDs, and 30 KIIs were conducted in the TIS+ settlements of Hawlwadag, Wadajir, Warta Nabada, and Abdiaziz, and in the non-TIS+ settlements of Bondhere, Dharkenley, Hodan, Huriwa, Karan, Kahda, Shibis, Waberi, Hamar Jajab, and Dayniile. HH Survey Sample TIS+ Settlements Baseline Midline Hawlwadag 18 99 Wadajir 34 177 Warta Nabada 25 148 Total 77 424 HH Survey Sample Non-TIS+ settlements Baseline Midline Bondhere 13 15 Dharkenley 15 18 Hodan 22 26 Huriwa 99 122 Karan 125 153 Kahda 21 25 Shibis 16 22 Waberi 16 17 Yaqshid 29 34 Total 356 424 258 FINDINGS Q1: TIS+’s Contribution towards Intended Results Assessment Question 1: Did TIS+ contribute to any of the following: (a) Community ownership of development projects and processes; (b) Effective shared asset management; (c) Improved delivery/satisfaction of services; (d) Reduced conflict over resources; (e) Greater citizen confidence in government; (f) Reduced support for violent extremism? 1(a): Community Ownership of Development Projects and Processes To examine changes in community ownership of processes and development projects and the contribution of TIS+, the assessment focused on the level of awareness of community planning events, participation across groups in those events, and community perceptions that the projects selected met priority community needs. Based on an analysis of both the quantitative survey data and qualitative interviews, the team found that: 1. In both TIS+ and non-TIS+ areas, level of awareness and participation generally increased since baseline across all the groups with the exception of clan elders. • The majority of FGDs conducted confirmed awareness of, and participation in, community planning events in both TIS+ and non-TIS+ areas. • KIIs conducted also indicated an increase in participation in TIS+ interventions. As a government official who worked with TIS+ from Banadir stated, “The communities were actively involved since these [projects] came from [their] priorities in the first place . . . based on the community action plans. . . . During the early phase of the project, segments of the community met and were asked about their priorities . . . [and] if they were willing to participate in the facilitation of these priorities. Previously, NGOs came with projects and approached the district administration who then executed the project” (KII Interview, Government, Banadir). • As a male elder from Warta Nabada stated regarding work with TIS+, “I believe I took on a great role when I was assigned the monitoring of the playground rehabilitation and evaluated this project. We declined projects that were wrongly conceived and requested them to do better since the project was ours. We were not engineers, but we expressed our thoughts and they acknowledged all our recommendations.” 2. Both TIS+ and non-TIS+ settlements showed strong results on community perceptions that projects were selected based on community planning events and that selected projects met community needs albeit increasing demand of participation of women and minority clans. • Between 2017 and 2018, the awareness of community planning events increased significantly by an average of 58% and 38% in TIS+ and non-TIS+ settlement groups respectively (see Figure 6 next page) from HH surveys. This was 20% higher in TIS+ settlements than in non-TIS+ settlements. • Assessing changes on whether selected projects addressed community priorities in TIS+ settlements was not feasible as only 10% (8/77) of the respondents indicated they were aware of the selection of projects through planning meetings. • At midline, there was a strong level of satisfaction with projects meeting community needs. A large 92% of the HH survey respondents said “some” to “all” selected projects addressed community priorities. • However, FGD respondents reported that greater participation of women and minority clans was needed, particularly in project selection and meaningful decision-making and prioritization. "The only activity that I believe the government does loyally is arranging for public consultations.” – Elder Men FGD, Wadajir 259 Figure 6: Percentage of participants who reported projects being selected based on community planning events 1(b) Effective Shared Asset Management The assessment team examined changes on “effective shared assessment management” along three criteria: (1) community awareness of project oversight committees; (2) community perceptions of the effectiveness of those project oversight committees; and (3) other actors (apart from project oversight committees) responsibility for shared asset management. Based on its analysis, the team found that: 1. Community members were generally more aware of project oversight committees across the cluster, however the perceived effectiveness of POCs varied by settlement and appear to be hampered by a need for additional capacity-building of POC members, particularly women and youth. • Household data reported significant increases in awareness of oversight committees in both TIS+ (+46%) and non-TIS+ settlements (+36%). • However, there were no significant results for more effective management of shared assets by TIS+ oversight committees, while the effectiveness of non-TIS+ oversight committees increased for schools (+9%), market facilities (+13%), sports arenas (+12%), and roads (+3%). • According to TIS+ program documentation and interviews with staff, POCs appear not to have received any substantive training until July 2018, mostly after construction of infrastructure finished. This practice ran counter to the process outlined in TIS+’s own “Operational and Management Guide,” which explains that POCs should be established and trained “before the groundbreaking of the project to enable, and encourage, the POC to play its vital role in an effective way.”13 • Women and youth FGD participants confirmed a desire for more community training and capacity￾building on shared management and oversight. • FGD interviews also cited a need for improved security for some shared asset management roles, noting that AS can influence community access to certain facilities as well as manipulate community involvement in the management of these assets, particularly stadiums and public market spaces. 13 AECOM, “General Principles of the O&M Plans for TIS+ Supported Infrastructure Activities in Somalia,“ (February 2018). 260 • In interviews with elder and young men, there was an interest in increasing the roles and responsibilities of women in managing shared assets. As one youth from an FGD conducted in Hawlwadag stated, “I believe we have to involve women in the management of the shared assets because they deserve it. Earlier, we used to say they are not educated. But now [we know] they are very talented and even better than the educated men are. Men are always corrupting management, so let us give women a chance.” 2. State and local government saw strong increases in their perceived involvement in the management of shared-assets, with particularly strong results in TIS+ areas. • There were strong increases since baseline in the reported roles of state and local government (BRA) in managing shared assets in TIS+ areas, and positive, but less pronounced, increases in non-TIS+ settlements. Technical Coordinators (TCs) were noted as being particularly helpful in working with the BRA. • Encouragingly, this included strong results in the management of roads and sports arenas, which were key TIS+ PIKAs in Banadir. • Likewise, respondents (FGD and KIIs) reported increased levels of community involvement in both TIS+ and non-TIS+ settlements. Generally, government (district administration) is seen as having the first line of responsibility and then delegating management responsibilities of certain assets to community members. • Interestingly, in the HH survey data, the business community—often described as the most important service provider—saw decreased perceived roles in TIS+ and non-TIS+ areas (see Table 6). • This was in contrast to qualitative FGD and KII interviews that mostly reported that NGOs and the private sector were more accountable than state government and BRA regarding the quality and management of shared assets. Table 6: Who else is responsible for the management of the following services/assets? “Consolidating the current activities of the district administration and even increasing the speed in which the administration does things are important factors to consider in managing the public assets effectively. Capacity building seminars for those youths involved in the management and maintenance of the public assets would also help in maintaining these assets more effectively.” – Youth Boys FGD, Abdiaziz Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Clan Elders 0% -1% 3% 3% 6% -1% -6% -7% -6% -5% -5% -5% Militia groups 0% 1% -1% 2% 0% -7% 1% -1% -3% -2% -1% -3% State government 16% 5% 33% 24% 27% 18% -1% -4% 28% 7% 8% 1% Religious leaders -13% 4% 2% 6% 1% 4% 2% -1% -4% -3% -1% -4% Local administration 16% 8% 31% 24% 23% 19% 3% 6% 29% 25% 5% 18% Ugaas/Sultan 9% 5% 8% 6% 5% 6% 2% 3% 5% 2% 2% 3% Diaspora 13% 13% 2% 8% 10% 7% 3% 5% 1% 1% 2% 2% Businesses/Traders -6% -12% -2% -16% -12% -4% -15% -19% -2% -16% -8% -2% Local community -4% -4% -1% -1% 0% 0% -1% -1% 0% -2% 0% 0% NGOs 3% -9% 2% 0% -1% 0% 0% -3% 3% 0% 0% 0% International organisations 0% 0% 0% 0% 0% 0% 0% 0% -2% 0% 0% -2% Youth 0% -1% 0% -1% -4% 0% -4% 6% 0% 0% -8% 0% TIS+ non TIS+ 261 1(c) Improved Delivery and Satisfaction of Services To analyze this result area, the team examined community perceptions around (1) service provision by government and other actors; (2) satisfaction with services provided; and (3) perceptions on “security as a service.” 1. There were mixed results on perceptions of service provision by the state, non-state actors and the business community at the cluster level. • The household data recorded a substantial increase in the perception since baseline that local government (BRA) provided services in both TIS+ (across all services) and non-TIS+ (all services except education). See Table 7 below. • The perception that local people and NGOs/INGOs provided services also saw larger increases in TIS+ settlements. Interestingly, services provided by the business community mostly decreased across both TIS+ and non-TIS+ areas. It is important to note that TIS+ settlements formed the inner core of Mogadishu’s settlements. As such, they benefitted from being the target of government and NGO interventions due to their accessibility and relative safety. • Services by state government decreased across most services in TIS+ areas, except for water, electricity, and financial services (see Table 2 next page). Since BRA does not offer utility services such as water and electricity, the change can be attributed to increased and improved regulation of these services by the municipality. For example, the municipality inspects businesses for licenses and also issues permits for new boreholes and overhead powerline installations. Table 7: Service provision changes between 2017 and 2018 non TIS+ Health Education Water Roads Electricity Sanitation Security Financial services Health Education Water Roads Electricity Sanitation Security Financial services Clan Elders -3% -2% 0% 3% 0% -4% 2% 1% -8% 8% -5% -6% -2% -2% -4% -4% State Government -10% -8% 8% -9% 7% -8% -10% 5% -5% 14% 6% -13% 7% -2% -9% -2% Local people 7% 11% 21% 6% 14% 28% 2% 22% -2% -15% 0% 3% 2% 25% 7% 24% Local NGOs -9% 7% 13% 11% 13% 18% 5% 18% 0% -2% 0% -2% 0% 5% 3% 7% International NGOs -6% 9% 11% 10% 12% 13% 5% 15% 2% -7% 0% 0% 1% 4% 1% 9% Religious leaders 3% -1% 2% 0% 1% 5% 1% 2% -2% 37% -1% -2% 0% -1% -2% -2% Local administration 26% 23% 39% 8% 39% 14% 7% 26% 22% -31% 23% 12% 21% 17% 18% 23% Police 1% 0% 0% -18% 0% -1% 6% 1% 1% 0% 1% -11% 1% -1% 1% -1% Local Courts 0% 1% 1% -2% 0% 1% 7% 0% -1% 42% 0% -2% 0% -2% -4% 0% Business community -15% -29% -34% 11% -25% -7% 3% 9% -19% -64% -27% -7% -30% -12% -7% 10% Ugaas/Sultan 1% 2% 0% 5% 1% 6% 2% 0% 0% 4% -1% 1% 0% 0% -2% 0% Diaspora 8% 7% 4% 5% 4% 6% 0% 14% 1% -2% 2% 3% 3% 3% 0% 6% Foreign Governments 1% 0% -1% 0% 0% 0% 1% 5% 0% 1% -1% 0% 0% 0% 1% 1% Women support groups 4% -2% 2% 2% 0% 24% 2% 2% 2% -1% -1% 0% 0% 11% 1% 1% TIS+ “I think the government is not up to the level of the NGOs or the other service providers because the NGOs and the other service providers have enough funds to implement their projects without fear. The government waits upon the support of the NGOs and as such, you can tell who deserves to be supported.“ – Elder Women FGD, Warta Nabada 262 2. There were mixed results on perceptions of service satisfaction by the local communities, state and non-state actors. • Based on the household data, satisfaction with services saw non-significant decreases since baseline in TIS+ areas across most services, except for small increases for “other than drinking water” (+2%) and “sanitation” (+3%). • FGD respondents widely preferred international NGOs and private sector providers and were often critical of government services, particularly for roads and health. • FGD respondents also felt that government did not do enough to monitor and maintain services. This was consistent in TIS+ and non-TIS+ settlements. 3. In terms of “security as a service,” there was a noted decrease since baseline in the perception of state and federal government provision of security in TIS+ settlements. There was an increase over baseline in non-TIS+ settlements with noticeable gender variations. • Perceptions of security vary by gender, with women consistently rating security as a higher concern than men. • FGD respondents reported a decrease in security in TIS+ areas and an increase from male respondents in non-TIS+ areas. • There was an expressed frustration (KII and FGDs) with the increased insecurity in Banadir, particularly AS incidents. However, most participants in KIIs and FGDs made clear that they believed the responsibility for security rests on the state and federal government. 1(d) Reduced Conflict over Resources To analyze this result area, the team examined changes in perceptions on (1) the level of conflict over resources and (2) trust across various actors to resolve conflicts over resources. 1. There was mixed evidence in resource-based conflicts across the cluster with data either showing that it either decreased, remained stable and/or persisted across the cluster since baseline. • Based on the HH data, there were no significant changes in the perception of decreased conflict over resources in either TIS+ or non-TIS+ areas since baseline. • FGD respondents generally reported decreases in resource conflict. However, responses reveal that conflicts over property, land, housing, and water persist. • There is no evidence that TIS+ programming directly works in this area. There are no direct examples of TIS+ programming aimed at conflict mitigation as a targeted intervention. 2. There were large increases across the cluster in perceived trust placed in local structures (clan elders, peacebuilding committees, local administration, religious leader, and Ugaas/sultans) as well as formal structures (national army) to resolve conflicts. Most importantly, the trust placed in the militia groups and the diaspora to resolve conflicts considerably decreased since baseline across the cluster. • Based on the HH data, there was increased confidence in local structures to assist in conflict resolution (i.e., clan elders, peacebuilding committees, local administration, religious leader, and Ugaas/sultans), as well as the national army across both TIS+ and non-TIS+ areas. Encouragingly, militia groups saw large decreases (-56% and -40%) in both areas as well. • TIS+ works indirectly on conflict resolution, as this is not a targeted area of implementation planning. For example, key initial steps in the community consensus process involve facilitating conflict and “There are people who are always busy making fake documents about ownership of land. In short, there are a lot of land disputes and the government, through the parliament, is yet to pass a good and effective rule which could reduce or eliminate such disputes.” -Youth Boys FGD, Dayniile 263 resource mappings and then discussing these with community members as they identify and select their priority projects. However, the assessment found little evidence of TIS+ directly mediating resource-based conflicts. • From the HH survey, there was decreased trust in diasporas to resolve conflict across both TIS+ (- 22%) and non-TIS+ areas (-40%), which echo findings from secondary studies on macro conflict drivers. • Diasporas play an important role in recovery in Somalia society, but there is evidence from interviews in Banadir that this is sometimes contentious and can cause tensions as well between diaspora community members and Somali’s that have stayed through the conflict. 1(e) Greater Citizen Confidence in Government To analyze this result area, the team examined changes in (1) the team’s “Good Governance Index;” (2) citizen confidence towards government; and (3) perceptions on extent of corruption. 1. Both TIS+ and non-TIS+ settlements demonstrated increased scores on their “Good Governance Index;” with no significant differences between TIS+ and non-Tis settlements. • The themes contributing to the composite good governance index included confidence in government, service delivery, and stakeholders’ partnerships. • The computed scores are presented in Figure 7 below. The closeness of the index scores between TIS+ and non-TIS+ settlements can be explained by the homogeneous nature of the Banadir Cluster. Figure 7: Good Governance Index 2. There was a perceived negative trend in citizen confidence towards state government in both TIS+ and non-TIS+ areas since baseline, but non-statistically significant changes for both federal and local governments amid citizens’ apathy on a host of issues. • Based on HH data, confidence in state government decreased in both TIS+ (-3%) and non-TIS+ areas (-4%). • Both areas saw nonsignificant changes in confidence toward both federal and local (BRA) levels. 41% 39% 54% 51% 0% 10% 20% 30% 40% 50% 60% 70% 80% TIS+ non-TIS+ 2017 2018 +13% <0.0001 <0.0001 regression analysis p - values +12% 264 • FGD and KII respondents reported a long list of frustrations, including perceptions of nepotism and clannism, unfulfilled election promises, corruption, security lapses coupled with fear of victimization by AS, injustices and weak judiciary, poor service delivery, and an outsized influence of the diaspora. 3. Mixed results were reported since baseline on perceptions of corruption across the cluster at various levels of government: local, federal, and state governments with a yearning among the citizens for wide engagement, participation and information sharing and transparency. • FGDs and KIIs confirmed that greater participation and engagement in public forums and public prioritization of shared assets would increase trust levels in local, state, and federal government. • Based on HH data, in TIS+ areas, perception of corruption increased toward federal (+6%) and BRA government (+8) but decreased in non-TIS+ areas (-4% and -3% respectively). • There was also a significant decrease in perception of ”local administration sharing information” in both TIS+ (-17%) and non-TIS+ (-24%) areas. • Based on FGDs, public sharing and information transparency is critical for community confidence, participation, and improved perceptions of governance. 1(f) Reduced Support for Violent Extremism To analyze this result area, the team examined perceptions around (1) youth vulnerability to extremist recruitment and (2) extent the respondents reported extremist groups were “not at all welcomed.” 1. There was an increase across the cluster of the perception of youth vulnerability to recruitment by extremist groups. • There was an increase in respondents reporting that youth were more prone to recruitment by extremist groups in both TIS+ (+23%) and non-TIS+ areas (+14%). While this variation can be attributed to a number of reasons, including the fact that since the TIS+ settlements are in the inner core of the city and primarily inhabited by government offices, Al Shabaab is likely to target youth in this area for its operations. However, a more credible explanation comes from the fact that government security services have been heavy handed in maintaining security in these settlements including multiple cases of extrajudicial killings of young people who operate motorcycle taxis causing a sense of disenfranchisement amongst young people. Al Shabaab has in the past targeted this sense of disenfranchisement to heavily recruit in urban areas. There was no evidence found that TIS+ activities contributed to youth being more susceptible to recruitment in TIS+ areas. This results area is at the far end of the results chain and it is expected that other factors would be more likely to influence the results, and which may need further investigation. • There were non-significant results for people reporting that extremist groups were “not welcomed at all.” • Based on FGDs, there is still evidence that AS employs youth. However, this was less evident than in the interviews during the baseline data collection stage. 265 Q2: Contribution of TIS+ Approaches Assessment Question 2: Were TIS+ approaches used in the cluster, e.g. the Community Consensus Process itself, instrumental in contributing to the achievement of TIS+ results described in Question 1? TIS+ has produced several guiding documents outlining some standardized approaches that the program planned to utilize across its clusters. As these approaches were intended to be program-wide, the assessment team analyzed these approach through the use of “contribution stories” (see Section X). These stories critically examined the “contribution claims” of the extent to which TIS+ contributed to higher-level results through the use of three specific approaches (1) Community Planning and Oversight (CPO); (2) Technical Coordinators (TCs), and (3) Sports for Peace and Development (SPD). While the contribution stories reference examples across the clusters, this section will focus on highlighting additional examples and influencing factors that impacted TIS+’s approaches in Banadir. It will examine TIS+’s CPO approach, which includes the use of community consensus processes; TIS+’s TCs approach; and the use of the “cluster” approach itself for geographically focusing activities in Banadir. Results from TIS+’s “Gender and Social Inclusion” (GESI) approach will be also be briefly examined under Question 4. Community Planning and Oversight (CPO) Approach A detailed description of TIS+’s CPO approach can be found in the Gedo/Bay Cluster Report, as well as in the CPO Contribution Story which provides a detailed case-study of the program-wide CPO approach and examines the strength of TIS+’s contribution claims to higher level results. However, to provide a brief background, the use of community planning and oversight is at the core of TIS+’s programming approach. It emphasizes the importance of the process not just the end product and is key to distinguishing TIS+ as a stabilization program rather than a more traditional development and infrastructure program. In short, the approach specifies how the program will engage community and government actors to plan, implement, oversee, and utilize PIKA activities and how this interaction will help the program achieve higher-level outcome results, such as increased citizen engagement and confidence in government, and reduced conflict over resources and support for violent extremism. TIS+’s specific approach to CPO is articulated in a number of guiding program documents and throughout its periodic reports. A critical feature of TIS+’s CPO process is the formation of POCs. The central role that POCs play in the larger CPO processes is documented in a number of TIS+ documents. It is also highlighted in the final evaluation of TIS+’s predecessor program, TIS, which recommended that “the POCs’ role at the community level should be strengthened and institutionalized. POCs should be provided with adequate training to allow adequate monitoring of projects. The contractor and the community must properly understand the POC’s role.”14 Findings of Results of CPO Approach 1. TIS+ took active steps to ensure that communities in Banadir were supported to collectively identify and prioritize their shared community needs. • The three TIS+ settlements in Banadir all demonstrated particularly strong results on “community awareness and participation.” • There was documentary evidence that TIS+ held a 3-day “Bridging Plan Stakeholders Consultation” workshop with over 60 participants in Banadir. • The midline assessment found an increase of 58% in respondents indicating that they were aware of community planning events and a significant increase in participation by all groups with exception of the clan elders. 14 IBTCI, "Final Performance Evaluation of the “Transition Initiatives for Stabilization” (Tis) Project," ed. Somalia Program Support Services (June 2016). 266 • Likewise, TIS+ settlements demonstrated a very strong level of satisfaction with projects meeting community needs with an impressive 92% of respondents reporting that “some” or “all” selected projects addressed community priorities. • The midline also identified a significant increase in awareness of oversight committees in both TIS+ settlements (+46%); however, notably, there was no change in the perceived effectiveness of those POCs. 2. TIS+ implemented most of its intended CPO steps but substantially deviated from the original design when it came to the establishment, training, and use of the POCs. • Interviews with TIS+ staff indicate that district-level POC members were identified during the last day of the bridging plan workshop. • However, interviews with the same TIS+ staff members also revealed that very limited training was provided to the POC members during these workshops and that they mostly just discussed the anticipated role of the POCs as they looked to recruit volunteers. • While TIS+ had originally intended to form POCs at the district level, the program decided in late 2018 that this approach might lead to a concentration of power in the hands of a limited group of people and thereby present opportunities for corruption and that it would be more prudent to establish Pool of In-Kind Activities (PIKA)-level POCs. Accordingly, PIKA-level POCs were established in November 2018 in Banadir. • This meant that all of the new POCs established in Banadir were established after the completion of construction. This suggests that the POCs had very little formal engagement in the activities in their intended capacities and did not have the opportunity to fulfill their intended role of providing community oversight and regular engagement with local authorities. • While the capacity-building and function of the POCs deviated considerably from the original design, both interviews and the assessment team’s review of sign-in sheets confirm that the POC membership looks to be inclusive of both women and youth, and in most cases included the participation of government officials and representatives from the business community. • In terms of inclusion, the midline assessment confirmed that all TIS+ POCs included the participation of women and youth, with an average membership rate of 40% for women and 30% for youth. 3. There was a strong increase in the reported roles of state and local governments in managing shared assets, particularly in the management of roads and sports arenas, which were PIKAs. • There were also fairly positive results in terms of the community interaction and performance of local (BRA) and state government in Banadir. • Encouragingly, the midline also saw a substantial increase in perception of the BRA providing services, however, most of these services were not related to the activities supported by TIS+. • On the less positive side, the perception of services provided by state government decreased across most areas and there was no change in the reported citizen satisfaction with those services. • Likewise, there were no notable changes in the perception of decreased conflict over resources and perceptions of corruption actually increased toward both the federal and local government (BRA). 4. There was a wide and diverse set of groups of actors implementing activities in each of the TIS+ settlements reviewed in Banadir, most of whom reported utilizing some aspect of community engagement, planning, and oversight. • The assessment team identified at least seven different organizations working in Baardheere, 11 organizations working in Dinsoor, and over 20 different organizations working in Banadir. • Both KIIs and FGDs conducted during the midline indicate that there has been a marked increase from these development partners to more systematically engagement with communities at the outset and involve community members in the planning and design of activities. • Likewise, other major partners, such as Danish Refugee Council, Finnish Church Aid, Nordic International Support Foundation (NISF), and International Committee of the Red Cross, as well as 267 UN agencies such as the Food and Agriculture Organization and World Health Organization report specifically using POCs or established community structures such as the Peacebuilding Committees to guide the planning and oversight of their committees. 5. Carry-Over Effects from TIS: The strong performance of Banadir on results related to community awareness and participation is in large part likely explained by strong carry-over effects from the activities of TIS. • Mogadishu was one of the first areas in Somalia to pilot CPO approaches during the initial TIS program and TIS conducted numerous community consensus planning sessions, trauma healing sessions, as well as investments in infrastructure, including rehabilitation of district administration buildings and sports facilities during its program life. • The TIS final evaluation concluded that these activities led to notable improvements in both community awareness and participation. • Specifically, the evaluation found that “TIS’s innovative essence was the consultative and participatory process that brought communities (civil society and business people), local administration, and central government together to discuss and identify the community’s priority needs;” “TIS proved to be a strong catalyst in encouraging greater engagement between communities;” and that “TIS contributed significantly toward increasing collaboration between local government and stakeholders.”15 6. Qualitative interviews (both KII and FGD) clearly conveyed that the unique socioeconomic characteristics of TIS+ settlements in Banadir played a large role in the results seen during the midline. • Interviewees identified the unique characteristics of the three TIS+ settlements in Banadir to explain the high levels of both government and development partner interest in supporting activities in these areas. • They explain that these districts are closer to the city center and are key revenue drivers for the local (BRA) and state government, accounting for up to 70% of revenue generation by some estimates. 7. Interviewees also explained that the building of trust between communities and respective levels of government following decades of conflict is a long-term endeavor and requires generational change. • Interviewees explained that while TIS+ activities might facilitate community and government interaction on the planning and oversight of specific shared assets, there is a long list of grievances and frustrations that still need to be overcome. • These include factors such as widespread perceptions of nepotism and clannism, unfulfilled election promises, corruption, security lapses coupled with a fear of victimization by AS, injustices and a weak judiciary, poor service delivery, and the outsized influence of diasporas. TIS+’s Technical Coordinators Approach In the original TIS+ documents on coordination and engagement, TIS+ funded Technical Coordinators (TCs) embedded within Ministries, were designed to support TIS+ Objectives 1, 2, and 3 and to help enhance the efficacy of TIS+ interventions. The TCs were also intended to deliver wider work in supporting communication between levels of government more broadly. The practice of embedding technical coordinators and advisors in Ministries and government agencies has been long utilized in Somalia (see TC Contribution Story – Section X for examples). This practice is set against the background of nascent institutions and limited capacity in the civil service, it is seen as an effective 15 IBTCI. “Final Evaluation of ‘Transition Initiatives for Stabilization’ (TIS)”. 268 way of facilitating donor engagement with government institutions whilst also building the capacity of the local staff in those institutions. TIS+’s TCs are expected to perform three central functions: 1. To support coordination and collaboration of TIS+ activities with government activities; 2. To support better communication with government at all levels and with other stabilization programs; 3. Under Objective 3, TCs are also expected to increase knowledge and gain support for locally-based Objective 3 activities at all levels of government. Since its inception, TIS+ has provided 27 TCs to different government institutions16 including the: • Banadir Regional Administration; • Ministries of Interior in Mogadishu, Baidoa and Kismayo; • Ministries of Gender, Human Rights and Family Affairs in Mogadishu and Kismayo; • Ministry of Public Works and Reconstruction; • Ministry of Youth and Sports; and • Ministry of Information. Findings on TCs Approach 1. TCs had helped improve coordination between the government and TIS+ as well as increase capacity of the government to engage with stabilization actors and donors. TCs also had contributed to better planning and strategy development by the government notwithstanding the apparent and inherent risk of capacity substitution and sustainability in the long term. • TCs have helped support the development of an extensive list of government policies and strategies and have supported vital state government processes such as district council formation processes. • KIIs with government conveyed appreciation for the work of TCs and saw them both as providing a point of contact with TIS+ programming and promoting government ownership of TIS+ interventions. • KIIs revealed that many other programs that work with government generally heard good feedback on TIS+ TCs from government partners, and some who had encountered TCs in Kismayo, or been involved in events organized by TCs in Jubaland, were positive about these experiences. Some of these respondents did, however, suggest that following up on these events and deliverables was important. • In addition to their individual documentary output and coordination roles, TCs and government partners noted their contributions beyond the individual Terms of Reference. • However, based on information from interviews, TCs and donor partners felt there is a risk of capacity substitution and sustainability challenges in the long-term. This is particularly of concern for stabilization programs that have to deal with frequent alignment of priorities because of changing dynamics. 2. TCs contribution was more visible at the state level ministries but is less visible at the Federal level due to the quantity of various actors providing an assortment of technical assistance. • In most of the Ministries in the states where TIS+ TCs are deployed, they were often the only ones at the Ministry with the exception of the Ministries of Interior in both states where there were a few other TCs deployed by IDLO and UN JPLG. • Levels of existing capacity within the Ministries where TCs are based has a clear impact on the contribution TCs will make. 16 AECOM, “FY2018 TIS+ Annual Report,” (2018). 269 • Jubaland was the first of the Emerging Federal States to be established and as a result of the relative stability, it has seen a steady influx of professionals from Mogadishu and abroad moving to Kismayo. • However, there has been no major donor investment in the capacity of the civil service beyond programs focused on public financial management. Short term technical assistance has come through IOM's MIDA especially in public healthcare but there remains a serious lack of capacity especially at ministerial level as a result of the limited available wage bill and the lack of donor investment when compared to Mogadishu and Puntland. • The South West administration’s expansion plans are uncertain, especially as the date of the move of the capital to Barawe remains unclear. The administration has not moved much from the vicinity of Baidoa airport where the Presidency is located. Limited resources at federal and state level have resulted in poorly capacitated ministries. The fairly nascent state of the administration has meant that there has been little investment in institution building and technical assistance to the Ministries. However, the establishment of the civil service commission in Baidoa may improve the capacity of the ministries in the near future. 3. TCs benefit from carry-over effects from TIS, particularly at the Federal level in Mogadishu, where the program was first piloted. • The final evaluation of TIS concluded that the program had “contributed significantly toward increasing collaboration between government and stakeholders.”17 • TIS utilized the ‘Dan Guud’ model, where it embedded professionals into the Banadir Regional Authority (BRA). These professionals were paid for by TIS but effectively served as the nascent cadre of civil servants that improved the capacity of the local government’s functions. • With the end of TIS, many of these local professionals were absorbed into the BRA as government civil servants. 4. The deteriorating political and security environment may have had a negative influence on the contribution of the TCs. • In particular, the frequent political crises which contributed to the high turnover in the leadership of the Ministries, often constrained the momentum built by the TCs. • This is further compounded by an unpredictable security situation, particularly in Mogadishu, where the Federal level technical coordinators are based. 5. The lack of understanding of TCs by USAID and the low level of their utilization by TIS+ to date mainly in the decision-making processes may have hindered the TCs effectiveness and contribution. • TCs expressed that they could play a much more pronounced role in TIS+’ decision making on strategic interventions as well as in improving the relationship with government stakeholders. They noted that in a lot of instances, they were informed of decisions after they had already been made, particularly whenever the program made realignments which affected planned activities. • Many of the TCs felt that USAID did not properly understand their role, citing that they had been often asked by USAID staff during TIS+ strategic review sessions to clarify what exactly they did in the Ministries and also added that the lack of an efficient feedback loop with the TIS+ program team sometimes led to missed opportunities, especially in situations where the governments’ unmet expectations had to be managed. • Some of these challenges and missed opportunities have been acknowledged by TIS+. One of the main outcomes from the 2018 annual Strategic Review Session (SRS) held by TIS+ was to “define the best possible way-as a best practice-to engage the TCs and Advisors in the FGS and Emerging Federal States (EFS) ministries.”18 17 IBTCI. “Final Evaluation of ‘Transition Initiatives for Stabilization’ (TIS)” 18 AECOM, “FY2018 TIS+ Annual Report,” (2018). 270 • As part of the realignment, TIS+ has implemented systematic reporting from the TCs and incorporated their participation as part of TIS+ progress on improving links to the host government. 6. TIS+ did not focus on capacity building for government in the work of its TCs, but the need made their work less effective as capacity injection could not address the gaps. • This was confirmed by other respondents who said TIS+ advisers had made a positive impact to the development of stabilization strategies, but this should be matched with an assessment of the real capacities but also motivations and leverage of the political actors in the ministries to implement these. • Other donors working on capacity injection in Somalia have noted that there has often been a failure of capacity injection interventions in Somalia to be tailored to specific needs, which are different in different regions.19 • TCs themselves did feel that they were contributing to capacity in their ministries but felt that the needs in this respect were very large. TIS+’s and USAID’s “Cluster” Approach Another critical feature of TIS+ programing approaches features the selection of geographic “cluster” in which to focus implementation. As explained in TIS+’s “Process of Engagement” guide, “under USAID’s direction and in coordination with relevant FGS ministries, a ‘clustering’ approach to district/community selection will often be employed, in which a tight grouping of districts/communities will be selected so to magnify stability impacts across a region.”20 The guide goes on to explain that the “the cluster approach and geographic expansion will allow the program to marshal resources to critical geographic areas, maximize coordination efforts and capitalize on existing resources and security assets within specific regions.” Interviews with USAID and TIS+ staff further clarified that the cluster approach was designed with the aim of having interventions that could contribute to the opening up and stabilizing areas around main supply routes that linked regions. Findings on Results of “Cluster” Approach 1. Cluster Approach in Banadir was not utilized despite the opportunity presented by the Mogadishu-Afgoye corridor which was opened by military campaigns aimed at mitigating the deteriorating security situation in Banadir. • Since Banadir was considered as being further along the stabilization continuum and had received considerable investment in the previous TIS program, TIS+’ approach was to leverage those existing investments and develop bridging plans. • However, as seen in the previous section, the security situation in Banadir deteriorated during the assessment period and the threats to Banadir’s stability emerged from nearby regions such as Lower Shabelle. The cluster approach would have been feasible in this instance since previous military campaigns had opened up the Mogadishu-Afgoye corridor. • In line with the cluster approach, TIS+ could have contributed to opening up and stabilizing the main corridor connecting Banadir and Lower Shabelle, which would not only improve stability but also contribute to economic growth by allowing farmers in the agriculturally rich region of Lower Shabelle to access the capital. 19 World Bank (2015) Somalia- Capacity Injection Project (English). Washington, D.C.: World Bank Group 20 AECOM, "Transition Initiatives for Stabilization Plus (Tis+) Process of Engagement," (March 2016). Pg. 3. 271 Q3: Government Involvement in Shared-Asset Management Assessment Question 3: Did Government at all appropriate levels contribute to asset management in communities in which TIS+ was implemented? If so, how? Did Government at all appropriate levels have both the capacity and the will to contribute? To help answer the preceding question, we present the results of our analysis in three sections showing: (1) the extent government contributed to the provision of shared assets; (2) the extent government contributed to management of shared assets; and (3) what was the capacity and will of government to do so. Findings on Provision of Shared Assets 1. The perception in federal government’s role in providing shared assets was minimal and centered mostly on the provision of security. • Most FGD respondents expressed the view that security is the main asset provided by federal government. TIS+ primarily supports this results area by providing direct technical support to government staff working in shared asset management. • TIS+ supported 27 Technical Coordinators (TCs) at the federal and state levels since inception. Fourteen TCs are active now. • KII Interviews identified TCs supporting “softer” results such as organizational strengthening, staff capacity-building, and gap-filling in civil service. • However, outcome level results are difficult to isolate to TCs, in large part by design, because the aim is for results to be attributed to government. • Key challenges have included frequent ministerial turnover, heavy workloads, uncertainty around funding and continued TIS+ support, and a lack of strategic approach for sharing skills and learning across TCs, including on how to standardize TIS+’s Gender Equality and Social Inclusion (GESI) approach. 2. Generally, across the cluster, the perception of State government’s level of provision of shared assets decreased with the exception of a few assets, such as water and electricity, which were outside of TIS+’s influence. • According to the HH survey, the provision of services by state government was generally reported to have decreased across most areas. • Exceptions include perceived increases in the provision of water and electricity in both TIS+ and non￾TIS+ settlements. 3. Across the cluster, the perception of local government’s (BRA’s) level of provision of shared assets increased considerably for most of the service areas with the exception of education. • Except for education in non-TIS+ areas, local government (BRA) was perceived to have increased the provision of services across all areas. Findings on Management of Shared Assets 1. Across the cluster, and with the exception of security, federal government was perceived to have a limited role in the management of shared assets. • Based on KIIs and FGDs conducted, security was cited as an enabling factor allowing others to manage assets. • FGD and KII respondents also mentioned that the federal government provides some limited financial resources for maintenance of roads, etc. 2. The perception in State government’s role in the management of all assets, largely in TIS+ areas increased and with stronger results. 272 • According to the HH survey, state government was perceived to have an increased role in the management of all assets in TIS+ areas, as well as all assets in non-TIS+ areas, except for small decreases in the management of schools (-1%) and health centers (-4%). • Encouragingly, TIS+ settlements reported stronger results in the management of roads (+17%) and sports facilities (+19%) compared with non-TIS+ areas. 3. Across the cluster, the perception of local government’s role in the management of all assets soared, and with stronger results in TIS+ settlements. • Based on the HH survey, Local administrations were perceived to have increased roles across all assets in both TIS+ and non-TIS+ areas. This was also confirmed in FGDs, although there were mixed views based on the settlements. • Again, encouragingly, TIS+ settlements reported stronger results against baseline survey results in the management of roads (+1%) and sports facilities (+18%) compared with non-TIS+ areas. Findings on Capacity and Will of Government 1. FGD and KII respondents acknowledged the will of various levels of government in assets managed, but chiefly the district administration, while equally identifying resource and systemic capacity challenges. Respondents constantly stressed that asset management thrives under good governance and is a shared responsibility between the government and the community, and strongly recommended for stronger public-private partnerships. • Qualitative interviews observed that the various levels of government had a role to play in asset management. They were however quick to point out that shared assets require good management and resources which could be realized through the synergy created by public-private partnerships that upholds the communities’ needs and priorities. • FGD and KII respondents further identified gaps in the governments’ will and capacity to manage public assets of which they stressed the need to ensure good governance, involve various stakeholders like youth, women, clan elders and religious leaders and provide regular capacity￾building of those involved in the management of shared assets. • Our qualitative interviews further stressed the need of the government to improve security, enact relevant legislation and policies on effective management of shared assets, and encourage the payment of maintenance taxes by citizens. • They further urged the government to promote a culture of honesty, transparency and social responsibility as well as ensure equal employment opportunities for individuals charged with asset management through meritocracy and gender parity while avoiding clannism. • Illustrative of the general trend, one FGD respondent in Wadajir, noted “I think the freshly graduated young men should be given a chance. The government and the community members should also stop supporting people according to the clans they belong to.” “The current administration has done a lot towards the managing and maintaining of these public assets. Abdiaziz district is one of the advanced districts in the Banadir region. To effectively manage these public assets, good governance should be practiced.” – Youth Boys FGD, Abdiaziz 273 Q4. Efforts to Promote Inclusion Assessment Question 4: Were TIS+ activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? Findings on Inclusion of Women 1. There was an increase within TIS+ settlements in the perception of those reporting that they believed that women were considered equal to men. • There was a significant increase in the percentage of respondents who reported in the HH survey that “women were considered equal to men,” with slightly stronger results in TIS+ settlements (+50%) than in non￾TIS+ settlement (+43%). 2. There were notable efforts by TIS+ to ensure women’s participation in POCs though without an organized training component • All six POCs established in areas reviewed included women members (average of 40%). • There is little evidence that women members of POCs received training on a regular basis to specifically improve their contribution beyond just participation on oversight and management, based on a review of TIS+ reporting and field interviews with TIS+ staff. Findings on Inclusion of Youth 1. There was notable youth participation in community planning and oversight of shared assets which was however limited to sport facilities. • All six POCs also include youth representatives (average of 30%). TIS+ did not collect any data on the participation of youth from minority clans. • Youth also reported being active in monitoring and oversight of shared assets, particularly in the management of sports facilities which were a key part of TIS+ youth engagement strategy (see “SDP Contribution Story” in Annex X for details). • Most FGD respondents also felt there should be greater capacity building for women and youth to ensure more effective asset management where community involvement was required. Findings on Inclusion of Minority Clan Inclusion and Improved Community Cohesion 1. This study finds that there was very scanty data from TIS+ on minority clan involvement or how this influenced programming. • TIS+ did not collect any data on the participation of women and youths from minority clans. • Women in qualitative interviews (both elder and youth) commented on the need for more women involvement in management of shared assets. This view was supported in several youth male FGDs conducted. 2. Community Cohesion Index scores increased in both TIS+ and non-TIS+ settlements • Based on HH data, both TIS+ (+6%) and non-TIS+ (+4) settlements showed increases in their Community Cohesion Index scores since baseline. See Figure 8. • Factors including interpersonal and institutional trust, social and political inclusiveness, and network diversity appear to have contributed to these increases. In the development of index scores, the three factors were considered to contribute equally to the overall CC index score. We thought this was a reasonable assumption, as there was no literature providing guidance on how these factor “I think the freshly graduated young men should be given a chance. The government and the community members should also stop supporting people according to the clans they belong to.” – Youth Boys FGD, Wadajir ”Until the Somalis respect the females, services will be of low quality. The rest of the world offers good chances to the females and the other discriminated communities. The district administration says it doesn’t like recruiting the Jareer people, which is totally against humanity.” – Elder Women FGD, Wadajir 274 scores would be weighted. The assumption of equal contribution seemed plausible after a sensitivity analysis presented in Annex D. • At the interpersonal level, the HH survey reported a slight increase of (+2%) of those “who do not trust one another across clans” but also an increase (+7%) in people who “who would cooperate on development initiatives.” • There were no significant changes on people reported to be “working collectively towards problems solving.” Non-TIS+ settlements reported decreases for all three of these indicators (-4%, -4%, and - 1% respectively). Figure 8: Banadir Community Cohesion Index measures/changes from 2017 to 2018 53% 50% 59% 54% 0% 10% 20% 30% 40% 50% 60% 70% 80% TIS+ non-TIS+ 2017 2018 +6% <0.0001 0.0003 regression analysis p - values +4% 275 CONCLUSIONS Q1: TIS+’s Contribution towards Intended Results Community Awareness and Participation • TIS+ provided a strong contribution to increased community and awareness in Banadir but also benefitted from carry-over effects, the unique characteristics of the settlements it worked in, and the work of other actors that support similar community awareness and participation activities. There was a general trend of increased community awareness and participation across the cluster, however results were particularly strong in TIS+ settlements. While these results are encouraging, TIS+’s contribution claim is tempered slightly by the fact that alternative explanations cannot be fully eliminated. It is reasonable to assume part of the results are influenced by carry-over effects from TIS activities. Likewise, the unique characteristics of these clusters likely also help explain the results. TIS+s districts are generally closer to the city center and strong revenue-drivers for the BRA and therefore key focus areas for government and civil society activities. Banadir is also an area that has had a significant USG investment and a proliferation of other donors working in a small and largely urban area. Approaches and techniques appear to be similar across both TIS+ and non￾TIS+ areas. For example, the use of project oversight committees appears to be a common feature of both local NGO and INGO efforts. Findings from FGD interviews indicate that there is now a clear emphasis from INGOs to make more of an effort to first consult with local communities rather than just come with a list of activities they want to implement. This is not unique to TIS+, but is an approach significantly valued at the local level. Findings from KII and FGD interviews also indicated that district officials conduct monthly consultations with community groups (women, youth, elders) and less frequent but larger public consultations across both TIS+ and non-TIS+ areas. Results might be explained by holdover effects from TIS activities that have now had more time to demonstrate effect. It is also evident that state governments are seeking to consult more with citizens overall. Effective Shared Asset Management • TIS+ has made a small contribution to increased awareness of POCs in Banadir. While there was a general increase of awareness of POCs across the cluster, this increase was slightly higher in TIS+ settlements. However, this did not translate into improved perceptions of performance likely due to inconsistencies in POC formations and implementation and the types of community infrastructure selected. There were significant increases in awareness of oversight committees in both TIS+ and non-TIS+ settlements, and these results were slightly higher in TIS+ settlements. TIS+ seemed to have followed most of its intended CPO steps but deviated from the original design when it came to the establishment, training, and use of the POCs. While it is true that POCs form only part of the overarching CPO process, it is nevertheless a crucial part to ensure regular dialogue with authorities, community monitoring and oversight, and for the building of a sense of ownership and responsibility for the sustainability of the activities within the community. The recent formation and training of POCs in Banadir (July 2018) by TIS+ makes it unlikely that these measures can be attributed to TIS+. Lack of changes in the effectiveness of POCs is most likely explained by the fact that PIKA-level POCs appear not to have been substantially trained until July 2018 as well, mostly after construction of various PIKAs had already finished. Although the community contracting approach by TIS+ was utilized and had been adopted in some measure by government counterparts, there has not been significant training at the local level to account for changes in this cluster, particularly linked to increased awareness of projects. POCs meant infrequently and all POCs were developed after interventions were completed. Qualitative interviews 276 consistently indicated a desire for more community training and capacity-building on shared management and oversight, an area of improvement required for TIS+. Both TIS+ and non-TIS+ areas recorded variations on other actors responsible for shared asset management with the perception that various levels of government play unique roles, but mixed results on non-state actors. For example, there were strong increases in the reported roles of state and local governments in managing shared assets in TIS+ settlements and positive, but less pronounced, increases in non-TIS+ settlements. Interestingly but encouragingly, this included strong results in the management of roads and sports arenas, which were key TIS+ PIKAs in Banadir. Findings from interviews indicated increased levels of community involvement in both TIS+ and non-TIS+ settlements. Generally, government was seen as having the first line of responsibility and then delegating management responsibilities of certain assets to community members. However, the need for greater involvement of minority clans, women, and youth was stressed as well as more targeted training for these populations. TIS+ has been working in Quarters 3 and 4 of 2018 to increase these statistics and to give women and youth more prominent roles, particularly in infrastructure projects. There is, however, a noted absence of monitoring mechanisms for minority clan inclusion in shared asset management. TIS+ is also meant to work with the private sector and has done so in Banadir for most PIKAs implemented. Interestingly, the business community—often described as the most important service provider—saw decreased roles in TIS+ and non-TIS+ areas in the HH survey. This contrasted with findings from qualitative interviews that mostly reported that NGOs and the private sector were more accountable than state and local government in relation to the quality and management of shared assets. Improved Delivery of / Satisfaction with Services • There was no observable evidence of TIS+ contribution to improved community perceptions around service-delivery and satisfaction due to inconclusive results with mixed patterns being observed across different service areas. There was a substantial increase in the perception of local government providing services in both TIS+ (across all services) and non-TIS+ (all services except education). There is little evidence that TIS+ contributed to these changes at the cluster level, but there were clear effects on improved coordination and information sharing within PIKA project environments. TIS+ PIKA activities focused mostly on roads, stadiums, and meetinghouses and not on services such as health or education, so it is likely that the TIS+ contribution at the cluster level was limited. It is apparent, however, that TIS+ partnerships with the BRA and work on several national level PIKAs contributed to improvements in coordination at the local and regional level in Banadir. The presence of TCs have also contributed to improvements in local government response as well as information sharing with citizens. However, this effect is limited and is unlikely to have contributed significantly to the changes observed at the cluster level. There was also a general perception that local people and NGOs/INGOs were responsible for improved service delivery in TIS+ settlements whereas services by state government had decreased across most services, except for water, electricity, and financial services. This included the provision of roads, which was a PIKA focus area. This finding is an indication that road activities delivered by TIS+ are regarded as services provided by NGOs/INGOs. This would be a negative result for TIS+ because the intention is to increase the role of the government in service provision and management. If TIS+ is seeking to influence community- level perceptions of services being provided more directly by the state, road construction does not seem to be an area that significantly impacts the measures that the TIS+ project is seeking to influence. There appear to be limited opportunities for community and government interaction. Despite having a community contracting mechanism, TIS+ interventions do not seem to impact community perceptions of government responsibility for enhancement of services. As evidenced in the baseline there are infrastructure 277 efforts that are largely seen to be led by INGOs as well as the private sector. FGD participants also expressed concern about the maintenance and quality of roads and the ability of the government to maintain them. Security is seen as a service that is largely provided by the federal government as well as the state. Answers varied by gender in almost all measures, requiring TIS+ to consider more analysis of the impacts of security on women. In both the HH survey and qualitative interviews there were varied perspectives on security. Women felt that security had significantly decreased and the variation between male and female responses was notable. There is a clear indication that particular projects, such as solar lighting, and improved market security features impact women’s perception of improvements in security provision. Reduced Conflicts over Resources • There was no observable evidence of TIS+ contribution to reduced conflict over resources at the cluster given similar results in both TIS+ and non-TIS+ areas. . Although there was a decrease in conflicts over resources at the cluster level, qualitative interviews showed a consistent concern about the impact of micro conflicts (conflicts over housing, land, and water) on community cohesion and interclan relationships. There is no evidence that TIS+ addresses these micro conflict issues. Because infrastructure relates directly to land use, it is critical that these tensions be monitored when considering future interventions. Micro conflicts also impact interclan dynamics. Several respondents cited the influence of these issues on minority clans as well as women. Notably, militia groups saw large decreases in both TIS+ and non-TIS areas as entities charged with conflict resolution. Interestingly, findings from the HH survey and in FGDs pointed large decreases in perceptions that the diaspora contributes to conflict resolution across both TIS+ and non-TIS+ areas, which echo findings from secondary studies on macro conflict drivers. This secondary data states that local structures are critical for conflict prevention and escalation of conflict in Somalia. There was expressed concern, similar to the baseline, that clannism is still a problem at the community level as well as within government. According to findings from KIIs, this practice can cause micro- and macro-level conflicts. Evidence has shown this was particularly expressed by women, who feel that there are some minority clans in Banadir that are not recognized by the government and are consistently denied access to services and consultation. The pending determination of the status of Banadir in Somalia’s federal structure further entrenches the feelings of exclusion for particular clans. Greater Citizen Confidence in Government • There was no observable evidence of TIS+’s unique contribution to greater citizen confidence in government as results were similar in both TIS+ and non-TIS areas. In general, there was a decrease in confidence towards state government across the cluster (with no significant change for federal government and local government (in this case BRA) although Governance Index measures recorded more positive results. Findings from interviews however pointed a long list of frustrations, including perceptions of nepotism and clannism, unfulfilled election promises, corruption, security lapses coupled with fear of victimization by AS, injustices and weak judiciary, poor service delivery, and outsized influence of diasporas. We would expect to see a unidirectional relationship for this result: trust must be built before confidence is guaranteed. In other words, trust is a prerequisite to confidence in government. Building trust is a process with multiple factors built by working closely with local community, allowing people to be heard, etc. TIS+ is working in a fluid environment characterized by corruption, nepotism, and clannism, and these factors also contribute to varied perspectives in confidence cited. While findings pointed that local government saw strong increases in the perception of its provision and management of services, local government performed more poorly on other governance measures (increased corruption and decreases in sharing information). Lower scores here on “sharing information” run counter to previous findings and are difficult to explain. One explanation may be that in Banadir the increased access to information makes citizens 278 more critical of government and more aware of corruption as a significant problem impacting development interventions. This seemed evident in the baseline as well as the midline as respondents of FGDs and KIIs saw more transparent information sharing as being directly related to increases in their confidence in government and helpful in reducing corruption, particularly around issues of perceived clannism. Reduced Support for Violent Extremism • There was no observable evidence that TIS+ contributed to reduced support for violent extremism in this cluster. However, potential changes related to countering violent extremism are at the far end of TIS+ intended results chain and are therefore expected to have indirect and minimal, if any, influence over these results. TIS+ activities and interventions did not contribute directly to these measures, but they are indications that AS still holds sway with youth populations in Banadir. AS has a complex administrative and logistical system that influences recruitment, but this cannot be aligned with TIS+ programming. TIS+ programming has attempted to have interventions with a P/CVE lens, for example through sports and youth empowerment, but there is no evidence that this is directly contributing to reduced support for violent extremism. There were non-significant results for people reporting that extremist groups were “not welcomed at all.” These measures have remained consistent since baseline. Q2: Contribution of TIS+ Approaches Community Planning and Oversight • TIS+’s CPO approach contributed to increased levels of community awareness and participation but did not result in higher-level cluster outcomes given deviations from its original design on the timing of the establishment, capacity-building, and engagement of its POCs. The midline assessment found evidence of TIS+ achieving notable results in terms of community awareness and participation, as well as local and state government service provision and management of shared assets. However, TIS+’s contribution claim is also limited by the lack of a consistent implementation approach across the program and within this cluster. While the community consensus workshops look to have been consistently implemented, the form and timing of other key parts of the CPO process varied across settlements. This includes a difference in the way that community contracting took place across the settlements, and most notably, significant deviations on how POCs were formed, trained, and utilized throughout the process. TIS+’s own program design documents clearly highlight the central role that POCs play in the overarching CPO process; however, evidence indicates that TIS+ did not prioritize their role until late in the PIKA process, almost always only after the construction of the different structures was already complete. Technical Coordinators • TIS+’s TC’s supported continued coordination and collaboration of TIS+ plans and activities with federal and state government plan, helped with the coordination with other stabilization programs, and improved communication especially on the government’s stabilization plans. However, their contribution claim is limited by the fact that other actors, and in particular, stabilization programs also embedded Technical Coordinators and Advisors to carry out similar functions such as supporting state and federal level coordination of stabilization programs. There was also limited evidence that the TIS+ TCs contributed as expected to higher level results such as increasing confidence in government. Findings point that there is evidence that the work of TIS+ TCs contributed to improving TIS+ interventions, improved coordination, and also injected capacity in their host Ministries. However, the high-level expected contributions such as increasing confidence in government presents a mixed picture. Moreover, the results 279 chain connecting the TCs inputs to these results is so long, and is decisively influenced by many other variables, that it would be unfair to expect any contribution at this level. While it was evident from findings on program documents, interviews, and the output of the TCs that they implemented as per the plan, there was also evidence that pointed TCs were not adequately utilized in situations where they would have greatly contributed to. A number of other influencing factors and alternative explanations contributed to the results seen. These included the activities of other actors, the carry over effects from the previous TIS program as well as the highly dynamic political and security situation. “Cluster” Approach • The “cluster” approach would have been feasible in this cluster but was not utilized. The major corridors linking Mogadishu with Lower and Middle Shabelle presented opportunities for intervention that could contribute to the stability of Banadir. Banadir was considered as being further along the stabilization continuum and had received considerable investment in the previous TIS program, TIS+’ approach was to leverage those existing investments and develop bridging plans. The security situation in Banadir deteriorated during the assessment period and the threats to Banadir’s stability emerged from nearby regions such as Lower Shabelle. The cluster approach would have been feasible in this instance since previous military campaigns had opened up the Mogadishu￾Afgoye corridor. In line with the cluster approach, TIS+ could have contributed to opening up and stabilizing the main corridor connecting Banadir and Lower Shabelle, which would not only improve stability but also contribute to economic growth by allowing farmers in the agriculturally rich region of Lower Shabelle to access the capital. Q3: Government Involvement in Shared-Asset Management Provision of Shared Assets • Local government (BRA) was widely regarded to have increased its service provision. This is a result of improved governance of private services by the BRA as opposed to directly providing services. The Federal government’s role was limited to the provision of security. TIS+’s efforts to strengthen the perception of Federal and state government service-provision seem to have been hampered by a variety of factors both within and outside of the program’s control. TIS+ primarily supports this results area by providing direct technical support to government staff working in shared asset management. TIS+ supported 27 Technical Coordinators (TCs) at the federal and state levels since inception. Fourteen TCs are active now. KII Interviews identified TCs supporting “softer” results such as organizational strengthening, staff capacity-building, and gap-filling in civil service. However, outcome level results are difficult to isolate to TCs, in large part by design, because the aim is for results to be attributed to government. Key challenges have included frequent ministerial turnover, heavy workloads, uncertainty around funding and continued TIS+ support, and a lack of strategic approach for sharing skills and learning across TCs, including on how to standardize TIS+’s Gender Equality and Social Inclusion (GESI) approach. Most FGD respondents expressed the view that security is the main asset provided by federal government. According to the HH survey, the provision of services by state government was generally reported to have decreased across most areas. Exceptions include perceived increases in the provision of water and electricity in both TIS+ and non-TIS+ settlements. TIS+ also supported TCs at this level to support the capacity of the nascent institutions since state administrations are relatively new and not properly capacitated. At local level, service provision across all service areas increased with the exception for education in non-TIS+ areas. 280 Management of Shared Assets • While the federal government was regarded to have a limited role in the management of shared assets, community members perceived state and local government to have increased their role since baseline, with particularly positive results in TIS+ areas (roads and sports facilities) while the federal government played a major role in managing security. Based on findings from KIIs and FGDs conducted, security was the main asset managed by the federal government, and which was as an enabling factor allowing others to manage assets. Findings pointed that the federal government provides some limited financial resources for maintenance of roads, etc. According to the findings from HH survey, state government was perceived to have an increased role in the management of all assets in TIS+ areas, as well as all assets in non-TIS+ areas, except for small decreases in the management of schools and health centers. Encouragingly, TIS+ settlements reported stronger results in the management of roads and sports facilities compared with non-TIS+ areas. On the other hand, Local administrations were perceived to have increased roles across all assets in both areas according to the HH survey. This was also confirmed in FGDs, although there were mixed views based on the settlements. Again, encouragingly, TIS+ settlements reported stronger results against baseline survey results in the management of roads and sports facilities compared with non-TIS+ areas. Capacity and Will of Government • The BRA figured prominently compared to federal and state governments as having the will to manage assets but often limited by capacity, resources and systemic challenges that needed to be filled through a shared responsibility between government and the community as well as stronger public-private partnerships. FGD and KII respondents acknowledged the will of various levels of government in assets managed, chiefly the district administration, but also noted systemic capacity challenges that need to be filled for asset management to thrive in this cluster. Respondents indicated that good governance is a shared responsibility between government and the community, and strongly recommended stronger public-private partnerships. Q4: Efforts to Promote Inclusion Inclusion of Women • TIS+ has taken a number of steps to increase the participation of women in its activities and has succeeded in facilitating women’s participation across its POCs, as well as influencing perceptions in its settlements that women were considered equal to men. However, evidence from the midline also suggests that there is a need for additional capacity-building of POC members, with youth and women identified specifically as needing more capacity-building. There was a significant increase in the percentage of respondents who reported in the HH survey that “women were considered equal to men,” with slightly stronger results in TIS+ settlements than in non-TIS+ settlement. All six POCs established in areas reviewed included women members (average of 40%). However, there is little evidence that women members of POCs received training on a regular basis on oversight and management, based on a review of TIS+ reporting and field interviews with TIS+ staff. Insecurity since baseline has only increased, and despite increased AS attacks and periodic problems with movement, participation is highly valued in Banadir. Numerous interviews during data collection discussed the importance of increasing participation, particularly for women and youth. This measure significantly changed from the baseline and shows a marked increase in the emphasis on youth and women inclusion. The decrease in the participation of clan elders likely reflects this shifting emphasis on these target groups as well. Inclusion of Youth 281 • While TIS+ has succeeded in facilitating the participation of youth in community discussions and planning for the construction of shared-assets, the program has mostly limited youth’s participation in the management and oversight of shared-assets to those involving sports facilities and playgrounds. All six POCs also include youth representatives. TIS+ did not collect any data on the participation of youth from minority clans. Youth also reported being active in monitoring and oversight of shared assets, particularly in the management of sports facilities which were a key part of TIS+ youth engagement strategy (see “SDP Contribution Story” in Section X for details). Most FGD respondents also felt there should be greater capacity building for youth to ensure more effective asset management where community involvement was required. Inclusion of Minority Clan Inclusion and Improved Community Cohesion • There was lack of systematic monitoring of minority clan involvement, or at least documentation of that monitoring in TIS+ interventions There was very limited data from TIS+ on minority clan involvement or how this influenced programming as TIS+ did not collect any data on the participation of women and youths from minority clans. Indeed women in qualitative interviews (both elder and youth) identified a gap in minority inclusion and commented on the need for more women involvement in management of shared assets. This view was supported in several youth male FGDs conducted. 282 RECOMMENDATIONS Strengthening TIS+ and/or USAID’s Contribution to Intended Results 1. TIS+ and USAID should ensure that the program follows its intended programming approach consistently and that these approaches are periodically monitored for alignment with TIS+’s overarching Theory of Change. The frequent realignment of the program and its interventions makes it difficult to effectively assess TIS+ interventions. An applied ToC approach requires not only flexibility, but also a strategic alignment based on accurately observed changes in the context informed by regular field monitoring and management. Stabilization programs require adaptive monitoring systems that complement an applied TOC approach. This in turn allows for greater program adaptation over time based on field-based analysis. Priority: High Responsible entity: TIS+ and USAID Timeframe: Medium (6-12 months) Resources Required: Minimal financial resources with sustained implementation effort. 2. USAID should endeavor to better understand the unique dynamics of violent extremism in Somalia and utilize cross-cutting interventions across all its programs to address violent extremism. The broad portfolio of USAID programming in Somalia (not just TIS+) should focus on a more coordinated approach to violent extremism and its prevention. An intervention designed around a single program such as TIS+ will not have attributable results as evidenced by this assessment but increasing coordination between implementing partners working in Somalia, particularly in AS-controlled areas, will greatly enhance impact and effects. Priority: High Responsible entity: USAID Timeframe: Medium (6-12 months) Resources Required: Moderate financial resources with sustained and deliberate effort to learn 3. TIS+ should consider strengthening its efforts to better appreciate the dynamics around violent extremism before tailoring interventions. There is no evidence that TIS+ interventions have contributed to reducing violent extremism. Analysis of program documents indicates a cursory understanding of the dynamics and that there is no direct linkage between the dynamics and the design of interventions such as Sports for Development and Peace interventions or youth outreach activities. Priority: High Responsible entity: TIS+ Timeframe: Medium (6-12 months) Resources Required: Moderate financial resources with sustained and deliberate effort to learn Strengthening TIS+ and USAID Approaches 4. TIS+ should prioritize the community consensus approach as an essential step in Community Planning and Oversight (CPO) and ensure the entire process is implemented with fidelity. 283 There was a significant increase in awareness of local projects being selected through community planning events across Banadir, and there is a clear preference for INGOs to make more of an effort to first consult with local communities rather than just come with a list of activities they want to implement. Where this approach was utilized by TIS+ and partners there are strong levels of satisfaction with projects meeting community needs. Priority: High Responsible entity: TIS+ Timeframe: Medium (6-12 months) Resources Required: Minimal financial resources with sustained and deliberate effort to involve and consult with communities in all their projects 5. TIS+ should mainstream Technical coordination (TCs) into their programs for better utilization. TCs enhance coordination and information exchange in TIS+ Interventions. Interviews and analysis clearly showed the utility of the TCs and their contribution to the results achieved by the program. During the strategic review sessions, TIS+ acknowledged the need to better define the role of the TCs and improve their linkage to the program. Better utilization of the TCs can be achieved by first, ensuring their role and funding within the program is sustainable. Secondly, TCs can play an integral role in better understanding the political and security dynamics, allowing TIS+ to better forecast and plan its interventions Priority: Medium Responsible entity: TIS+ Timeframe: Medium (6-12 months) Resources Required: Minimal financial resources, staff with regular and continuous education 6. USAID should work with TIS+ to encourage greater conflict sensitivity integration into program planning. There was no clear evidence that TIS+ staff have a consistent system for integrating conflict analysis into their program operations, which is concerning for a stabilization program working in areas vulnerable to both AS and ISIS influence. A deliberate conflict-sensitive design will allow the program to benefit from the immense local knowledge of the program staff and allow for more nuanced approaches that yield results. This may help mitigate some of the targeted attacks on infrastructure that occurred in this cluster as well as increase awareness of the need for more minority clan engagement in key areas. Priority: High Responsible entity: USAID and TIS+ Timeframe: Medium (6-12 months) Resources Required: Moderate financial resources with sustained and deliberate effort to learn and adapt Strengthening Government Involvement In Shared-Asset Management 7. TIS+ should facilitate the visibility of local government in planning and decision-making platforms. Promoting the government’s role as a facilitator can be a more effective and efficient use of TIS+ resources and can be an area where small interventions lead to big results, for example, supporting regulations and standards such as the material testing labs, solar maintenance, or the government’s communication and outreach activities. Additionally, it is essential that the government is seen as facilitating the planning and decision-making meetings as well so that citizens see government officials as valuing collaboration with communities. 284 Priority: High Responsible entity: TIS+ Timeframe: Medium (6-12 months) Resources Required: Minimal financial resources with sustained advocacy and collaborative efforts that bring together the local people and various forms of government Strengthening Efforts to Promote Inclusion 8. TIS+ should focus on activities that emphasize the inclusion of women, youth, and minority clans. A focus on softer activities that enhance the participation of women, youth, and minority clans may be helpful in the final phases of the project, particularly activities that increase interclan collaboration. Priority: High Responsible entity: TIS+ Timeframe: Short (0-6 months) Resources Required: A moderate budget. 9. TIS+ should ensure regular trainings forums for women and youths working in project oversight committees (POCs) and asset management through a program that is implemented with fidelity. Increases in participation are distinctly linked to citizens feeling more competent to engage in decision-making and management and oversight. Long-term CDD approaches are familiar in this cluster, but the quality of participation is still a challenge for the TIS+ project. Increased emphasis on training will capacitate community members to engage more meaningfully in the oversight and management of shared assets and lead to higher￾quality citizen and government interaction. Priority: High Responsible entity: TIS+ Timeframe: Medium (6-12 months) Resources Required: A moderate budget, goodwill as well as well as a deliberate effort to train women and youths. 285 VIII. GEDO/ BAY CLUSTER Background and Context of Cluster The Gedo Region is the second largest in Somalia and lies on the border with Ethiopia and Kenya; it shares borders with Bay, Bakool, Middle Juba, and Lower Juba regions. The primary settlements in this cluster are in the Gedo Region, save one district, Dinsoor, which is in Bay. Gedo is part of the Mandera Triangle and is a vibrant cross-border area critical to economic growth and development for not only Somalia, but also Kenya and Ethiopia. More than any other Somali border area, the Mandera Triangle constitutes an integrated cross￾border economy. This area contains critical corridors for commerce and livestock sales for the entire region. Clan conflict over territory and resource control has been a major feature in this cluster since the collapse of the Somali government in 1991. Conflicts in this region are greatly impacted by minority clan dynamics and exacerbated by Al-Shabaab (AS). While most of Gedo’s districts have been captured from Al Shabaab, towns such as Baardheere and parts of El Wak, Luuq, Garbaharey continue to deal with insecurity, especially along main supply routes. Gedo and Bay are inextricably linked, not just by their geographic proximity, but also because of the elite political plays around Somalia’s emerging federal structure. Bay is a major source of fighters for AS. It is also an important geographic space as AS manipulate sentiments of marginalization by the clans in the area, as well as economic inequities. Targets of this marginalization and manipulation include non-Marehan sub-clans as well as the Hawarsame and Fiqi Yaqub (both are Marehan sub-clans but are also considered to be “lower-caste” groups). Gedo is the staging point for AS’s cross-border attacks into Kenya. Additionally, AS has sought to cash in on the lucrative cross-border trade, particularly seeking to control the flow of contraband goods. Intense resource conflicts in Gedo due to water shortages have had a negative effect on the region’s stability, leading to dozens of chronic micro-conflicts. There are frequent conflicts between Gabaweyn (Digil/Rahanweyn) farming communities and the pastoralist Marehan. Climate change and forced migration due to drought cycles and reduced access to resources have increased tensions and dramatically impacted pastoralist communities and those reliant on livestock trade. TIS+ Interventions Table 8 (next page) provides the list activities, by settlement, by which TIS+ sought to engage government and stakeholder partners to ensure that community cohesion and governance improvements were observed in Gedo and Bay. Stabilization and security are slowly improving in Gedo, but there is still a long way to go because AS controls significant areas of this region. This still challenges development partners working in this cluster. There are fewer stabilization actors working in this region; however, there were a large number of development projects and some cross-border projects that were considered in analyzing the midline effects of TIS+. Gedo is deemed to have extremely high Water, Hygiene, and Sanitation (WASH) vulnerability (except for El Wak, which has very high vulnerability) based on Acute Watery Diarrhea (AWD)/cholera risk, flood risk, drought risk, number of internally displaced persons (IDPs), water, and sanitation coverage. 286 Drought has prompted migration into urban centers in Gedo since November 2016 and is likely to impact TIS+ programming and analysis of impact in this cluster. Table 8: Gedo and Bay PIKA Activities Other development activities and partners have also been active in this cluster. Prior USG investments are evident, and this has influenced data analysis of this cluster. As shown in Figure 9, the following actors and activities were evident during the WAVE II data collection period. Figure 9: Gedo and Bay, Summary of Other Actors and Interventions Actors Activities • Private sector (telecom, bank/remittance, construction) • International Organization for Migration (IOM) • Nordic International Support Foundation (NISF) • International Committee of the Red Cross (ICRC) • Intersos • Troicare • De Garas • Gredo • Lifeline • Norwegian Church Aid (NCA) • Adeso • Rainwater Association of Somalia (RAAS) • World Food Programme (WFP) • Football and basketball grounds • Rehabilitation of district hospital • Women’s vocational program • Road and culvert construction • Rehabilitation of government offices • Seed distribution • Solar lights • Microloans • Various humanitarian projects Methodological Considerations Findings are primarily aggregated at the cluster level, but where relevant, individual settlement information is provided to illustrate to USAID and TIS+ the diversity of these settlements and the anticipated effect on stabilization planning Settlement TIS+ Activity/Intervention Baardheere JSS Baardheere Capital City Community Consensus Building Session JSS Baardheere Capital City Roads Bridge, Airstrip Assessment Study and Design JSS Baardheere Capital City Community Orientation Center Construction Youth Motivational Sessions Dinsoor Ministry of Interior and Dinsoor District Community Consensus Building Session Ministry of Public Works and Reconstruction Dinsoor Town Women’s Center Construction Ministry of Public Works and Reconstruction Dinsoor Road and Airstrip Upgrading Survey Dinsoor District Capital Football Stadium Construction Dinsoor District Dinsoor Capital City Airstrip Rehabilitation Dinsoor District Dinsoor Capital City Main Road Rehabilitation Baidoa ISWA Regional and District Stability Assessment and Actions Capacity Building Ministry of Information and Social Awareness Peace and Stability/Arts and Culture SWS Ministry of Women, Family Affairs, and Human Rights Promoting Women’s Roles in Leadership and Stability in ISWA Ministry of Interior and Local Governments Stability Department Institutional Capacity Building SWS Ministry of Youth and Sports Baidoa Youth Day SWS Ministry of Planning and International Cooperation Furniture and Equipment SWS Ministry of Interior and Local Governments PIKA Implementation Support Unit SWS Barawe District Town Peace Committee Youth Iftar 287 There is considerable variation in the findings between settlements, therefore generalizability at the cluster level is difficult. TIS+ and non-TIS+ settlements were analyzed for comparison and to assess cluster level contribution. In addition to the HH survey, 20 FGDs and 21 KIIs were conducted in the TIS+ settlements of Dinsoor, Baardheere, Luuq, and Baidoa and non-TIS+ settlements of Garbaharey and El Wak (see Table 9). Table 9: Gedo and Bay HH Survey Samples, TIS+ and non-TIS+ Settlements Gedo/Bay Cluster TIS+ Settlements Baseline Midline Dinsoor 82 116 Baardheere 233 309 Total 315 425 Non-TIS+ Settlements Baseline Midline Garbaharey 99 246 El Wak 86 200 Total 185 446 288 FINDINGS Q1: TIS+’s Contribution Towards Intended Results Key Assessment Question 1: Did TIS+ contribute to any of the following: (a) Community ownership of development projects and processes; (b) Effective shared asset management; (c) Improved delivery/satisfaction of services; (d) Reduced conflict over resources; (e) Greater citizen confidence in government; (f) Reduced support for violent extremism? 1(a): Community Ownership of Development Projects and Processes To examine changes in “community ownership of processes and development projects,” examined three areas: (1) participation of different stakeholders; (2) awareness of projects being selected based on community planning events; and (3) the extent to which community members felt that the selected projects meet priority community needs. Based on an analysis of both the quantitative survey data and qualitative interviews, the team found that: 1. Perceived participation across stakeholders generally increased across the cluster with the exception of Dinsoor, which witnessed a decrease across groups. However, this decrease is likely better explained by the actions of local political leaders rather than a direct result of TIS+ activities. • The reasons for Dinsoor’s unique and adverse performance were not immediately apparent upon an initial review of the midline data. • Interviews and desk review confirmed that TIS+ held a Dinsoor district consensus-building workshop from June 6-8, 2016 that included 43 participants of diverse backgrounds. Further, qualitative interviews confirmed that a district-level POC was established during the final day of the consensus￾building workshop and that community members report regularly providing both financial and in-kind support to development projects in their areas. • However, further interviews conducted as part of the team’s contribution analysis identified that the actions of the District Commissioner (DC) likely serve as a more credible proximate factor explaining the negative trend witnessed in Dinsoor. Interviewees described the DC as a strong personality who was rumored to regularly coerce and intimidate vendors working in Dinsoor. • Interviews with TIS+ staff revealed that the program has received more letters of complaint against the Dinsoor DC than they have received for all other areas combined, resulting in TIS+ senior staff twice requesting a meeting with the President of South West State to ask for his intervention and possible replacement of the DC. • Interviews with TIS+ staff also highlighted concerns that the DC might have had undue influence on the selection of the district-level POC members and a decisions was therefore taken in January 2019 to break up the POC into four separate PIKA-specific POCs with varied membership. 2. Non-TIS+ settlements showed stronger results on community perceptions that projects were selected based on community planning events compared to TIS+ settlements. Baardheere was the only settlement demonstrating a negative trend, likely influenced by a more substantial number of activities being implemented in the district, some of which are likely not to have included community planning events. • Survey results showed that there was a non-significant increase in Dinsoor (+2%) but a significant reduction in Baardheere (-20%) compared to significant increases of 35% in El Wak and 52% in Garbaharey (see Figure 10 below). 289 Figure 10: Do you know whether local projects are selected through community planning events? • Again, as part of its overarching contribution analysis, the team explored what factors might explain why Baardheere was the only settlement assessed to show a negative trend on this criteria. The team found that several factors, both within and outside of TIS+’s control, are likely to have contributed to this result. These factors include: 1. A recent proliferation of development activities including at least 19 separate activities implemented by at least eight different NGO, INGO, and UN actors. It is not clear that all of these activities included explicit community planning events. 2. TIS+ community consensus workshop identified six community priorities. While the selected PIKAs of the rehabilitation of the airstrip and airport road, as well as the construction of community orientation center, were part those priorities, the top two were related to the rehabilitation of bridges as well as a final priority for the construction of a Maternal Child Health (MCH) center. 3. There was an 11-month time lag between the TIS+’s community consensus workshop (May 2016) and the start of construction on the PIKA activities (April 2017) and a 20-month period between the community consultation and substantial completion of the PIKAs (January 2018). Qualitative interviews conducted as part of the midline confirmed community frustrations with ongoing delays and perceptions that the activities were not sufficiently “community led.” 4. Local politics and clan relations also look to have had a strong influence. Interviewees explained that Baardheere is regarded as being divided between two dominant clans, in part exemplified by an ongoing power struggle between the DC and Deputy DC. 3. With the exception of Garbaharey where a substantial number of projects are being implemented with a complete initial conflict assessment, there was a general perceived decrease across cluster in respondents indicating that “‘most’ to ‘all’ of the selected projects effectively addressing community priorities.” • Survey responses indicate negative trends for Dinsoor (-10%), Baardheere (-8%), and El Wak (-24%). Garbaharey, on the other hand, saw an increase (+22%). • The team hypothesized that two key factors likely help explain this results: (1) Garbaharey is the administrative capital of Gedo region and thereby home to a substantially higher number of development projects addressing community needs; and (2) Garbaharey was selected by JPLG as one of its Phase 1 sites for the formation of district councils and thereby might have experienced higher levels of discussion and commitment on meeting community needs. • Subsequent evidence discovered as part of the team’s contribution analysis confirms that the first factor but disconfirms the second. The team found that there is indeed a substantially higher number of development activities being implemented in and around Garbaharey. This includes more than 25 290 projects implemented by a wide range of actors including government, UN, NGO, INGO, private sector, and diaspora actors. However, interviews with both implementing partners and outside observers of JPLG, revealed that although an initial conflict assessment has been completed in Garbaharey, further implementation of the district council formation process has stalled, and little tangible implementation has taken place to date. 1(b): Effective Shared Asset Management The assessment team examined changes on “effective shared assessment management” along three criteria: (1) community awareness of project oversight committees; (2) community perceptions of the effectiveness of those project oversight committees; and (3) other actors (apart from project oversight committees) responsible for shared asset management. Based on its analysis, the team found that: 3. Communities members’ perceived awareness of project oversight committees was higher in non￾TIS+ areas compared to TIS+ areas. The lower levels of perceived awareness in TIS+ areas is likely explained by the formation of POCs at district-level, not activity-level, and likely also by the type of activity that the POCs are responsible for managing. • Data collected during the midline indicates that not only are other programs in addition to TIS+ establishing community-based project oversight committees, but also that awareness of these committees actually appears to rising in non-TIS+ areas while falling in TIS+. • According to the household survey, El Wak (+17%) and Garbaharey (+32%) saw increases from the baseline, while Dinsoor (-49%) and Baardheere (-10%) both saw decreases. • The team’s contribution analysis identified two main reasons for these results. 1. The TIS+ POCs in both Dinsoor and Baardheere were initially established only at the district level and was responsible for the oversight of multiple PIKAs. It was not until November 2018 (Baardheere) and January 2019 (Dinsoor) that activity level POCs were established, both of which dates fall after data was collected for this midline report. 2. The type of activity that the POCs are overseeing will in all likelihood affect the frequency with which the POC will engage with the community. It can reasonably be assumed that other POCs responsible for overseeing more direct service-delivery focused projects— for example, the building of hospitals, health centers, and schools—will have more frequent interaction with the community than the TIS+ activities focused on constructing roads and air strips. 4. The perceived effectiveness of TIS+ POCs remain mostly unchanged since baseline with exception of their role in maintenance quality. This is likely explained by the same factors mentioned above which likely limited the direct interaction of POCs and community members. • According to the household survey, oversight committees mostly saw non-significant changes in the perception of their effective management for most assets, except for a decrease (-18%) in management of market facilities. In Baardheere, the only significant result was an increase (+9%) in the management of health facilities. • While the assessment team had expected to see changes in perceptions of oversight committee’s effectiveness in the management of at least roads and meeting houses in the TIS+ areas given that these were the type of assets that TIS+ PIKAs focused on, these results did not materialize. • These results (or rather lack of) are most likely explained by the fact that activity level TIS+ POCs were not established in these areas until after data collection for the midline (October 2018). • The results also indicate that district-level POCs are not an effective mechanism for increasing perceptions of the effectiveness of oversight committees, at least in the short-term. • Interestingly, apart from Dinsoor (no significant changes), there were increases in the maintenance quality by actors other than oversight committees across nearly all assets. 291 5. There were large variations across TIS+ and non-TIS+ settlements in perceived roles of other actors (see Figure 11) responsible for shared asset management. Notably, however, the perception of both local and state governments role in the management of shared assets decreased in both TIS+ areas, including on PIKA-focused assets. • Table 9 below summarizes the household results on respondent perceptions of actors outside of the project oversight committees involved in the management of shared assets. Table 9: Who else is responsible for the management of the following services/assets? Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Clan Elders -35% -40% -85% -87% -33% -84% -17% -16% -16% -16% -18% -15% Militia groups 0% 0% -26% -28% -5% -59% -2% -1% -2% -1% -1% -3% State government 15% 12% -10% -37% -7% -18% -14% -9% -12% 7% -16% -7% Religious leaders -75% -2% -1% -2% 0% 0% -2% -4% -6% -4% -5% -4% Local administration -67% -72% 1% -63% -2% -3% -46% -35% -42% -19% -36% -43% Ugaas/Sultan -32% -42% -51% -73% -2% -54% 0% 0% 0% 1% -1% 2% Diaspora 0% 0% 0% 0% 0% 0% 1% 4% 0% 1% 0% 1% Businesses/Traders -1% 0% 0% 0% 0% 0% -3% -3% -3% -3% -3% -3% Local community 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% NGOs 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% International organisations 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% Youth 0% 0% 0% 0% -77% 0% 0% 0% 0% 0% -1% 0% Dinsoor Baardheere Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Clan Elders -29% 7% -20% -19% -21% -28% -26% 33% -1% 0% 0% 1% Militia groups 0% 0% 0% 0% 0% 0% 0% 0% 0% -1% 0% 0% State government 14% 7% 20% 5% 26% 20% 22% 3% 40% 29% 12% 39% Religious leaders 10% -21% -3% -4% -4% -4% 38% 0% 3% 0% -1% 2% Local administration 27% 28% 15% 15% 11% 34% -15% -10% -16% -17% -28% -15% Ugaas/Sultan 10% 5% 3% 1% 2% 3% 0% 0% 0% -1% 0% 0% Diaspora -39% 0% 0% 0% -1% 0% 21% 14% 0% -1% 0% 11% Businesses/Traders 0% -1% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% Local community 0% 0% 0% 0% 0% 0% -5% 0% 0% 0% -1% -1% NGOs 0% -8% -1% 0% 0% 0% -1% -17% -2% 0% 0% -1% International organisations 0% -8% 0% -1% 0% 0% -1% -2% -3% -5% -5% -3% Youth 0% 0% 0% 0% -28% 0% 0% 0% 0% 0% -15% 0% El Wak Garbaharey 292 • The results paint a mixed pictures across each of the sites which challenges identifying cluster-level findings. However, one important finding that can be drawn is that both local and state governments saw large decreases in management of nearly all assets TIS+ areas compared with increases for state government in both non-TIS+ areas, as well as increases for local government in El Wak. • Interestingly, the qualitative interviews diverge from the survey findings and convey a sense of shared responsibility between the communities and local and state government in the management of shared assets. • They also emphasize the role that NGOs and private sector play currently in the management of shared assets. • Notably, while the majority of respondents felt there was a shared responsibility, the majority also noted that government held the “first line of responsibility” for management and would then delegate secondary levels of responsibility to the community. 1(c): Improved Delivery of / Satisfaction with Services To analyze this result area, the team examined community perceptions around (1) service provision by government and other actors; (2) satisfaction with services provided; and (3) perceptions on “security as a service.” 1. There was a perceived increase at cluster level on service provision by international NGOs. However, , this corresponded with a decrease in perception of the role of both state and local government for road provision in both Dinsoor and Baardheere (key PIKA activities). • Table 10 below summarizes changes in respondent perceptions on actors responsible for the provision of various services. Figure 10(a): Actors Responsible for Service Provision (changes between 2017 and 2018) Baardheere Health Education Water Roads Electricity Sanitation Security Financial services Health Education Water Roads Electricity Sanitation Security Financial services Clan Elders -46% -39% 0% -85% 0% -13% -89% 0% -6% -3% -1% -2% -1% -3% -6% -4% State Government 13% 15% 16% -58% 14% 14% -61% 16% -23% -20% -5% -17% -10% -11% -7% -7% Local people 1% 1% 6% -8% 5% -3% -4% 1% -48% -49% -20% -53% -23% -8% -50% -24% Local NGOs 25% 27% 30% 24% 28% 27% 17% 30% -7% -4% -12% -10% -11% -11% -21% -7% International NGOs 91% 84% 88% 90% 77% 95% 60% 89% 12% 11% 6% 6% 3% 16% 0% 25% Religious leaders -1% -44% 1% 0% 0% 1% -1% 0% -4% -4% -5% -2% 0% -7% -6% -1% Local administration -69% -63% 19% -71% 32% -34% -43% 15% -11% -18% 5% -11% 4% 17% 3% 17% Police 0% 0% 0% -77% 0% -2% -52% 0% -2% 0% 0% -3% 0% -3% 41% 0% Local Courts 0% 0% 0% 0% 0% 0% 0% 0% 0% -1% 1% 0% 1% 1% 3% -2% Business community 0% -4% -4% 0% 6% 0% 0% -2% -31% -33% -28% -24% -19% -15% -22% -15% Ugaas/Sultan -43% -26% 0% -29% -1% -4% -30% 0% -2% 1% 0% -1% 0% 0% 0% 0% Diaspora 0% 0% 0% 0% 0% 0% -1% 0% -2% -7% -3% -3% -2% -6% -8% -15% Foreign Governments 0% 0% 0% -33% 0% 0% -62% 2% -1% -1% -1% 0% -2% -1% -24% -3% Women support groups 0% 0% 0% 0% 0% -1% 0% 1% 0% 0% 0% 1% 0% 2% 0% 0% Dinsoor 293 Figure 10(b): Actors Responsible for Service Provision (changes between 2017 and 2018) • While the results vary by settlement, at least two significant trends can be identified. 1. INGOs are seen as having an increased role in the provision of nearly all types of services across the cluster. 2. The role of both state and local government in the provision of roads, key PIKA activities in both areas, decreased in both Dinsoor and Baardheere. • The central role played by INGOs and NGOs in service provision was confirmed in qualitative interviews. FGD and KII interviews indicated the existence of other providers, including local and international NGOs, diaspora community, local businesses, humanitarian organizations, telecommunication companies, money remittance companies, banks, and water and electricity companies that were involved in service provision besides the government. • Illustrative of the general trend across the interviews, an elder woman from an FGD in Baardheere aptly summarized, “There are many NGOs in the district that help in food aid distribution while some offer cash aid for the orphans in the community. There are also individuals that set up INGOs that work in various sectors within the community. The most important ones are the youths that mobilize themselves to develop their community. For example, these youth organizations work in schools and offer affordable education to the people living outside the town who cannot afford to attend the well-established schools.” 2. In terms of satisfaction with services, there was a perceived improvement across the cluster with the exception of Baardheere (decrease in roads, sanitation, electricity, security, financial services, police, and drinking water). TIS+ was specifically criticized by respondents for poor quality of roads and airstrip. • According to the household survey, satisfaction levels across most services generally increased for Dinsoor, El Wak, and Garbaharey. Baardheere, on the other hand, generally saw a decrease across services, including significantly decreases between 5 – 35% for drinking water, roads, sanitation, electricity, security, financial services, and police. Garbaharey Health Education Water Roads Electricity Sanitation Security Financial services Health Education Water Roads Electricity Sanitation Security Financial services Clan Elders -35% -33% -40% -33% -40% -36% -5% -38% -1% 0% 0% 0% 0% 5% 11% 0% State Government 4% 5% 6% 4% 7% 10% 18% 2% -15% -14% -22% -9% -24% -21% 9% -18% Local people 56% 56% 31% 32% 28% 30% 31% 30% 34% 47% 44% 16% 42% 19% 27% 26% Local NGOs 49% 49% 30% 30% 16% 21% 21% 35% 6% -9% -30% 2% -16% 14% 8% -7% International NGOs 16% 8% 14% 7% 2% 39% 1% 30% 10% 7% -16% 9% -8% 20% 8% 0% Religious leaders 8% 8% 8% 8% 8% 8% 22% 7% 1% 47% -1% 1% -1% 16% 13% 5% Local administration 25% 27% 23% 38% 18% 26% 27% 20% -4% -34% -24% -5% -28% -1% -6% -26% Police 0% 0% 0% 0% 0% -1% -6% 0% 0% 0% 0% 11% 0% 12% 3% 2% Local Courts 0% 0% 0% 0% 0% 0% 0% 0% 0% -1% 0% -1% 0% 3% 11% -1% Business community 14% 9% 25% 8% 29% 1% 0% -8% 9% -7% -35% 5% -47% 16% 7% 19% Ugaas/Sultan 0% 0% 0% 0% 0% 0% 13% 0% 0% 0% 0% 0% 0% 2% 9% 0% Diaspora 12% 23% 11% 13% 8% 5% 0% 13% 13% 11% -3% 8% -1% 2% 2% -9% Foreign Governments 0% 0% 0% 0% 0% 0% 0% 0% 0% 1% 0% 0% 0% 0% 7% 1% Women support groups 0% 0% 0% 0% 0% 0% 0% 0% 3% 1% 0% 3% 0% 7% 7% 2% Elwak 294 • The qualitative interviews provide some insight into the decreasing trends in Baardheere. TIS+ was specifically criticized by respondents for poor quality of roads and airstrip and were compared to services provided by other actors. • Interviewees explained the FAO and ICRC had a long presence in Baardheere that they usually worked through local partners, particularly when focused on more short-term humanitarian assistance. • They further praised the private sector and diaspora members for their support of community development projects. • Echoing findings from the household survey, only a few respondents mentioned any service provision by government and even these few mentions were mostly limited to the provision of security which in turn allowed other partners to provide services. 3. In terms of “security as a service,” there were an increase in the perception of state government provision of security in non-TIS+ settlements but a decrease in TIS+ settlements. • Factors explaining the divergent results between non-TIS+ and TIS+ settlements were not immediately apparent based on the quantitative and qualitative collected during the midline. • However, as part of the team’s additional interviews for its contribution analysis, interviewees identified that the location of the non-TIS+ settlements likely help explain the increasing perception of state government providing security. • Garbaharey is the administrative capital of Gedo region and therefore expected to have a larger state security presence, while El Wak, located along with boarder with Kenya, has been a recent theatre of operations by Kenyan Defense Forces, likely working in partnership with Somali security forces. 1(d): Reduced Conflict over Resources To analyze this result area, the team examined changes in perceptions on (1) the level of conflict over resources and (2) trust across various actors to resolve conflicts over resources. 1. There was a notable perceived reduction in resource-based conflicts in Dinsoor, which ran counter to the general trend of increased resource-based conflict across other settlements in the cluster. • The reasons for Dinsoor’s notable success in reducing resource-based conflicts are not immediately apparent based on the HH survey data collected. • While qualitative interviews (FGDs and KIIs) credit the local Peacebuilding Committee, clan elders, and district and state government with helping to resolve conflict, these interviews equally conveyed a sense that resource-based conflicts continue to linger in Dinsoor, particularly around issues related to land, housing, water, and access to pasture land, with the minority clans bearing the brunt of such conflicts. • This was evident in numerous interviews. As stated by one young man from an FGD in Dinsoor, "Yes, there are disagreements over land disputes, which occur in the district due to annexation of clan lands. This becomes a base of misunderstandings in the district and often affects the minority clans in the district. These minority clans are always looked down upon by the major clans. This happens repeatedly.” 2. There were large perceived increases across cluster in trust placed in local structures to resolve conflicts (clan elders, religious leaders, and Ugaas/sultan). Positively, TIS+ settlements also demonstrated large increases in trust in state and local government levels to resolve conflicts. • This finding was confirmed by qualitative data (KIIs and FGDs), which emphasized the centrality of local structures (including peace committees) in resolving conflicts. • As one male elder from Dinsoor stated, “The elders, the chiefs, the clan leaders, the peace committee, and most importantly, the local authority resolve the conflicts.” The assessment team was not able to find any specific examples of TIS+ activities directly supporting state or local governments helping to resolve conflict. 295 1(e): Greater Citizen Confidence in Government To analyze this result area, the team examined changes in (1) the team’s “Good Governance Index;” (2) citizen confidence towards government; and (3) perceptions on extent of corruption. 1. Both TIS+ and non-TIS+ settlements demonstrated increased scores on their “Good Governance Index” mainly in service provision, confidence in government, and collaboration with community members; however non-TIS+ settlements demonstrated slightly stronger results. • The “Good Governance Index” (GGI) is a composite measure that considers changes to indicators along three themes: (1) confidence in government, (2) service delivery, and (3) stakeholders’ partnership. The individual scores for each settlement in both TIS+ and non-TIS+ areas is provided in Figure 11 below. Because the index score was a composite measure, the direction of effects varied, positive and negative between baseline and midline data, in the individual survey questions constituting the good governance index. • The observed positive trends in TIS+ settlements were mostly explained by survey questions on service provision, public trust, transparency (the local administration sharing information), accountability (reduction in corruption levels both at federal and local administration levels), and representation (federal officials not being directed by clan interests in decision-making). • Overall, there were more positive changes for survey questions explaining good governance in non￾TIS+ settlements as compared to TIS+ settlements. 2. There were no significant changes in perceived citizen confidence towards government (all levels) in Dinsoor but statistically significant decreases for both state (-19%) and local governments (- 16%) in Baardheere. • Interestingly, the qualitative interviews for Baardheere were much more positive than survey results would indicate, particularly for local government, and to lesser degree for state government. • Respondents stated that they had increased trust in government because of involvement in development planning, improved international and foreign policy, and provision of security/other services as well as functional judicial system and fidelity to the rule of law. • This variation indicates a lack of impact of TIS+ activities at the cluster level, particularly in areas where settlements vary in context, political climate, and access to service Figure 11: Gedo Bay Good Governance Index Measures, Changes from 2017 to 2018 2017 2018 2017 2018 TIS+ non TIS+ 296 3. Through improved government information sharing with communities and improved social service delivery, the perceptions of corruption at local and federal levels was reduced across the cluster. • Encouragingly, perceptions of government corruption, at both local and federal levels, decreased between baseline and midline across the cluster, with Dinsoor showing particularly large improvements. • Qualitative interviews (both KII and FGD) indicate that community members recognized government efforts to increase their and that this was attributed to helping reduce overall corruption. • Importantly, interviews also emphasized that efforts by government to improve social service￾delivery, as well as to help mediate and resolve local conflicts, helped strengthen the relationships between government and communities, validating several of TIS+’s key underlying programming assumptions. 1(f): Reduced Support for Violent Extremism To analyze this result area, the team examined perceptions around (1) youth vulnerability to extremist recruitment and (2) extent the respondents reported extremist groups were “not at all welcomed.” 1. There were wide variations across the settlements on the perception of youth being vulnerable to extremist recruitment with Baardheere reporting a slight increase in youth susceptibly to recruitment and El Wak reporting a large decrease. • The household survey showed there was an increase in Baardheere (+9%), considerable decrease in El Wak, (-34%), and non-significant results in Dinsoor and Garbaharey. • The qualitative interviews do not immediately indicate either why there was a slight increase in Baardheere or such a substantial decrease in El Wak. • However, follow-on interviews conducted for the contribution analysis postulated that the large decrease in El Wak is most likely explained by the intense and ongoing military campaigns in campaign and presence of Kenya Defense forces. 2. TIS+ settlements showed no significant changes in respondents indicating extremist groups were “not welcomed at all” in their settlements whereas non-TIS+ settlements both showed notable increases in respondents indicating extremist groups were not welcomed. • While the qualitative interviews did not identify any immediate explanation for the differing results between TIS+ and non-TIS+ settlements, the location of these settlements is likely a significant contributing factor. • Both Garbaharey and El Wak, one the administrative capital and the one a key area of ongoing military intervention, are likely to host higher levels of security personnel and also to hold a higher number of public awareness campaigns combating violent extremism. 297 Q2: Contribution of TIS+ Approaches Key Assessment Question 2: Were TIS+ approaches used in the cluster, e.g. the Community Consensus Process itself, instrumental in contributing to the achievement of TIS+ results described in Question 1? TIS+ has produced a number of guiding documents which outline some standardized approaches that the program will implement across its implementation clusters. As these approaches were intended to be program-wide, the assessment team analyzed these approach through the use of “contribution stories” (see Section X). These stories critically examine the extent to which TIS+ has contributed to improvements in higher-level results through the use of three specific approaches (1) Community Planning and Oversight (CPO); (2) Technical Coordinators (TCs), and (3) Sports for Peace and Development (SPD). While the contribution stories reference examples across the clusters, this section will focus on highlighting additional examples and influencing factors that impacted TIS+’s approaches in Gedo/Bay. Given its importance to TIS+’s overall programing approach, this section will focus on analyzing the extent to which TIS+ implemented its intended CPO process, which includes the community consensus process, in Gedo/Bay. The section will then turn to a brief analysis of the “cluster approach” itself and its relevance to the Gedo/Bay context. TIS+’s approach for working with TCs will be covered in the Banadir cluster report and TC contribution story, and results from TIS+’s “Gender and Social Inclusion” (GESI) approach will be examined under Question 4. Community Planning and Oversight (CPO) Approach The use of community planning and oversight is at the core of TIS+’s programming approach. It emphasizes the importance of the process not just end product and is key to distinguishing TIS+ as a stabilization program rather than a more traditional development and infrastructure program. In short, the approach specifies how the program will engage community and government actors to plan, implement, oversee, and utilize PIKA activities and how this interaction will help the program achieve higher-level outcome results, such as increased citizen engagement and confidence in government, and reduced conflict over resources and support for violent extremism. TIS+’s specific approach to CPO is articulated in a number of guiding program documents and throughout its periodic reports. • “Process for Engagement” document explains that “the TIS+ model is based strongly on a community/government programming approach. TIS+ activities will work to foster the relationship between community members and government representatives and supporting Somali leadership in those processes. Particularly, supporting Somali self-efficacy in governance is a priority, which supports USAID’s goals in Somalia and New Deal Principles. TIS+ activities will contribute to improvements in government service delivery, leading to strengthened governance and community cohesion. Communities will be supported with access to livelihoods, income generation, and basic community infrastructure.”21 • “Community Contracting Guide” explains that TIS+’s CPO approach is intended to complement FGS’s ongoing District Council Formation process and emphasizes the “leadership role of the inclusive community, including women, youth, traditional authorities, and minority clans.”22 Communities are expected to collectively prioritize their needs through consensus planning and then undertake a “more detailed planning to directly participate in the implementation and long-term management of the project, as well as to help manage resources with appropriate participation and oversight from the local authorities.”23 The guide posits that “as a result of this process, and with appropriate support and training, the communities gain confidence, and organizational and negotiation skills, enabling them to engage with external partners and the local authorities on an equal footing. 21 AECOM. 22 MOIFAR, "Community Contracting Guide," (2018). 23 Ibid. 298 The direct involvement of the community leaders, with local, state and federal government counterparts and [TIS+], is intended to build confidence and accountability to address nation-wide stabilization concerns and enhance local ownership of projects and processes. As stabilization projects are successfully implemented in a transparent and accountable process, trust is built between the community and all the three levels of government . . . to deliver basic services and to construct local government infrastructure, in a more participatory and accountable manner.”24 • Finally, the “Consensus Process Guide” explains that a proper CPO process can also help mitigate and reduce conflict over resources. The guide outlines a 10-step process for implementing the initial community consensus process, which includes initial steps working with communities to conduct conflict and resource maps and introduce the concepts of “Conflict Sensitivity” and “Do No Harm.” The guide then instructs facilitators on how to support communities to identify, prioritize, and reach consensus on activities, form Project Oversight Committees, and how to conduct strategic communication and dialogue events.25 A critical feature of TIS+’s CPO process is the formation of POCs. As explained in TIS+’s “Operational and Management Guide.” “A [POC] is a community-based entity created for the Transition Initiative for Stabilization Initiative (TIS+) Program. The POC is a Joint Government–Community Implementation Unit created for a project location to oversee the project delivery from start to finish. The POC works with other stakeholders, including other international organizations, which serve to strengthen government and community interaction. The POC acts as a monitoring and accountability mechanism while mobilizing the community and increasing its awareness of the project for joint implementation. In addition, POCs add an element of sustainability by providing an opportunity for communities to take ownership of project maintenance through the formation of a lasting relationship with state and federal government officials.”26 The important role played by the POCs was emphasized in the final evaluation of TIS, which found that POCs play a vital role in ensuring community participation and oversight. Specifically, the evaluation recommended that “the POCs’ role at the community level should be strengthened and institutionalized. POCs should be provided with adequate training to allow adequate monitoring of projects. The contractor and the community must properly understand the POC’s role.”27 Findings of Results of TIS+ Approach 1. TIS+ implemented most of its intended CPO steps but substantially deviated from the original design when it came to the establishment, training, and use of the POCs. • In Gedo/Bay, TIS+ held a 3-day “District Consensus Building” workshop in Baardheere on May 2–4, 2016, involving 42 participants28 and a similar 3-day session in Dinsoor on June 6–8, 2016, involving 43 participants of diverse backgrounds.29 • Interviews further confirmed that the workshops followed the 10-step process outlined in the “Community Process Guide.” Interviews with TIS+ staff further indicate that district-level POC members were identified during the last day of each workshop and the agendas from the workshops confirm that sessions were indeed planned to identify these members, although the assessment team could not independently verify that these sessions actually took place (it should be noted that the team has no reason to suspect that they did not). 24 Ibid. 25 Federal Government of Somalia Ministry of Interior and Federal Affairs, "Consensus Process Guidebook." 26 AECOM, "General Principles of the O&M Plans for Tis+ Supported Infrastructure Activities in Somalia," (February 2018). 27 IBTCI, "Final Performance Evaluation of the “Transition Initiatives for Stabilization” (Tis) Project," ed. Somalia Program Support Services (June 2016). 28 Participants included stakeholders from the Jubaland State of Somalia, Somali Federal Government, the Local Administration, youth, women, elders, traditional leaders, religious group and the local private sector among others. 29 Participants included representatives from the interim South West Administration, the Somali Federal Government, the Local Administration, youth, women, elders, traditional leaders, religious groups, and the local private sector. 299 • However, interviews with the same TIS+ staff members also revealed that very limited training was provided to the POC members during these workshops and that they mostly just discussed the anticipated role of the POCs as they looked to recruit volunteers. • Based on both document review and interviews with TIS+ staff, substantive training of the POCs did not take place until nearly 24 months after the establishment and the subsequent reconstitution of the POCs. • While TIS+ had originally intended to form POCs at the district level, the program decided in late 2018 that this approach might lead to a concentration of power in the hands of a limited group of people and thereby present opportunities for corruption and that it would be more prudent to establish Pool of In-Kind Activities (PIKA)-level POCs. • Accordingly, PIKA-level POCs were established in November 2018 in Baardheere and January 2019 in Dinsoor. This meant that all of the new POCs established in Gedo/Bay were established after the completion of construction, with the exception of the POC in Baardheere, which was established just two months before the completion of construction. • The evidence, therefore, suggests that the POCs had very little formal engagement in the activities in their intended capacities and did not have the opportunity to fulfill their intended role of providing community oversight and regular engagement with local authorities. • While the capacity-building and function of the POCs deviated considerably from the original design, both interviews and the assessment team’s review of sign-in sheets confirm that the POC membership looks to be inclusive of both women and youth, and in most cases included the participation of government officials and representatives from the business community. • Interviews also confirmed that although the community contracting was not led by the POCs as intended, community contracting did occur in each of the Wave II areas. 2. TIS+’s CPO approach produced uneven results across the Gedo/Bay cluster suggesting the approach was implemented inconsistently across settlements or was unable to successfully mitigate other influencing factors specific to each settlement. • While the participation of different stakeholder groups in community consultations generally increased in both TIS+ and non-TIS+ areas across the cluster, they decreased considerably in Dinsoor. • However, despite reporting decreased participation across groups, respondents showed an increase in the perception that projects were selected based on community planning events. • Paradoxically, Baardheere demonstrated the exact inverse, reporting an increase in the participation rates across groups but a decrease in the perception of projects being selected based on community planning events. • Troublingly, there was a decrease in both Dinsoor and Baardheere in respondents reporting that “most” or “all” of the selected projects effectively addressed community priorities, as well as a substantial decrease in the awareness of POCs and no change in their perceived effectiveness. • Likewise, the results were not generally very positive related to shared asset management and quality of services. • The midline showed a perceived decrease in the role of state and local government in the management of nearly all assets, including in roads and meetinghouses, which were key PIKA activities. • In terms of service provision, international NGOs are seen as having an increased role in the provision of nearly all types of services across the cluster and the role of both state and local government in the provision of roads, key PIKA activities in both areas, decreased in both Dinsoor and Baardheere. • While most settlements across the cluster reported an increase in citizen satisfaction levels with services, Baardheere saw a decrease and the TIS+ constructed road and airstrip were specifically identified as being of poor quality. 300 • On the positive side, Dinsoor was noted as a unique “positive deviant” and witnessed a substantial reduction in resource-based conflicts were other settlements across the cluster saw increases. Likewise, both Dinsoor and Baardheere reported reduced levels of perceived corruption by both the federal and local government. 3. There were several other actors utilizing similar CPO approaches across the cluster. • There was a wide and diverse set of groups of actors implementing activities in each of the TIS+ settlements reviewed during Wave II, most of whom reported utilizing some aspect of community engagement, planning, and oversight. • The assessment team identified at least seven different organizations working in Baardheere and 11 organizations working in Dinsoor. • Both KIIs and FGDs conducted during the midline indicate that there has been a marked increase from these development partners to more systematically engagement with communities at the outset and involve community members in the planning and design of activities. • Likewise, other major partners, such as Danish Refugee Council, Finnish Church Aid, Nordic International Support Foundation (NISF), and International Committee of the Red Cross, as well as UN agencies such as the Food and Agriculture Organization and World Health Organization report specifically using POCs or established community structures such as the Peacebuilding Committees to guide the planning and oversight of their committees. 4. TIS+’s CPO approach supported positive but small improvement in citizen awareness, engagement, and satisfaction with services but has not yet had a measurable influence on the program’s other intended higher-level result, such as increased confidence in government. • While the midline assessment found evidence of TIS+ achieving notable results in terms of community awareness and participation, as well as local and state government service provision and management of shared assets, these results varied across settlements in Gedo/Bay. • TIS+’s results are also limited by the lack of a consistent implementation approach across the program. While the community consensus workshops look to have been consistently implemented, the form and timing of other key parts of the CPO process varied across settlements. • This includes a difference in the way that community contracting took place across the settlements, and most notably, significant deviations on how POCs were formed, trained, and utilized throughout the process. • TIS+’s own program design documents clearly highlight the central role that POCs play in the overarching CPO process; however, evidence indicates that TIS+ did not prioritize their role until late in the PIKA process, almost always only after the construction of the different structures was already complete. • To its credit, TIS+ appears to recognize the need to systemized its CPO approach. In its 2018 annual report, TIS+ identified a key lesson learned so far is the need to “invest more in foundational governance through processes and programs [to]l strengthen transparency and participation… There will be a greater emphasis on community accountability, participatory O&M and linking stabilization initiatives to transitional governance.”30 TIS+’s “Cluster” Approach Another critical feature of TIS+ programing approaches features the selection of geographic “cluster” in which to focus implementation. As explained in TIS+’s “Process of Engagement” guide, “under USAID’s direction and in coordination with relevant FGS ministries, a ‘clustering’ approach to district/community selection will often be employed, in which a tight grouping of districts/communities will be selected so to magnify stability impacts across a region.”31 The guide goes on to explain that the “the cluster approach and geographic expansion will allow the program to marshal resources to critical geographic areas, maximize 30 TIS+ “2018 Annual Report,” pg. 43. 31 AECOM, "Transition Initiatives for Stabilization Plus (Tis+) Process of Engagement," (March 2016). Pg. 3. 301 coordination efforts and capitalize on existing resources and security assets within specific regions.” Interviews with USAID and TIS+ staff further clarified that the cluster approach was designed with the aim of having interventions that could contribute to the opening up and stabilizing areas around main supply routes that linked regions. Findings on Results of “Cluster” Approach 1. The “cluster” approach was not effective in Gedo/Bay cluster as it did not properly consider distinct demographic and administrative differences between Gedo and Bay as well as the remaining influence of AS when combining these areas into one cluster. • Gedo and Bay's demographics are disparate including the predominant clans in the region as well as the dialect (in Bay, the predominant dialect is the ‘Maay’, while in Gedo, the more prevalent ‘Maxaa Tiri’ dialect is used). • The regions also belong to two different federal member states. While there have been some cross￾border clashes in the two regions, most of the conflicts in this area have been micro-conflicts in nature, often pitting sub-clans within the same clan against each other. • The obvious differences are further compounded by the fact that supply routes linking the two have remained blockaded by AS for years. 2. Planned progress by AMISOM military campaigns and the opening of “corridors” between clusters did not occur as expected and hampered the utility of the cluster approach. • The cluster approach was contingent on variables that were out of the control of TIS+, such as planned military campaigns by AMISOM. A lot of the planned activities did not materialize because the military campaigns did not take place. • However, because the cluster approach was engrained in the TIS+ design, TIS+ had to implement within this approach, despite the fact that main routes connecting the settlements had not opened up and the settlements did not share many similarities. • In fact, some of the clusters were in different federal states and did not share distinct characteristics that would have been present had the supply corridors opened up, which defeated the purpose of the approach. 3. The cluster approach was hampered in Gedo/Bay as it did not adequately take into consideration entrenched national and regional political interests that resulted in heavy contestation, and cross border conflicts. • Gedo and Bay have been historically of interest to neighboring Kenya and Ethiopia with armies from both countries regularly maintaining presence in these regions. • Neighboring countries often ensured they could exert a sphere of influence by propping up local political elites who then had outsize influence on the stabilization of the regions. • The result has been heavy contestation, cross border conflict and deeply entrenched interest that constrain the ambitions of the cluster approach. If anything, these dynamics show that TIS+ may have been better suited to addressing micro-stabilization issues at settlement level since the cluster level attracted the entrenched interests. It’s important to note that other stabilization actors’ plans, while distinctly not referred to as ‘cluster approaches’ were also reliant on the opening up of the transport corridors. Many stabilization actors planned around AMISOM’s much publicized ‘Jubba Valley’ offensive to open up parts of Gedo and Bay but this did not materialize. 302 Q3. Government Involvement in Shared-Asset Management Key Assessment Question 3: Did Government at all appropriate levels contribute to asset management in communities in which TIS+ was implemented? If so, how? Did Government at all appropriate levels have both the capacity and the will to contribute? To help analyze this question, the assessment team divided it into three parts: to what extent did government contribute the (1) provision of shared assets; (2) management of shared assets; and (3) what was the capacity and will of government to do so. Findings on Provision of Shared Assets 1. Federal government, which was mainly credited with security provision, was perceived to have a minimal role directly in the provision of shared assets. • According to the data collected during the midline, public perception of the federal government provision of shared assets in Gedo/Bay appears to be minimal. • Interviews rarely referred assets provided but the federal government other than crediting them with providing improved security which they noted was an important enabling factor that allowed other actors to the provide various services. 2. Perceptions of state government’s provision of shared assets varied across settlements. • The midline data indicates large variations between the role of state government in service provision among the settlements assessed in Gedo/Bay. • State government was perceived to have a considerably increased role on the provision of most assets in Dinsoor (TIS+) and El Wak (non-TIS+); however, was then also perceived to have considerably decreased in Baardheere (TIS) and Garbaharey (non-TIS+). • Interestingly, state government provision of security decreased in both TIS+ settlements but increased in both non-TIS+. State government provision of roads decreased across most settlement, with the exception of a small increase in El Wak. 3. Perceptions of local government’s provision of shared assets was dependent on specific asset and their location. • In TIS+ settlements, there was a consensus that local government had increased its role in the provision of water, electricity, and financial services, assets which are all outside the scope of TIS+’s support. • Worryingly, local government was perceived to have a strongly decreased role in the provision of roads, which was a key focus of PIKAs in both Dinsoor and Baardheere. • Local government in El Wak (non-TIS+) was perceived to have performed uniquely well and saw increased positive perceptions of their involvement in the provision of all assets measured. Findings on Management of Shared Assets 1. Federal government, which was mainly credited with security provision, was perceived to have a limited role in the management of shared assets. • As with service provision, the Federal government was widely perceived as having a very limited role in the management of shared assets. • The limited mention of the Federal government’s role in the management of shared assets were mostly limited to the provision of security which allowed other actors to manage shared assets. 2. State government’s role varied between TIS+ and non-TIS+ settlements in cluster. • TIS+ saw a general decrease for state government management of shared assets (except for increases in schools and health centers in Dinsoor and market facilities in Baardheere), whereas non-TIS+ settlements saw increases in all types of assets listed. 303 3. Local government generally saw a perceived decrease across cluster except for El Wak. • Respondents generally reported a decreased role of local government across the cluster, except for in El Wak where local administration saw notable increases across all assets listed. • However, the qualitative interviews were more positive and credited local administrations with improvements in their coordination, management, and oversight of shared assets. Findings on Capacity and Will of Government 1. Respondents acknowledged the will of local and state governments to improve service delivery but also that they were substantially limited in their resources to effectively provide, manage, and maintain shared assets. • Respondents generally recognized the limited resources that state and local governments currently have to help provide and manage shared assets. • With these resource limitations in mind, respondents also frequently explained that the government should continue to help attract and approve projects funded by donors and implemented by NGOs/INGOs. 2. Recognition that management is a shared responsibility between government and community, and of the need for a strong role for NGOs, private sector, and diaspora. • Respondents recognized that Gedo/Bay has long been marginalized with very limited government provision or management, and that this role has generally been filled by INGOs/NGOs, private sector, and diaspora. • Respondents also recognized that the effect management of assets was a shared responsibility that relies on actions by community actors to help support government representatives. 3. TCs provide a viable strategy to increasing both the will and capacity of government to participate in shared asset management; however, the roles and linkages of TCs to the program can be strengthened. • The assessment team found evidence across program documents, outputs from the TCs, and interviews with government partners that the work of the TCs contributed to both (1) improved coordination between the government and TIS+; and (2) increased capacity by the government to engage with stabilization actors and donors and better planning by the government. • Interviews with government counterparts also conveyed an appreciation of the work of TCs and saw them both as providing a point of contact with TIS+ programing, and promoting government ownership of TIS+ interventions, as well as building capacities in their ministries, but the clear emphasis was on the need for, and importance of, institution building. • However, the midline assessment also found that the TCs have faced significant challenges thus far. Interviews with TCs confirmed the challenge of dealing with high turnover often preceded by a political impasse, which often disrupts the project timelines. • There is also the challenge of sustaining positive results beyond the short term and managing expectations in order to achieve long term capacity and effective coordination. • TIS+ TCs highlighted that they constantly worried about the own tenure and funding priorities, which affected their ability to commit to long term visions by the government institutions they were supporting. • Both the TIS+ program and TCs noted the challenge of subsuming the authority of the government institutions in some instances, with TCs acting as representatives of the government, rather than externally injected capacity. • Finally, the political and security environment had a strong influence on the contribution of the TCs. In particular, the frequent political crises which contributed to the high turnover in the leadership of the Ministries, often constrained the momentum built by the TCs. 304 Q4: Efforts to Promote Inclusion Key Assessment Question 4: Were TIS+ activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? Findings on Inclusion of Women 1. With the exception for Dinsoor, there was a perceived increase across the cluster in those reporting that they believed that women were considered equal to men. • Survey results indicate a 43% increase in Baardheere and 17% increase in each of the non-TIS+ settlements. • Dinsoor was a strong, negative outlier witnessing a 32% decrease in respondents reporting that women were equal to men. • The reasons for this significant decrease could not be immediately explained by the data available in the midline. • Although women and men in this cluster had relatively similar views of gender equality, qualitative evidence (FGDs) indicated that clan rules limit the participation and engagement of women in community planning and consultation events. • However, other FDG gave a contrasting view emphasizing increasing role of women in politics, peacebuilding, and asset management, monitoring, and oversight. 2. TIS+ has succeeded in ensuring women’s participation in POCs although further effort is needed to ensure equal participation. • The assessment team consulted the original sign-in sheets and confirmed that all five POCs in Gedo/Bay included women members, usually accounting for around one third of the membership. • While this number is below TIS+’s overall target of 50-50 gender participation on the POCs, it nevertheless demonstrates a concerted effort by TIS+ to ensure the involvement of women. Findings on Inclusion of Youth 1. Youth were reported across the cluster to be actively involved in the discussion of community priorities and projects; however, their participation in the management and oversight of shared assets was mostly limited to sport facilities and playgrounds. • The team’s qualitative interviews (both FGDs and KIIs) suggest that youth play an active role in the development of their communities and that most settlements have their own “youth committees.” • However, while youth appear to have played a prominent role in the identification and prioritization of shared-assets in TIS+ areas, their role in the management of those assets are often confined to sport facilities and playgrounds. • This was particularly problematic in Gedo/Bay as the only sports facility built at the time of the midline was the Dinsoor Sports stadium which unfortunately was blown up shortly before midline data collection. • While the role of youth in the management of shared assets is expected to improve with the role out of TIS+’s new “Sport for Development and Peace” (SDP) strategy, this strategy was finalized after the midline data collection and those its potential results could not be captured here. “I think the community members should all be aware of the necessary projects being implemented. This will allow them to have positive thoughts about what is being implemented in their district. On the other hand, NGOs should increase the way they are respecting the community members and they have to share the full information about the projects. There was a time [when] the NGOs used to start the implementation phase of the projects without sharing the information with the community, but this time the community members have a good chance of involvement in the planning phase of the interventions. Currently, I believe the community members are involved in feasibility studies of the projects.” – Women's leader, Dinsoor 305 2. Similar to baseline findings, several FGDs with youth expressed frustration in feeling that their involvement was marginalized and limited in community planning and development across the cluster (both TIS+ and non-TIS+). • Some of the factors impacting this perception among youth during midline were (1) lack of focus on economic opportunity and employment for youth, (2) dominance by elders in decision-making and planning, and (3) weak understanding of local conditions and youth pressures by INGOs and stakeholders working in Gedo. Findings of Inclusion of Minority Clan Inclusion and Improved Community Cohesion 1. There was very limited data from TIS+ on minority clan involvement or how this influenced programming. • There was little evidence that TIS+ systematically monitors conflicts that impact minority clan engagement and that this informs programming. • However, there was evidence from qualitative interviews (both FGDs and KIIs) that clan dynamics as well as a history of cross-border trade and migration and competition over resources impacts political settlements, trust, and civic engagement among clan groups in the Gedo/Bay region. 2. For clans and sub-clans, tactical affiliation with AS has been a useful means of countering a dominant rival clan. AS has successfully exploited minority clans and other grievances. • From FGDs and KIIs conducted three aspects of this form of identity politics are worth highlighting, particularly as it relates to minority clans: 1. Affiliation or alliance with AS is generally very tactical and fluid across Gedo; 2. Accusations that a clan is “pro-AS”— whether true or not—is an easy way to harness external (Ethiopian, Kenya, African Union, US) support against a local rival; and 3. At least some clashes that appear to be AS vs. Somali National Army in this cluster are actually clan clashes and not affiliated with AS, which continues to impact security. • This dynamic has increased local levels of violence, particularly among small sub-clans, and has been the main cause of instability in Gedo in recent years, causing significant displacement. • This, combined with environmental stress, is causing micro-conflicts to be more serious for local administration and state governance as evidenced in KIIs. 3. Community Cohesion Index scores increased in non-TIS+ settlements but decreased across TIS+ settlements due to a large decrease in Baardheere. • As with the “Good Governance Index,” this index is also a composite measure, but this time focused on measuring changes in interpersonal and institutional trust, social and political inclusiveness, and network diversity. • As illustrated in Figure 12 on the next page, Dinsoor (+13%) and Garbaharey saw increases (+9%) in the index scores, while El Wak (non-TIS+) saw non-significant changes and Baardheere (TIS+) saw a decrease (-11%). • Interviews indicate that competition among two dominant clans in Baardheere likely accounts for the decrease in the community cohesion scores. • This perception is supported by the survey findings which found that there was a general decrease of “people who do not trust one another across clans,” except Baardheere which saw an 8% increase. 306 Figure 12: Gedo Bay Good Community Cohesion Index Measures/Changes from 2017 to 2018 307 CONCLUSIONS Q1: TIS+ Contribution Towards Intended Results Community Awareness and Participation • TIS+ provided a small but positive contribution to increased community awareness and participation in development processes in the Gedo/Bay cluster. However, this contribution did not translate into improved perceptions of projects meeting priority needs, which were likely tempered by local socio-political factors, prevalence of other actors undertaking similar activities, and considerable lag time between planning processes and the completion of activities. There was evidence that TIS+ completed many of its intended community planning activities. The program held community planning sessions in both Dinsoor and Baardheere that included a relatively large and diverse group of participants, selected district-level POCs at the end of these community planning sessions, and implemented its community contracting procedures to initial construction on its infrastructure projects. However, varied results across the two TIS+ settlements and comparatively stronger results in non-TIS+ settlements indicate that TIS+’s implementation and use of community oversight could be improved. There was a yearlong gap between the community planning session and start of PIKA activities. Additionally, community oversight was provided mostly through the use of district-level POCs which resulted in the participation of relatively few community members in project monitoring and oversight. While TIS+ did identify and address this issue, the use of PIKA-level POCs did not occur until the completion of PIKA activities in all but one isolated case. Further, TIS+ settlements actually witnessed a decrease in community perceptions that development projects were meeting priority needs. At least three likely contributing factors were identified as helping explain this result: (1) the influence of local socio-political factors in both settlements; (2) the top two priorities in the Baardheere community consensus process were not selected; and (3) a prevalence of similar development partners and activities. While these factors are difficult to mitigate entirely, TIS+ could have taken steps to limit their repercussions through more consistent, transparent, and inclusive community planning and oversight processes. The program could also have taken steps to better identify and address issues related to local political and clan tensions which ended up delaying activities and negatively impacting perceptions of bias. USAID, likewise, bears some of the responsibility for TIS+ not selecting the top community priorities in Baardheere as advised TIS+ to prioritize needs around the airstrip and community hall. Effective Shared Asset Management • There was no evidence that TIS+ contributed to more effective shared-asset management across the cluster, with results are generally stronger in non-TIS+ areas. The long delay in forming PIKA￾level POCs, the types of community infrastructure projects selected, were identified as a likely contributing factors to explain this result. The results on effect shared asset management were actually stronger in non-TIS+ areas than TIS+ areas at midline. TIS+ settlements saw both a decrease in awareness of oversight committees, as well as a decrease in the participation of local and state government officials, while non-TIS+ settlements generally saw an increase in these indicators. While the assessment team had expected to see changes in perceptions of oversight committee’s effectiveness in the management of at least roads and meeting houses in the TIS+ areas given that these were the type of assets that TIS+ PIKAs focused on, these results did not materialize. These results (or rather lack of) are most likely explained by the fact that activity level TIS+ POCs were not established in these areas until after data collection for the midline (October 2018). The results also indicate that district-level POCs are not an effective mechanism for increasing perceptions of the effectiveness of oversight committees, at least in the short-term. Worryingly, both local and state governments saw large 308 decreases in management of nearly all assets in TIS+ areas compared with increases for state government in both non-TIS+ areas, as well as increases for local government in El Wak. One key explanatory factor looks to have been TIS+’s initial use of district-level instead of activity-level POCs and then long delay in providing substantive training to these POCs, nearly 24 months after the initial community consensus sessions and after construction on the various activities had already started. Another is that the type of infrastructure chosen may also have influenced the frequency and quality of the POC in engaging in shared asset management. Interviews indicated that other POCs responsible for overseeing more direct service delivery-focused projects—such as the building of hospitals, health centers, and schools—had more frequent interaction with the community compared to POCs for TIS+ activities focused on constructing roads and air strips. One a more positive note, however, findings from qualitative interviews indicate a more nuanced understanding of effective shared-asset management among community members across the cluster and emphasized a shared sense of responsibility with local and state government in the management of shared assets. They also emphasize the role that NGOs and private sector play currently in the management of shared assets. That said, the majority of respondents maintained that government held the “first line of responsibility” for management and would then delegate secondary levels of responsibility to the community. Improved Delivery of / Satisfaction with Services • TIS+ provided a small but positive contribution to improved community perceptions around service-delivery and satisfaction with services in Dinsoor; however, non-TIS+ performed more consistently across the cluster and Baardheere witnessed a substantial decrease in satisfaction with both regular services and “security as a service.” There was evidence of important outcome-level results based on the services provided in Dinsoor. Interviews indicate that as a result of the airstrip renovation, flights in and out of Dinsoor are now more frequent and with fewer delays and cancellation. Qualitative interviews also indicate a strong appreciation for the Women’s Center in providing a safe venue for women to meet and discuss their needs and priorities. However, findings from qualitative interviews, as well as from separate references in the external MV&R reports indicate that poor contractor performance resulting in lower quality work and delays look to have been a significant contributing factor, especially related to road construction in Baardheere. A fair assessment of TIS+’s performance in this result areas also needs to consider the contextual factors that likely influenced these results. Although there is currently relative stability in some limited areas of the region, there are incidences of insecurity and clashes where inter- and intra-clan conflicts have reemerged following the removal of AS in key districts. Social services in Gedo are minimal, and although there has been increased effort in recent years to improve roads and access, the region is very remote and lacks even the most basic infrastructure except in some border towns. The proliferation of these border towns has put stress on the need for services and access to health care, education, and livelihoods. At baseline, service provision was low in all primary community development areas. Despite modest improvements, satisfaction with security and police was very low and AS provides a certain measure of security that the regional and state governments do not. In data analysis, it was clear that there is little community interface with government officials around community prioritization of development planning, and where this does take place it was primarily initiated by one of the stability actors working in the region. Likewise, the central role played by INGOs and NGOs in service provision and the influence this has on perception of government services should be recognized. Findings from FGDs and KIIs indicated the existence of plethora of other providers, including local and international NGOs, diaspora community, local businesses, humanitarian organizations, telecommunication companies, money remittance companies, banks, and water and electricity companies, many of who are better resourced than their government counterparts. 309 Reduced Conflicts Over Resources • There was no evidence that TIS+ contributed to reduced conflict over resources, and these resource conflicts have generally increased across the cluster since baseline. The mediation of resource-based conflicts also does not appear to have been a priority for the program and communities generally preferred the use of local structures (clan elders, peacebuilding committees, and local and even state administrations) to mediate these conflicts. While some of TIS+’s guiding documents generally reference the program’s intention to contribute to the identification of conflict issues and their potential resolution, the assessment found very limited evidence of the program actively mediating resource-based conflicts. While there were isolated examples of TIS+ staff helping to address ad-hoc community conflicts, there was no evidence that these interventions were symmetrized across the program and most conflict-resolution interventions seemed to be more focused on ensuring conflict sensitivity and “Do No Harm.” Findings from qualitative interviews (both FGD and KIIs) emphasized the prominent role that local structures currently play in mediating these conflict, perhaps indicating that TIS+’s reduced focus on this result area is justified. However, these same interviews also indicated that underlying community conflict issues continue to fester across the cluster and that the impact of the recent drought continues to put severe stress on some communities around issues such as food production and water for livestock and animals. This raises a potential concern for TIS+ to monitor throughout its continued implementation. Greater Citizen Confidence in Government • The assessment could not establish TIS+ contribution to greater citizen confidence in government given mixed evidence across the cluster. Overall, findings point that there was a decrease in confidence towards local and state government across cluster (no change for federal government) although specific findings in Good Governance Index measures and qualitative interviews indicated more positive results. Corruption was perceived to have decreased across cluster, in part because of more active citizen monitoring. However, TIS+ is part of a much larger group of program implementers using community monitoring and all but one of its POCs, a critical part of the program for supporting citizen monitoring, were not established until after the construction of all but one project. Encouragingly, findings from interviews (both KII and FGD) indicate that community members recognized government efforts to increase their information sharing with communities and that this was attributed to helping reduce overall corruption. Importantly, interviews also emphasized that efforts by government to improve social service-delivery, as well as to help mediate and resolve local conflicts, helped strengthen the relationships between government and communities, validating several of TIS+’s key underlying programming assumptions. Reduced Support for Violent Extremism • There was no evidence that TIS+ contributed to reduced support for violent extremism. TIS+ actually performed worse than non-TIS+ in reducing youth vulnerability to extremist recruitment and showed no significant changes in extremist groups “not welcomed” in TIS+ areas. However, potential results related to countering violent extremism are at the far end of the TIS+ intended results chain and is therefore expected to have indirect and minimal, if any, influence over these results. 310 Long-standing clan conflicts still cause political issues and severe mistrust. As pointed in the findings, AS is seen by some FGD and KII respondents as a “manipulator” of these sentiments. Minority clans also use this to their advantage and seek to align themselves strategically to gain influence over a rival clan or to gain resource access. This acts as an indirect system of support for AS. Findings suggested that most respondents, both male and female, described the security situation among settlements in this cluster as unstable. Respondents ranked security services very low. Findings from quantitative data suggest that youth in this cluster are not prone to forced recruitment in violent extremist groups, but some respondents in FGDs and KIIs noted that youth work for AS by providing logistical and administrative support. As pointed in the findings, there were wide variations across the settlements on the perception of youth being vulnerable to extremist recruitment with Baardheere reporting a slight increase in youth susceptibly to recruitment and El Wak reporting a large decrease. TIS+ settlements showed also showed no significant changes in respondents indicating extremist groups were “not welcomed at all” in their settlements whereas non-TIS+ settlements both showed notable increases. While findings from the qualitative interviews did not identify any immediate explanation for the differing results between TIS+ and non-TIS+ settlements, the location of these settlements is likely a significant contributing factor. Both Garbaharey and El Wak, one the administrative capital and the one a key area of ongoing military intervention, are likely to host higher levels of security personnel and also to hold a higher number of public awareness campaigns combating violent extremism. Q2: Contribution of TIS+ Approaches Community Planning and Oversight • TIS+’s CPO approach contributed to increased levels of community awareness and participation but did not result in higher-level outcomes given deviations from its original design on the timing of the establishment, capacity-building, and engagement of its POCs. While the midline assessment found evidence of TIS+ achieving notable results in terms of community awareness and participation, as well as local and state government service provision and management of shared assets, these results were usually only at the individual settlement level and were not consistent across clusters. TIS+’s contribution claim is also limited by the lack of a consistent implementation approach across the program. While the community consensus workshops look to have been consistently implemented, the form and timing of other key parts of the CPO process varied across settlements. This includes a difference in the way that community contracting took place across the settlements, and most notably, significant deviations on how POCs were formed, trained, and utilized throughout the process. TIS+’s own program design documents clearly highlight the central role that POCs play in the overarching CPO process; however, evidence indicates that TIS+ did not prioritize their role until late in the PIKA process, almost always only after the construction of the different structures was already complete. TIS+ seems to have followed most of its intended CPO steps but deviated substantially from the original design when it came to the establishment, training, and use of the POCs. While it is true that POCs form only part of the overarching CPO process, it is nevertheless a crucial part to ensure regular dialogue with authorities, community monitoring and oversight, and for the building of a sense of ownership and responsibility for the sustainability of the activities within the community. To its credit, the program decided in late 2018 that the approach of district-level POCs might lead to a concentration of power in the hands of a limited group of people and thereby present opportunities for corruption and that it would be more prudent to establish Pool of In-Kind Activities (PIKA)-level POCs. Accordingly, PIKA-level POCs were established in November 2018 in Baardheere and January 2019 in Dinsoor. This meant that all of the new POCs established in Gedo/Bay were established after the completion of construction, with the exception of the POC in Baardheere, which was established just two months before the completion of construction. The evidence, therefore, suggests that the POCs had very little formal engagement in the activities in their 311 intended capacities and did not have the opportunity to fulfill their intended role of providing community oversight and regular engagement with local authorities. “Cluster” Approach • The “cluster” approach was not effective in this cluster as it did not properly consider distinct differences between Gedo and Bay demographics and administrative location when combining these areas into one cluster. The approach was also hampered by the lack of expected military progress of AMISOM military campaigns and the opening of “corridors” between settlements. The cluster approach hampered the pace of implementation and limited TIS+’ ability to flexibly intervene in other opportunities at the settlement level as they arose such as the formation of district councils. Qualitative interviews with TIS+ staff and other stabilization confirmed the sentiment of frustration in the slow pace of military campaigns, which did not materialize despite actors planning and committing resource in anticipation. The main enabler of the cluster approach was conducive security along the main supply routes connecting the different settlements and this was out of TIS+’ control. With the knowledge of the pace and lessons learnt from past military campaigns such as ‘Operation Indian Ocean’ in 2014, the cluster approach should not have been in consideration as a strategic approach by TIS+. Q3: Government Involvement in Shared-Asset Management Provision of Shared Assets • Federal government was regarded to have a minimal role in the provision of shared assets, while the state and local government provision was based on specific assets and locations. The midline assessment showed that the public perception of the federal government’s role in the provision of shared assets in Gedo/Bay appears to be minimal. Interviews rarely referred assets provided but the federal government other than crediting them with providing improved security which they noted was an important enabling factor that allowed other actors to the provide various services. The data was more varied on the role of state government in service provision among the settlements assessed in Gedo/Bay. Interestingly, state government provision of security decreased in both TIS+ settlements but increased in both non-TIS+. Worryingly, local government was perceived to have a strongly decreased role in the provision of roads, which was a key focus of PIKAs in both Dinsoor and Baardheere. Management of Shared Assets • Perceptions of the role of government at all levels (federal, state, and local) generally decreased in TIS+ settlements. While the Federal government was regarded as having a limited role in the management of shared assets, apart for contributing to increase security which allowed other groups to manage assets, both state and local government was perceived to have a decreased role since baseline. As with service provision, the Federal government was widely perceived as having a very limited role in the management of shared assets. The limited mentions of the Federal government’s role in the management of shared assets were mostly focused on the provision of security which allowed other actors to manage shared assets. TIS+ saw a general decrease for state government management of shared assets (except for increases in schools and health centers in Dinsoor and market facilities in Baardheere), whereas non-TIS+ settlements saw increases in all types of assets listed. Respondents generally reported a decreased role of local government across the cluster. However, findings from the qualitative interviews were more positive than those from survey responses and credited local administrations with improvements in their coordination, management, and oversight of shared assets. 312 Capacity and Will of Government • Local and state governments were generally perceived as having the will to improve service delivery but were recognized to have limited capacity and resources to effectively provide, manage, and maintain shared assets. Encouragingly, this has resulted in community members also recognizing that they had a shared responsibility to help manage these resources and that other civil society and private sector actors were stepping in to fill the void left by the under￾resourced government actors. Findings from interviews generally pointed the limited resources that state and local governments currently have to help provide and manage shared assets. With these resource limitations in mind, findings also explained that these governments are and should continue to help attract and approve projects funded by donors and implemented by NGOs/INGOs. findings also suggested that Gedo/Bay has long been marginalized with very limited government provision or management, and that this role has generally been filled by INGOs/NGOs, private sector, and diaspora. They reported that the effect management of assets was a shared responsibility that relies on actions by community actors to help support government representatives. • TCs provided a crucial link between the government and the program, increasing the will and capacity of the government to participate in shared asset management. However, TCs were not adequately utilized and linked more strategically into the program. The assessment team found evidence across program documents, outputs from the TCs, and interviews with government partners that the work of the TCs contributed to both (1) improved coordination between the government and TIS+; and (2) increased capacity by the government to engage with stabilization actors and donors and better planning by the government. However, the assessment also found that the TCs have faced significant capacity challenges within their ministries, as well as in their own capacity, and are limited by a lack of clarity on their roles and contributions within the larger TIS+ program. Q4: Efforts to Promote Inclusion Inclusion of Women • TIS+ has taken a number of steps to increase the participation of women in its activities, and there was an increase in those reporting that they believed that women were considered equal to men with the exception of Dinsoor. TIS+ has ensured that women were represented on all five of its POCs across the Gedo/Bay cluster and generally accounted for one third of POCs members. While this number is below TIS+’s overall target of 50- 50 gender participation on the POCs, it reflects the realities of some of the prevailing social norms that TIS+ is facing in its communities and also demonstrates that the program has undertaken a concerted effort to ensure the involvement of women in its PIKA activities. The reasons for the decrease in Dinsoor of people reporting that women were equal to men was not immediately clear on data available in the midline but represents and area that TIS+ should explore and will also be further analyzed in the assessment team’s forthcoming Synthesis Report. Inclusion of Youth • Youth are actively involved in the discussion of community priorities across the Gedo/Bay cluster, however, youth also reported feeling marginalized in the project process and that their participation in the management and oversight of shared assets was mostly limited to sport facilities and playgrounds. 313 Youth appear to play an active role in the development of their communities and also have their own “youth committees” in most settlements visited. However, while youth appear to have played a prominent role in the identification and prioritization of shared-assets in TIS+ areas, their role in the management of those assets are often confined to sport facilities and playgrounds. This was particularly problematic in Gedo/Bay as the only sports facility built at the time of the midline was the Dinsoor Sports stadium which unfortunately was blown up shortly before midline data collection. While the role of youth in the management of shared assets is expected to improve with the role out of TIS+’s new “Sport for Development and Peace” (SDP) strategy, this strategy was finalized after the midline data collection and those its potential results could not be captured here. Some of the factors impacting this perception among youth during midline were (1) lack of focus on economic opportunity and employment for youth, (2) dominance by elders in decision-making and planning, and (3) weak understanding of local conditions and youth pressures by INGOs and stakeholders working in Gedo. Minority Clan Inclusion and Improved Community Cohesion • The lack of systematic monitoring of minority clan involvement, or at least documentation of that monitoring, limited the assessment’s ability to conclude on this results area or how this has influenced programming. There was little evidence that TIS+ systematically monitors conflicts that impact minority clan engagement and that this informs programming. However, there was evidence from qualitative interviews (both FGDs and KIIs) that clan dynamics as well as a history of cross-border trade and migration and competition over resources impacts political settlements, trust, and civic engagement among clan groups in the Gedo/Bay region. Additionally, there was evidence from the FGDs and KIIs conducted that AS has successfully exploited minority clans and other grievances through: (1) affiliation or alliance with AS is generally very tactical and fluid across Gedo; (2) accusations that a clan is “pro-AS”— whether true or not—is an easy way to harness external (Ethiopian, Kenya, African Union, US) support against a local rival; and (3) at least some clashes that appear to be AS vs. Somali National Army in this cluster are actually clan clashes and not affiliated with AS, which continues to impact security. This dynamic has increased local levels of violence, particularly among small sub-clans, and has been the main cause of instability in Gedo in recent years, causing significant displacement. This, combined with environmental stress, is causing micro-conflicts to be more serious for local administration and state governance as evidenced in KIIs. 314 RECOMMENDATIONS Strengthening TIS+ and/or USAID’s Contribution to Intended Results 1. TIS+ should ensure that the program follows a more consistent and complete community planning and oversight approach. Critically, this involves providing timely training and capacity building to POC members so that they can properly serve in their oversight roles for improved community ownership and effective shared asset management. . Increases in participation and capacity are distinctly linked to citizens feeling more competent to engage in decision-making, management, and oversight. Long-term CPO approaches are familiar in this cluster, but the quality of participation is still a challenge for the TIS+ project. While there is evidence of increased awareness of POCs in this cluster, there were few results demonstrating the increased effectiveness of the POCs. There is therefore a need for TIS+ to further train and support POC oversight and responsibility. POCs should be formed prior to construction. Increased emphasis on training will capacitate community members to engage more meaningfully in the oversight and management of shared assets and lead to higher-quality citizen and government interaction. Priority: High Responsible entity: TIS+ Timeframe: Short-Term (0-6 months) Resources Required: Minimal given that guiding documents and budget allocations have already been created and approved by USAID. 2. USAID should consider focusing on supporting more day-to-day government-led service delivery activities and infrastructure projects if it aims to influence higher-level results related to increased citizen confidence in government or improved perceptions of service delivery. The midline assessment indicated that TIS+ has made progress in increasing community awareness and participation in development processes but that its efforts are not translating into higher level results. If USAID is interested in supporting outcome level results related to citizen confidence in government in the future, it should consider supporting interventions that are focused on more frequent interactions between citizenries and their elected officials. Priority: High Responsible entity: USAID Timeframe: Medium-Term (6-12 months) as it plans its next stabilization program. Resources Required: Minimal as this mostly relates to internal strategy discussions. 3. TIS+ should continue its informal conflict monitoring and PESA assessments but take steps to systematically document how results from these assessments influence changes in its programming. While TIS+ undertakes efforts to identify conflict issues and their potential mitigation strategies during their community consensus process, as well as sometimes mediates community-level disputes on an ad-hoc basis, TIS+ should consider conducting more regular conflict assessments or settlement/community level PESAs and documenting how this information is used to inform programming. The midline assessment identified that underlying conflict issues continue to linger through Gedo/Bay, and it will be important for TIS+ to carefully track these ensure conflict sensitivity when implementing in this cluster. Priority: Medium Responsible entity: TIS+ Timeframe: Short-term (0-6 months) 315 Resources Required: Minimal as this would entail mostly increasing effort to document what staff are already doing informally. Strengthening Specific TIS+ and/or USAID Approaches 4. USAID and future implementers should carefully consider the unique local contexts and socio-political dynamics within and across settlements when considering future clustering approaches. Evidence from the midline suggests that the “cluster approach” was not relevant for the Gedo and Bay context given both the unique socio-political differences between the areas and also lack of military progress in the areas more generally. In the future, USAID should consider ways of ensuring that these differences are better accounted for in its strategy and geographic grouping of activities. Interventions should not be planned around large-scale military interventions such as the ‘Jubba Valley’ offensive which rely on multiple stakeholders but should instead focus to leverage small-scale clearing operations such as the US-supported, Somali-led operations in the Lower Shabelle which have opened up settlements such as Sabiid and Anole. Responsible entity: USAID Timeframe: Medium-Term (6-12 months) as it plans its next stabilization program. Resources Required: Minimal. Strengthening Government Involvement In Shared-Asset Management 5. TIS+ and USAID should explore opportunities to better utilize TCs to encourage the participation of various levels of government in shared-assessment management, as well as ensure enhanced coordination and information exchange. Interviews and analysis clearly showed the utility of the TCs and their contribution to the results achieved by the program. During the strategic review sessions, TIS+ acknowledged the need to better define the role of the TCs and improve their linkage to the program. Better utilization of the TCs can be achieved by first, ensuring their role and funding within the program is sustainable. Secondly, TCs can play an integral role in better understanding the political and security dynamics, allowing TIS+ to better forecast and plan its interventions. Priority: Medium Responsible entity: TIS+ and USAID Timeframe: Short-term (0-6 months) Resources Required: Minimal as this would involve mostly internal discussions on how to better utilize the TCs. Strengthening Efforts to Promote Inclusion 6. TIS+ should focus on activities that emphasize the inclusion of women, youth, and minority clans during the final stages of the project. TIS+ interventions, particularly in infrastructure planning and management, have sought to increase the inclusion of women and youth. There is a GESI strategy for the project, but limited planned forums for increasing inclusion, particularly at a cluster level for women and youth. Larger infrastructure projects such as roads and airstrips have less opportunity for community engagement and decision-making than community￾based infrastructure. A focus on softer activities that enhance the participation of women, youth, and minority clans may be helpful in the final phases of the project, particularly activities that increase inter-clan collaboration. Priority: High Responsible entity: TIS+ Timeframe: Short-Term (0-6 months) 316 Resources Required: Minimal as this would entail mostly implementing TIS+’s GESI strategy which has already been approved. 7. USAID and TIS+ should strengthen its M&E efforts to better track minority clan and other disadvantaged groups’ engagement in this cluster. The midline assessment was challenged in commenting on the extent to which minority clans were involved in TIS+ activities. If this is an indicator that USAID and TIS+ are interested in tracking and reporting on, they should consider updating their current M&E tools to better track the participation of these groups. Priority: High Responsible entity: USAID and TIS+ Timeframe: Short-Term (0-6 months) Resources Required: Minimal as this would entail only a small change to current M&E tools. 317 IX. BAKOOL CLUSTER Background and Context of Cluster The Bakool region covers an area of approximately 26,000 km2 and consists of five districts: Hudur (the headquarters of the region), Wajid, Tieglow, El Barde, and Rabdhure. The region is primarily agro￾pastoralists, who make up 70% to 80% of the population and are predominantly located in Hudur, Wajid, and Tieglow districts with a small proportion found in Rabdhure district, and mainstream pastoralists, who make up 20% to 30% of the population and are mainly located in Rabdhure and El Barde districts.32 Bakool is largely seen as a cosmopolitan region that hosts sub-clans within the Rahanweyne as well as other Somali clans who live in the region and have equal representation in the administration. The other clans include the Ogden’s Aulihan and Reer Afgab sub-clans and Buurashadley in Yed, the Aulihan and Jejele clan in El Barde and Hudur, and the Gadsan (Dir) in Rabdhure district. The major political actors in the region are the African Union Mission in Somalia (AMISOM) Ethiopian Defense Forces, and the Interim South West State Administration (ISWA), who controls Hudur, El Barde, and Wajid. Al-Shabaab (AS) still controls Rabdhure and parts of Tieglow. The region is currently under the control of various actors including AMISOM and the Somali National Army (SNA) under the ISWA. AS remains present in Bakool, particularly in Rabdhure district and the rural areas of the other three districts. Although government and AMISOM forces control most towns in the region, most rural territory remains under the control of AS’s mobile units. These dynamics pose a security challenge to peace and stability in Bakool. Summary of Interventions in Bakool There were no TIS+ interventions in this cluster. Other development activities and partners have been active in this cluster. Prior USG investments are evident, and this has influenced data analysis of this cluster. At baseline, there were several stabilization projects operating in Bakool, but fewer than in other clusters due to its instability and newly liberated status. The following actors and activities were evident during the WAVE II data collection period (see also Figure 13):  The Somalia Stability Fund had a presence in Xudur working on community consultations for development planning. This work is now being continued by Finn Church Aid.  The EU is funding efforts and they have expanded to include peace building, civic education, and governance capacity building efforts.  UNDP is supporting on FGS stabilization planning efforts and has a Local Governance/ Community Liaison Officer working in Xudur and Wajid.  Nordic International Support Foundation is doing infrastructure development in this cluster. 32 FSNAU 2015 Bakool Region Food Security Analysis. 318 Figure 13: Bakool, Summary of Other Actors and Interventions Actors Activities • World Food Program (WFP) • Food and Agriculture Org. (FAO) • Int. Org. for Migration (IOM) • Int. Com. of the Red Cross (ICRC) • DRC/DDG • Medecins sans Frontiéres (MSF) • MARDO, GREDO, ACF • Road construction • Solar lights • Seed distribution • Pond drilling • Football ground construction • Maternal Child Health center construction • WASH activities • Cash transfers The sections below provide an analysis of HH survey data collected from Bakool and triangulation of this information with the KIIs and FGDs conducted in the settlements in which data was collected. Contribution analysis was not directly applied because this cluster had no TIS+ implementation. Findings are primarily aggregated at the cluster level, but where relevant, individual settlement information is provided where the results might be outliers and to illustrate to USAID the diversity of these settlements and the anticipated effect on future USG program planning. Methodological Considerations TIS+ did not implement in this cluster so CA will inform about how other donors and USG efforts have impacted changes observed since baseline. The findings section of this cluster report follows the format used at baseline and compared changes over time. The conclusion and recommendations section highlights observations made by the assessment team that may be helpful for planned USAID/Somalia programming in this area. The Assessment questions answered for Bakool vary slightly from the previous two cluster sections. The assessment questions for the Bakool midline are outline in Table 11 below. Table 11: Bakool Midline Assessment Questions No Bakool Midline Key Assessment Questions (KAQs) 1 What have been the changes in the following from BASELINE: (a) Community ownership of development projects and processes; (b) Effective shared asset management; (c) Improved delivery/satisfaction of services; (d) Reduced conflict over resources; (e) Greater citizen confidence in government; (f) Reduced support for violent extremism? 2 Did Government contribute to asset management in communities in this cluster? If so, how? Did Government have both the capacity and the will to contribute? 3 Were activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? Note: Task Order Question 2: ‘Were TIS+ approaches used in the cluster (e.g., the community consensus process) instrumental in contributing to the achievement of TIS+ results described in Question 1?’ was not answered as TIS+ did not implement in this cluster. Table 12 below shows the areas where data was collected for the HH survey. In addition to the HH survey, 8 FGDs and 18 KIIs were conducted in the following settlements: Xudur, El Barde, and Wajid. Bakool Index measures were used by the team to support or triangulate findings from the qualitative interviews as well as the desk review and secondary literature review. Table 12: Bakool Cluster Sample Size for Wave II, Baseline – Midline All Non-TIS+ Settlements Baseline Midline Xudur 175 148 El Barde 37 36 Wajid 240 240 Total 452 424 319 Bakool cluster had three settlements: Wajid, Xudur, and El Barde. This analysis is disaggregated by Wajid and Xudur only because they had modest sample sizes as allocation was based on population sizes, both at baseline and midline. El Barde had a baseline sample size of 37 and therefore drawing inferences at the settlement level would not be completely reliable; however, it is included as part of the total sample size. Findings compared baseline and midline percentages and the team weighted data to adjust for population differences within the settlements. The team used regression analysis to identify distinctions between “significant“ and “nonsignificant“ results at midline. 320 FINDINGS Q1: Changes from Baseline in Intended Result Areas Assessment Question 1: What have been the changes in the following from BASELINE: (a) Community ownership of development projects and processes; (b) Effective shared asset management; (c) Improved delivery/satisfaction of services; (d) Reduced conflict over resources; (e) Greater citizen confidence in government; (f) Reduced support for violent extremism? 1(a): Community Ownership of Development Projects and Processes To examine changes in community ownership of processes and development projects, the assessment focused on community consensus-building, consultation/participation in decision-making, and overall management of shared assets. Participation in Decision-Making • Between 2017 and 2018, at the cluster level, the awareness of community planning events increased significantly by an average of 16% with the increase being eight times higher in Xudur (+40%) than in Wajid (+5%, p – value = 0.015). • These variations show the differences in settlement responses to the HH survey. Xudur also has more implementing partners compared to Wajid. • Both FGDs and KIIs agreed on the need for enhanced awareness and social mobilization. Information exchange remains an issue in the surveyed areas across the cluster • Both FGDS and KIIs confirmed participation of various community actors in planning and development events. Youth and women FGDs reported increases in participation, and this held true across the cluster in both local and district community events. However, youth also noted that meaningful decision-making and oversight was still limited for youth and women. • As stated by one youth leader from Hudur, “To encourage greater community involvement, I think there should be more awareness raising done for the community in which they are educated about the importance of these projects and that these projects that are being implemented is for their benefit. They should also be given skills and better training so that they can take part in these projects and be consulted in the development activities occurring in their settlement.” • Youth respondents cited a need for more training, particularly for the voices of youth and women to be utilized and heard in regard to development planning. Selection of Projects Based on Community Planning Events • At baseline, the HH survey found that awareness of local oversight or planning committees (in the assessed areas, especially Hudur and El Barde) was higher in this cluster and there was awareness of community planning events – see Figure 14. • The previous TIS work at the community level contributed to this. • From baseline, projects selected through community planning events increased by an average of 16% with the increase being eight times higher in Xudur (+40%) than in Wajid (+5%). • In Wajid, there was an increase (+23%) in respondents indicating “most” to “all” of the selected projects effectively addressing community priorities. • In Xudur, there was only a small, nonsignificant increase (+4%). • There was evidence that practices in Xudur resulted in the exclusion of women. There was a view from some male elders that NGOs and the district administration largely controlled community planning events as well as project and site selection. “Actually, there are no women who are actively involved in management and maintenance of public assets and most of these public asset or services are managed by men only and the role of women is less. We pray to Allah to give us good administration with justice so that they keep the equality among men and women because it’s in the goodness and the interest of the community.” – Elder Women FGD, El Barde 321 • As one male elder from an FGD in Xudur stated, “The role of the community is to listen to what the NGOs and the district administration comes up with in public meetings. If you enquire more on that, they will silence you. This happens indirectly. The activities are always done through the consent of the district administration and the NGOs.” • And another male elder supported this view. “The NGOs decide when projects are to be implemented, the location, who are to benefit and what is to be done. We have to give you the right information since we are representing the community.” Figure 14: Do you know whether local projects are selected through community planning events? 1(b): Effective Shared Asset Management Relating to Oversight Committees • From baseline there was a marked increase in the awareness of the role of oversight committees in shared asset management. • However, there was evidence from the FGDs that some shared assets are not managed as inclusively as they could be. There was a particular concern for the role of women. • As stated from a KII with a women’s leader in Xudur, “The public assets and services are managed and maintained by the local government, community elders, and other people who have been put in charge of the management and maintenance of these assets. There are people who are assigned to manage and maintain the public assets. Health professionals manage MCH centers, school managers manage schools, and a selected committee, which is specifically trained for the management and maintenance of the water well, manages the water.” • In Wajid, there was a slight increase in awareness of oversight committees to manage services/assets (+2%); however, there were generally large increases in perceptions of their effectiveness. o This included increases in the management of health centers (+9%), meetinghouses (+14%), market facilities (+13%), and roads (+21%). o There was a significant reduction in effective management of sports arenas (-22%). Based on qualitative interviews this seemed to be due to increased insecurity in this particular settlement. • In Xudur, there was a significant increase in the awareness of oversight committees (+51%), however no significant change in perceptions of their effectiveness. “The community actively took part in all the projects. Normally, whenever the district administration receives new projects, they often delegate it to specific community leaders and influencers such as the youth or women leaders who mobilize the community. These leaders also closely oversee and administer projects, whether they are rehabilitating roads or those who dig wells.” 322 Relating to Other Actors: Non-Oversight Committees • In Wajid, maintenance quality for the following assets improved significantly: schools (+17%), Health centers (+17%), meetinghouses (+22%), market facilities (+20%), and roads (+29%) - see Table 11. o Again, there was a significant reduction in maintenance quality for sports arenas by 25%. • Maintenance quality for all the assets in Xudur had no significant changes between 2017 and 2018. • In FGD and KIIs, there were interesting insights as the types of assets that seemed to contribute to improved governance. Health centers, schools and MCH centers seemed to be facilities that allowed for more interface with local and district administrations – see Table 13. • As one business leader from Xudur commented, “The public assets and services are managed and maintained by the local government, which manages the education sector and the ministries of South West state government, which then manages the health sector. There has recently been progress made in terms of good governance.” • For effective management of shared assets, FGD respondents recommended a closer working relationship between the local government and the community, introduction of rules and regulations, and increased awareness on the prioritization process of public assets. • FGD respondents also recommended increased involvement of experts and members of the diaspora to support the role of women and youth in taking a more active role in asset management. Table 13: Who else is responsible for the management of the following services/assets? 1(c): Improved Delivery and Satisfaction Services Relating to Service Delivery – Provision • In Wajid, clan elders, state-level government, local people, religious leaders, and police were perceived to have significantly increased the provision of all services. • In Xudur, there were mixed patterns of service provision; for instance, health services are increasingly being provided by the local administration and international NGOs; education is increasingly provided by clan elders, local administration, and the business community; and water and sanitation are increasingly provided by local people and local/international NGOs. Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Schools Health Centers Meeting Houses Market Facilities Sports Arenas Roads Clan Elders 10% 15% 16% 13% 7% 17% 24% 14% 35% 16% 15% 19% Militia groups 20% 23% 23% 24% 11% 22% 0% 0% 1% 0% 0% 0% State government 27% 34% 31% 35% 18% 37% 33% 11% -10% -2% 0% -4% Religious leaders 5% 26% 26% 28% 15% 29% 9% 17% 11% 15% 11% 27% Local administration 8% -12% -10% -14% -32% -8% 10% 13% 26% 25% 24% 36% Ugaas/Sultan 13% 17% 16% 16% 9% 20% 2% 3% 11% 5% 4% 3% Diaspora 5% 6% 9% 8% 3% 12% -12% 7% -1% 23% 9% 1% Businesses/Traders 0% 0% 0% -3% 0% 0% -2% -1% 0% 0% 0% 0% Local community -5% 0% 0% 0% 0% 0% 0% -2% 0% -1% 0% 0% NGOs -2% 0% 0% 0% -2% 0% 0% 0% 0% 0% 0% -1% International organisations 0% 0% 0% 0% 0% 0% -1% 0% 0% -2% -1% -2% Youth 0% 0% -1% 0% 0% 0% 0% 0% 0% 0% 0% 0% Wajiid Xudur 323 Relating to Service Delivery – Satisfaction • Satisfaction in service delivery by local/international NGOs, business community, and local administration significantly increased in Wajid. • In Wajid, the survey respondents were increasingly satisfied with the majority of services: health (+17%), education (+24%), drinking water (+3%), water for other purposes (+11%), roads (+29%), sanitation (+21%), electricity (+44%), police (+27%), and financial services (+10%). • In Xudur, satisfaction levels increased for most services: health (+44%), education (+35%), water for other purposes (+16%), roads (+1%), electricity (+15%), financial services (+13%), and police (+10%). Sanitation saw a 22% reduction in satisfaction levels. • Based on FGDs, there was a significant improvement in the satisfaction of service delivery, with noted exceptions being security where respondents wanted greater security provided for ease of movement.. • As a male elder from Wajid stated, “The roads, football grounds, and the solar lights installed were all great. The roads facilitated transport. The community was very happy and took part in the construction of the football ground because it was now sure that the youth would not be tempted to join the Al-Shabaab. Solar lights benefitted both the male and the females. People can now move around freely even at night because the transport system has now improved.” ‘Security’ as a Service Delivery • In Wajid, there was an increase (+3%) in respondents who felt there was an improvement in the general security; however, in Xudur, these perceptions decreased (-7%). See Figure 15. • The household assessment also specifically examined the security situation between settlements: calmness increased by 11% in Wajid but decreased by 10% in Xudur. • As discovered in FGD discussions, security is still very challenging for local populations. As one female elder respondent stated, “Xudur is the capital city of Bakool region. Though we had problems, there are some good things that also happened. The problem is lack of security. It is so poor. The administration is trying its best though. Ethiopian troops made security in the district better, but it is very challenging outside.” • The security situation in the region has improved, but AS continues to control some critical corridors, making movement in this area still a cause of concern for citizens. • Although movement to Bakool by air is not restricted, there is a sporadic siege on most of the districts by AS and there is poor access to some towns and settlements. • Because of this siege and restrictions there is weak market access by roads to and from these towns. • This restriction on movement takes its toll on local populations, sometimes limiting the flow of essential commodities and driving up the price of goods. Improved market access is a critical concern for citizens. Figure 15: Do you feel that security has improved in your settlement over the past year? 324 325 1(d): Reduced Conflict over Resources Resource Conflicts • According to interviews conducted (FGDs), local conflicts (e.g., family conflicts, small-scale land conflicts) seem to be resolved as they arise by local and religious leaders, but there is increasing stress over access to water and grazing rights that may require local and state government intervention. • Respondents in both Wajid and Xudur believed conflicts over resources had reduced, though not significantly, between 2017 and 2018 (see Figure 16). • In Wajid, respondents reported increased trust in conflicts resolved by the following: clan elders (+38%), national army (+29%), peace committee (+17%), oversight committee (+18%), state government (+7%), religious leaders (+28%), local administration (+28%), police (+22%), local courts (+15%), and traditional mediation (+22%). o There was significantly decreased trust by 22% in conflicts solved by the diaspora. • In Xudur the results were as follows: peace committee (+17%), oversight committee (+15%), state government (+10%), religious leaders (+5%), local administration (+2%), police (+29%), local courts (+6%), and traditional mediation (+8%). o There was decreased trust by -2% and -22% in conflicts solved by the national army and diaspora. Figure 16: Compared to last year, have conflicts over shared resources increased or decreased among various groups? 1(e): Greater Citizen Confidence in Government Confidence in Government • At the cluster level, there were higher levels of confidence in federal and member state government and local administration to work on local development, but clan interests were considered a factor in decision-making and access to planning and development. • However, results varied based on the settlement. • In Wajid, confidence in federal government increased by a significant 11%, and member state government by a nonsignificant 3%. There was a nonsignificant decrease of 2% in confidence in local administration. • In Xudur, confidence increased in federal government by 28%, member state government by 21%, and local administration by 20%. • FGD and KII respondents had clear ideas as to how confidence and trust could be built at the various levels of government. • A business leader KII from Xudur said, “There are specific factors that have contributed to the increase. These include collaboration and better working relationship between the departments of state governments such as 326 South West state and the federal government. There is also transparency and accountability in the taxation system.” • For youth this seemed to be influenced by access to better economic and educational opportunities. • As a youth girl from Xudur stated, “There are several factors that contributed to the increased confidence in the government. In order for the community to have confidence, the government developed education centers where the essential survival of the nation started. The government improved the common curriculum of the schools in the district. This led to the upbringing of a fruitful society which is well educated.” Public Trust • In Wajid, there was a significant increase in the percentage of respondents reporting the following: o local administration prioritized community needs (37%), local/district officials were trusted in utilizing funds for development (20%), (federal officials were not guided by clan interests (12%), local administration shared information (38%), government encouraged people to participate in planning (34%), and corruption at local and federal levels reduced by 2% and 17%. • In Xudur, respondents reported the following reductions: o local administration prioritizing community needs (-2%), local/district officials being trusted in utilizing funds for development (-8%), and local administration sharing information (-7%). • In Xudur, there were some positive changes noted from the baseline: o The proportion of those reporting federal officials were not guided by clan interests and government encouraged people to participate in planning increased by 22% and 1% respectively. • Corruption at local and federal levels reduced significantly by 45% in Wajid and 52% in Xudur. 1(d): Reduced Support for Violent Extremism • The influence of AS is still prevalent in Bakool and was evident at the cluster level in the HH survey responses, and also in FGD and KIIs conducted. • Respondents were asked if young people were prone to violent extremism, and changes in perceptions were similar between Wajid and Xudur. At the cluster level, there was a significant increase by 22%. See Figure 17. • Between 2017 and 2018, at the cluster level, there was a reduction by 19% [Wajid (-22%) and Xudur (- 18%)] in respondents indicating extremist groups were “not welcomed at all.” • Sports facilities were mentioned by male elders and male youth as being particularly useful in mitigating recruitment and collaboration with AS for young men. • As stated by one youth from Xudur, “The youth, who are the most vulnerable to join the Al-Shabaab, got sports support after a beautiful football ground was built for them by the International Organization of Migration.” Figure 17: Do you think that young people in your settlement are highly prone to violent extremist group? 327 Q2: Government Involvement in Shared Asset Management Assessment Question 2: Did Government contribute to asset management in communities in this cluster? If so, how? Did Government have both the capacity and the will to contribute? 2(a): Contribution to Asset Management • From baseline there was a marked increase in the awareness of the role of oversight committees in shared asset management, however there was a persistent perception in qualitative interviews that government (state and local) still is limited in their role. • At the cluster level, there were higher levels of confidence in federal and member state government and local administration to work on local development, but clan interests were considered a factor in decision-making and access to planning and development. • Qualitative interviews reflected a concern that certain projects were influenced by clan conflicts and that the government at State and local levels discriminated against some clans. • Local administration was largely seen as responsible for the administration of shared assets, particularly security services. In qualitative interviews, there was consistent acknowledgement of improvement of local officials in engaging communities, but also an awareness of their limitations in terms of staffing and resources. • While the diaspora plays a significant role in contributing to shared assets there is no cooperation between local government, FMS and the diaspora. Qualitative interviews as well as HH data showed that this had gotten worse since baseline. • Qualitative interviews noted that community expectations had increased due to the dominance of INGO and NGO development-led planning that emphasizes participation by government. In qualitative interviews, NGOs and businesses were often described as service delivery mechanisms. • Although movement to Bakool by air is not restricted, there is a sporadic siege on most of the districts by AS and there is poor access to some towns and settlements. • Because of this siege and restrictions there is weak market access by roads to and from these towns. This restriction on movement takes its toll on local populations, sometimes limiting the flow of essential commodities and driving up the price of goods. • Qualitative interviews expressed a need for more government involvement in security and access issues. • Respondents stated that this was critical for economic improvements as well as to increase interaction between settlements. • Attacks have become more violent. There is increased evidence of community expectations for enhanced security to be provided by the FMS government. This an area of shared concern by government as well as community respondents. • Bakool has been severely affected by drought, and this has worsened local resource conflicts, particularly over land and water. Local conflicts (e.g., family conflicts, small-scale land conflicts) seem to be resolved as they arise by local and religious leaders, but there is increasing stress over access to water and grazing rights. Several respondents wanted greater cooperation with state and local government in resolving these conflicts. • A conflict between the Rahanweyn and Ogaden clans is particularly destabilizing. Essential political and power sharing arrangements still need to be formed and government is seen to have a central role in this. This impacts perceptions at the local level that the government (state and district) are invested in equal access for minority clans to services and assets. 2(b): Capacity and Will • Development planning at the state and local levels is gradually improving, but quantitative and qualitative (KIIs and FGDs) data confirmed that capacity is still low, particularly at the State and district level. This impacts participation in state and district/ local systems, particularly for women, youth and minority groups. 328 • Several qualitative interviews also noted the dominance of the male voice in planning for shared assets and community engagement. • As stated by one female elder from an FGD in Xudur, “We are still facing gender discrimination because the assets are shared in a bad way. Men always access the majority of the assets such as in projects of the NGOs and in those for the government”. • Qualitative interviews also highlighted that there is a need for women to persist in their efforts to have greater say in development planning. As one elder women from Xudur noted, “The decision-making process is not such a big thing, but the government usually gives a chance to those it is closer to and those as higher officials. I encourage females not to be intimidated by the way the district administration treats females”. • In several KII interviews with government officials and with youth it was also noted that despite improvements in this cluster, that the inclusion of women in decision-making was still not prioritized by the local administration and with State government. 2(c): Governance Index and Changes from Baseline The themes contributing to good governance index included confidence in government, service delivery, and stakeholders’ partnerships – and variables under each of these have been analyzed in question number one. ● Good governance index increased by 9% (p – value < 0.0001) and 13% (p – value < 0.0001) in Wajiid and Xudur respectively. Figure 18: Good governance index changes between baseline and midline data collection (2017 to 2018). 329 Q3: Efforts to Promote Inclusion and Community Cohesion Assessment Question 3: Were activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? 3(a): Inclusion of Women and Gender Equality Attitudes: There was a significant increase by 1% (p – value = 0.575) in Wajid for those who believed women were considered equal to men. In Xudur, there was a large significant increase by 79% (p – value < 0.001). • This difference is largely due to the higher number of development actors working in Xudur compared to Wajid and El Barde. Assessments within each of these settlements asked further questions on why women were considered not equal to men. • Respondents in Wajid decreasingly supported reasons in Table 15, most of which are closely related with cultural beliefs at the clan level. In Xudur, even though positive attitude toward women seem to increase, there is a significant increase (16%) in those who believe women are not allowed to make decisions without men. • As evident in Table 14, access to planning events is not as inclusive as it should be. For instance, in Xudur the youth and even NGOs had reduced access. Table 14: Why are women not considered equal to men? • Although women and men in this cluster had relatively similar views of gender equality, there was clear evidence that clan rules limit the participation and engagement of women in community planning and consultation events, although this was said to be improving. • Many FGD and KII respondents felt that inclusion of women and youth had increased over the past year, however, there were diverging views and opinions across gender lines. • Qualitative interviews (FGD and KIIs) stated that despite increased efforts to include more women that there were consistent gaps, particularly in the role of women in government and leadership. This was cited to be at the State and local (district) level. • As stated by one male youth leader from Xudur, “No, women do not have equal access to public assets and services in this settlement, as they are not fairly represented in the local administration. They are not involved in positions of leadership apart from ministry of women's affairs. There are cultural stigmas attached to women being in leadership positions. They are marginalized when it comes to power sharing agreements.” • As stated by one female youth leader from an FGD in El Barde, “No, we don’t know any women involved in the monitoring and overseeing of the developmental projects/activities in the area. Only men are those who are involved in those activities. There are five council members who are all men.” WHY WOMEN ARE NOT EQUAL TO MEN Total Male Female Wajiid Xudur Women are not as educated as men -8% -7% -8% -14% 2% Women’s sole responsibility is to raise the children -13% -12% -13% -26% -7% Women are necessary to maintain the household -10% -11% -9% -29% -4% Women are not able to speak in public 6% 8% 6% -28% 3% Women are cannot make decisions for men -13% -9% -18% -31% -6% Women are not allowed to make decisions without men -6% -5% -10% -37% 16% Women are dependent on men for their protection and surviv -7% -6% -8% -33% -20% Women lack leadership skills 0% 5% -4% -37% 0% 330 • There were expressed views within the FGDs, particularly with elder women, that women are not given equal access to positions within the government and also on NGO committees. 3(b): Youth Engagement and Inclusion • Youth expressed a feeling that their involvement was being more recognized. Security had improved, which promoted more opportunities for youth to engage in public planning and project development, but training and access to meaningful decision-making is still lacking. • By most accounts interface with the government and youth and women had improved greatly. • However, youth leaders were more articulate as to the specific problems that women and youth face in Bakool. Primarily this centered around inclusion in decision-making. • Most projects in the settlements surveyed had criteria for minority and youth inclusion in project management, maintenance, and selection. 3(c): Minority Clan Inclusion • AS appeals to disgruntled Somali minority clans in this cluster. This is particularly compelling as leverage for some clan-based power dynamics at the local level in Bakool. • There was evidence in several interviews across settlements that AS manipulates minority sentiments, and minority clans affiliate with AS to resolve specific conflicts around land or water access in specific areas. • This is particularly evident around the acquisition of land and access to water resources critical for livelihoods of minority groups. The drought has only amplified this effect according to some sources. • Some AS leadership is also from this region, so the reach and structure of AS in this region may be more sophisticated, according to officials interviewed. • Recent political events in this cluster have negatively impacted the belief that the government is treating AS defectors as potential collaborators for improved political inclusion. 3(d): Community Cohesion Findings The themes contributing to community cohesion index included interpersonal and institutional trust, social and political inclusiveness, and network diversity (see Figure 20). Findings on institutional trust to resolve resource related conflicts will be presented in this section of this report. ● Community cohesion index significantly increased for Wajid (+15%) and Xudur (+7%) See Figure 19. “No, women do not have equal access to public assets and services in this settlement, as they are not fairly represented in the local administration. They are not involved in positions of leadership apart from ministry of women’s affairs. There are cultural stigmas attached to women being in leadership positions. They are marginalized when it comes to power sharing agreements.” – Youth leader, Hudur Figure 19: Bakool Community Cohesion Index Measures. Changes from 2017 to 2018. 331 Social and Political Inclusion • Clan dynamics and an embedded history of migration and competition over resources (e.g., land and water access) impacts political settlements, trust, and civic engagement among clan groups in this region. There is significant competition over grazing rights and water access, which can impact whole communities. • Development planning at the national and state level is gradually changing, and as more established district and local systems are developed, it is likely that expectations of participation and inclusion will increase. Much of this is due to the dominance of INGO and NGO development-led planning. • There were higher levels of confidence in member state government and local administration to work on local development, but clan interests were considered a factor in decision-making and access to planning and development. Network Diversity The following results were observed concerning network diversity: • Perceptions of peaceful coexistence between clans increased in Wajid (+1%) and in Xudur (+8%). • The percentage of people believing that clans only cooperated when necessary decreased in Wajid (-3%) and in Xudur (-8%). • Male and youth FGDs recognized sports facilities and activities as being helpful in improving peaceful coexistence and social integration. • Specific clan conflicts still cause political issues and severe mistrust. AS is a masterful “manipulator” of these sentiments, particularly around land conflicts. • Security continues to be an ongoing cause of concern for both the government as well as local communities. AS continues to provide security in some rural areas. • Although movement to Bakool by air is not restricted, there is a sporadic siege on most of the districts by AS and there is poor access to some towns and settlements. Market access is severely weakened in this cluster for critical roads and corridors. • According to FGDs and KIIs, this impacts security for women as well as interaction between clan groups in this cluster as travel or corollary roads leaving town can be unsafe and limits movements between areas where varied clan groups live. • This restriction on movement takes its toll on local populations, sometimes limiting the flow of essential commodities and driving up the price of goods. This seems to cause competition and some minor conflict over market access. 332 CONCLUSIONS Q1: Contribution Towards Intended Results Community Awareness and Participation • There was a general trend in surveyed settlements in this cluster of increased community awareness and participation, however results were mixed for the increased participation of women, youth and minority clans. There was a general trend of increased community awareness and participation across the cluster, however results were mixed for women, youth and minority clans. While result changes from baseline were encouraging, this is tempered by the fact that there are still significant gaps in participation across key groups. This perception was shared across the settlements surveyed and also in KIIs with government and community leaders. Business leaders felt that awareness had increased, but women were still consistently left out of participating at levels equal to men. Effective Shared Asset Management • While there was a general increase of awareness of POCs across the cluster, communities surveyed were consistently aware of gaps in government management capabilities, particularly at the State and local (district) levels. There were significant increases in awareness of oversight committees in the settlements in this cluster, however management of shared assets was largely dominated and led by INGOs and NGOs and there was a consistent recognition that state and local governments were less able to participate in the effective management of these assets. Qualitative interviews consistently indicated a desire for more community training and capacity-building on shared management and oversight, an area of improvement required for increased participation. There was increased recognition of the role of local communities in managing assets in collaboration with state and non-state actors. For example, there were strong increases in the reported roles of local community members in managing shared assets, but less pronounced local and state level management roles. Interviewees reported increased levels of community involvement in these settlements. However, government is still seen by interviewees as having the first line of responsibility and then delegating management responsibilities of certain assets to community members, INGOs, NGOs, and the business community. Interviewees felt the need for greater involvement of minority clans, women, and youth as well as more targeted training for these populations. Several partners interviewed, tracked monitoring mechanisms for minority clan inclusion in shared asset management. Improved Delivery of / Satisfaction with Services • There was a modest increase in the perception of local government providing services, and strong evidence that communities still see INGOs and NGOs as frontline providers of services. There was also a general perception that local people and NGOs/INGOs were responsible for improved service delivery in Bakool settlements surveyed, whereas services by state government had decreased across most services, except for water and electricity. Security is seen as a service that is largely provided by the federal government as well as the state. There is a clear indication that particular projects, such as solar lighting, and improved market security features impact the community’s perception of improvements in security provision. Security was cited as a critical service for improvement in this cluster as access to economic opportunity as well as movement between settlements in severely limited by AS attacks and general insecurity on road leading away from major towns and urban centers. 333 Reduced Conflicts Over Resources • Although there was a decrease in conflicts over resources at the cluster level, qualitative interviews showed a consistent concern about the impact of micro conflicts (conflicts over land, grazing rights and water) on community cohesion and interclan relationships. There was expressed concern, similar to the baseline, that clannism is still a problem at the community level as well as within government. According to KIIs, this practice can cause micro- and macro-level conflicts. This was particularly expressed by women and elders, who feel that there are some minority clans in Bakool that are consistently denied access to services and whose conflict concerns are not heeded by local administrations. At baseline this was flagged as a serious concern for stability and sentiments continue at midline. Greater Citizen Confidence in Government • In general, there was a decrease in confidence towards state government across the cluster (with no significant change for federal government and local government although Governance Index measures recorded more positive results. Clannism was a significant factor in citizen perception. While local government saw increases in the perception of its provision and management of services, local government performed more poorly on other governance measures (increased corruption and decreases in sharing information). One explanation may be that in Bakool the increased access to information makes citizens more critical of government and more aware of corruption as a significant problem impacting development interventions. Citizens were also concerned with security as a provision of service and seemed to feel that the federal government could do more to protect citizens and assist state and local administrations. This seemed evident in the baseline as well as the midline as respondents of FGDs and KIIs saw increased security as being directly related to increases in their confidence in government and helpful in reducing corruption, particularly around issues of perceived clannism. Reduced Support for Violent Extremism • Most of the stabilization interventions in this cluster with indirect results aimed at reducing youth engagement with AS focused on youth engagement in sports facilities. Youth however, also wanted more interventions focused on increasing economic opportunity. Most activities and interventions surveyed did not directly contribute to these measures, but there are indications that partners designed interventions with the prevention of violent extremism (PVE) inherent in their programs. Reintegration of youth as well as implementing activities targeted for youth prone to recruitment was an aspect of some programming in Bakool. AS still holds sway with youth populations in Bakool and community members interviewed were thankful for sports facilities developed to engage youth. AS has a complex administrative and logistical system that influences recruitment, and several youth surveyed wanted more interventions that focused on economic opportunities, job creation and training. Some programming attempted to have interventions with a PVE lens, for example through sports and youth empowerment, but there is no evidence that this is contributing to reduced support for violent extremism. There were nonsignificant results for people reporting that extremist groups were “not welcomed at all.” These measures have remained consistent since baseline. Q2: Government Involvement In Shared-Asset Management Provision of Shared Assets • Security was an important shared asset cited by citizens surveyed. Discussions of security were linked to service delivery, access, and development planning. In general, federal government was regarded to have a direct role in providing security, with state and local governments reporting decreasing and increasing patterns in service provision respectively. 334 Results across levels of government showed increasing government contribution, whether through facilitation of security or through increased participation in local community governance models. Most FGD and KII respondents expressed the view that security is the main asset provided by federal government. According to the HH survey, the provision of services by state government was generally reported to have decreased across most areas. Exceptions include perceived increases in the provision of water and electricity in both settlements surveyed. State administration is relatively new and not properly capacitated. At a local level, service provision across all service areas increased with the exception of security and this was seen as a shared responsibility with the federal and state government. Management of Shared Assets • Across the cluster, both the state and local government were perceived to have increased roles in the management of most shared assets, with citizens recognizing the inherent limitation in human as well as financial resources. Based on KIIs and FGDs conducted, security was cited as the main asset managed by the federal government, and which was as an enabling factor allowing others to manage assets. FGD and KII respondents also mentioned that the federal government provides some limited financial resources for maintenance of roads, etc. and that local communities also contributed in collaboration with district administration. Local administrations were perceived to have increased roles across all assets, save security, according to the HH survey. This was also confirmed in FGDs, although there were mixed views based on the settlements. Settlements reported stronger results against baseline survey results in the management of roads and sports and health facilities. Capacity and Will of Government • District Administration figured prominently compared to federal and state governments as having the will to manage assets but often limited by capacity, resources and systemic challenges that needed to be filled through a shared responsibility between government and the community as well as stronger public-private partnerships. FGD and KII respondents acknowledged the will of various levels of government in assets managed, chiefly the district administration, but also noted systemic capacity challenges that need to be filled for asset management to thrive in this cluster. Respondents indicated that they recognized the limitation, particularly of district administrations, and that it is a shared responsibility between government and the community, and strongly recommended stronger public-private partnerships. Q3: Efforts to Promote Inclusion and Community Cohesion Inclusion of Women • Partners in Bakool have taken a number of steps to increase the participation of women in development planning, however decision making, and leadership is still limited. There is a need for more engagement of women in local governance as well as political negotiations that may increase the voice of women in planning and access. There was a significant increase in the percentage of respondents who reported in the HH survey that “women were considered equal to men. Insecurity since baseline has only increased, and despite increased AS attacks and periodic problems with movement, participation of women is highly valued in Bakool. Numerous interviews during data collection discussed the importance of increasing participation, particularly for women and youth in political negotiation, governance (local and district) and also decision-making regarding development planning. 335 Inclusion of Youth • Although youth were actively involved in the discussion of community priorities and projects across the cluster, they reported feelings of frustration in the sense that their participation in the management and oversight of shared assets was most of the time limited to sporting facilities and playgrounds. Youth reported being active in monitoring and oversight of shared assets, particularly in the management of sports facilities, but also expressed a desire for greater responsibility. Most FGD respondents also felt there should be greater capacity building for youth to ensure more effective asset management where community involvement was required. They also wanted a greater role in development planning so that projects would focus on economic opportunity and job creation. Inclusion of Minority Clan Inclusion and Improved Community Cohesion • Minority clan involvement is still limited and needs serious consideration in development planning in Bakool. Minority clan involvement in this cluster was cited as critical in improving shared asset management and community inclusion. Several partners collected data on the participation of women and youth from minority clans and have increased the visibility of these groups in their development planning. Indeed women in qualitative interviews (both elder and youth) identified a gap in minority inclusion and commented on the need for more women involvement in management of shared assets to minimize conflicts and perceived inequities and access to resources. This view was supported in several elder and youth male FGDs conducted. 336 RECOMMENDATIONS USAID Programs that Contribute to Intended Results 1. Youth engagement programs are needed in Bakool and should engage youth in civic efforts. There is a need to support youth engagement in civic and development efforts in this cluster. There is a need to look at ways to elevate the role of youth and to engage them more actively in community life and development planning. Youth in this cluster have expressed an interest in more effective community level planning and engagement, particularly at the district and state level. 2. USAID should focus on quick-impact projects that mitigate the economic impacts of AS in critical corridor communities in Bakool. Settlements in Bakool have been hit the most by the Al￾Shabaab strategy of blocking off roads leading into the towns. Quick-impact projects such as access to water and facilities that allow the nascent administration to project legitimate authority will further the stabilization agenda in these towns. Specifically smaller community driven efforts that encourage access and improve freedom of movement outside and between towns is critical. Strengthening USAID Approaches 3. A Youth Preventing Violent Extremism (PVE) program would be useful in Bakool that focuses specifically on youth engagement in decision making at the local and state levels. There is also a need to support youth engagement in development efforts in this cluster. There was an expressed concern in this cluster regarding youth vulnerability to recruitment with militia groups and to engage in conflict. In this cluster there is a need to look at ways to elevate the role of youth and to engage them more actively in community life and development planning. Youth need greater access to development and community planning activities that they can lead in the areas in which they live. Bakool is an area that needs more targeted programs for youth engagement and leadership. Youth also would benefit from interclan and cross-cultural experiences with youth from outside of Bakool. 4. Bakool programs should specifically target the enhanced engagement of minority clans and women. Women and minority clan participation in district-level planning efforts is particularly small in this cluster. Interventions need to focus on increasing the transparency and opportunities for these groups to take part in development planning and implementation. District level planning is one way to ensure there is an institutionalized way to encourage greater inclusion for these groups. Strengthening Government Involvement In Shared-Asset Management 5. Programs need to plan interventions that seek to enhance the leadership of women in government and at the local and state level. There is a need for programs and efforts that focus on the leadership of women in government planning as well as local political representation. 6. There is a need for intervention efforts that focus on ensuring consistent engagement between minority and/or marginalized community members and district and state officials. There is a need for programs interventions that focus explicitly on the inclusion of women and minority clans. There is a need for more regular forums for these citizens to engage with government, particularly at the district and state level. Programs that focus on greater inclusion and target enhanced participation of these groups also need monitoring and measurement methods that clearly show the influence of these inclusion efforts on more inclusive policies, services, and local governance practices (public engagement forums, inclusion in decision making etc.). Strengthening Efforts to Promote Inclusion 7. Programs need to plan interventions that explicitly target the inclusion of minority member clans. There is minimal opportunity for inclusion and leadership in development planning for minority 337 clan groups in this cluster. Interventions need to focus on increasing the transparency and opportunities for these groups to take part in development planning and implementation and reduce the likelihood that these clans will be influenced and manipulated by AS. 338 X. CONTRIBUTION CLAIMS AND TIS+ The following “contribution stories” will critically examine the extent to which TIS+ has contributed to improvements in anticipated higher-level effects. Each story is structured according to the five criteria identified by John Mayne, the originator of the Contribution Analysis methodology, to analyze the extent of a “contribution claim” (or “plausible association”) between TIS+ activities and the results witnessed during the midline assessment. These criteria include: 1. Plausibility: Is the theory of change plausible? 2. Implementation per plan: Has the program been implemented with high fidelity? 3. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? 4. Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? 5. Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? The contribution story will then conclude with an overarching assessment of the strength of TIS+’s “contribution claim” to the results relevant to the identified results chain area in the midline assessment. Contribution Story 1: Community Planning and Oversight TIS+ from other more traditional infrastructure programs. While the program aims to identify and address priority community needs through the delivery of quick-impact projects, most of which are infrastructure￾focused, the process for how TIS+ engages community and government actors to plan, implement, oversee, and utilize these activities is what distinguishes it as a stabilization program. Simply put, the “how” (processes used to plan, implement, and oversee activities) is more important than the “what” (specific activities supported in each settlement) for achieving TIS+’s intended higher-level results. These include increased citizen engagement and confidence in government, and reduced conflict over resources and support for violent extremism. PLAUSIBILITY OF THE TIS+ CPO THEORY OF CHANGE The initial step in building a contribution story involves identifying an intervention’s intended Theory of Change and then assessing its plausibility. The majority of the main elements behind TIS+’s ToC for community planning and oversight are derived from the concept of “Community-Driven Development” (CDD), an approach to supporting international development projects systematized by the World Bank during the 1990s but utilized and refined by a wide range of other international financial institutions and development partners since then. At its core, CDD focuses on establishing partnerships between communities and local government units to improve the transparency, inclusivity, accountability, sustainability, and local capacity to provide basic services and manage shared assets. According to the World Bank, “CDD approaches are particularly prominent in conflict and fragile situations—CDD programs operate in 22 countries on the list of fragile and conflict-affected situations, and an additional seven countries with internally displaced populations, refugees, or conflict zones.”33 TIS+’s CPO Approach TIS+’s approach to CPO is articulated in a number of guiding program documents and throughout its periodic reports. As explained in its “Process for Engagement,” one of the program’s foundational design documents, “The TIS+ model is based strongly on a community/government programming approach. TIS+ activities will 33 The World Bank Group, “Community Driven Development,” https://www.worldbank.org/en/topic/communitydrivendevelopment#2. 339 work to foster the relationship between community members and government representatives and supporting Somali leadership in those processes. Particularly, supporting Somali self-efficacy in governance is a priority, which supports USAID’s goals in Somalia and New Deal Principles. TIS+ activities will contribute to improvements in government service delivery, leading to strengthened governance and community cohesion. Communities will be supported with access to livelihoods, income generation, and basic community infrastructure.”34 According to the “Community Contracting Guide,” a document jointly developed by TIS+ and the Ministry of Interior, Federal Affairs, and Reconciliation (MOIFAR), TIS+’s CPO approach is intended to complement FGS’s ongoing District Council Formation process and emphasizes the “leadership role of the inclusive community, including women, youth, traditional authorities, and minority clans.”35 Communities are expected to collectively prioritize their needs through consensus planning and then undertake a “more detailed planning to directly participate in the implementation and long-term management of the project, as well as to help manage resources with appropriate participation and oversight from the local authorities.”36 The guide posits that “as a result of this process, and with appropriate support and training, the communities gain confidence, and organizational and negotiation skills, enabling them to engage with external partners and the local authorities on an equal footing. The direct involvement of the community leaders, with local, state and federal government counterparts and [TIS+], is intended to build confidence and accountability to address nation￾wide stabilization concerns and enhance local ownership of projects and processes. As stabilization projects are successfully implemented in a transparent and accountable process, trust is built between the community and all the three levels of government . . . to deliver basic services and to construct local government infrastructure, in a more participatory and accountable manner.”37 According to the “Consensus Process Guide,” a proper CPO process can also help mitigate and reduce conflict over resources. The guide outlines a 10-step process for implementing the initial community consensus process, which includes initial steps working with communities to conduct conflict and resource maps and introduce the concepts of “Conflict Sensitivity” and “Do No Harm.” The guide then instructs facilitators on how to support communities to identify, prioritize, and reach consensus on activities, form Project Oversight Committees, and how to conduct strategic communication and dialogue events.38 Project Oversight Committees (POCs) A critical feature of the CPO process is the formation of POCs, introduced under TIS+’s predecessor program, “Transition Initiatives for Stabilization” (TIS). As explained in TIS+’s Operational and Management Guide (OM Guide), “A [POC] is a community-based entity created for the Transition Initiative for Stabilization Initiative (TIS+) Program. The POC is a Joint Government–Community Implementation Unit created for a project location to oversee the project delivery from start to finish. The POC works with other stakeholders, including other international organizations, which serve to strengthen government and community interaction. The POC acts as a monitoring and accountability mechanism while mobilizing the community and increasing its awareness of the project for joint implementation. In addition, POCs add an element of sustainability by providing an opportunity for communities to take ownership of project maintenance through the formation of a lasting relationship with state and federal government officials.”39 According to the OM Guide, POCs are instrumental for providing the following: • A Mechanism for Community Mobilization – allowing “for a range of interests to be addressed and a diversity of stakeholders to have input into a project, which creates a stronger and more cohesive community identity...By coalescing local community voices around a project, the new 34 AECOM, “Transition Initiatives for Stabilization Plus (TIS+) Process of Engagement,” (March 2016). 35 MOIFAR, “Community Contracting Guide,” (2018). 36 Ibid. 37 Ibid. 38 Federal Government of Somalia Ministry of Interior and Federal Affairs, “Consensus Process Guidebook.” 39 AECOM, “General Principles of the O&M Plans for TIS+ Supported Infrastructure Activities in Somalia,” (February 2018). 340 community linkages may remain after the project is finished…POCs serve as an access point between the community and government, allowing communities to better respond to issues affecting the community as a whole and in the future. POCs also help build social capital that will enable a community to develop better public services and/or resist negative influence.” • Monitoring and Accountability – “POCs serve as a monitoring body, verifying that the work on a project is completed, according to basic specifications. . . . POCs lay the foundation for the real￾time monitoring and community’s accountability of the tender work . . . since the community members have constant access to the project site, while government officials and TIS+ representatives are usually based elsewhere, community-based members provide an invaluable first￾line monitoring capability.” • Facilitating Government-Community Interaction – “By providing a forum for community and government representatives to interact, TIS+ ensures that there is regular communication during project implementation and, hopefully, after project completion. POC meetings encourage active government involvement in project implementation and increase visibility among key community stakeholders. In addition, for the government to receive adequate recognition for its involvement, it is the responsibility of the POC to actively communicate project developments to the rest of the community.” • Improving Project Implementation – “POCs help TIS+ implement quicker and better projects. POCs resolve issues that pose obstacles to the swift completion of implementation.” • Operations of the infrastructure – POCs should help lead the running of facilities and should include “a representative from the District Administration, for a better liaison and understanding.” • Maintenance – “The POC is the main mechanism to ensure the maintenance as in most cases there is an evolving governance structure and it requires the ownership, legitimacy, and support from the communities. . . . The POC provides that platform to be utilized, constituted and supported, in terms of the capacity to be the main institution providing the maintenance service after the project completion.” The centrality of the POCs as the key nodal point in the CPO process is aptly summarized in Figure 1, which also provides a useful summary of TIS+’s overarching CPO process. Figure 1: Overview of TIS+’s CPO Process 341 Training and Capacity-Building of POCs Given the importance of POCs for ensuring the quality of community participation and oversight, the OM Guide explains, “It is important for TIS+ to carry out the aforementioned trainings, before the groundbreaking of the project to enable, and encourage, the POC to play its vital role in an effective way. During these training sessions, identifying the specific focal persons for the technical and operational/financial aspects of the O&M will establish the needed link between the investment and its sustainability right from the start” (emphasis added).40 The OM Guide further outlines five specific five areas where TIS+ will aim to build the capacity of POCs (see Figure 2). Figure 2. Areas for TIS+ Capacity Building of POCs No Training Support Required Proposed Steps 1. Training on their roles and responsibility (training on how to conduct POC meeting, take minutes, and keep records, including notes on decisions made, schedule of meetings, and overall agreements/outcomes. Provide training and facilitate sessions to build the capacity of the POC to understand and meet the demands of the role (including record keeping). 2. Increase POC knowledge of Public Finance Management concepts to support fundraising efforts and ensure that community contribution is used for the betterment and sustainability of projects. For example, funding to support the implementation of Operations and Maintenance (O&M) of the projects, post-completion. Train POCs on the financing modalities and resource mobilization aspects for project sustainability and accountability. 3. Ensure POCs understand Do No Harm (DNH) principles. POC should avoid assuming a parallel role to the local administration, but rather limit themselves to their roles on the project to support the local governance structure. Following and engage DNH principles with POC, throughout project implementation phases, in order to curb community conflict that may arise during and after the project handover. 4. Train POCs on legitimate government documentation needed for the community contracting processes. This will enable POCs to play an active custodianship role, during the community contracting process. Train POCs on correct and legitimate documentation requirements, particularly for construction awards. For example, local administration, Ministry of Public Works or federal government certificates.) This will prepare POCs to be watchdogs during the implementation process. 5. Train POCs on the respective O&M arrangements, for each project. This shall include two key components, including the community–government partnership for the resource mobilization required for the maintenance and the technical O&M part. Before the project is handed over, it is important for TIS+ to establish this important liaison role and working partnership between the local administration and the POC, preferably in writing. This will help with long-term sustainability and support long term O&M prospects. The above capacity-building plan directly responds to one of the key recommendations from the final evaluation of TIS, which found that POCs play a vital role in ensuring community participation and oversight. Specifically, the evaluation recommended that “the POCs’ role at the community level should be strengthened and institutionalized. POCs should be provided with adequate training to allow adequate monitoring of projects. The contractor and the community must properly understand the POC’s role.”41 Theory of Change Building off the elements described above, the midline assessment team conducted a one-day workshop with TIS+ field and headquarters staff in June 2017, as well as the USAID/Somalia lead for the TIS+ program to develop to clearly articulate the intended results, causal logic, and underlying assumptions guiding TIS+’s overarching activities, but also those specifically focused on CPO. 40 Ibid. Pg. 8. 41 IBTCI, “Final Performance Evaluation of the “Transition Initiatives for Stabilization” (TIS) Project,” ed. Somalia Program Support Services (June 2016). 342 Figure 3 below outlines that Theory of Change with the light blue boxes depicting intended results and the grey boxes highlighting some of the underlying assumptions that need to be fulfilled to progress along the causal logic. Figure 3: Theory of Change for TIS+’s Community Planning and Oversight 343 Assessment of Plausibility: Moderate While TIS+’s CPO approach is informed by decades of similar CDD work implemented by the World Bank and a wide range of other development partners, a major recent study by the International Initiative for Impact Evaluation (3ie) questions many of the principles behind the CDD approach and the extent to which it leads to tangible results. The study, a meta-analysis of 25 impact evaluations covering 23 programs across 21 low- and middle-income countries finds that while “CDD programs have made a substantial contribution to improving the quantity of small-scale infrastructure,” they achieve few of their intended higher-level outcomes, with the exception of results around improved water supply.”42 Furthermore, and particularly problematic for TIS+, “CDD programs have little or no impact on social cohesion and governance. This synthesis study shows that the assumption that the entire community participates in the program is not valid. Data show that participation in decision-making is limited to a small number of community members. There is a clear ‘funnel of attrition’; many people may be aware of the program and the community meeting, but few participate in the meeting and fewer still speak or participate in decision-making. People participated in making bricks, not decisions.”43 As expected, the meta-analysis study has set off a fierce debate within the development community. The report has received both widespread praise and criticism, leading the authors to publish a series of clarifications of their original findings. The authors readily acknowledge the success of CDD approaches for delivering small-scale infrastructure and the immediate benefits that these structures bring to those community members who use them. However, the authors restate that there is limited evidence of “higher order social outcomes,” particularly for social cohesion and governance. “Our review finds that CDD has no impact on social cohesion. There is no heterogeneity there. The lack of effect on social cohesion is consistent across contexts. It is in building social cohesion that CDD has not worked. This is where meta￾analysis is so useful, as it clearly illustrates the consistency of this finding. . . . As we say in the report, the lack of impact on social cohesion is not a new finding. Indeed, one of us was a co-author of the 2002 World Bank review of social funds – including the CDD-like Malawi and Zambia funds – which reported no impact on social capital, as did the OED CDD report three years later. The review confirms this finding now that we have additional evidence from high-quality impact evaluations.”44 In sum, while the anticipated results, underlying assumptions, and causal logic behind TIS+’s CPO Theory of Change follows decades of conventional thinking around CDD and is logically plausible, evidence amassed from impact evaluations conducted over the past decade indicates that the programming logic is fundamentally flawed and that CDD does not contribute to improved community cohesion or local governance. That said, findings from 3ie’s synthesis study is passionately contested and future parallel studies might in all likelihood find contrary evidence supporting the use of CDD approaches for small￾scale community infrastructure projects. Therefore, the team concluded that the plausibility of TIS+ CPO Theory of Change is, at present, uncertain. 42 Howard White, “Community-Driven Development: Does It Build Social Cohesion or Infrastructure? A Mixed-Method Evidence Synthesis,” International Initiative for Impact Evaluation (March 2018). 43 Ibid. 44 Howard White and Radhika Menon, “Community Driven Development: Howard White and Radhika Menon Respond to Scott Guggenheim,” Oxfam (July 2018). “CDD programs may be using social cohesion rather than building it. Numerous factors may affect community involvement, such as the role played by the elite or prime movers in the community, intra￾community divisions and the perceived benefits of participation. CDD’s impact on governance is sometimes undermined by the creation of parallel structures for the sake of the program. The function of these governance structures is not clear once the community projects end.” ~Howard White et al. “Community Driven Development: Does It Build Social Cohesion or Infrastructure? A Mixed-Method Evidence Synthesis.” 344 FIDELITY OF IMPLEMENTATION The next step in developing the contribution story involves assessing the extent to which implementation actually followed the intended design of the Theory of Change discussed above. Assessment of Fidelity: Moderate. The midline assessment found clear evidence that TIS+ took active steps to ensure that communities in both Banadir and Gedo/Bay were supported to collectively identify and prioritize their shared community needs. There was documentary evidence that TIS+ held a 3-day “Bridging Plan Stakeholders Consultation” workshop with over 60 participants in Banadir. In Gedo/Bay, TIS+ held a 3-day “District Consensus Building” workshop in Baardheere on May 2–4, 2016, involving 42 participants45 and a similar 3-day session in Dinsoor on June 6–8, 2016, involving 43 participants of diverse backgrounds.46 Interviews further confirmed that the workshops followed the 10-step process outlined in the “Community Process Guide.” Interviews with TIS+ staff indicate that district-level POC members were identified during the last day of each workshop and the agendas from the workshops confirm that sessions were indeed planned to identify these members, although the assessment team could not independently verify that these sessions actually took place (it should be noted that the team has no reason to suspect that they did not). However, interviews with the same TIS+ staff members also revealed that very limited training was provided to the POC members during these workshops and that they mostly just discussed the anticipated role of the POCs as they looked to recruit volunteers. Based on both document review and interviews with TIS+ staff, substantive training of the POCs did not take place until nearly 24 months after the establishment and the subsequent reconstitution of the POCs. While TIS+ had originally intended to form POCs at the district level, the program decided in late 2018 that this approach might lead to a concentration of power in the hands of a limited group of people and thereby present opportunities for corruption and that it would be more prudent to establish Pool of In￾Kind Activities (PIKA)-level POCs. Accordingly, PIKA-level POCs were established in November 2018 in Banadir and Baardheere and January 2019 in Dinsoor. This meant that all of the new POCs established in both Banadir and Gedo/Bay were established after the start of construction and even after the completion of construction except for the POC in Baardheere, which was established just two months before the completion of construction. The evidence, therefore, suggests that the POCs had very little formal engagement in the activities in their intended capacities and did not have the opportunity to fulfill their intended role of providing community oversight and regular engagement with local authorities. While the capacity-building and function of the POCs deviated considerably from the original design, both interviews and the assessment team’s review of sign-in sheets confirm that the POC membership looks to be inclusive of both women and youth, and in most cases included the participation of government officials and representatives from the business community. Interviews also confirmed that although the community contracting was not led by the POCs as intended, community contracting did occur in each of the Wave II areas. In sum, TIS+ seems to have followed most of its intended CPO steps but substantially deviated from the original design when it came to the establishment, training, and use of the POCs. While it is true that POCs form only part of the overarching CPO process, it is nevertheless a crucial part to ensure regular 45 Participants included stakeholders from the Jubaland State of Somalia, Somali Federal Government, the Local Administration, youth, women, elders, traditional leaders, religious group and the local private sector among others. 46 Participants included representatives from the interim South West Administration, the Somali Federal Government, the Local Administration, youth, women, elders, traditional leaders, religious groups, and the local private sector. 345 dialogue with authorities, community monitoring and oversight, and for the building of a sense of ownership and responsibility for the sustainability of the activities within the community. EVIDENCE SUPPORTS INTENDED RESULTS Having outlined the intended Theory of Change and extent to which implementation followed that intended design, the next step in building the contribution story is to examine the extent to which key elements of the Theory of Change are confirmed by evidence. Assessment of Supporting Evidence: Moderate As could be expected given the fluid and unique contexts of individual settlements across Somalia, there was a great deal of variation in results across the Wave II areas. As demonstrated below, there were inconsistent results both within and across TIS+ clusters reviewed during Wave II. The evidence was supportive of some intended results in some settlements—e.g., community participation and awareness in Banadir—but then contradictory in others—e.g., participation in Dinsoor and awareness in Baardheere. Thus, while there was supportive evidence of TIS+’s CPO approach achieving intended results in a specific settlement, the results were too diverse to draw any overarching conclusions on the effectiveness of TIS’s CPO approach across multiple clusters and diverse contexts. The three TIS+ settlements in Banadir demonstrated particularly strong results on “community awareness and participation.” The midline assessment found an increase of 58% in respondents indicating that they were aware of community planning events and a significant increase in participation by all groups with the exception of clan elders. Likewise, TIS+ settlements demonstrate a very strong level of satisfaction with projects meeting community needs with an impressive 92% of respondents reporting that “some” or “all” selected projects addressed community priorities. The midline also identified a significant increase in awareness of oversight committees in both TIS+ settlements (+46%); however, notably, there was no change in the perceived effectiveness of those POCs. This evidence would seem to support the finding in the “Fidelity of Implementation” section that the POCs did not perform as had been originally anticipated. There were also fairly positive results in terms of the community interaction and performance of local and state government in Banadir. Encouragingly, the midline saw a strong increase in the reported roles of state and local governments in managing shared assets, particularly in the management of roads and sports arenas, which were PIKAs. There was also a substantial increase in perception of local government providing services, however, most of these services were not related to the activities supported by TIS+. On the less positive side, the perception of services provided by state government decreased across most areas and there was no change in the reported citizen satisfaction with those services. Likewise, there were no notable changes in the perception of decreased conflict over resources and perceptions of corruption actually increased toward both the federal and local government. In terms of inclusion, the midline assessment confirmed that all TIS+ POCs included the participation of women and youth, with an average membership rate of 40% for women and 30% for youth. The results were considerably more varied, and often less positive, in TIS+ settlement in Gedo/Bay. While the participation of different stakeholder groups in community consultations generally increased in both TIS+ and non-TIS+ areas across the cluster, they decreased considerably in Dinsoor. However, despite reporting decreased participation across groups, respondents showed an increase in the perception that projects were selected based on community planning events. Paradoxically, Baardheere demonstrated the exact inverse, reporting an increase in the participation rates across groups but a decrease in the perception of projects being selected based on community planning events. Troublingly, 346 there was a decrease in both Dinsoor and Baardheere in respondents reporting that “most” or “all” of the selected projects effectively addressed community priorities, as well as a substantial decrease in the awareness of POCs and no change in their perceived effectiveness. Likewise, the results were not generally very positive related to shared asset management and quality of services. The midline showed a perceived decrease in the role of state and local government in the management of nearly all assets, including in roads and meetinghouses, which were key PIKA activities. In terms of service provision, international NGOs are seen as having an increased role in the provision of nearly all types of services across the cluster and the role of both state and local government in the provision of roads, key PIKA activities in both areas, decreased in both Dinsoor and Baardheere. Finally, while most settlements across the cluster reported an increase in citizen satisfaction levels with services, Baardheere saw a decrease and the TIS+ constructed road and airstrip were specifically identified as being of poor quality. On the positive side, Dinsoor was noted as a unique “positive deviant” and witnessed a substantial reduction in resource-based conflicts were other settlements across the cluster saw increases. Likewise, both Dinsoor and Baardheere reported reduced levels of perceived corruption by both the federal and local government. Gedo/Bay reported similar results as Banadir in terms of inclusion with women and youth being present on all TIS+ POCs and accounting for roughly 30% of the membership for each group. INFLUENCING FACTORS AND ALTERNATIVE EXPLANATIONS The final step in building the contribution story then is to analyze the extent to which “other influencing factors” and “alternative explanations” explain the identified results. Assessment of Influence of Alternative Explanations: High. As demonstrated by the inconsistency in the results highlighted above, a number of other influencing factors and alternative explanations account for the variety of results, including the following: • Activities of Other Development Partners. There was a wide and diverse set of groups of actors implementing activities in each of the TIS+ settlements reviewed during Wave II, most of whom reported utilizing some aspect of community engagement, planning, and oversight. The assessment team identified at least seven different organizations working in Baardheere, 11 organizations working in Dinsoor, and over 20 different organizations working in Banadir. Both KIIs and FGDs conducted during the midline indicate that there has been a marked increase from these development partners to more systematically engagement with communities at the outset and involve community members in the planning and design of activities. Likewise, other major partners, such as Danish Refugee Council, Finnish Church Aid, Nordic International Support Foundation (NISF), and International Committee of the Red Cross, as well as UN agencies such as the Food and Agriculture Organization and World Health Organization report specifically using POCs or established community structures such as the Peacebuilding Committees to guide the planning and oversight of their committees. • Carry-Over Effects from TIS: The strong performance of Banadir on results related to community awareness and participation is in large part likely explained by strong carry-over effects from the activities of TIS. Mogadishu was one of the first areas in Somalia to pilot CPO approaches during the initial TIS program and TIS conducted numerous community consensus planning sessions, trauma healing sessions, as well as investments in infrastructure including rehabilitation of district administration buildings and sports facilities during its program life. The TIS final evaluation concluded that these activities led to notable improvements in both community awareness and participation. Specifically, the evaluation found that “TIS’s innovative essence was the consultative and participatory process that brought communities (civil society and business people), local administration, and central government together to discuss and 347 identify the community’s priority needs;” “TIS proved to be a strong catalyst in encouraging greater engagement between communities;” and that “TIS contributed significantly toward increasing collaboration between local government and stakeholders.”47 • Unique Characteristics of Settlements. Qualitative interviews (both KII and FGD) clearly conveyed that the unique historical, socioeconomic, and geopolitical characteristics of each settlement played a large role in the results seen during the midline. For example, interviewees identified the unique characteristics of the three TIS+ settlements in Banadir to explain the high levels of both government and development partner interest in supporting activities in these areas. They explain that these districts are closer to the city center and are key revenue drivers for the local and state government, accounting for up to 70% of revenue generation by some estimates. Likewise, clan dynamics and the influence of powerful individuals were identified as reasons for the variations between community awareness and participation in both Dinsoor and Baardheere. • Historical and Contextual Factors. Interviewees also explained that the building of trust between communities and respective levels of government following decades of conflict is a long￾term endeavor and requires generational change. Interviewees explained that while TIS+ activities might facilitate community and government interaction on the planning and oversight of specific shared assets, there is a long list of grievances and frustrations that still need to be overcome. These include factors such as widespread perceptions of nepotism and clannism, unfulfilled election promises, corruption, security lapses coupled with a fear of victimization by AS, injustices and a weak judiciary, poor service delivery, and the outsized influence of diasporas. ASSESSMENT OF CONTRIBUTION CLAIM Summary of Assessment Criteria Criteria Assessment Plausibility: Is the theory of change plausible? Moderate. The anticipated results, underlying assumptions, and causal logic behind TIS+’s CPO Theory of Change follows decades of conventional thinking around CDD and is logically plausible, however recent evidence from a meta-analysis of 25 impact evaluations suggests that this programming logic is fundamentally flawed and that CDD does not contribute to improved community cohesion or local governance. Implementation per plan: Has the program been implemented with high fidelity? Moderate. TIS+ followed most of its intended CPO steps but deviated substantially from its original design when it came to the establishment, training, and use of the POCs. While it is true that POCs form only part of the overarching CPO process, it is nevertheless a crucial part to ensure regular dialogue with authorities, community monitoring and oversight, and for the building of a sense of ownership and responsibility for the sustainability of the activities within the community. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? Moderate. Inconsistency in results across locations means that cross￾cluster conclusions on the effectiveness of TIS+’s approach cannot be determined. 47 IBTCI. “Final Evaluation of ‘Transition Initiatives for Stabilization’ (TIS)”. 348 Identification and examination of other influencing factors: To what extent have other influencing factors been identified? Identification of alternative explanations: To what extent do alternative explanations explain results? High. As demonstrated by the inconsistency in the results, a number of other influencing factors and alternative explanations contributed to the results seen during the midline. These factors include (1) activities of other development partners, (2) carry-over effects from TIS, (3) unique characteristics of settlements, and (4) historical and contextual factors. Summary Assessment of Contribution Claim: Weak. At present, there is limited evidence that TIS+’s Community Planning and Oversight approach is consistently contributing to higher-order, outcome-level results related to improved citizen engagement and confidence in government, as well as reduced conflict over resources or support for violent extremism. While the midline assessment found evidence of TIS+ achieving notable results in terms of community awareness and participation, as well as local and state government service provision and management of shared assets, these results were usually only at the individual settlement level and were not consistent across clusters. TIS+’s contribution claim is also limited by the lack of a consistent implementation approach across the program. While the community consensus workshops look to have been consistently implemented, the form and timing of other key parts of the CPO process varied across settlements. This includes a difference in the way that community contracting took place across the settlements, and most notably, significant deviations on how POCs were formed, trained, and utilized throughout the process. TIS+’s own program design documents clearly highlight the central role that POCs play in the overarching CPO process; however, evidence indicates that TIS+ did not prioritize their role until late in the PIKA process, almost always only after the construction of the different structures was already complete. 349 Contribution Story 2: Technical Coordinators and TIS+ TIS+ is a complex program with different interventions that allow it to adopt a flexible approach to programming in Somalia. In 2017, the TIS+ program and assessment teams developed a program-wide Theory of Change (ToC). Nested within this larger ToC were three sub-ToCs which specifically focused on key approaches utilized by the program to achieve stabilization results. One of these approaches was the use of Technical Coordinators (TCs) embedded within host-country government ministries at the federal and state level. This “contribution story” will focus on critically examining the role of the TCs and any influence they may have had on the achievement of higher-level stabilization, governance, and community cohesion results. In the original TIS+ documents on coordination and engagement, TIS+ TCs, as TIS+-funded positions embedded within Ministries, were designed to support TIS+ Objectives 1, 2, and 3. These were explicitly seen as roles at least partly designed to enhance the efficacy of TIS+ interventions, for example, adding value to the projects delivered under our “core” contribution story on community planning and oversight of infrastructure. The TCs were also intended to deliver wider work in supporting communication between levels of government more broadly. These two functions are summarized in the TIS+ Approach to Engagement document from 2016: “To support continued coordination and collaboration of TIS+ plans and activities with federal and state government plans, TIS+-funded Technical Coordinators are proposed to sit within government ministries. These Technical Coordinators will facilitate not only planning coordination but will support better communication across levels of government and with other stabilization programs in support of more effective stabilization activities.”48 In assessing the TC contribution claim, we will therefore be looking for evidence that TCs have demonstrably improved TIS+ or other interventions, and that they have improved communications between levels of government, as an early step in the results chain, and for evidence that they have contributed to results under the three TIS+ objectives below: 1. Increase confidence in governance based on equitable participation in decision-making and management of community assets. 2. Empower community and government representatives to engage with the private sector and development actors in a collaborative process for community growth. 3. Increase Somali engagement in creating a more stable future (federal-level Ministry TCs only). The contribution story is structured according to the five criteria to analyze the extent of the contribution claim between the TIS+ activities and results witnessed during the midline assessment. PLAUSIBILITY OF THE TIS+ TCS THEORY OF CHANGE The practice of embedding TCs and advisors in ministries and government agencies has been long utilized in Somalia. Against the background of nascent institutions and limited capacity in the civil service, it is seen as an effective way of facilitating donor engagement with government institutions while also building the capacity of the local staff in those institutions. This approach has not been without challenges, which have forced a rethink of the interventions to make them more effective. 48 TIS+ (March 2016) ‘TIS+ Process of Engagement’, p. 4. 350 United Nations Development Programme (UNDP) Somalia49 identified the following obstacles: • Radically weakened institutions with little legitimacy • Dramatic variation in the political landscape from region to region with shifting power and allegiances over time • Security threats • The challenge of maintaining neutrality in a complex situation The approaches to technical assistance delivered in Somalia has varied from utilizing diaspora experts [International Organization for Migration (IOM)/UNDP’s “Migration for Development in Africa” (MIDA)], project implementation units (Aid Coordination Units), Technical Advisors/Coordinators (UN, Care, Concern International, GIZ, USAID) and policy development. TIS+ approach is in line with this thinking to provide embedded capacity within the government ministries at the state and federal level. TIS+ Approach Across TIS+’ three objectives, TCs are expected to do the following: • Support coordination and collaboration of TIS+ activities with government activities. • Support better communication with government at all levels and with other stabilization programs. • Increase knowledge and gain support for locally based Objective 3 activities at all levels of government. Figure 4 on the next page shows the causal steps through which the TCs were expected to contribute to TIS+ objectives and the assumptions on which these steps are based. 49 UNDP, “Supporting capacity development in conflict and fragile contexts,” https://www.undp.org/content/dam/undp/library/crisis%20prevention/2012SupportingCapacityDevelopmentinConflictFragileSetti ngs.pdf 351 Figure 4: TC Theory of Change 352 Since its inception, TIS+ has provided 27 different TCs to different government institutions50 including the following: • Banadir Regional Administration • Ministries of Interior in Mogadishu, Baidoa, and Kismayo • Ministries of Gender, Human Rights, and Family Affairs in Mogadishu and Kismayo • Ministry of Public Works and Reconstruction • Ministry of Youth and Sports • Ministry of Information Some of these TC positions have already been scrapped either due to funding or strategic realignment of the program. The work and output of the TCs will be highlighted in the later section of this document. Assessment of Plausibility: Moderate. The approach of utilizing embedded experts in government institutions across Somalia is common. That said, there are challenges associated with implementing this sort of assistance that constrains the overall prospects for institution and state building. For example, an approach paper commissioned by the World Bank Group and authored by the Overseas Development Institute warns that “in fragile contexts when institutional capacity has been eroded, the strategic positioning and short-term calculations of leadership figures and elite groups carry disproportionate weight in shaping policy directions and outcomes in what are frequently fluid and rapidly changing circumstances.”51 In the absence of strong institutions, policy is often dictated by strong personalities and interests. The success of TCs placed in government institutions depends more on the personality and network of the individuals than on their skills. High turnover in these ministries due to dynamic political changes further affects the influence and progress made on delivering key interventions. Interviews with TIS+ TCs confirmed the challenge of dealing with high turnover often preceded by a political impasse, which often disrupts the project timelines. “We often deal with new Ministers who stay between 6 months and a 1 year and it takes time to bring them up to speed. Sometimes, these are political appointments and focusing too much on the work without cozying up politically can threaten the buy-in of a project. As a Technical Coordinator, you’re seen as having divided loyalties between the project and the institution.”52 There is also the challenge of sustaining positive results beyond the short-term and managing expectations in order to achieve long-term capacity and effective coordination as envisioned in the TIS+ Theory of Change. In interviews, TIS+ TCs highlighted that they constantly worried about their own tenure and funding priorities, which affected their ability to commit to long-term visions by the government institutions they were supporting. “As a Technical Coordinator, my biggest challenge is the uncertainty of the work. We have temporary contracts. Mine runs out next year in February. The Ministry relies on us and we are developing plans going beyond our contract term, which is a bit difficult to do if you know you may not be there to implement them.”53 The challenge of divided loyalties and inherent capacity substitution effects has been noted by other partners supporting technical assistance interventions. In its “Guidelines for Technical Assistance” document, the 50 AECOM, “FY2018 TIS+ Annual Report,” (2018). 51 Overseas Development Institute, “An Approach Paper for Capacity Development in Fragile States,” (2009). 52 Interview with TIS+ Federal TC. (2019). 53 Interview with TIS+ South West TC. (2018). 353 Danish International Development Agency notes that “program coordination and supervision must be vested in the partner institution. The partner institution may be supported in the overall planning, coordination, monitoring and reporting tasks related to program management by technical assistance on the basis of the partner organization’s own efforts to build up such systems.”54 Both the TIS+ program and TCs noted the challenge of subsuming the authority of the government institutions in some instances, with TCs acting as representatives of the government, rather than externally injected capacity. What is evident, however, is that the choice of institutions receiving the TIS+ technical assistance is in line with the priorities of the government. This was confirmed in government interviews, as well as a review of government strategies. For example, the FGS National Stabilization Strategy identifies the TIS+ supported Radio Wadajir platform, designed and produced by a TIS+ TC, as “the main strategic communication initiative supporting the stabilization strategy.” The validation in the government’s plans is an indicator for the expected buy-in and political will from the government for the work of the specific TC in this instance. “The Technical Coordinators are one of the main conduits through which the government engages TIS+. They provided much-needed capacity in the Ministries. At the national level, the institutions are struggling due to lack of capacity. It is even worse at the state level because, for the first time in the history of the country, we are trying to have fully functioning ministries in the states. Previously, we had local administrations managed centrally from Mogadishu. To establish the functioning institutions in Kismayo, we need to make double the effort in a short time to make this a reality.”55 FIDELITY OF IMPLEMENTATION The below summary of some of the outputs by the TCs highlights their contribution: Jubaland State: At the Ministry of Interior and Security, the two TCs listed their deliverables as follows: • Drafting of the Jubaland Stabilization Plan • Drafting of the Ministry of Interior’s Briefing Note on Jubaland Public Expenditure Management Toolkit • Organizing a Countering Violent Extremism (CVE) Workshop • Drafting of the Jubaland State Public Expenditure Management and Planning of the District PEM Cycle • Organizing a training on the Public Expenditure Management Toolkit • Supporting the Sanguuni District Community Consultation organized by SSI/OTI • Delivering a Presentation on the Jubaland Stabilization Plan at the bi-monthly Stabilization Meeting in Mogadishu • Organizing the Jubaland Quarterly Stabilization Meeting • Developing the Organogram for the Ministry • Drafting the Local Government Financial Management Manual At the Ministry of Youth and Sports in Jubaland, the TC’s activities and deliverables were listed as follows: • Developing the Ministry’s Strategic Plan • Supporting the development of the National Sports for Development Framework • Supporting the Kismayo Bridging Plan Consultation Workshop • Presenting on the Sports for Development Framework at the Coordination Meeting in Mogadishu • Supporting the Kismayo Book Fair • Supporting the Jubaland Youth Conference in Doolow • Organizing tournaments for Girls’ Basketball and Boys Soccer 54 DANIDA, “Guidelines for Technical Assistance,” (2009). 55 Interview with former Director General, Ministry of Interior, Jubaland State of Somalia. (2018). 354 The TC at the Ministry of Women, Family Affairs and Human Rights in Jubaland listed the following deliverables: • Development of the Ministry’s three-year strategic plan • Contribution to the drafting of the Sexual Offenses Bill • Drafting of Standard Operating Procedures for Prevention and Response to Gender-Based Violence • Delivering a Training on Leadership and Management for Women Leaders • Organizing a Workshop for Civic Engagement on Women’s Constitutional Rights • Supporting the Opening Ceremony of the Garbahaarey Women’s Center The TC at the Ministry of Public Works and Housing was relatively new but is working on a capacity building plan for the Ministry as well as a strategic plan in addition to supervising the construction of public buildings currently undergoing rehabilitation through the support of various donors. South West State: In Baidoa, the two TCs at the Ministry of Interior and Local Governance listed the following among their deliverables: • Development of the Stabilization Strategic Plan for South West State • Development of the Local Government’s Employment Regulations and the Ministry’s Organogram • Development of Terms of Reference for the Ministry’s departments • Development of the Ministry’s Human Resources Policy • Development of the Finance Policy • Drafting of the Local Government Financial Manual for Trainers • Delivering Leadership Training in Hudur and Bardale Districts • Delivering Induction Training for the new District Council Members in Hudur At the federal level in Mogadishu, TCs contributed to: • Development of the Wadajir Radio Program • Development and launch of the Wadajir Framework for Local Governance • The Mogadishu Book Forum • The National Women’s Basketball Tournament • Gedo Reconciliation Meeting • Supporting the District Council Formation Processes • Marka Corridor Stabilization Plan In addition to the individual documentary output and coordination roles, TCs and government partners noted their contributions beyond the individual Terms of Reference. Noting the inherent low capacity in the institutions they were embedded in, TCs often played larger-than-life roles in the day-to-day running of the Ministries. While this had a positive short-term impact on the capacity of these institutions, it presents the risk of capacity substitution and sustainability challenges in the long-term. This is particularly of concern for stabilization programs that have to deal with the frequent alignment of priorities because of changing dynamics. As mentioned above, TCs regularly highlighted their concerns over their tenure in the long-term: “When I arrived, there were about four staff members in the entire Ministry, so the items on my job description are minuscule compared to the work I ended up doing. My formal title is Strategic Governance Advisor and Technical Program Coordinator. My real work has ended up being the de facto Chief of Staff at the Ministry because I facilitated discussions between the Minister and the international partners, and I also coordinated stabilization activities with partner government agencies.”56 Despite their contributions in the institutions they were embedded in, TCs expressed frustration that their strategic role was not adequately utilized by TIS+ and USAID. Technical Contributors expressed that 56 Interview with TIS+ JSS TC. (2018). 355 they could play a much more pronounced role in TIS+’ decision making on strategic interventions as well as in improving the relationship with government stakeholders. TCs noted that in a lot of instances, they were informed of decisions after they had already been made, particularly whenever the program made realignments which affected planned activities. Many of the TCs felt that USAID did not properly understand their role, citing that they had been often asked by USAID staff during TIS+ strategic review sessions to clarify what exactly they did in the Ministries and also added that the lack of an efficient feedback loop with the TIS+ program team sometimes led to missed opportunities, especially in situations where the governments’ unmet expectations had to be managed. In such instances, TCs felt that TIS+ could have relied on their understanding of the local dynamics and individual relationships to navigate these challenges. “There is still some work to be done on improving the relationship between the Nairobi office and the technical coordinators. Because of this, sometimes we do not have visibility on how the decision-making process takes place. We are at the Ministries and have managed to build a solid relationship in the last two years and sometimes we feel there is lost opportunity to take advantage of the goodwill and sort out any potential misunderstanding or managing expectations.”57 It is important to note that some of these challenges and missed opportunities have been acknowledged by TIS+. One of the main outcomes from the 2018 annual Strategic Review Session (SRS) held by TIS+ was to “define the best possible way as a best practice—to engage the TCs and Advisors in the FGS and Emerging Federal States (EFS) ministries.”58 As part of the realignment, TIS+ has implemented systematic reporting from the TCs and incorporated their participation as part of TIS+ progress on improving links to the host government. TIS+ lists the contribution and participation of the TCs as a core approach to improving the program’s linkage with the government institutions it works with. Assessment of Fidelity of Implementation. Moderate. It is evident from the review of the program documents, the output from the TCs, and interviews with government partners that the work of the TCs contributed to both improved coordination between the government and TIS+ and increased capacity by the government to engage with stabilization actors and donors and better planning by the government. Evidence Support Intended Results. Moderate. The next step is to examine the extent to which key elements of the Theory of Change are confirmed by evidence. TCs were designed to improve existing TIS+ interventions, and to improve coordination in government in order to contribute to Program Objectives 1 and 2: 1. Increase confidence in governance based on equitable participation in decision-making and management of community assets. 2. Empower community and government representatives to engage with the private sector and development actors in a collaborative process for community growth. Ultimately, the TCs are aiming to contribute, alongside other TIS+ interventions, to increased confidence in state government. We have a mixed picture of confidence in the government from qualitative and quantitative surveys. Moreover, the results chain connecting the TCs’ inputs to these results is so long and is decisively influenced by so many other variables, that it would be unfair to expect any contribution at this level in any survey data. Our results chain for the TCs’ work has therefore concentrated on intermediate steps, such as improving TIS+ interventions, improving coordination, and, crucially, injecting sustainable capacities in the relevant ministries. 57 Interview with TIS+ JSS TC. (2018). 58 AECOM, “FY2018 TIS+ Annual Report,” (2018). 356 The TCs themselves certainly confirmed that their activities had been focused both on supporting TIS+ interventions and on promoting coordination. This is supported by the evidence of their produced documents, interviews with government officials and other partners as well as the program documents. TCs have also worked on processes of vital importance to the state government, such as district council formation processes. There was evidence that government counterparts appreciated the work of TCs and saw them both as providing a point of contact with TIS+ programming, and promoting government ownership of TIS+ interventions, as well as building capacities in their ministries but respondents emphasized the importance of larger scale institution building. “At [the] national level, the institutions are struggling due to lack of capacity. It is even worse at the state level because, for the first time in the history of the country, we are trying to have fully functioning ministries in the states. . . . The Technical Coordinators provide their own skills and experiences as required. In addition, they also train and build the capacity of the few available civil servants in the Ministries.”59 Broader CA interviews revealed that many other programs that work with government generally heard good feedback on TIS+ TCs from government partners, and some who had encountered TCs in Kismayo or been involved in events organized by TCs in Jubaland, were positive about these experiences. Some of these respondents did, however, suggest that following up on these events and deliverables was important. Others suggested that TIS+ could benefit from a more explicit focus on capacity building in the work of its TCs. This was confirmed by other respondents who said TIS+ advisors had made a positive impact to the development of stabilization strategies, but this should be matched with an assessment of the real capacities but also motivations and leverage of the political actors in the ministries to implement these. An approach more focused on improving capacities beyond the lifetime of supported TC roles would likely include such an assessment. Other donors working on capacity injection in Somalia have noted that there has often been a failure of capacity injection interventions in Somalia to be tailored to specific needs, which are different in different regions.60 TCs themselves did feel that they were building capacities in their ministries but felt that the needs in this respect were very large: “We provide a lot of support especially in terms of capacity for the Ministry because the skillset is not available because the South West State and the system of administration is less than five years old and when you think of a civil service in a country, you’re talking about building up capacity over 50 years or more.”61 Assessment of Influence of Alternative Explanations: High. The next stage in our analysis is to ask what other influencing factors could be accounting for the changes observed. These include other externally funded advisors as well as positive and negative factors affecting coordination and capacities in the relevant ministries. Qualitative research showed that in most of the Ministries where TIS+ TCs are deployed, they were often the only ones at the Ministry, with the exception of the Ministries of Interior in both states, where there were a few other TCs deployed by IDLO and the United Nations Joint Program for Local Governance (UNJPLG). TIS+ coordinates very closely with programs such as Somalia Stability Fund (SSF), which deliberately does not duplicate advisors in the same ministries where TIS+ TCs are placed. As a result, the TCs contribution to the state-level Ministries is visible but there is a mixed picture at the federal level due to multiple actors providing an assortment of Technical Assistance to the institutions in the form of short-term consultants as well as long term technical coordinators. 59 Interview with former Director General, Ministry of Interior, Jubaland State of Somalia. (2018). 60 World Bank (2015) Somalia-Capacity Injection Project (English). Washington, D.C.: World Bank Group. 61 Interview with TIS+ South West TC. (2018). 357 Levels of existing capacity within the Ministries where TCs are based has a clear impact on the contribution TCs will make. One of the main challenges that the Federal Member States have to contend with is the capacity gaps in the civil service. The advent of federalism has meant that the powers and services have been devolved to the regions, but this did not come with any transfer of capacity or resources from the center. Mogadishu itself struggles with lack of capacity in the federal civil service. However, there has been heavy investment through programs such as the Capacity Injection Program by the World Bank and the deployment of short-term and long-term technical assistance by a variety of donors. Jubaland was the first of the FMS to be established, and as a result of the relative stability, it has seen a steady influx of professionals from Mogadishu and abroad moving to Kismayo. However, there has been no major donor investment in the capacity of the civil service beyond programs focused on public financial management. Short-term technical assistance (STTA) has come through IOM’s MIDA especially in public healthcare but there remains a serious lack of capacity especially at ministerial level as a result of the limited available wage bill and the lack of donor investment when compared to Mogadishu and Puntland. The South West administration’s expansion plans are uncertain, especially as the date of the move of the capital to Barawe remains unclear. The administration has not moved much from the vicinity of Baidoa airport where the Presidency is located. Limited resources at federal and state level have resulted in poorly capacitated ministries. The fairly nascent state of the administration has meant that there has been little investment in institution building and technical assistance to the Ministries. However, the establishment of the civil service commission in Baidoa may improve the capacity of the ministries. Traditionally, the overwhelming investment in technical assistance and capacity injection has been focused at the federal level in Mogadishu as well as in more established regions such as Somaliland and Puntland. Programs including IOM MIDA, UNDP’s Multi-Sectoral Capacity Development Program, the World Bank’s Capacity Injection Project, and SSF, as well as USAID-funded programs including Strengthening Somali Governance (SSG) and Growth, Employment, Economy and Livelihood (GEEL) have provided various forms of technical assistance in the aforementioned regions. However, as of 2018/19, there has been increasing investment in the Emerging Federal States (EFS) which will likely improve the capacity of state-level institutions in the long-term. In addition to the contribution of other development partners, the assessment also acknowledges the carry￾over effect of TIS, particularly at the federal level in Mogadishu, where the program was first piloted. The evaluation of the program concluded that the program had “contributed significantly toward increasing collaboration between government and stakeholders.”62 TIS utilized the “Dan Guud“ model, where it embedded professionals into the Banadir Regional Authority (BRA). These professionals were paid for by TIS but effectively served as the nascent cadre of civil servants that improved the capacity of the local government’s functions. With the end of TIS, many of these professionals were absorbed into the BRA as government civil servants. The political and security environment had a strong influence on the contribution of the TCs. In particular, the frequent political crises which contributed to the high turnover in the leadership of the Ministries often constrained the momentum built by the TCs. This is further compounded by an unpredictable security situation, particularly in Mogadishu, where the federal-level TCs are based. 62 IBTCI. “Final Evaluation of ‘Transition Initiatives for Stabilization’ (TIS)”. 358 ASSESSMENT OF CONTRIBUTION CLAIM Summary of Assessment Criteria Criteria Assessment Plausibility: Is the theory of change plausible? Moderate. The approach of utilizing embedded experts in government institutions across Somalia is common. However, there are challenges associated with the approach including the potential for elite capture, frequent turnover of government leadership, and the challenge of sustaining positive results beyond the short term. Implementation per plan: Has the program been implemented with high fidelity? Moderate. It is evident from the review of the program documents, interviews, and the output of the TCs that they implemented as per the plan. However, there was also evidence that TCs were not adequately utilized in situations where they would have greatly contributed to. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? Moderate. There is evidence that TC work contributed to improving TIS+ interventions, improved coordination and injected capacity in their host Ministries. However, the high-level expected contributions, such as increasing confidence in government, presents a mixed picture. Moreover, the results chain connecting the TCs inputs to these results is so long and is decisively influenced by so many other variables that it would be unfair to expect any contribution at this level. Identification and examination of other influencing factors: To what extent have other influencing factors been identified? Identification of alternative explanations: To what extent do alternative explanations explain results? High. A number of other influencing factors and alternative explanations contributed to the results seen. These include the activities of other actors, the carryover effects from the previous TIS program, and the highly dynamic political and security situation. Summary of Assessment of Contribution Claim: Moderate. Despite missed opportunities and the impact of external factors, there was evidence that TIS+ TCs accomplished the following: • Supported continued coordination and collaboration of TIS+ plans and activities with federal and state government plans • Supported coordination with other stabilization programs • Improved communication especially on the government’s stabilization plans The contribution claim is also limited by the fact that other actors, and in particular, stabilization programs, also embedded TCs and Advisors to carry out similar functions such as supporting state- and federal-level coordination of stabilization programs. There was also limited evidence that the TIS+ TCs contributed as expected to higher-level results such as increasing confidence in government. 359 Contribution Story 3: Sports for Development and Peace (SDP) The use of sports is a key feature of TIS+’s youth engagement strategy. The program has made sizable investments in either building or rehabilitating football stadiums and basketball courts as a shared community asset and to provide a venue for the community, and youth, in particular, to gather and interact. TIS+ has also undertaken efforts to develop and roll out a national “Sport for Development and Peace” (SDP) strategy through its Technical Coordinators and an outside SDP consultant. According to its 2018 Annual Report, TIS+’s investments in “sports and youth facilities reaches thousands of youth across Somalia and provides them with opportunities to meet, break down barriers of clan and geography, develop a national identity, participate in peaceful social interactions and learn to negotiate losses, compromise and non-violently resolve conflict-all of which directly counter the divisive narrative extremists propagate.”63 The following “contribution story” will critically examine the extent to which TIS+ has contributed to the positive engagement of youth and reduced support for violent extremism through the use of SDP. It is structured according to the five criteria identified by John Mayne, the creator of the Contribution Analysis methodology, to analyze the extent of a “contribution claim” (or “plausible association”) between TIS+ activities and the results witnessed during the midline assessment. These criteria include the following: 1. Plausibility: Is the theory of change (ToC) plausible? 2. Implementation per plan: Has the program been implemented with high fidelity? 3. Evidentiary confirmation of key elements: To what extent are the key elements of the ToC confirmed by new or existing evidence? 4. Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? 5. Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? The contribution story will then conclude with an overarching assessment of the strength of TIS’s contribution claim to the results relevant to increased youth engagement and reduced support for violent extremism identified in the midline assessment. PLAUSIBILITY OF TIS+’S CPO THEORY OF CHANGE Theory of Change and Underlying Rationale TIS+ did not have a defined ToC specifically focused on SDP at the outset of the program. Rather, the renovation of sports stadiums was covered under its rationale for supporting quick-impact, small-scale infrastructure projects to promote the use and management of shared community assets. Likewise, the hosting of tournaments and other larger-scale sports activities was also covered under TIS+ more general community engagement approach and strategic communication plans. However, in order to enable a Contribution Analysis of TIS+’s results, the assessment team worked with the TIS+ team to develop an SDP ToC in June 2018 (see Figure 5 below). The resulting ToC highlighted some of the key intended results, causal logic, and underlying assumptions guiding TIS+’s SDP-related activities. However, the ToC was rather weak and included substantial leaps in the causal logic and several unrealistic assumptions. Identifying an opportunity to be more strategic in its approach to SDP, as well as wanting to support the Ministry of Youth and Sports (MOYS) develop a national sports policy, TIS+ contracted an outside SDP expert to work with the MOYS to develop a national SDP policy. A final draft of the “Somali Ministry of Youth & Sport (MOYS) Sport for Development and Peace (SDP) Strategy & Implementation Framework” was produced in August 2018. 63 AECOM, “Transition Initiatives for Stabilization Plus: Fy 2018 Annual Progress and Financial Report,” (2019). Pg. 78. 360 Figure 5: TIS+’s SDP Theory of Change 361 The SDP Strategy draws on international good practice and global experience on positive youth development (PYD) through the use of sports and highlights several factors that make it particularly appropriate for the Somali context. The Strategy posits that SDP can help reduce youth propensity toward violent extremism by offering an “attractive alternative” and using sports as a “hook” to attract at-risk youth.64 SDP was also assumed to bring additional benefits, such as physical and mental benefits of physical exercise; a mechanism for developing life and social skills such as teamwork, sportsmanship, and comradery; the possibility to link with educational and vocational opportunities to both attract youth and also strength their capacity and potentially employability; and even to promote community-wide civic engagement through the use of Community Planning and Oversight approaches to the selection, implementation, and oversight of SDP projects, as well as an opportunity to promote shared-asset management between the community and local authorities. The Strategy even identified opportunities to use SDP to promote other cross-sector approaches supporting gender equity, social inclusion, and trauma healing. Figure 6 below provides an overview of the Strategy’s conceptual framework. The Strategy also included an updated ToC behind the intended SDP activities (see textbox below). 64 Paraphrased from Stephen Reynard, “Somali Ministry of Youth & Sport (MOYS) Sport for Development and Peace (SDP) Strategy & Implementation Framework,“ ed. AECOM/TIS+ (August 2018). Figure 6: SDP Conceptual Framework Revised SDP Theory of Change “If the federal and state MOYS are able to implement the Holistic SDP Conceptual Framework, then youth will develop relevant professional, life, and social skills and feel more empowered and connected to local development issues, which will enable them (youth) to counter the negative physical, psychological, and emotional effects of their environments and will help them (youth) to contribute to improved social cohesion, social inclusion, and positive peace.” ~ “Sport for Development and Peace (SDP) Strategy and Implementation Framework” 362 Specific to Preventing and Countering Violent Extremism (P/CVE), SDP was seen as providing a “structure that young people detached from their communities and schools may need. It can also act as a ‘diversionary activity from violent and illicit activities’. Building resilience to radicalization can be viewed in a similar light as crime prevention in that there is ‘crossover in the potential influencing factors that may contribute to an individual becoming radicalized, or engaging in crime.’”65 Relevance with Somali Context and National Efforts Somalia has a long history of using sports as a mechanism for community-mobilization and developing a shared identity (see textbox). Sports have also been used a tool to promote peace by several Somali government ministries, such as MOYS, the Somali Olympic Committee (SOC), the Somali Football Federation (SFF), as well as by a variety of national and international NGOs. Private companies—such as Hormuud Telecom, NationLink Telecom, Premier Bank, and Dahabshiil Money Transfer—have also hosted sports tournaments or funded renovations to community sports fields and stadiums as Corporate Social Responsibility (CSR) and community outreach activities. The use of sports is also widely regarded as an effective way to engage youth, a key demographic with over 70% of Somalia’s population under the age of 30 according to UNICEF Somalia.66 This is reflected in Somalia’s “National Youth Policy” which states that “Sports and recreation are very important factors for youth since this provides opportunities to socialize, build social connections, counter boredom, spend their time productively, develop character and learn new skills and teamwork. They also shape the mental and physical wellbeing of the youth.”67 The Policy recommends several sports-focused interventions, which strongly align with those utilized by TIS+, including (1) “strengthening sports and restarting the interregional, regional and national tournaments for both male and female athletes;” (2) “increase the opportunities for young women and men to engage in a diverse range of sports and sporting competitions including soccer, basketball, volleyball, and athletics;” (3) “provide greater support and direction to youth and sports centers to enable them to become more friendly and conducive to young people with special needs; and” (4) “instigate media awareness programs to raise awareness amongst parents and community and religious leaders of the importance of recreational and sporting programs to the personal development of both young men and women.” Additionally, the TIS+ SDP approach is also consistent with other Somali policy frameworks, including the Somali National Development Plan and the Somalia Youth for Peace Pact. Evidence from Other SDP Projects 65 Ibid. 66 UNICEF Somalia, “Education in Somalia.” 67 Ministry of Youth and Sports of the Federal Government of Somalia, “The National Youth Policy of the Federal Government of Somalia.” History of Sport in Somalia 1969–1991: Sport was used as a “mobilization and orientation” tool by the Siad Barre regime, mostly focused in Mogadishu and in a few major cities. All sports clubs were government funded; private sports clubs not encouraged. 1982–1990: Following the war with Ethiopia and subsequent political and economic effects, government-funded sport deteriorated. 1991–2009: Despite civil war and destruction of sports infrastructure, the private sector-initiated sports programs to promote peace. Somalia was represented at various international sports competitions. 2010–2016: Banadir Stadium and Aden Yabarow Wish Sports Center were rehabilitated by the Fédération Internationale de Football Association (FIFA) and the International Olympic Committee (IOC); Sports for Peace was included in SOC’s strategic plan. 2017 – present: Development of 1st National Sports Policy (pending Cabinet approval). ~ “Sport for Development and Peace (SDP) Strategy and Implementation Framework” 363 There are a number of studies that have examined the use of SDP both in Somalia and across the globe. An independent review of the International Labor Organization’s (ILO’s) “Youth for Change” project, which worked with over 6,000 youths determined to be a risk of Al-Shabaab recruitment, found that the project helped “ facilitated healthy competition and cohesion amongst the groups” and helped “reduce aggression among participants.”68 Further, an independent evaluation of the USAID-funded “Somalis Harmonizing Inter￾and Intra-Communal Relationships” (SHIIR) program implemented by PACT found that the program contributed to positive changes regarding the beliefs and attitudes of the 480 male youth participants towards other clans” but that “to have a meaningful impact on youth, the soccer drills should be accompanied by an in-depth educational program with a long-term perspective.”69 The MOYS SDP Strategy and Implementation Framework also cited a number of other international studies that demonstrates SDP’s potential to building social cohesion and promote “positive peace,” mend “broken social relationships,” promote psychosocial healing, address social exclusion, and challenge negative gender stereotypes and mitigate potential gender￾based violence. Assessment of Plausibility: Plausible The updated SDP ToC developed by TIS+ presents a convincing argument supporting its intended results, causal linkages, and underlying assumptions. The ToC is aligned to the Somali context and follows a deep and widespread history of using sports to promote community engagement, cohesion, and outreach. It is also aligned with host-government strategies such as the National Youth Policy and obviously also the SDP Strategy within which it is presented. Finally, the ToC is also supported by evidence from independent evaluations and secondary studies of SDP-related programs in both Somalia and across the globe. FIDELITY OF IMPLEMENTATION TIS+ has implemented a number SDP-related activities, including the construction or renovation of football and basketball stadiums, organization of tournaments and other sports events, the establishment of sport facility Project Oversight Committees, and also the distribution of sports equipment to both community and government stakeholders. Figure 7 below provides an overview of all SDP-related PIKAs implemented at the time of the midline assessment. Figure 7: List of SDP-related PIKAs Location PIKA No. and Title Description Status at Midline Abdiaziz (BRA004) BRA Districts Sports Facilities and Youth Engagement 59-member District Sports Facilities & Programming Bridging Plan Task Force (BRA, district consensus groups, technical experts, donor/partner organizations and civil society representatives) formed Completed Abdiaziz (BRA012) BRA Abdiaziz District Basketball Mini Stadia Rehabilitation Rehabilitation of basketball stadium, setting up and training IFOC (formerly POC) Ongoing Hawlwadag (BRA013) Mogadishu Hawlwadag District Basketball Court Rehabilitation Rehabilitation of basketball stadium, setting up and training IFOC Ongoing Warta Nabada (BRA011) Mogadishu Warta Nabada District Basketball Court Rehabilitation Rehabilitation of basketball stadium, setting up and training IFOC Ongoing Kismayo (JSS013) JSS Kismayo National Youth Day Youth Engagement National Youth Day support with football, basketball, bicycle race, peace march in Kismayo Closed Kismayo (JSS026) JSS Ministry of Youth and Sports Technical Coordinator/Sports Technical Coordinator support Ongoing Garowe (PSS002) PSS Garowe Town Basketball Court Upgrading Construction of basketball stadium Ongoing 68 Lilla Schumicky-Logan, “Addressing Violent Extremism with a Different Approach: The Empirical Case of at-risk and Vulnerable Youth in Somalia,” Journal of Peacebuilding & Development (August 2017). 69 Reynard. 364 Baidoa (SWS008) SWS Ministry of Youth and Sports Baidoa Youth Day Supported National Youth Day celebration Closed Dinsoor (SWS016) SWS Dinsoor District Football Stadium Construction Construction of football stadium; setting up and training IFOC Completed Barawe (SWS015) SWS Ministry of Youth and Sports Barawe District Football Stadium Construction Development of engineering documents Ongoing National (SOM014) Somalia Somali Women’s Basketball Support Regional Training 8 training sessions for already established women/girls’ basketball teams; providing new equipment, uniforms, and training in preparation for their participation in a national tournament (96 girls trained) Closed National (SOM015) Somalia Somali National Women’s Basketball Tournament A national women’s basketball tournament, with 8 regional teams (96 young women) participating, and the selection of a women’s national team Closed National (SOM028) Somalia Sports for Development and Peace Framework STTA STTA for FGS MOYS Sport for Development and Peace Framework and a rollout plan for Somalia in consultation with the FMSs MOYS and Sports Federations; SDP approach for TIS+ produced Ongoing In 2016, TIS+ supported preparatory basketball training camps for 119 young women (including in Kismayo) and then supported the Somali National Olympic Committee to organize a national women’s basketball tournament, which took place in the court rehabilitated by TIS+ in Garowe. Working through federal and regional MOYS, TIS+ supported logistics, transport, and accommodation for the second national men’s soccer tournament in December 2017. Both these tournaments generated significant interest and impact on new social media. Respondents interviewed during the midline assessment confirmed that the men’s soccer tournament had been a major event that the nation had stopped to watch. However, as evident in the PIKA list above, the majority of TIS+’s SDP-related activities have focused on the construction or renovation of simple community sports facilities. Only one of these facilities has finished construction at the time of the midline and unfortunately, that facility, a sports stadium in Dinsoor, was blown up by AS militants and almost completely destroyed. As noted in the ToC section, TIS+ did not have a defined strategy to guide its SDP work until mid-2018. This has resulted in a trend for most SDP activities to be stand-alone with few follow-on or complementary activities to support the achievement of higher-level community cohesion and youth engagement results. That said, the wide range of actors currently supporting SPD activities or interested in support SDP activities in the future, combined with capacity challenges in the MOYS, suggest that the TIS+ approach of working on a strategic framework for SDP for MOYS, as well as providing technical support to MOYS, was appropriate. Assessment of Fidelity: Weak. TIS+’s implementation of the SDP strategy and ToC has to date been limited mostly due to the recent development of these strategies. Interviews with TIS+ staff as well as evidence recent periodic program reports suggest that the program is committed to its new SDP strategy and endeavor to undertake a more systemic approach to implementing activities following this strategy. EVIDENCE SUPPORTING INTENDED RESULTS According to TIS+’s 2018 Annual Report, the program has achieved the following SDP-related results: • Facilitate the development and rollout of the Somalia Sports for Development and Peace (SDP) Framework while utilizing TIS+ investments in sports facilities and sectors. • Complete 20 infrastructure projects: Five sports stadiums. 365 • Hold four special celebrations including Somali National Youth Day and the International Day of Sport for Development and Peace in Baidoa (SWS), Galkayo (PSS) and Kismayo (JSS), with 1 celebration held in Q4. • Launch one National Sports for Development Framework in Mogadishu. • One Men’s National Soccer Tournament in Mogadishu attended by over 30,000 people. • Capacity-building and technical support provided by TIS+ Technical Coordinator to the Jubbaland Ministry of Youth and Sports. However, evidence from the midline assessment indicates that these output-level results have not yet translated into clear outcome-level results. There was a consensus among key informants working on sports￾related activities that conducting or facilitating sports events alone was not enough to deliver desired results. Physical education does deliver positive health benefits by itself, but the SDP framework is not a health intervention first and foremost. There may be some inherent benefits for social cohesion of providing sports venues and events. As one business leader in Kismayo noted: “People from different clans meet and interact with each other in places such as the sports stadiums, restaurants, businesses, and mosques. These venues provide an opportunity for people with different views to exchange ideas, learn from each other, reconcile and bridge differences.” Additionally, simply having sports venues without additional supporting activities was regarded by interviewees as insufficient for achieving higher-level results. “Awareness creation is important as well as having stadiums and youth centers.” This message was reinforced by several interviewees who had worked on, or had an interest in, sports. As one specialist in youth programming put it, “It is important not just to support events—it has to be more than kicking a football around.” It was seen as very important for these interventions to be conducted in a purposive way. As much as bringing people together in events and tournaments could support a sense of shared identity, these events needed to be more intentional in their design to be fully inclusive, as well as to avoid potential perverse outcomes from interstate games that could exacerbate regional tensions. The recent SDP approach developed by TIS+ focuses on using sport to deliver or to boost the delivery of other skills and messages. These can be through side-by-side programming in which educational, civic, or vocational training is provided alongside sport, or through “pedagogical sports drills” in which sport training sessions are used as a platform to diffuse educational and life skills messages. This approach appears to have been used by TIS+ in its support to the men’s football tournament in December 2017: “The Sports for Development and Peace approach uses sports to create social change, to facilitate community development, to increase cohesion, and to support peace and conflict mitigation efforts. . . Four SDP drills were conducted to show how working together can facilitate healthy competition that will extend off the playing field. . . . Before today we were just playing football for health and personal skills, but today we have learned that sports are a great tool for mitigating conflict and fostering peace.” The main challenge to the suggested mechanism in the SDP results chain is the focus on individual youth choices and decisions. This assumption was seen as crucial by some of our expert, peer donor, and program interviewees. As one such interviewee noted: “As for sports for development and peace, I don’t know if this would work according to my understanding of how recruitment happens. If we are saying sports will keep youth distracted and keep them away from AS – perhaps this could work when we have heard some cases of individual recruitment. But these are not many. In many cases, recruitment to AS is a collective decision, for example of a sub-clan who decides to franchise with Al-Shabaab because they think that in their particular circumstances, they best represent their interests. When that happens, it is the whole clan militia that is franchised to Al-Shabaab. They have to undergo training, most of which is ideological indoctrination, and out of this process a good proportion become hardcore jihadi extremists.” Related to gender, midline interviews suggested that Somali society is generally accepting of men’s participation in sports events. However, the participation of women and girls is much more controversial. 366 AS prohibits women and girls from playing sports, but disapproval of women and girls’ participation is much wider than AS. For example, before the TIS+-supported basketball tournament in Garowe in December 2016, the Somali Religious Council, a group of influential Somali clerics, issued a warning to women not to engage in sports: “We warn that the women basketball violates Islamic law, culture, and its values, and it is a place where women can be easily corrupted.”70 The tournament was conducted with high-security costs. From a “do no harm” perspective, this has clear implications for the protection of women and girls’ safety during sports activities and their protection from subsequent stigma or threats because of their involvement in such activities. The TIS+ 2016 basketball tournament and the training camps that preceded them are some examples of interventions targeted specifically at women and girls. TIS+ supported basketball training camps that involved 119 young women between the ages of 16 and 25 who came from seven different regions (states) in Somalia. TIS+ supported the Somali National Olympic Committee to organize a national women’s basketball tournament, the first such event in over two decades. The tournament was held on a TIS+ rehabilitated basketball court in Garowe and was watched by an estimated 900 viewers per day, many of whom were women. In addition to the direct participants and spectators, the tournament reached over one million people through social media was broadcast live on national and regional television, and sparked significant debate about women and sports across Somalia. Our respondents confirmed that these interventions had had a sustainable impact on women’s’ basketball in Somalia, which was progressing at a federal level, as well as at the Jubbaland level, where women’s basketball teams were still playing. Other respondents confirmed the huge positive impact that access to sport could have on women and girls who previously had no opportunity to engage in physical activity. Assessment of Supporting Evidence: Weak. TIS+’s SPD-related results to date have been primarily at the output level and there is little evidence of TIS+ implementing an integrated and intentional strategy that explores and utilizes potential complementarities between activities. That said, the development of the SDP Strategy and Implementation Framework is a key recent result and is expected to help systematize TIS+’s SDP approach going forward and increase the prospects of achieving outcome-level results. INFLUENCING FACTORS AND ALTERNATIVE EXPLANATIONS Our assessment indicates that although many stabilization and development actors have supported sports activities often in an ad hoc manner, efforts to develop a more consolidated programming approach to sport interventions are nascent. TIS+’s own stakeholder mapping, conducted by the SDP consultant in 2018, revealed a range of actors working on sports in Somalia and identified a wide range of actors: ● UNSOM, UNFPA, and UN-Habitat, which use sport as a tool to engage women and youth ● The EU, which is planning to build cultural centers with sport as a component ● Somalia Olympic Committee, which is planning to develop a Sports for Peace program (2017– 2020) ● The Pact SHIIR program, which piloted pedagogical sports drills in Jubbaland ● IOM, which has used sports interventions in its programs for rehabilitating AS defectors ● CARE Somalia, Mercy Corps, and Concern Worldwide, which were also identified as working or having worked on sports71 ● Private sector companies such as Hormuud Telecom, NationLink Telecom, and Premier Bank, and Dahabshiil Money Transfer, which have sponsored the Somali Olympic Committee, and 70 https://www.voanews.com/a/somali-clerics-warn-women-aginst-playing-basketball/3646679.html 71 TIS+. (2018). Sport for Development and Peace Strategy and Implementation Framework stakeholder mapping PowerPoint presentation. 367 Somali Football Federation, and SSF, as well as conducted a CSR program focused on education and health in the Horn of Africa ● Other donor, program, or implementing partner actors such as US OTI, GIZ, SSF, and the NISF, who have worked on sports interventions or have plans to do so. This range of actors who have worked/are working on sports interventions are certain to affect TIS+ future results and there was some evidence that these initiatives have already influenced perceptions of TIS+’s activities. The Kismayo sports stadium, mentioned by several Kismayo interviewees as a positive asset, was reconstructed by NISF in 2014 with UK funding and has more recently been upgraded in 2017 by the American Refugee Committee and the United Nations High Commission for Refugees. Sports tournaments were also mentioned by respondents as positive developments. TIS+ will certainly have contributed to these through its basketball tournament (which was in 2016) and the men’s soccer tournament in 2017.72 However, there are many other actors supporting tournaments, which may have had a larger effect on responses because they are more recent. For example, in May 2018, Save the Children Somalia, in partnership with MOYS, co-organized an interschool sports tournament for 14 public schools. The football and athletics tournament were funded by the German Federal Foreign Office.73 Results are also likely to be substantially affected by factors in the political economic and security context that affect sports events specifically. There are many supporting factors in the Somali environment that would enhance the effectiveness of sports interventions for youth. The fact that Somalis are big sports fans was mentioned by a number of expert, peer donor, and program interviewees. A number of respondents noted that there were positive Somali role models on the international sporting arena, such as Mo Farah (athletics), Abdi Bile (athletics), Maryam Nah Muse (athletics), Abdisalam Ibrahim (soccer), and Mukhtar Ali (soccer), and other Muslim sporting heroes such as Mo Salah who were wildly popular in Somalia. A further positive factor, also made by TIS+, is the existence of several private sector sports football stadiums and investors in sports events, including banks and telecom companies. This means that sports facilities and events would take place without donor intervention in Somalia. These interventions are sustainable after the initial investment. There are also threatening and challenging factors in the Somali environment that negatively affect sporting interventions and pose serious risks to them. These are related to AS threats and attacks and to the broader acceptance of women’s involvement in sport. This is a threat is exemplified by the AS attack on the TIS+ Dinsoor stadium in 2018. It is also demonstrated by the closure of 30 football fields in Mogadishu follow threats by AS. AS also handed down terms in AS prisons for some of those defying these orders.74 These orders are an effective way for AS to demonstrate and project its symbolic power into areas not formally under its control. There have also been AS attacks on sporting events, such as the bombing of a friendly match in a Barawe football stadium in April 2018, which killed five people.75 This does not indicate that AS has an inherent opposition to football. There have been stories reporting that AS allows and encourages the playing of “halal” football between AS fighters and local teams in areas under its control.76 However, it does illustrate the security risks of public events and venues, especially those connected to the government, and the need to protect participants. Assessment of Influence of Alternative Explanations: Moderate. There are several other actors who are engaged in sports activities in Somalia, including donors and development programs, but much of this work has been ad hoc in nature. TIS+ could maximize its impact in this area by playing a coordination role among donors and programs, working with the MOYS. The popularity of the sport among Somalis and the proliferation of private sector venues suggest that sport would be positively regarded by Somalis with or without interventions by donors and programs, and are likely to 72 https://www.kawowo.com/2017/12/24/jubbaland-wins-somalia-inter-states-championship/ 73 https://somalia.savethechildren.net/news/enhancing-integration-through-inter-school-football-athletics-tournament-kismayo 74 https://www.aljazeera.com/indepth/features/somalia-al-shabab-shuts-football-pitches-mogadishu-180607045144830.html 75 https://www.aljazeera.com/news/2018/04/blast-somalia-stadium-kills-football-fans-180413070232458.html 76 https://www.aljazeera.com/blogs/africa/2014/06/99086.html 368 enhance the impact of interventions. However, AS attacks pose a risk to sports venues and events, and social and pervasive resistance to the participation of women in sports threatens to undermine the objectives of sports interventions, especially those engaging women. ASSESSMENT OF CONTRIBUTION CLAIM Summary of Assessment Criteria Criteria Assessment Plausibility: Is the theory of change plausible? Plausible. The updated SDP ToC developed by TIS+ presents a convincing argument supporting its intended results, causal linkages, and underlying assumptions. It is aligned to the Somali context and government strategies such as the National Youth Policy. It is also supported by evidence from independent evaluations and secondary studies of SDP-related programs in both Somalia and across the globe. Implementation per plan: Has the program been implemented with high fidelity? Weak. TIS+’s implementation of the SDP strategy and ToC has to date been limited mostly due to the recent development of these strategies. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? Weak. TIS+’s SPD-related results to date have been primarily at the output level and there is little evidence of TIS+ implementing an integrated and intentional strategy that explores and utilizes potential complementarities between activities. That said, the development of the SDP Strategy and Implementation Framework is a key recent result and is expected to help systematize TIS+’s SDP approach going forward and increase the prospects of achieving outcome-level results. Identification and examination of other influencing factors: To what extent have other influencing factors been identified? Identification of alternative explanations: To what extent do alternative explanations explain results? Moderate. There are several other actors who are engaged in sports activities in Somalia, including donors and development programs, but much of this work has been ad hoc in nature. TIS+ could maximize its impact in this area by playing a coordination role among donors and programs, working with the MOYS. However, AS attacks pose a risk to sports venues and events, and social and pervasive resistance to the participation of women in sports threatens to undermine the objectives of sports interventions, especially those engaging women. Summary Assessment of Contribution Claim: Weak. The midline assessment found that any potential claim for TIS+’s SPD activities to have influenced higher￾order outcomes related to community cohesion or positive youth development to date are not well substantiated by current evidence. However, there is room for optimism as TIS+’s recent SDP Strategy and Implementation Framework demonstrated not only a commitment to more systematic link SDP activities to achieve higher-level results but also provides a concrete road map for how to do so. It is expected that TIS+’s contribution claim around its SDP results is likely to be strengthened in the coming years. 1 SOMALIA PROGRAM SUPPORT SERVICES TRANSITION INITIATIVES FOR STABILIZATION PLUS MIDLINE ASSESSMENT WAVE II REPORT - ANNEX 1 IDIQ AID-623-I-14-00009 TASK ORDER AID-623-TO-16-00015 May 31, 2019 This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by International Business & Technical Consultants, Inc. (IBTCI), under IDIQ AID-623-I-14-00009, Task Order AID-623-TO-16-00015. 370 SOMALIA PROGRAM SUPPORT SERVICES (SPSS) TRANSITION INITIATIVES FOR STABILIZATION PLUS MIDLINE ASSESSMENT WAVE II REPORT – ANNEX 1 DELIVERABLE #20 IDIQ AID-623-I-14-00009, TASK ORDER AID-623-TO-16-00015 Submitted: May 31, 2019 Authors: Sarah Wood, Lucas Malla, Patrick Okwarah, Suleiman Omar, and Mathias Kjaer International Business & Technical Consultants, Inc. (IBTCI) In US: 8618 Westwood Center Drive, Suite 400 Vienna, VA 22182 USA +1.703.749.0100 In Kenya: 8th Floor Block C Westcom Point Building Mahiga Mairu Road, off Waiyaki Way Nairobi, Kenya +254 (0) 737 544 912 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 371 Table of Contents ANNEX A – Evaluation Statement of Objectives ......................................................... 372 ANNEX B – Places Visited; List of Organizations and People Interviewed ............... 391 ANNEX C. Evaluation Design and Data Collection Methodology .............................. 392 Good Governance Index............................................................................................................392 Community Cohesion Index .....................................................................................................392 Attachment C1 .......................................................................................................................................................... 395 Attachment C2 .......................................................................................................................................................... 396 Attachment C3 .......................................................................................................................................................... 397 Attachment C4 .......................................................................................................................................................... 398 ANNEX D. Recalculated Wave II Significance Levels .................................................. 399 ANNEX E – Contribution Stories ................................................................................... 401 Contribution Story 1: Community Planning and Oversight.......................................................401 Contribution Story 2: Technical Coordinators and TIS+...........................................................401 Contribution Story 3: Sports for Development and Peace (SDP) .............................................401 372 ANNEX A – Evaluation Statement of Objectives STATEMENT OF WORK C.1 PURPOSE OF THE TIS+ EVALUATION The primary purpose of the TIS+ evaluation is to generate a better understanding of the extent to which TIS+ interventions have achieved their objectives and contributed to TIS+’s (and USAID’s) goal of increased stability in Somalia, both within and across activity clusters, through participatory processes that promote good governance and community cohesion. Another purpose of this evaluation is to learn lessons about conducting stabilization activities in Somalia to improve future programming in this area. The evaluation will provide comprehensive answers to six key evaluation questions (see Section C.3) in the process of assessing the extent to which the activity goal and objectives were achieved and identifying lessons learned. C.2 AUDIENCE AND INTENDED USE The audience for the TIS+ evaluation is USAID Somalia, the FGS (including the Ministry of Interior and Federal Affairs, a key TIS+ partner), Emerging Federal Members States (e.g., Jubaland State of Somalia, South West State of Somalia), AECOM, and other USAID offices, donors, or development organizations supporting Somalia. The TIS+ evaluation will help guide USAID in improving the design of its future stabilization interventions in Somalia. TIS+ evaluation reports will be uploaded to the Development Experience Clearinghouse and shared with the above stakeholders. In addition, USAID Somalia will develop an action plan to track any changes made to programming based on this evaluation. C.3 KEY EVALUATION QUESTIONS The TIS+ evaluation focuses at two levels of analysis, each with its own set of evaluation questions: within cluster analysis (cluster-level analysis) and cross-cluster analysis (comparative cluster analysis). The Statement of Operations (SOO) identified five clusters where TIS+ activities have either commenced or will soon commence, which USAID changed to five in an email on July 14, 2016. Within each of these five clusters, the evaluation will answer the following four cluster-level evaluation questions: 1. Did TIS+ contribute to any of the following anticipated results: a. Community ownership of development projects and processes? b. Effective shared asset management in the communities in which they were implemented? c. Improved delivery of services targeted by TIS+? Improved satisfaction with delivery of services? d. Reduced conflict over resources? e. Greater citizen confidence in government? f. Reduced support for violent extremism? 2. Were TIS+ approaches used in the cluster (e.g., the community consensus process itself) instrumental in contributing to the achievement of TIS+ results described in Question 1? 3. Did government at all appropriate levels contribute sufficiently to asset management in communities in which TIS+ was implemented? How? Did government at all appropriate levels have both the capacity and the will to contribute? 4. Were TIS+ activities inclusive of women, youth, and minority clans, as applicable to the context of the cluster being evaluated? Besides the above cluster-level evaluation questions, the evaluation will answer two additional cross-cluster evaluation questions as part of the comparative analysis scheduled for the first half of 2019: 5. In the aggregate, did TIS+ contribute to the anticipated results described above? What range of results can be observed across assessed clusters? 373 a. Which cluster-level factors contributed most significantly to achievement of results? For example, did successful inclusion of women, youth and minority clans amplify the effectiveness of TIS+ activities? b. Were results greatest in newly liberated clusters or in bridge clusters? 6. Were some kinds of interventions more effective than others in improving citizen confidence or reducing support for violent extremism? Note that the two cross-cluster evaluation questions are, in essence, restatements of the cluster-level evaluation questions applied at the aggregate cluster level. However, for the purpose of making a clear distinction between the cluster-level and cross-cluster components of the TIS+ evaluation, we have elected the simplifying expedient of referring to them as distinct evaluation questions. Annex A contains a Data Collection Matrix that links each of the above six evaluation questions to a set of more detailed sub-questions, the type of evidence that will be used to answer the questions, and the data sources, data collection tool, data collection method, sampling approach, and data analysis method that will be used to answer each question. The Data Collection Matrix thus provides a concise summary of the evaluation methodology presented here. The SOO further holds out the possibility of conducting a final TIS+ evaluation. Although the SOW for the TIS+ baseline/midline evaluation does not include the final evaluation, the evaluation methods described in this technical proposal have also been designed to provide sufficient comparative potential for the final evaluation to answer the following two final evaluation questions: • Were results to which TIS+ activities contributed sustainable over time? In clusters implemented early in TIS+ (2016–2017), were the results that were observed at the conclusion of activities in those clusters still observed at TIS+ activity end? • Did the cluster-based approach itself contribute to the effectiveness of TIS+ interventions? Gender considerations will be taken into account in answering each of the above key evaluation questions. C.4 METHODOLOGY The TIS+ evaluation will use a mixed-methods approach that integrates a suite of quantitative and qualitative data collection methods implemented over four implementation phases to answer the above key evaluation questions. As described in Section C.4.3, data collection methods will consist of a household survey (HH), secondary research, key informant interviews (KIIs), focus group discussions (FGDs), and community checklists. Evaluation questions will be answered using a combination of these five data collection methods via the triangulation of evidence and the use of contribution analysis to estimate TIS+’s unique contribution to observed results. Contribution analysis is a pragmatic and methodologically rigorous approach to situations in which the activity is one of many potential contributory causes and where it is not possible to measure attribution directly via creation of a scientifically-valid counterfactual (as is the case here). Instead, contribution analysis assesses attribution in terms of the contribution that TIS+ activities are making to observed outcomes based on the totality of evidence. The essential value of contribution analysis in this case is that it offers an approach designed to reduce uncertainty about the contribution TIS+ is making to the observed results through an increased understanding of why the observed results have occurred (or not) and the roles played by TIS+ activities and other internal and external factors.1 (For more on contribution analysis, see Section C.4.5.) C.4.1 DESCRIPTION OF EVALUATION APPROACH 1 John Mayne. (2008). “Contribution Analysis: An Approach to Exploring Cause and Effect,” http://dmeforpeace.org/sites/default/files/0501_Contribution_Analysis_ILAC.pdf. 374 As noted above in Section C.1, the purpose of the TIS+ evaluation is to generate a better understanding of the extent to which TIS+ activities have achieved their objectives and contributed to TIS+’s goal of increased stability in Somalia, both within and across the five TIS+ clusters. The unit of analysis for the evaluation is the specific communities within these five clusters where TIS+ activities will be implemented. Table 1 shows IBTCI’s current understanding of the clusters and communities in which TIS+ activities will be implemented. As seen in Table 1, the evaluation will cover 38 communities in which implementation will be undertaken in the early stages of the process.2 Within the different clusters and communities, TIS+ is expected to pursue a variety of objectives and implement an even wider variety of activities falling under those objectives. Thus, it is imperative that the evaluation achieves a sufficiently representative coverage of clusters and communities so as to capture this expected variety. One option to do this is to conduct data collection activities in a representative sample of communities, along with objectives and activities. Sampling is appropriate in such cases when the number of communities is too large to allow full coverage of communities within existing resource constraints. Another option is to conduct data collection activities in a census of communities, which will provide a fully comprehensive coverage of communities, objectives, and activities. A census approach is appropriate when the number of communities is not so large that comprehensive coverage is possible within existing resource constraints. The latter appears to be the case here; 38 communities is not so large that it falls outside the scope of available resources. Thus, we propose to conduct both quantitative and qualitative data collection activities in all 38 TIS+ communities during both baseline and endline evaluation rounds because this will allow us to develop a fully comprehensive picture of TIS+ results, which, we believe, will ultimately prove more useful in informing USAID programming decisions. Table 1. Number of TIS+ Communities/Districts per Cluster Cluster Region District Settlement Estimated No. of Households Estimated Population 1 Lower Shabelle Wanleweyne Wanleweyne 5,308 36,625 Lower Shabelle Afgooye Afgooye 12,037 83,055 Lower Shabelle Qoryooley Qoryooley 10,692 73,775 Lower Shabelle Barawe Barawe 3,007 20,748 Lower Shabelle Marka Marka 7,960 54,924 Lower Shabelle Kurtunwaarey Kurtunwarey * 1,890 13,041 Lower Shabelle Afgooye Awdeghle * N/A N/A 2 Lower Juba Afmadow Afmadow 6,880 39,216 Lower Juba Kismayo Kismayo 21,824 124,397 Lower Juba Jamame Jamame* 1,832 10,442 Middle Juba Jilib Jilib* N/A N/A 3 Gedo Baardheere Baardhere 4,610 29,965 Gedo Garbahaarey Garbaharey 3,852 25,038 Bay Dinsoor Dinsoor N/A N/A Gedo Ceel Waaq El Wak* 2,102 13,663 2 TIS+ activities will be implemented in other clusters beyond the five specified while this evaluation is still ongoing. The evaluation will not cover all activities during the timeline, and it is not anticipated that additional clusters will be added to the evaluation TO. 375 Gedo Baardheere Geriley* N/A N/A 4 Banadir Abdiaziz Abdiaziz 1,775 12,070 Banadir Bondhere Bondhere 4,447 30,240 Banadir Daynile Daynile 10,033 68,224 Banadir Dharkenley Dharkenley 13,744 93,459 Banadir Hamar Jajab Hamar Jajab 8,682 59,038 Banadir Hamar Weyne Hamar Weyne 2,331 15,851 Banadir Hodan Hodan 13,027 88,584 Banadir Howl Wadag Howl Wadag 7,327 49,824 Banadir Huriwa Huriwa 5,239 35,625 Banadir Kaxda Kaxda N/A N/A Banadir Karan Karan 12,073 82,096 Banadir Shangani Shangani 1,134 7,711 Banadir Shibis Shibis 4,034 27,431 Banadir Waberi Waberi 8,568 58,262 Banadir Wadajir Wadajir 18,929 128,717 Banadir Warta Nabada Warta Nabada 11,166 75,929 Banadir Yaqshid Yaqshid 14,083 95,764 5 Bakool Xudur Xudur* 3,075 20,295 Bakool Tayeeglow Tayeglow* 7,271 47,989 Bakool Ceel Barde El Barde* 1,780 11,748 Bakool Rab Dhuure Rabdurre* N/A N/A Bakool Waajid Wajiid* 9,958 65,723 Total 5 36 38 240,670 1,599,469 * To be determined Region Multipliers (Average Household Sizes) Lower Shabelle 6.9 Lower Juba 5.7 Gedo 6.5 Banadir 6.8 Bakool 6.6 Bay 4.8 Middle Juba 4.0 Note: Numbers of Districts (36) and Settlements (38) are preliminary. Catchment area of Bakool has to be determined. Note that in Somalia the last population census was conducted in 1987 due to the collapse of the Federal Government of Somalia in 1989 and the subsequent conflicts that followed. The population data shown in 376 Table 1 represents the United Nations Population Fund’s (UNFPA)3 best estimates. The population figures for communities shown in Table 1 will be used to guide sample size distribution. C.4.2 EVALUATION PHASES This section of the SOW describes each of the four evaluation phases to be implemented under the proposed evaluation design. C.4.2.1 Phase 1 Evaluation Phase 1 will involve a thorough desk review of activity data and documents generated by TIS+, in addition to in-depth interviews with USAID officials and key members of the TIS+ and implementing partner (IP) management teams. A critical objective of Phase 1 activities will be to reach an agreement with TIS+ on the final list of communities in the five clusters selected by USAID for implementation of baseline and endline assessments. With the knowledge gained from the desk review and interviews, we will create an in-depth and fully specified activity ToC (necessary for the contribution analysis, see Section C.4.5) and finalize all details related to the evaluation design including the creation of a set of cluster-level toolkits, which include the quantitative and qualitative data collection instruments, a data analysis plan, and a final work plan, which includes a projected timeline, logistical arrangements, sampling plan, and data collection methods. C.4.2.2 Phase 2 During evaluation Phase 2, we will implement the baseline evaluation within each of the five clusters and 38 communities following the design and using the data collection methods and instruments developed during Phase 1. As specified in the evaluation SOO, the Phase 2 baseline data collection will involve a single data collection wave to be launched soon after the final work plan is approved. At the conclusion of the baseline data collection, we will analyze the baseline data, with a focus on describing and explaining the baseline conditions in the cluster using the methods described in Section C.4.3, and prepare the baseline evaluation report. The analytic focus in the baseline evaluation will be on describing the initial conditions in each of the clusters with regards to the key outcome variables related to EQs 1–4. Within clusters, all data will be analyzed in the aggregate and disaggregated by gender, age, and membership in minority clans. Once all baseline data collection has been completed, we will prepare a baseline evaluation report incorporating the findings from all five clusters. C.4.2.3 Phase 3 Phase 3 of the evaluation involves the endline data collection in the same five TIS+ clusters and 38 communities using the same data collection methods as in the Phase 2 baseline evaluation. Unlike the baseline evaluation, the endline evaluation will occur on a rolling basis as TIS+ phases out its activities within each of the clusters over time. The SOO assumes that endline data collection will occur over three separate data collection waves “based on the assumption that at least two clusters will complete activities in close enough proximity that they can be evaluated by the same team.”4 Because the number of cluster has changed from seven to five, for the purpose of cost containment, SPSS anticipates two waves: one wave will cover three clusters, the other will cover two clusters. One possible sequencing of the endline data collection waves is suggested on pages 16–17 of the SOO and is as follows: (1) Mogadishu (Banadir) in mid-2017; (2) Lower Juba 2017–2018; and (3) Lower Shabelle, Gedo, and Bakool in 2018–2019. This is, however, only one of several possible sequencing possibilities, depending on the timing of activities within each cluster and community. In any case, it is not possible at this point to propose with any precision when activities in the different clusters and communities will conclude and thus the number, timing, and composition of the endline data collection waves. For convenience sake, our proposed evaluation design accepts utilizing the same data collection team. This assumption will be thoroughly 3 http://countryoffice.unfpa.org/somalia/drive/Population-Estimation-Survey-of-Somalia-PESS-2013-2014.pdf 4 There could be a months-long lag in the endline evaluation of some communities as it’s very likely that TIS+ work in communities will end at different dates based on circumstances on the ground. In other words, the endline of some communities will have to wait for TIS+ to finish in other communities. 377 vetted during the initial discussions with TIS+ and its IPs during Phase 1 and the evaluation design and data collection work plan adjusted accordingly depending on the results of these discussions. At the conclusion of each endline data collection wave, we will analyze the data and prepare an endline evaluation report for each cluster covered by that wave, resulting in five separate cluster-level endline evaluation reports. The analytic focus in the endline evaluation will be on (1) measuring changes in key outcome variables, (2) assessing TIS+’s contribution, and that of other factors, to the changes in the key outcome variables, and (3) based on the findings/conclusions to 1 and 2, answering EQs 1–4. C.4.2.4 Phase 4 In Phase 4, we move from the within-cluster evaluation to the cross-cluster evaluation to produce a synthetic comparative analysis of results aggregated across the five clusters. Phase 4 of the evaluation will occur in close succession after the completion of Phase 3 and will begin with creating an updated data analysis plan, after which we will aggregate all of the data collected during Phases 2 and 3, analyze it, and prepare a final, comparative evaluation report, along with a set of actionable recommendations to improve the design of future stabilization interventions in Somalia. The analytic focus during Phase 4 will be similar to that in Phase 3, except that the focus will now be on measuring changes in key outcome variables and assessing TIS+’s contribution to them across all five clusters to answer evaluation questions 5–6. Cross-cluster data will be analyzed in the aggregate and disaggregated by gender, age, membership in minority clans, and cluster. C.4.3 Data Collection Methods Under our proposed mixed-methods design, we will implement a combination of quantitative and qualitative data collection methods to answer the above key evaluation questions. The sole quantitative method is the household survey, which will be administered to residents living in communities where TIS+ activities are implemented. The qualitative data collection instruments include secondary research, in-depth KIIs with diverse TIS+ stakeholders, FGDs with persons participating in and benefiting from the TIS+ activities, and community checklists. This combination of quantitative and qualitative data collection methods will allow us to construct a robust evidence base that will be analyzed to provide comprehensive answers to EQs 1–4 within clusters and EQs 5–6 across clusters. A description of each data collection method is provided below. C.4.3.1 Household Survey We will administer a repeated cross-sectional household survey in the baseline and endline to a randomly selected group of 424 residents living in communities participating in TIS+ activities within each of the five clusters covered by the TIS+ evaluation for a total sample size of 2,200.5 Under the cluster-level evaluation approach specified in the evaluation SOO, each cluster is to be treated as a separate analytical population for sampling purposes. This means that for each cluster examined, the baseline and endline survey samples must be large enough to draw statistically meaningful conclusions about the results of TIS+ activities within that cluster. This will further allow full disaggregation by cluster when the survey data from each cluster are consolidated for the comparative analysis in Phase 4. To calculate the required sample size in each cluster, we are using the sample size calculation formula for determining the appropriate sample size for detecting a difference between two proportions.6 The first step in calculating the sample size using this approach is to determine the key outcome variables and the minimum detectable effect (MDE) to be measured.7The SOO identifies a number of outcome variables of interest (e.g., improved satisfaction with delivery of services, greater citizen confidence in government, reduced support for violent extremism), which can be used for this purpose. The question then becomes: What amount of 5 A repeated cross-sectional survey interviews a different random sample of respondents in each evaluation round. In contrast, a longitudinal or panel survey interviews the same cohort of randomly selected respondents in each evaluation round. The relative strengths and weaknesses of these two approaches are discussed in Section C.4.8. 6 See, for example, https://select-statistics.co.uk/calculators/sample-size-calculator-two-proportions/. 7 MDE is the smallest true effect or change for a given outcome variable, which is detectable for a given level of power and statistical significance. 378 change (effect size) does USAID expect to observe over time in the relevant outcome variable as a result of TIS+ activities? For example, if in the baseline 50% of survey respondents are satisfied with the delivery of local government services, by how much does USAID expect this to increase by the endline as a result of TIS+ activities? TIS evaluation findings have been used to estimate expected effect size for any of the key outcome variables. The assumptions include the following: Effect Size: 10 percentage points (Note that the assumed effect size is the smallest effect size that we consider to be relevant for informing USAID programming decisions that is feasible within reasonable budget parameters.) Confidence Level: 95%8 Statistical Power: 80%9 The above assumptions produce a required sample size of 385 households per cluster, which multiplied by the five clusters covered by the TIS+ evaluation yields a total baseline and endline sample size of 2,200.10 To this total we next add a 10% oversampling factor, which is done to ensure a sufficient number of responses for each survey question given an expected 10% non-response rate that is typical for household surveys in Somalia. Adding this 10% oversampling factor pushes the sample size to 424 per cluster and 2,120 across the five clusters. The survey instrument will be modeled in part on a past stabilization survey fielded in Somalia, which was used to generate the TIS Stabilization Report in 2013–2014, and the TIS performance evaluation done by SPSS in 2016. In addition to these two surveys, the survey instrument will draw on other surveys done to evaluate USAID local governance projects in other countries, many of which use methodologies similar to TIS+ to develop the capacity of local government institutions to deliver local services and to strengthen civil society structures within those countries. The survey instrument for TIS+ will be tailored to ensure that it accurately reflects TIS+ activities and their expected outcomes, while also answering the evaluation questions, within the constraints of the method. The following points summarize the key characteristics of the household survey: • Sample Universe: The adult population in communities where TIS+ activities are implemented • Sampling Frame: The 38 communities across the five clusters in which TIS+ activities are being implemented • Target Population: Adult members of TIS+ communities 16 years and older • Statistical Power: 80% • Confidence Level: 95% • Sample Size: N = 424 per cluster, 2,120 total • Effect Size: 10 percentage point increase (50%-60%) • Stratification: 50/50 male/female, female and male interviewers will conduct interviews with gender￾matched respondents • Sample Distribution: Within each cluster, the assignment of the 424 sample households among the selected TIS+ communities will be allocated using the probability proportionate to size (PPS) 8 Statistical power is the probability of detecting a significant change when one exists. With a power of 80%, this means that there is a 20% probability of failing to detect a significant change when one exists such as a false negative result (otherwise known as Type II error). For example, with an expected effect size of 10 percentage points, a statistical power of 80% means that we have an 80% probability of finding a 10 percentage point change, if one exists. 9 The confidence level is confidence with which we would like to detect a significant difference between the two proportions. If the confidence level is 95%, then this means there is a 5% probability of incorrectly detecting a significant difference when one does not exist, such as a false positive result (otherwise known as Type I error). 10 Including a pilot test of questionnaire with total sample size of 80 respondents. 379 approach. 11 During the cluster-level analysis, the sample will be weighted to be representative of the entire cluster population using weights developed with United Nations Population Fund (UNFPA) community-specific population data. During the comparative cross-cluster analysis, the consolidated data set will be re-weighted, again using the derived population data, to be representative of the entire study population. • Unit of Observation: One adult 16 years and older selected randomly from the members of each household who is systematically selected at each sampling point. • Household Selection Method: Random walk systematic selection. The sampling plan will specify a randomly selected starting location, walking direction, and the first household to be contacted for an interview in each community. After the first household is selected, the male interviewer selects every fifth household on the right-hand side and the female interviewer selects every third household on the left-hand side along the walk-in rural areas, and every tenth household in urban areas. • Respondent Selection Method: Random selection using the two-step Kish Grid procedure. First all eligible members of the household (over 16 years old and either male or female depending on the sex of the interviewer) are listed. Then the random numbers in the grid cells are used to randomly select the respondent from the list. C.4.3.2 Secondary Research The first data collection method undertaken by the team will be to conduct a comprehensive review of internal and external documents of relevance to the TIS+ activity. We will work in coordination with the TIS+ management team to identify, collect, and review documents over the course of the evaluation that include the following: • Statements of work • Activity designs, training curricula, event agendas, and other key documents for understanding the content of activities • Annual Work Plans (initial and subsequent) • Activity Monitoring and Evaluation Plan (initial and revised) • Quarterly and annual reports • Results from TIS+ participatory M&E activities • SPSS monitoring and verification reports • Additional documents provided with the evaluation SOO • External publications by donors, academic researchers, NGOs, etc. The results from TIS+’s participatory M&E activities will further comprise a primary source of performance data for the cluster-level evaluations. TIS+ M&E data will be integrated with the data generated from the other data collection methods and will be used as a source for triangulation to answer the key evaluation questions. The evaluation team will incorporate information from the secondary research in the enumerator training to ensure that enumerators understand the background and context of TIS+ and are attentive to the need for data to answer the primary evaluation questions. Of particular importance is the geography of TIS+, its M&E framework, progress over time in implementing activities, key challenges reported by the project for achieving its objectives, and other key information for understanding the activity and how best to evaluate its performance using the other methods described below. C.4.3.3 Key Informant Interviews (N = 85, plus 20 [four in each cluster] for validation including the contribution analysis = 105) 11 The universe will be the population size of the TIS+ communities. Table 1 presents population estimates by community as best estimated by UNFPA PESS 2014. IBTCI will estimate boundaries of these communities and also follow a similar approach used by UNFPA to estimate population data for four communities whose population figures were not presented in UNFPA reports. 380 KIIs are in-depth, semi-structured interviews conducted with individual activity stakeholders. The evaluation team will use the knowledge gained from the desk review and in-depth interviews conducted during evaluation Phase 1 to inform the design of the KII discussion guides and selection of key informant interviewees. Interviews will be conducted in both the baseline and endline with key informants from the following stakeholder groups, with other stakeholder groups and key informants added in during both evaluation rounds if certain key information can be gained from others not listed below: • USAID officials in the USAID Somalia Office, N = 4 • AECOM senior management, regional offices staff, and subcontractor staff, N = 10 • Officials from Interim Regional Government Authorities and relevant government actors at the central and district levels in Somalia, N = 51 interviews (5 Ministry of Interior representatives, 10 FGS officials, and 1 district senior representative from each of the 36 sampled communities) • Key staff from other stabilization programs that interact with TIS+ or support programming that shares its objectives and geography (for example the Office of Transition Initiatives’ Somalia Stabilization Initiative, World Bank, Did), N = 10 • Representatives from private sector, nongovernmental, or governmental donor organizations collaborating with or asked to collaborate with TIS+ communities, N=10 Participants will be selected purposively for the additional four KIIs for each cluster, which will take place after initial data has been collected and analyzed and preliminary findings produced and presented. The purpose of this second round of KIIs is to validate the preliminary findings and gather additional data required to complete the contribution analysis for the cluster. The key informants from each of the groups listed above and their locations and their numbers will be finalized in coordination with USAID and TIS+ during evaluation Phase 1 and again during Evaluation Phase 3 at the beginning of the endline data collection. It should be noted that the list of stakeholder groups and number of KIIs per group found above are estimates based on information currently available and are subject to revision based on discussions with USAID and TIS+. The KII interview guides will follow an open-ended format that will give the interviewees the space to describe their experiences in their own words and allow the interviewer to probe for information that will uncover new insights into key aspects of TIS+’s performance. The interview guides will be modular so that certain question modules will be asked of all respondents while other question modules will be tailored to specific groups of respondents. C.4.3.4 Focus Group Discussions (FGDs) (N = 68 + 10 [two in each cluster] for validation including the contribution analysis = 78) FGDs are in-depth, moderated discussions with small groups of TIS+ community representatives. FGDs are a key method for measuring program performance as perceived by beneficiaries because in-depth information can be collected on the effects of participation in activity interventions. A major advantage of the FGD method, as opposed to other forms of qualitative research, is its flexibility for in-depth exploration of the knowledge and attitudes of the group toward TIS+, processes used during TIS+ activity implementation (e.g., representativeness, transparency, tendering, procurement, and participation by women, youths, minority clans, and members of other marginal groups) and the outcomes of the activities in which they participated. Four FGDs will be conducted in each selected community with the following groups: • Female beneficiaries • Male beneficiaries • Male youth beneficiaries • Female youth beneficiaries 381 We propose to conduct four FGDs in up to 50% of the TIS+ communities within a particular cluster. This produces the total number of FGDs in each cluster shown in Table 2. The communities participating in the FGDs will be selected purposively guided by criteria to be established by the evaluation team in conjunction with TIS+ management and field staff. FGD participants within communities will be selected based on the criteria that they are permanent community residents and have participated in one or more of the activities implemented by TIS+ in their community. FGD participants will be drawn, for example, from participants in TIS+ consensus groups, monitoring committees, and local tendering committees, as well as other well￾informed community members. To the extent feasible, the FGDs will consist of homogenous groups. To get a fully representative picture of activity and implementation, the composition of FGD participants will be determined on a rolling basis in each successive community depending on the nature of activities implemented within those communities, previous issues identified within those communities (e.g., via TIS+ monitoring activities), and the coverage of TIS+ participants in prior FGDs. Participants will be selected purposively for the additional two FGDs for each cluster, which will take place after initial data has been collected and analyzed and preliminary findings produced and presented. The purpose of this second round of FGDs is to validate the preliminary findings and gather additional data required to complete the contribution analysis for the cluster. Table 2. Number of FGDs per Cluster Cluster Number of TIS+ Communities Number of Communities for FGDs Total Number of FGDs 1 7 3 12 2 4 2 8 3 5 2 8 4 17 8 32 5 5 2 8 Total 38 17 68 The FGD guide will give the discussion moderator a set of key questions and topics for guiding the discussions to uncover key information for answering the evaluation questions and determining whether TIS+ met its objectives, how TIS+ activities unfolded within the community, and the contribution of TIS+ activities to observed changes within the community. To keep respondents focused throughout the session, FGDs will be organized at the venues where participants will be comfortable to be recorded using digital audio recorders. The FGDs will include from 6 to 10 participants for a two-hour discussion. During each session, participants will be provided with refreshments. As with the KII interview guides, the FGD discussion guides will follow an open-ended format that will give the participants the space to describe their experiences in their own words and allow the interviewer to probe for information that will uncover new insights into key aspects of TIS+’s performance. The discussion guides will be modular so that certain question modules will be asked of all respondents, while other question modules will be tailored to specific groups of respondents, clusters, and objectives. C.4.3.4 Community Checklists (N = 380) Qualitative researchers will make direct observations about sample communities using a detailed checklist, which will be used to evaluate the extent to which physical structures were rehabilitated. C.4.4 Data Analysis Methods The following presents a summary of our proposed quantitative and qualitative data analysis methods. We will disaggregate all data analysis by gender, age, membership in minority clans, or other population characteristics, as appropriate. 382 C.4.4.1 Quantitative Data Analysis Methods The primary quantitative results of interest are the changes in key outcome variables related primarily to EQ1, which will be measured as the change in two proportions; for example, a change in the proportion of survey respondents who express confidence in government in the baseline compared with the proportion who express confidence in government in the endline. To analyze changes in key outcome variables and to understand the relationships between other variables in the survey data set, we will employ a range of statistical tests depending on the nature of the data. These tests include the following: • Tests of independent means for numerical and binary data. • The independent samples t-test will be used with numerical and binary data to determine whether the difference between two means is statistically significant. • Analysis of variance will be used with numerical and binary data to determine whether the difference between three or more means is statistically significant. • Ordinary least squares regression analysis, in which continuous outcome variables are dependent variables and survey data (for example, demographic variables for both within and across cluster analysis and location data for across-cluster analysis) are independent variables that will be used to analyze the factors that are associated observed outcomes; disaggregate the analysis by sex, age, clan, location, and other sample sub-population and control for possible intervening factors. • Logistic regression analysis (Logit), in which dichotomous outcome variables are dependent variables, and survey data are independent variables, will be used to analyze the factors that are associated observed outcomes; disaggregate the analysis by sex, age, clan, location, and other sample sub￾populations and control for possible intervening factors. • Non-parametric tests for ordinal data: A number of the household survey questions will be Likert scales, which produce ordinal data. • The Mann-Whitney U test is a non-parametric test that will be used to compare differences between two independent groups when the dependent variable is ordinal. • The Somers’ D test is a non-parametric measure of the strength and direction of association that exists between an ordinal dependent variable and an ordinal independent variable. • Ordinal logistic regression, in which ordinal outcome variables are dependent variables and survey data are independent variables, will be used to analyze the factors that are associated observed outcomes; disaggregate the analysis by sex, age, clan, location, and other sample sub-populations and control for possible intervening factors. • Tests of categorical data: The Pearson’s chi-square test will be used to test whether there is a difference in observed and expected frequencies and, therefore, whether two samples being tested are from the same population. For the purposes of this evaluation, p-values of greater than or equal to 0.10 will considered to indicate a statistically significant difference. C.4.4.2 Qualitative Data Analysis Methods Data from the KIIs and FGDs will be analyzed using an inductive thematic approach, which reduces text to codes and identifies patterns among the codes. Analysis will be performed using the NVivo qualitative analysis software program. Using a standardized iterative process, a codebook will be developed, and codes will be systematically applied to the data. A matrix will be designed to identify themes to better understand (1) the ways in which TIS+ activities have affected outcomes related to the evaluation questions; and (2) other factors, both internal and external to TIS+, that may have affected these outcomes. At the same time, codes will be developed to facilitate the integration of findings across the qualitative and quantitative data. We will design the data analysis strategy for both quantitative and qualitative data before the baseline data collection begins to encourage team members to begin thinking about trends in findings from different data sets to determine whether findings are similar or divergent. Also, the team will consider which data relates to the contribution analysis at the settlement and cluster level to be part of answering EQs 1–4, and in turn, 383 how these data will be analyzed across sectors to answer EQs 5 and 6. Data analysis will employ a parallel, mixed-data approach in which quantitative data are independently analyzed from qualitative data, but both use coding related to the evaluation questions. This approach takes the findings and analysis from the qualitative data and uses it to inform and explain findings from the quantitative data, and vice versa. We will then seek to expand on the findings that emerge from the quantitative data by identifying qualitative trends to provide explanations and depth for those results. At the same time, we will take the themes and trends emerging from the qualitative data and look for existing quantitative results or undertake additional quantitative analysis to determine whether there is evidence to generalize the qualitative findings to the target populations. Finally, we will utilize different types of triangulation to validate findings, analyses, and conclusions including data triangulation using a variety of data sources12 and investigator triangulation involving the use of different evaluators who bring diverse perspectives and cultural and analytical skills. C.4.5 Contribution Analysis TIS+ operates within a wider system in which it is only one of a number of factors that potentially affect activity outcomes. TIS+’s interventions will, therefore, be at best a contributory cause to any observed outcomes. In other words, TIS+ activities are a necessary part of a “package” of causal factors that are together sufficient to produce the observed outcomes. It is important, therefore, to place project activities within the wider range of other potential causal factors and to understand that these other factors have a potentially important influence on the occurrence, scale, and nature of change. According to Jonathan Mayne, contribution analysis infers causality from the following evidence: • The project is based on a well-reasoned ToC, while the assumptions underlying the ToC are plausible, sound, informed by evidence, and agreed-on by key stakeholders. • The project implements activities as outlined in the ToC. • The evidence verifies both the occurrence of expected outcomes in the ToC and its underlying assumptions. • External factors, including context and rival explanations, are assessed and are shown not to have made a significant contribution to observed results, or, if they did, their relative contribution is recognized. One purpose of contribution analysis is to work through a set of contribution stories that assist in answering the relevant evaluation questions. In working through the contribution stories, the analysis must be guided by the following three considerations related to the nature of causality/contribution. First, project interventions do not operate in isolation but are part of a causal package that includes other interventions, events, or conditions external to the project. Thus, our interest in doing contribution analysis focuses less on whether and how TIS+ has contributed to a particular outcome but instead on whether and how the causal package, which includes TIS+ interventions, has contributed to the outcome. Second, any particular causal package comprises the intervention outputs plus the assumptions, which together are sufficient to bring about the intended impacts. The ToC will identify the support factors (assumptions) and confounding factors (risks). It also explains how and why intended impacts are expected to occur. The causal questions will address the following: (1) Has the intervention made a difference; in what way was it a contributory cause? (2) Why has the impact occurred? How did the causal factors bring about the result? What was the context and the mechanisms? What role did the intervention play (i.e., as a trigger) compared to contributions played by other influencing factors? 12 Analysis of background documents, FGDs, KIIs, and household survey. 384 Therefore, this approach should shed light on the questions about what the relative contribution of TIS+ approaches was to anticipated results, which is the first EQ, which leads to the second EQ regarding the extent to which TIS+ approaches were instrumental in achieving results. Third, assessing contribution must take into account the probabilistic nature of causality and the lack of certainty that often attends our attempts to measure it. This requires that we talk more in terms of likelihoods instead of certainties with regard to our conclusions about contribution. Thus, the standard of evidence to determine contribution that we will apply through the contribution analysis is that the causal package is likely a sufficient cause of the observed change, while TIS+ activities are in turn a “likely necessary” part of the causal package. Thus, in lieu of creating a statistically valid counterfactual, contribution analysis reduces uncertainty about the contribution TIS+ is making to observed changes through an increased understanding of why the observed changes have occurred (or not occurred) and the role TIS+ played in contributing to these changes relative to other internal and external factors. As specified by Mayne, and as to be implemented in the TIS+ evaluation, contribution analysis involves the following six-step approach to arrive at evidence-based conclusions about the factors likely contributing to observed changes. CONTRIBUTION ANALYSIS PROCESS TIS+ EVALUATION PHASE PRODUCTS Step 1: Set out the cause–effect issue to be addressed in a detailed ToC. This step maps out the cause-and-effect logic leading from project activities to expected outcomes and carefully traces out the expected outcomes, possible unexpected outcomes, and causal linkages, along with key assumptions and causal mechanisms underpinning the ToC. PH 1: PREPARATION Attribution problem statement: What needs to be addressed Articulate the cause-effect question Explore contribution expected Identify other influencing factors Identify plausibility of the contribution Articulate causal questions related to the EQs Step 2: Identify the risks to the postulated ToC, including rival explanations. This step assesses the strengths and weaknesses of the cause-and-effect logic underlying the ToC and assesses the nature and extent of TIS+’s contribution to each expected result, along with factors that imperil the hypothesized causal linkages and other possible contributory factors and rival explanations. PH 2: BASELINE PHASE Draft of the TOC with assumptions and risks and roles played by other factors. How are they contested in the TOC? Review results. Questions for inclusion in tools and analysis plan Step 3: Gather evidence. This step involves the implementation of the data collection methods described above to gather evidence for the contribution analysis, particularly (1) whether the expected results did or did not occur, (2) why expected results did or did not occur, and (3) the causal process and contextual and other contributory factors explaining why expected results did or did not occur. PH 3: ENDLINE Comparative analysis section for each cluster report following analysis plan Preliminary Findings and guides for follow up KIIs and FGDs Step 4: Assemble and assess the contribution claim and challenges to it. This step constructs the contribution story while assessing the strengths and weaknesses of the cause-and-effect logic in light of the evidence collected during the previous step, the relevance of other contributory factors, and the evidence for rival explanations. This step involves three analytical tasks: 385 (1) Bottom-up measurement: This task assesses how TIS+ activities have facilitated changes at the outcome level by analyzing the effects of interventions and their interaction with the context, including other actors, the local political economy, and other external factors. (2) Top-down measurement: This task assesses the key changes in outcomes and then investigates how activities and other factors have driven these changes. (3) Compare and triangulate findings: This task investigates the causal relationships between activities and observed outcomes by assessing the extent to which observed outcomes and contributory causal factors are consistent across data collection methods and contexts. Step 5: Seek out additional evidence. This step determines what kind of additional evidence is needed to enhance the credibility of the contribution claim and gather the appropriate evidence, as feasible. PH 3: ENDLINE Final sections with data second round of FGDs and KIIs Step 6: Revise and strengthen the contribution story. Having identified and addressed any challenges to our credibility, such as the strengths of certain assumptions and/or the roles of other influencing factors, this step builds a more credible contribution story. Because this process involves both bottom-up and top-down measurement of change, it provides triangulation of evidence for change up and down the project ToC. It also provides a reasonably robust approach to measure contribution and guard against “self-importance bias,” which occurs when the relative importance of other contextual factors is underestimated, and the importance of project-specific factors is overestimated. PH 4: SYNTHETIC COMPARATIVE ANALYSIS Result of analysis with added data C.4.6 Quality Assurance To reduce sources of bias and obtain valid and accurate responses from participants, we will implement a series of rigorous quality assurance (QA) methods for both quantitative and qualitative data. These methods are described below. Interviewer training. Central to overall QA will be providing training for our local research partner and its interviewers. Each member of the quantitative and qualitative field teams will participate in a two- to three￾day training workshop before entering the field during which time they will receive question-by-question instruction in research instruments, their purpose, and their implementation along with instruction and practice in the research protocols related to, for example, skip patterns, coding and use of handheld devices (quantitative training), and sound interview/moderation, probing, and participatory research techniques (qualitative training). Cognitive consistency. Instrument calibration will also be critical to ensure accurate results. Instruments will be tested on location with key informants and/or representatives of the target population. The purpose of this step is to fine-tune the translation and check the cognitive consistency and precision of the questions asked. This might result in re-phrasing of some questions, adjustment of vocabulary, or response options to better fit the local context. Use of technology. The evaluation will use both Mobile Data Collection (MDC) and traditional paper-and￾pencil data interview (PAPI) methods. MDC will be used with quantitative data collection instruments in communities that are considered very accessible. It will also be used to create digital audio recordings of KIIs and FGDs proceedings. PAPI will be used with quantitative data collection instruments in communities that 386 are considered moderately accessible and where the use of MDC would draw attention (thus risking the lives of interviewers). It will also be used for KIIs and FGDs as a backup record of KII and FGD proceedings. The MDC platform provides real-time data functionality and back-end verification capabilities to support data collection activities, while also enabling geo-tagging and time-stamping of interviews and quick analysis of data on an ongoing basis for quality control. Programming the devices for data collection and data processing will include range, logic, and consistency checks customized for the item types and expected responses. Range checks will ensure that continuous data will be entered within predefined boundaries or that interviewers only select categorical data from a predefined list of responses. Skip logic will be scripted to ensure that respondents receive the appropriate questions based on previous responses or data derivations. Internal consistency checks will be built into the computerized script to allow interviewers to make necessary corrections to data while interviewing the respondent. We are familiar with the challenges posed by both MDC and PAPI tools and as such will develop a set of rigorous data management and quality assurance protocols for each (complementing the QA methods described above) and will train all field researchers in their use. Pilot testing. We will conduct a pilot test of all quantitative instruments with the intended informants and target population and in circumstances similar to that of the study itself. The purpose of the pilot is threefold: (1) It tests the data collection protocol. (2) It further trains the survey team by giving its members the opportunity to get better acquainted with the content and the target population. (3) It permits further adjustments of the language in the instruments for better flow and to ensure internal consistency of the responses. Based on the findings of the pilot, changes will be made to the survey instrument and protocols. Household survey data checks. For the household survey, we will conduct routine quality and completion checks in the field during the household survey implementation to ensure enumerators follow protocols and ensure data accuracy. Conducting regular spot-checks will allow problems to be identified and resolved while the data collection team is on location. Each enumeration team will include one Field Supervisor to check that each record is completed before assigning other sample cases to the enumerator. A Data Collection Coordinator will oversee all enumeration teams and will check the data of each Field Supervisor at the end of each day. Based on this information, they will assign callbacks to Field Supervisors or re-conduct surveys themselves. Callbacks make sure data are collected correctly but also point to problems that are more systemic and may require refinements to the data collection protocols. Qualitative controls. Qualitative quality control methods will include standardizing introductory comments to create a common set of expectations for respondents, holding discussions in venues where participants feel comfortable and secure, excluding activity staff or local authority figures from FGDs, and monitoring a random sample of KIIs and FGDs to ensure that team members are following the moderation protocols. After completion of each FGD session, each evaluation team member will review the transcripts from the transcribers individually. Next, they will meet to discuss and review the transcriptions. If unanticipated information is uncovered, team members will design appropriate probing questions for the ensuing KIIs. Additional information collected through this approach will help the evaluation team in the final analysis during triangulation of data (findings). Finally, prior to processing the data, the qualitative evaluation team will review collected data to ensure all findings are properly recorded and documented. Data cleaning. At the end of each day, survey data will be uploaded to a local server as well as a server hosted by researchers. Back-office staff, both locally and remotely, will run analytic programs that perform data cleaning, check coding, and produce descriptive statistics. These analytic checks will be performed daily or at an adequate frequency so problems may be addressed immediately. 387 Debriefings. During the fieldwork, both quantitative and qualitative data collection teams will hold a debriefing at the conclusion of each field day to share notes, check data quality, ensure that instruments have been completed correctly, and troubleshoot problems. Using the real-time, back-end verification capabilities embedded into the MDC platform, the Field Supervisors and Data Collection Coordinator will verify the data collected and uploaded for that day. Check-ins. We will institute a regular check-in process consisting of weekly teleconferences between the Data Collection Coordinator and the IBTCI Evaluation Manager and weekly status reports from the Field Supervisors. Challenges arising in the field will be resolved on the spot by the Field Supervisors when appropriate. When this is not possible, the Field Supervisor will refer the problem to the Data Collection Coordinator for resolution. As a last resort, and for the most serious issues, the Data Collection Coordinator will refer the problem to the Evaluation Manager for resolution. C.4.5 Data Format and Sharing The evaluation data and information will be converted to the appropriate formats and shared with USAID per ADS 579, “USAID Development Data” or “USAID Open Data” requirements and guidelines. To ensure transparency and replicability, quantitative data will be submitted as annotated data sets clearly defined with codebooks and annotated analysis files. Unstructured qualitative data, such as interview transcripts, will be stored in text-based data entry templates, while structured quantitative data will be stored in Excel and SPSS files. To ensure respondent anonymity, survey data will be entered into two separate databases, one with identifying information and one without identifying information. Metadata will be generated in the form of codebooks and data summaries. The final data set will be compiled and submitted to USAID in accordance with relevant Agency guidelines. The evaluation team will also work with USAID Somalia’s Open Data Point of Contact to ensure that it submits the correct files and data in the appropriate format. C.4.6 Dissemination of Results To ensure that key findings and recommendations are harmonized into a realistic and meaningful body of analytical work, the evaluation team will vet findings with key stakeholders to get input and validate recommendations. At the conclusion of the evaluation, we will host a workshop for key project stakeholders to present our findings and receive feedback. The workshops and presentations will be an important tool in achieving stakeholder buy-in and ensuring that results are acceptable. We propose a second round of FGDs for each cluster to take place after the two waves of data collection. These will happen after analysis, preparation of preliminary findings, including each cluster contribution story and presentation to USAID and stakeholders in Nairobi. Additional questions may come out of those presentations; we will also review the topics that will be covered in the second round of FGDs and KIIs for their inputs. The second round of FGDs and KIIs will then take place, where the preliminary findings will be presented, and final contribution analysis information collected. These data will be used for the final reports for each cluster. A workshop is planned in Mogadishu in the final synthetic comparative analysis phase to present the evaluation report preliminary findings before finalizing the report. This will be similar to the preliminary findings workshop for this phase with stakeholders in Nairobi, but will allow more FGS and civil society representatives to attend. Final planning on these second round FGDs, KIIs, and the final workshop will be refined during the phase one preparation stage. C.4.7 Ethical Considerations All data collection will be conducted according to the professional standards of the American Evaluation Association. Signed informed consent will be obtained prior to the start of any data collection activity, while interviewees will be given the option not to respond to any questions or to decline any interview. To protect the interviewee’s anonymity, we will implement a number of safeguards. To begin with, all evaluation team members will be required to sign a certificate of confidentiality. Any information that could be directly linked 388 to an individual will not be used, including quotes provided in evaluation reports. Further, only general information (e.g., location, sex, age, etc.) will be utilized for analysis or reporting, while only members of the evaluation team will have access to the transcripts and raw data. Finally, data sets provided to USAID or other stakeholders will be cleaned with identifying information removed. C.4.8 METHODOLOGY STRENGTHS AND LIMITATIONS USAID’s Evaluation Policy states that any methodological strengths and limitations are to be communicated explicitly in SOWs. Some of the methodological strengths and limitations of the TIS+ evaluation are described below. C.4.8.1 Methodological Strengths The sampling design provides 95% confidence level and 80% statistical power, both of which adhere to established best practice in evaluation research, thus providing reliable data for decision-making purposes. Contribution analysis provides a credible alternative approach for dealing with the issue of attribution in the absence of a counterfactual. While contribution constitutes a lower standard of causality than attribution, contribution analysis is a second-best alternative that can still provide rigorous evidence of an activity’s specific contribution to observed outcomes. SPSS is mindful of the political polarization in Somalia, and will discuss and coordinate with USAID Somalia Office and TIS+ about selection of respondents and other issues that may arise. While bias can never be entirely eliminated from an assessment, there are a number of checks the evaluation team will engage in to mitigate and minimize bias. The KIIs will be implemented using question guides rather than a detailed interview guide that might “force” respondents to provide answers to questions about aspects of the activity for which they may have inadequate knowledge. Conveying independence of the evaluation team from TIS+ implementation and decision-making and the fact that the evaluation will have no direct influence on future project activities will strengthen the independence of the TIS+ evaluation. Evaluator conflicts of interests are another potential source of bias. SPSS certifies that no individual included in its evaluation team, including contractor staff employed to collect data, will possess a material or perceived conflict of interest that might influence the evaluation process, the quality of the data collected, or the findings of the evaluation. Data collection using mobile devices will allow for quality control and analysis in near real time, enabling fast decision making by the evaluation team for quality control and quick course corrections on fieldwork plans and data collection instruments based on clearly identified need and response to exigencies that may arise during fieldwork. Emotional attachment and professional reputation may enter as a different source of bias in the data that will be collected from IP management and staff. Reputational considerations and strong attachments to the Stabilization Project and the areas where it worked may create a tendency to answer questions in ways that cast a positive light on some of the more challenging aspects of TIS+. Triangulation of data from different sources is the key analytical technique for identifying and controlling this bias. That is, bias will be controlled by cross-referencing information from interviews with USAID officials, with information from IPs, with data collected from beneficiaries in the TIS+ communities, plus data from secondary sources including independent monitoring, verification, and reporting. The triangulation of data from these different sources will ensure that the answers to the evaluation questions reflect a distillation of the full range of dynamics surrounding TIS+ programming and the various interests of its stakeholders. 389 Insecurity is a persistent limitation that affects all types of work in Somalia, including evaluations. Insecurity may affect the mobility of evaluators and access to research subjects and activity sites. Insecurity may create bias in evaluation findings if it results in the systematic exclusion of certain types of activity sites and/or groups of respondents. If during the course of fieldwork insecurity compromises the evaluation team’s ability to fulfill the evaluation plan, then SPSS will immediately identify the best means of revising the evaluation plan such that inaccessible locations and/or respondents are replaced with comparable ones that are accessible. Thereby the evaluation team will control the potential source of sampling bias that might arise from insecurity. C.4.8.2 Methodological Weaknesses The evaluation SOO specifies that data collection will be limited to TIS+ clusters and communities. This requirement forecloses the possibility of creating a counterfactual, which would allow us to attribute findings with regard to EQ1 to TIS+ activities. In lieu of a counterfactual, we have proposed to use contribution analysis (see above). Notwithstanding best efforts to avoid or mitigate potential biases (see above), they remain a risk, particularly with regards to qualitative methods, which rely heavily on respondent perceptions and interviewer interpretation. The desire to attract follow-on programming among certain types of respondents is one such source of bias. This source of bias is often a feature of the community context where project beneficiaries and stakeholders attempt to provide the information that they think will prompt the delivery of additional benefits from a new project. More generally, KII and FGD respondents have their own biases that must be recognized and considered when analyzing their responses. If not moderated correctly, biases can also emerge during FGDs that can taint their usefulness. Finally, qualitative data collection methods are also subject to some important interviewer effects because the quality of the research is heavily dependent on the skill of the researcher and can be easily influenced by their personal idiosyncrasies and biases. Interviewer bias will be mitigated by convening daily team debriefs, rolling data analysis, and presenting transcripts with 24 hours of each qualitative data collection. Key informants from important stakeholder groups (e.g., government agencies) are likely to rotate in and out over time meaning that valued key informants during the baseline evaluation may not be available during the endline evaluation and, thus, key informants during the endline round may lack the institutional/historical knowledge or context to answer questions. In addition, the selection of key informants for KIIs may be challenging and findings can be biased if the informants are not carefully selected. When only a few people are involved in a particular activity, it may be difficult to demonstrate the validity of the findings. Within the clusters and communities covered by the evaluation, TIS+ will be implementing a range of activities tailored to the needs in each community falling under Objectives 1–4. At the time of the baseline data collection, the specific needs and priorities of communities—and therefore the specific content of the TIS+ interventions—may not yet be known, although they will be by the endline. Moreover, while the qualitative methods allow significant flexibility to adjust the discussions to issues of more specific relevance to individual communities, this does not mean that questions in the household survey must be written at a higher level of generality that may not allow a fine granularity of analysis. The repeated cross-sectional survey design is a cost-effective means for tracking changes in general population characteristics over time; however, it does less well than a longitudinal (panel) survey in terms of measuring the variations in the patterns of change at the household (or unit of observation) level or in explaining why the change is occurring. On the upside, a repeated cross-sectional survey does not suffer a loss in explanatory (statistical) power that is characteristic of longitudinal designs due to panel attrition (respondents who drop out of the panel from the baseline to the endline). 390 C.4.9 IMPLEMENTATION RISKS The use of consultants for assignments that are intermittent with possible schedule changes for reasons outside of anyone’s control (e.g., political changes in Somalia, IP implementation delays, security concerns), can lead to changes due to availability issues with consultants. SPSS mitigates this by staying in regular contact with the Evaluation Team, by using some team members who are known quantities, and by exercising strong technical oversight at the field and Home Office levels to ensure continuity. 391 ANNEX B – Places Visited; List of Organizations and People Interviewed Included in separate document – Final Respondent Data Annex 392 ANNEX C. Evaluation Design and Data Collection Methodology Assessment of Preconditions for TIS+ Goal The goal for the TIS+ project is to increase stability in the clusters of implementation. For stability to be achieved, two preconditions should be satisfied: (I) increased good governance and (II) increased community cohesion. This subsection, therefore, explains the development of index scores13 to measure these preconditions. Proposed Models Two models have been formulated for good governance and community cohesion. Model 1 is primary and is validated through sensitivity analysis. The sensitivity analysis process involves the formulation of Model 2 to check how estimated summary statistics and distributions would vary under a different scenario. This is to examine whether having an unequal number of questions within themes for the two preconditions would result into different results. Good Governance Index Model 1: Household questions have been grouped into three themes explaining good governance (confidence in government, service delivery, and stakeholders’ partnerships). See Model 1 (good governance) Excel file in Table 1 for specific questions under each of these themes. Each of the three themes is assumed to contribute equally to the overall good governance index score (each contributes 33.33%). Model 2: All questions in the three themes are assumed to contribute equally to the overall good governance index score. There are 15 questions across the three themes, and each contributes 6.67% (100% ÷ 15) regardless of the theme. Community Cohesion Index Model 1: Household questions have been grouped into the three themes explaining community cohesion (network diversity, interpersonal and institutional trust, and social and political inclusiveness). These themes for community cohesion were informed by the impact evaluation of Peace through Radio Programming conducted in Chad and Niger.14 In addition, the MISTI report15 provided further guidance on what would constitute community cohesion; relevant questions (and in other cases proxies) were identified and classified into the relevant groups documented in the Chad and Niger evaluation report. In other words, this index draws from both the MISTI and Chad/Niger evaluation reports. Please see Table 1 (Community Cohesion Excel file – Model 1) for all the survey questions grouped by network diversity, interpersonal and institutional trust, and social and political inclusiveness themes. Attachment A shows questions in the Niger/Chad report that are related to those in the TIS+ questionnaire. Attachment B shows questions in MISTI closely related to those in the TIS+ assessment questionnaire. Each of the three themes is assumed to contribute equally to the overall community cohesion index score (each contributes 33.33%). 13 Index score is a single measure derived from a set of variables. 14 https://pdf.usaid.gov/pdf_docs/pa00ktf3.pdf 15 http://www.d3systems.com/wp-content/uploads/2015/10/MISTI-Wave-5-Analytical-Report-FINAL_USAID.pdf 393 Model 2: All the questions in the three themes are assumed to contribute equally to the overall community cohesion index score. There are eight questions across the three themes, and each contributes 12.5% (100% ÷ 8) regardless of the theme. Table 1, Specific Contribution of Survey Questions in Models 1 and 2 Model 1 Model 2 Good governance index GG model 1.xlsx GG model 2.xlsx Community cohesion index CC model 1.xlsx CC model 2.xlsx Sensitivity Analysis This involves comparing the distribution of derived scores under models 1 and Model 3.16 All models for good governance and community cohesion seem plausible because the distribution and mean of index scores are approximately similar between the two models (see Figures 1 and 2). 16 This sensitivity analysis process used the TIS+ baseline assessment data set. Figure 1: Good Governance Frequency and Sensitivity 394 Figure 2: Community Cohesion Frequency and Sensitivity Analysis 395 Attachment C1 Linking Questions in the Niger/Chad Report to Those in TIS+ Questionnaire Niger/Chad Report TIS+ Questionnaire Theme a) Talk to people from other communities b) Talk to people from other religions c) Talk to people from another ethnicity 34. Which of the following is true at your personal level? Family members are married to another clan. Family members are not allowed to marry members of another clan(s). I attend meetings with other clan(s) in the same area. Clans in this settlement coexist peacefully. Clans in this settlement do not get along. Clans cooperate only when necessary and a third party (development or government actor) intervenes. Network Diversity a) Interpersonal trust b) Trust in local government c) Trust in central government d) Trust in religious leaders e) Trust in NGOs f) Trust in police 28. On a scale from 1 to 10, how much trust do you have in each of the following to resolve conflicts over shared resources and disputes among various groups in this settlement? 1 being “I do not trust them at all” and 10 being “I trust them a lot”. (clan elders, National Army, militia groups, Peace Committee, Oversight Committee, Central government, religious leaders, local administration, police, local courts, ugaas/sultan, diaspora, District Council members) Interpersonal and institutional trust a) Who makes decisions: Ordinary people, youth, women, my ethnic group, other ethnic groups b) Satisfaction with decisions and political participation 9. In your opinion, has people’s participation in decision￾making increased, decreased, or continued to remain the same, compared to the previous year? 10. Over the past year, who was most responsible for decisions concerning community development in this settlement? Who did not participate? Please check all that apply. (local community members, local administration, local community members participating in planning and contracting meetings, district council members, clan elders, religious leaders, ugaas/sultan, women, youth) 11. To what extent were you satisfied with the decisions made during the past year by the following people/ groups? (local community members, local administration, local community members participating in planning and contracting meetings, district council members, clan elders, religious leaders, ugaas/sultan, women, youth) Social and political inclusiveness 396 Attachment C2 Additional Questions Shows relationship between those questions in MISTI report and TIS+ assessment questionnaire MISTI TIS+ Theme 2.1.1.1 Ability to solve external problems (q34c) 2.1.1.2 Ability to solve internal problems (q35c) 2.1.1.3 How often villages work together (q36) 43. How far do you agree or disagree with the following? a) People work collectively towards problem solving when they know it will benefit the community. b) People do not trust one another across clans and within settlement areas. c) People will cooperate on development initiatives and efforts. 44. How far do you agree or disagree with the following? a) People work collectively towards problem solving which leave beneficial effect on the settlement. b) People are untrustworthy; therefore, development projects cannot be implemented. c) People are very cooperative. Interpersonal and institutional trust 2.1.2.1 Local leaders consider citizen interests (q37a) 2.1.2.3 Local leaders secure funds (q38)17 21. On a scale from 1 to 10, how would you rate the following statements? 1 being “I do not agree at all” and 10 being “strongly agree”. a) The local administration prioritizes community needs. b) The local/district officials are trustworthy for utilization of funds for the development of the settlement. Interpersonal and institutional trust 2.1.2.2 Local leaders consider women's interests (q37b) Participation of women in decision-making at the community level. 18 (This is already captured in questions 10 and 11 of ANNEX A – with “women” as a sub- question in both.) Inclusiveness 17 There is no direct survey question on local leaders securing funds in the TIS+ assessment though the constructed cohesion index assumes (as a proxy) that local officials would be careful spenders if they were the people securing the funds. 18 Also, this is used as a proxy for women’s interests because their involvement in decision-making provides them with a platform to represent women’s interests. 397 Attachment C3 Household survey questions that were in the baseline community cohesion index and are now excluded in the new CC index. 35. Do you think that young people in your settlement are highly prone to violent extremist groups? (Yes, No, DK) 36. To what extent are extremist groups welcomed in this settlement? (less welcomed, moderately welcomed, not at all welcomed, DK) 37. How often are cultural events organized in this settlement? (> 10 per year, 5–10 per year, 1–4 per year, Never, DK) 38. How often are sports activities/ events organized in this settlement? (> 10 per year, 5–10 per year, 1–4 per year, Never, DK) 39. To what extent do these cultural events contribute to the following: (peaceful coexistence, social integration, tolerance) 40. To what extent do these sports events contribute to the following: (peaceful coexistence, social integration, tolerance) 41. What is your preferred identity? (only Somali, Somali then member of a clan, equally Somali and member of this clan, only member of this clan) 42. Which of the following did you participate in the past year? (settlement meeting, volunteered to do settlement work) 45. To what extent do you agree or disagree with the following? (We are one nation and not a group of clans, we are proud of our country, life in Somalia is getting better for all people, I am proud to be a Somali, I fear that Somalia will never be a peaceful place for Somali people) 398 Attachment C4 Household survey questions that were in the baseline good governance index and are now part of the new CC index. 9. In your opinion, has people’s participation in decision-making increased, decreased, or continued to remain the same, compared to the previous year? 10. Over the past year, who was most responsible for decisions concerning community development in this settlement? Who did not participate? Please check all that apply. (local community members, local administration, local community members participating in planning and contracting meetings, district council members, clan elders, religious leaders, ugaas/sultan, women, youth) 11. To what extent were you satisfied with the decisions made during the past year by the following people/groups? (local community members, local administration, local community members participating in planning and contracting meetings, district council members, clan elders, religious leaders, ugaas/sultan, women, youth) 21. On a scale from 1 to 10, how would you rate the following statements? 1 being “I do not agree at all” and 10 being “strongly agree”. a) The local administration prioritizes community needs. b) The local/district officials are trustworthy for utilization of funds for the development of the settlement. 28. On a scale from 1 to 10, how much trust do you have in each of the following to resolve conflicts over shared resources and disputes among various groups in this settlement? 1 being “I do not trust them at all” and 10 being “I trust them a lot”. (clan elders, National Army, militia groups, Peace Committee, Oversight Committee, Central government, religious leaders, local administration, police, local courts, ugaas/sultan, diaspora, district council members) 399 ANNEX D. Recalculated Wave II Significance Levels As initially estimated in the proposal, a sample size of 848 (baseline – 424 and midline – 424) per cluster was considered sufficient to detect a minimum effect size of 10% at a power of 80% and significance level of 5%. Given the limited TIS+ implementation, the effective baseline sample sizes, that would be used for comparative analysis with the midline data, were reduced for all the five clusters. For example, see Tables 1 a and b. Table 1: Achieved wave II baseline and midline sample sizes in TIS+ settlements (a) Banadir Cluster TIS + Settlements Baseline Midline Hawlwadag 18 99 Wadajir 34 177 Warta Nabada 25 148 Total 77 424 The reduction in baseline sample sizes implied varying increases in significance levels across the clusters (with the assumption the power would be maintained at 80%). We recalculated the significance level (α) using the formula by refs: 𝛼𝛼 = Ф �� 𝑑𝑑 𝛿𝛿 �𝑛𝑛𝑛𝑛 × 𝑟𝑟 𝑟𝑟 + 1 � − 𝑧𝑧𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝� Where: d – is the effect size (this has been given as 10% in the proposal) 𝛿𝛿 – defines pooled standard deviation. This is derived from the anticipated proportions at baseline (0.5) and midline (0.6): �0.5×(1−0.5)+0.6 ×(1−0.6) 2 = 0.245 ns – size of the smaller group (either baseline or midline) r – ratio of the larger to smaller group 𝑧𝑧𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 – corresponds to power (0.84 = 80% power) Ф – defines the cumulative density function (c.d.f) Note: This formula is implemented in the user defined R code in the ANNEX. The recalculated significance levels were at least three times 5% (See Table 2). What this means is that if we are to run a full likelihood regression model then the levels to judge significance will be based on these new estimates. For instance, for estimates to be considered significant in Banadir TIS+ settlements, the corresponding p – values must be less than 0.43. The threshold even being lower for Dinsoor settlement (0.58). Significance level, by simple definition, is the chance to make error, hence a 58% would imply a huge chance to make error in distinguishing between significant and non – significant findings. The recalculated significance levels are therefore quite conservative and would result in less precise estimates. Table 2: Recalculated significance levels Baseline Midline Significance level Banadir TIS+ settlements19 77 424 43% 19 We did not disaggregate the recalculation of significance level for Banadir by settlement due to the limited settlement level sample sizes. Settlements in Banadir were instead considered homogeneous and analysis therefore aggregated. However, we disaggregated (b) Gedo/Bay Cluster TIS + Settlements Baseline Midline Dinsoor 82 116 Baardhere 233 309 Total 315 425 400 Gedo/Bay TIS+ settlements Dinsoor 82 116 58% Baardhere 233 309 14% Mitigation strategy to inconsistent significance levels In acknowledging the inconsistencies in significance levels, we estimated regression models using penalized maximum likelihood to minimise bias due to data sparsity (1) – as a result of smaller sample sizes. Through this approach, we were able to emphasize significant results at 5%, and were slow to reject non-significant results at 5% as mostly p – values were less than the recalculated significance levels reported in Table 2. If rejected, then a lot of true findings, with p – values more than 5% and less than Table 2 recalculated significance levels, are likely to be missed. References 1. Cole SR, Chu H, Greenland S. Maximum Likelihood, Profile Likelihood, and Penalized Likelihood: A Primer. Am J Epidemiol. 2013 09/16;179(2):252-60. the recalculation of Gedo/Bay significance level by settlement as there were substantial settlement level variations – with generalizability only being at the settlement level. 401 ANNEX E – Contribution Stories Contribution Story 1: Community Planning and Oversight Contribution Story 2: Technical Coordinators and TIS+ Contribution Story 3: Sports for Development and Peace (SDP) 402 CONTRIBUTION STORY 1: COMMUNITY PLANNING AND OVERSIGHT Community Planning and Oversight (CPO) is at the core of TIS+’s Theory of Change (ToC) and helps differentiate TIS+ from other more traditional infrastructure programs. While the program aims to identify and address priority community needs through the delivery of quick-impact projects, most of which are infrastructure-focused, the process for how TIS+ engages community and government actors to plan, implement, and oversee these activities is what distinguishes it as a stabilization program. Simply put, the “how” (processes used to plan, implement, and oversee activities) is more important than the “what” (specific activities supported in each settlement) for achieving TIS+’s higher-level results of increased citizen engagement and confidence in government, decreased conflict over resources, and reduced support for violent extremism. The following “contribution story” will critically examine the extent to which TIS+ has contributed to improved community planning and oversight. It is structured according to the five criteria identified by John Mayne, the creator of the Contribution Analysis methodology, to analyze the extent of a “contribution claim” (or “plausible association”) between TIS+ activities and the results witnessed during the midline assessment. These criteria include: 1. Plausibility: Is the ToC plausible? 2. Implementation per plan: Has the program been implemented with high fidelity? 3. Evidentiary confirmation of key elements: To what extent are the key elements of the ToC confirmed by new or existing evidence? 4. Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? 5. Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? The contribution story will conclude with an overarching assessment of the strength of TIS’s contribution claim to the results relevant to community planning and oversight identified in the midline assessment. PLAUSIBILITY OF THE TIS+ CPO THEORY OF CHANGE The initial step in building a contribution story involves identifying an intervention’s intended ToC and then assessing its plausibility. The majority of the main elements behind TIS+’s ToC for community planning and oversight are derived from the concept of “Community Driven Development” (CDD), an approach to supporting international development projects systematized by the World Bank during the 1990s but utilized and refined by a wide range of other international financial institutions and development partners since then. At its core, CDD focuses on establishing partnerships between communities and local government units to improve the transparency, inclusivity, accountability, sustainability, and local capacity of projects to provide basic services and manage shared assets. According to the World Bank, “CDD approaches are particularly prominent in conflict and fragile situations – CDD programs operate in 22 countries on the list of fragile and conflict-affected situations, and an additional seven countries with internally displaced populations, refugees, or conflict zones.”20 TIS+’s CPO Approach TIS+’s approach to CPO is articulated in a number of guiding program documents and throughout its periodic reports. As explained in one of the program’s foundational design documents, “Process for Engagement,” “the TIS+ model is based strongly on a community/government programming approach. TIS+ activities will work to foster the relationship between community members and government representatives and supporting Somali leadership in those processes. Particularly, supporting Somali self-efficacy in governance is a priority, which supports USAID’s goals in Somalia and New Deal Principles. TIS+ activities will contribute 20 The World Bank Group, "Community Driven Development," https://www.worldbank.org/en/topic/communitydrivendevelopment#2. 403 to improvements in government service delivery, leading to strengthened governance and community cohesion. Communities will be supported with access to livelihoods, income generation, and basic community infrastructure."21 According to the “Community Contracting Guide,” a document jointly developed by TIS+ and the Ministry of Interior, Federal Affairs and Reconciliation (MOIFAR), TIS+’s CPO approach is intended to complement FGS’s ongoing District Council Formation process and emphasizes the “leadership role of the inclusive community, including women, youth, traditional authorities, and minority clans.”22 Communities are expected to collectively prioritize their needs through community consensus planning and then undertake a “more detailed planning to directly participate in the implementation and long-term management of the project, as well as to help manage resources with appropriate participation and oversight from the local authorities.”23 The Guide posits that “as a result of this process, and with appropriate support and training, the communities gain confidence, and organizational and negotiation skills, enabling them to engage with external partners and the local authorities on an equal footing. The direct involvement of the community leaders, with local, state and federal government counterparts and [TIS+] is intended to build confidence and accountability to address nation-wide stabilization concerns and enhance local ownership of projects and processes. As stabilization projects are successfully implemented in a transparent and accountable process, trust is built between the community and all the three levels of government…to deliver basic services and to construct local government infrastructure, in a more participatory and accountable manner.”24 According to the “Consensus Process Guide,” a proper CPO process can also help to mitigate and reduce conflict over resources. The Guide outlines a ten-step process for implementing the community consensus process which includes initial steps for working with communities to conduct conflict and resource maps and introduce the concepts of “Conflict Sensitivity” and “Do No Harm.” The Guide then instructs facilitators on how to support communities to identify, prioritize, and reach consensus on activities, form Project Oversight Committees (POCs), and how to conduct strategic communication and dialogue events.25 Project Oversight Committees A critical feature of the CPO process is the formation of POCs, introduced under TIS+’s predecessor program, “Transition Initiatives for Stabilization” (TIS). As explained in TIS+’s “Operational and Management Guide,” “A [POC] is a community-based entity created for the Transition Initiative for Stabilization Initiative (TIS +) Program. The POC is a Joint Government-Community Implementation Unit created for a project location to oversee the project delivery from start to finish. The POC works with other stakeholders, including other international organizations, which serve to strengthen government and community interaction. The POC acts as a monitoring and accountability mechanism while mobilizing the community and increasing its awareness of the project for joint implementation. In addition, POCs add an element of sustainability by providing an opportunity for communities to take ownership of project maintenance through the formation of a lasting relationship with state and federal government officials.”26 According to the O&M Guide, POCs are instrumental for providing: • A Mechanism for Community Mobilization – allowing “for a range of interests to be addressed and a diversity of stakeholders to have input into a project, which creates a stronger and more cohesive community identity…By coalescing local community voices around a project, the new community linkages may remain after the project is finished. In addition, POCs serve as an access point between the community and government, allowing communities to better respond to issues 21 AECOM, "Transition Initiatives for Stabilization Plus (Tis+) Process of Engagement," (March 2016). 22 MOIFAR, "Community Contracting Guide," (2018). 23 Ibid. 24 Ibid. 25 Federal Government of Somalia Ministry of Interior and Federal Affairs, "Consensus Process Guidebook." 26 AECOM, "General Principles of the O&M Plans for Tis+ Supported Infrastructure Activities in Somalia," (February 2018). 404 affecting the community as a whole and in the future. POCs help build social capital that will enable a community to develop better public services and/or resist negative influence as a cohesive unit.” • Monitoring and Accountability – "POCs serve as a monitoring body, verifying that the work on a project is completed, according to basic specifications…POCs lay the foundation for the real-time monitoring and community's accountability of the tender work…since community members have constant access to the project site, while government officials and TIS+ representatives are usually based elsewhere, community-based members provide an invaluable first-line monitoring capability." • Facilitating Government-Community Interaction – “By providing a forum for community and government representatives to interact, TIS+ ensures that there is regular communication during project implementation and, hopefully, after project completion. POC meetings encourage active government involvement in project implementation and increase visibility among key community stakeholders. In addition, for the government to receive adequate recognition for its involvement, it is the responsibility of the POC to actively communicate project developments to the rest of the community.” • Improving Project Implementation – “POCs help TIS+ implement quicker and better projects. POCs resolve issues that pose obstacles to the swift completion of implementation.” • Operations of the infrastructure – POCs should help lead the running of facilities and should include “a representative from the District Administration, for a better liaison and understanding.” • Maintenance – "The POC is the main mechanism to ensure maintenance as in most cases there is an evolving governance structure and it requires the ownership, legitimacy, and support from the communities…The POC provides that platform to be utilized, constituted and supported, in terms of the capacity to be the main institution providing the maintenance service after the project completion.” The centrality of the POCs as the key nodal point in the CPO process is aptly summarized in Figure 1, which also provides a useful summary of TIS+’s overarching CPO process. Source: “Community Contracting Guide,” pg. 7. Figure 3: Overview of TIS+'s CPO Process 405 Training and Capacity-Building of POCs Given their importance for ensuring quality of community participation and oversight, the O&M Guide explains, “it is important for TIS+ to carry out the aforementioned trainings, before the groundbreaking of the project to enable, and encourage, the POC to play its vital role in an effective way. During these training sessions, identifying the specific focal persons for the technical and operational/financial aspects of the O&M will establish the needed link between the investment and its sustainability right from the start.” (emphasis added).27 The O&M Guide further outlines five specific five areas where TIS+ will aim to build the capacity of POCs (see Figure 2). Figure 4, Areas for TIS+ Capacity Building of POCs No Training Support Required Proposed Steps 1. Training on their roles and responsibility (training as to how to conduct POC meeting, take minutes and keep records, including notes on decisions made, schedule of meeting and overall agreements/outcomes. Provide training and facilitate sessions to build the capacity of the POC to understand and meet the demands of the role (including record keeping). II. Increase POC knowledge of Public Finance Management (PFM), concepts, in order to support fundraising efforts and ensure that community contribution is used for the betterment and sustainability of projects. For example, funding to support the implementation of Operations and Maintenance (O&M) of the projects, post completion. Train POC’s on the financing modalities and resource mobilization aspects, for project sustainability and accountability. III. Ensure POCs understand Do No Harm (DNH) principles. POC should avoid assuming a parallel role to the local administration, but rather limit themselves to their roles on the project to support the local governance structure. Following and engage DNH principles with POC, throughout project implementation phases, in order to curb community conflict that may arise during and after the project handover. IV. Train POCs on legitimate government documentation needed for the community contracting processes. This will enable POCs to play an active custodianship role, during the community contracting process. Train POCs on correct and legitimate documentation requirements, particularly for construction awards. For example, local administration, Ministry of Public Works or federal government certificates.) This will prepare POCs to be watchdogs during the implementation process. V. Train POCs on the respective O&M arrangements, for each project. This shall include two key components, including the community – government partnership for the resource mobilization required for the maintenance and the technical O&M part. Before the project is handed over, it is important for TIS+ to establish this important liaison role and working partnership between the local administration and the POC, preferably in writing. This will help with long-term sustainability and support long term O&M prospects. The above capacity-building plan directly responds to one of the key recommendations from the final evaluation of TIS, which found that POCs play a vital role in ensuring community participation and oversight. Specifically, the evaluation recommended that “the POCs’ role at the community level should be strengthened and institutionalized. POCs should be provided with adequate training to allow adequate monitoring of projects. The contractor and the community must properly understand the POC’s role.”28 Theory of Change Building off the elements described above, the midline assessment team conducted a one-day workshop with TIS+ field and headquarters staff, as well as the USAID/Somalia lead for the TIS+ program to develop to clearly articulate the intended results, causal logic, and underlying assumptions guiding TIS+'s overarching activities, but also those specifically focused on CPO. Figure 3 (next page) outlines that ToC with the light 27 Ibid. Pg. 8. 28 IBTCI, "Final Performance Evaluation of the “Transition Initiatives for Stabilization” (Tis) Project," ed. Somalia Program Support Services (June 2016). 406 blue boxes depicting intended results and the grey boxes highlighting some of the underlying assumptions needed to be fulfilled to progress along the causal logic. 407 Figure 5: Community Planning and Oversight Mini Results Chain 408 Assessment of Plausibility: Moderate While TIS+’s CPO approach is informed by decades of similar CDD work implemented by the World Bank and a wide range of other development partners, a major recent study by the International Initiative for Impact Evaluation (3ie) questions many of the principles behind the CDD approach and the extent to which it leads to tangible results. The study, a meta-analysis of 25 impact evaluations covering 23 programs across 21 low- and middle-income countries finds that while “CDD programs have made a substantial contribution to improving the quantity of small￾scale infrastructure,” they achieve few of their intended higher-level outcomes, with the exception of results around improved water supply.”29 Furthermore, and particularly problematic for TIS+, “CDD programs have little or no impact on social cohesion and governance. This synthesis study shows that the assumption that the entire community participates in the program is not valid. Data show that the participation in decision-making is limited to a small number of community members. There is a clear ‘funnel of attrition’; many people may be aware of the program and the community meeting, but few participate in the meeting and fewer still speak or participate in decision-making. People participated in making bricks, not decisions.”30 As expected, the meta-analysis study has set off a fierce debate within the development community. The report has received both widespread praise and criticism, leading the authors to publish a series of clarifications of their original findings. The authors readily acknowledge the success of CDD approaches for delivering small scale infrastructure and the immediate benefits that these structures bring to those community members that use them. The authors again, however, state that there is limited evidence of "higher order social outcomes," particularly for social cohesion and governance. “Our review finds that CDD has no impact on social cohesion. There is no heterogeneity there. The lack of effect on social cohesion is consistent across contexts. It is in building social cohesion that CDD has not worked. This is where meta-analysis is so useful, as it clearly illustrates the consistency of this finding…As we say in the report, the lack of impact on social cohesion is not a new finding. Indeed, one of us was a co-author of the 2002 OED review of social funds – including the CDD-like Malawi and Zambia funds – which reported no impact on social capital, as did the OED CDD report three years later. The review confirms this finding now that we have additional evidence from high-quality impact evaluations."31 In sum, while the anticipated results, underlying assumptions, and causal logic behind TIS+’s CPO ToC follows decades of conventional thinking around CDD and is logically plausible, evidence amassed from impact evaluations conducted over the past decade indicates that the programming logic is fundamentally flawed and that CDD does not contribute to improved community cohesion or local governance. That said, findings from 3ie’s synthesis study is passionately contested and future parallel studies might in all likelihood find contrary evidence supporting the use of CDD approaches for small-scale community infrastructure projects. Therefore, the team concluded that the plausibility of TIS+ CPO ToC is, at present, uncertain. 29 Howard White, "Community-Driven Development: Does It Build Social Cohesion or Infrastructure? A Mixed-Method Evidence Synthesis," International Initiative for Impact Evaluation (March 2018). 30 Ibid. 31 Howard White and Radhika Menon, "Community Driven Development: Howard White and Radhika Menon Respond to Scott Guggenheim," Oxfam (July 2018). “CDD programs may be using social cohesion rather than building it. Numerous factors may affect community involvement, such as the role played by the elite or prime movers in the community, intra￾community divisions and the perceived benefits of participation. CDD’s impact on governance is sometimes undermined by the creation of parallel structures for the sake of the program. The function of these governance structures is not clear once the community projects end.” ~Howard White, et al. “Community Driven Development: does it build social cohesion or infrastructure? A mixed-method evidence synthesis.” 409 FIDELITY OF IMPLEMENTATION The next step in developing the contribution story involves assessing the extent to which implementation actually followed the intended design of the ToC discussed above. Assessment of Fidelity: Moderate. The midline assessment found clear evidence that TIS+ took active steps to ensure that communities in both Banadir and Gedo/Bay were supported in collectively identifying and prioritizing their shared needs. There was documentary evidence that TIS+ held a 3-day “Bridging Plan Stakeholders Consultation” workshop with over 60 participants in Banadir. In Gedo/Bay, TIS+ held a 3-day “District Consensus Building” workshop in Baardheere from May 2-4, 2016 involving 42 participants32 and a similar 3-day in Dinsoor from June 6-8, 2016 involving 43 participants of diverse backgrounds.33 Interviews further confirmed that the workshops followed the 10-step process outlined in the “Community Process Guide.” Interviews with TIS+ staff indicate that district-level POC members were selected during the last day of each workshop and the agendas from the workshops confirm that sessions were indeed planned to identify these members although the assessment team could not independently verify that these sessions actually took place (the team has no reason to suspect that they did not). However, interviews with the same TIS+ staff members also revealed that there was very limited training provided to the POC members during these workshops and that they mostly just discussed the anticipated role of the POCs as they looked to recruit volunteers. Based on both document review and interviews with TIS+ staff, substantive training of the POCs did not take place until nearly 24 months after the establishment and the subsequent reconstitution of the POCs. While TIS+ had originally intended to form POCs at the district-level, the program decided in late 2018 that this approach might lead to a concentration of power in the hands of a limited group of people and thereby present opportunities for corruption and that it would be more prudent to establish PIKA-level POCs. Accordingly, PIKA-level POCs were established in November 2018 in Banadir and Baardheere and January 2019 in Dinsoor. This meant that all of the new POCs established in both Banadir and Gedo/Bay were established after the start of construction and even the completion of construction for all but the POC in Baardheere which was established just two months before the completion of construction for their project. The evidence, therefore, suggests that the POCs had very little formal engagement in the activities in their intended capacities and did not have the opportunity to fulfill their intended role of providing community oversight and regular engagement with local authorities. While the capacity-building and function of the POCs deviated considerably from the original design, both interviews and the assessment team’s review of sign-in sheets confirm that the POC membership was inclusive of both women and youth, as well as government officials and representatives from the business community in most cases. Interviews also confirmed that although the community contracting was not led by the POCs as intended, community contracting did occur in each of the Wave II areas. In sum, TIS+ looks to have followed most of its intended CPO steps but substantially deviated from the original design when it came to the establishment, training, and use of the POCs. While it is true that POCs form only part of the overarching CPO process, it is nevertheless a crucial part to ensure regular dialogue with authorities, community monitoring and oversight, and for the building of a sense of ownership and responsibility for the sustainability of the activities within the community. 32 Participants included stakeholders from Jubbaland State of Somalia, Somali Federal Government, the Local Administration, youth, women, elders, traditional leaders, religious group and the local private sector among others. 33 Participants included representatives from the interim South West Administration, the Somali Federal Government, the Local Administration, youth, women, elders, traditional leaders, religious group and the local private sector among others. 410 EVIDENCE SUPPORT INTENDED RESULTS Having outlined the intended ToC and extent to which implementation followed that intended design, the next step in building the contribution story is to examine the extent to which key elements of the ToC are confirmed by evidence. Assessment of Supporting Evidence: Moderate As could be expected given the fluid and unique contexts of individual settlements across Somalia, there was a great deal of variation in results across the Wave II areas. As demonstrated below, there were inconsistent results both within and across TIS+ clusters reviewed during Wave II. The evidence was supportive of some intended results in some settlements—e.g. community participation and awareness in Banadir—but then contradictory in others—e.g. participation in Dinsoor and awareness in Baardheere. Thus, while there was supportive evidence of TIS+'s CPO approach achieving intended results in a specific settlement, the results were too diverse to draw any overarching conclusions on the effectiveness of TIS's CPO approach across multiple clusters and diverse contexts. The three TIS+ settlements in Banadir demonstrated particularly strong results on “community awareness and participation.” The midline assessment found an increase of 58% in respondents indicating that they were aware of community planning events and a significant increase in participation by all groups with the exception of clan elders. Likewise, TIS+ settlements demonstrate very strong level of satisfaction with projects meeting community needs with an impressive 92% of respondents reporting that “some” or “all” selected projects addressed community priorities. The midline also identified a significant increase in awareness of oversight committees in both TIS+ settlements (+46%); however, notably, there was no change in the perceived effectiveness of those POCs. This evidence would seem to support the finding in the “Fidelity of Implementation” section that the POCs did not perform as had been originally anticipated. There was also fairly positive results in terms of the community interaction and performance of local and state government in Banadir. Encouragingly, the midline saw a strong increase in the reported roles of state and local governments in managing shared assets, particularly in the management of roads and sports arenas which were PIKAs. There was also a substantial increase in perception of local government providing services, however, most of these services were not related to the activities supported by TIS+. On the less positive side, the perception of services provided by state government decreased across most areas and there was no change in the reported citizen satisfaction with those services. Likewise, there were no notable changes in the perception of decreased conflict over resources and perceptions of corruption actually increased towards both the Federal and local government. In terms of inclusion, the midline assessment confirmed that all TIS+ POCs included the participation of women and youth, with an average membership rate of 40% for women and 30% for youth. The results were considerably more varied, and often less positive, in TIS+ settlement in Gedo/Bay. While the participation of different stakeholder groups in community consultations generally increased in both TIS+ and non-TIS+ areas across the cluster, they decreased considerably in Dinsoor. However, despite reporting decreased participation across groups, respondents showed an increase in the perception that projects were selected based on community planning events. Paradoxically, Baardheere demonstrated the exact inverse, reporting an increase in the participation rates across groups but a decrease in the perception of projects being selected based on community planning events. Troublingly, there was a decrease in both Dinsoor and Baardheere in respondents reporting that ‘most’ or ‘all’ of the selected projects effectively addressing community priorities, as well substantial decrease in the awareness of POCs and no change in their perceived effectiveness. 411 Likewise, the results were not generally very positive related to shared asset management and quality of services. The midline showed a perceived decrease in the role of state and local government in the management of nearly all assets, including in roads and meeting houses which were key PIKA activities. In terms of service provision, INGOs are seen as having an increased role in the provision of nearly all types of services across the cluster and the role of both state and local government in the provision of roads, key PIKA activities in both areas, decreased in both Dinsoor and Baardheere. Finally, while most settlements across the cluster reported an increase in citizen satisfaction levels with services, Baardheere saw a decrease and the TIS+ constructed road and airstrip were specifically identified as being of poor quality. On the positive side, Dinsoor was noted as a unique "positive deviant" and witnessed a substantial reduction in resource-based conflicts were other settlements across the cluster saw increases. Likewise, both Dinsoor and Baardheere reported reduced levels of perceived corruption by both the Federal and local government. Gedo/Bay reported similar results as Banadir in terms of inclusion with women and youth being present on all TIS+ POCs and accounting for roughly 30 percent of the membership for each group. INFLUENCING FACTORS AND ALTERNATIVE EXPLANATIONS The final step in building the contribution story is to then analyze the extent to which “other influencing factors” and “alternative explanations” explain the identified results. Assessment of Influence of Alternative Explanations: High. As demonstrated by the inconsistency in the results highlighted above, a number of other influencing factors and alternative explanations account for the variety of results. These include: • Activities of Other Development Partners. There was a wide and diverse set of group of actors implementing activities in each of the TIS+ settlements reviewed during Wave II, most of who reported utilizing some aspect of community engagement, planning, and oversight. The assessment team identified at least seven different organizations working in Baardheere, 11 organizations working in Dinsoor, and over 20 different organizations working in Banadir. Both KIIs and FGDs conducted during the midline indicate that there has been a marked increase from these development partners to more systematic engagement with communities at the outset and involve community members in the planning and design of activities. Likewise, other major partners, such as Danish Refugee Council (DRC), Finnish Church Aid (FCA), Nordic International Support Foundation (NIS), and International Committee of the Red Cross (ICRC), as well as UN agencies such as the Food and Agriculture Organization (FAO), World Health Organization (WHO) report specifically using POCs or established community structures such as the Peacebuilding Committees to guide the planning and oversight of their committees. • Carry-Over Effects from TIS: The strong performance of Banadir on results related to community awareness and participation is in large part likely explained by strong carry-over effects from the activities of TIS. Mogadishu was one of the first areas in Somalia to pilot CPO approaches during the initial TIS program and TIS conducted numerous community consensus planning sessions, trauma healing sessions as well as investments in infrastructure including rehabilitation of district administration buildings and sports facilities during its program life. The TIS final evaluation concluded that these activities led to notable improvements in both community awareness and participation. Specifically, the evaluation found that “TIS’s innovative essence was the consultative and participatory process that brought communities (civil society and business people), local administration and central government together to discuss and identify the community’s priority needs;” “TIS proved to be a strong catalyst in encouraging greater 412 engagement between communities;” and that “TIS contributed significantly toward increasing collaboration between local government and stakeholders.”34 • Unique Characteristics of Settlements. Qualitative interviews (both KII and FGD) clearly conveyed that the unique historical, socio-economic, and geopolitical characteristics of each settlement played a large role in the results seen during the midline. For example, interviewees identified the unique characteristics of the three TIS+ settlements in Banadir as helping to explain the high levels of both government and development partner interested in support activities in these areas. They explain that these districts are closer to the city center and are key revenue￾drivers for the local and state government, accounting for up to 70% of revenue generation by some estimates. Likewise, clan dynamics and the influence of powerful individuals were identified as reasons for the variations between community awareness and participation in both Dinsoor and Baardheere. • Historical and Contextual Factors. Interviewees also explained that the building of trust between communities and respective levels of government following decades of conflict is a long￾term endeavor and requires generational change. Interviewees explained that while TIS+ activities might facilitate community and government interaction on the planning and oversight of specific shared assets, there is a long list of grievances and frustrations that still need to be overcome. These included factors such as wide-spread perceptions of nepotism and clannism, unfulfilled election promises, corruption, security lapses coupled with a fear of victimization by AS, injustices and a weak judiciary, poor service delivery, and the outsized influence of diasporas. ASSESSMENT OF CONTRIBUTION CLAIM Summary of Assessment Criteria Criteria Assessment Plausibility: Is the theory of change plausible? Moderate. The anticipated results, underlying assumptions, and causal logic behind TIS+’s CPO ToC follows decades of conventional thinking around CDD and is logically plausible, however recent evidence from a meta-analysis of 25 impact evaluations suggests that this programming logic is fundamentally flawed and that CDD does not contribute to improved community cohesion or local governance. Implementation per plan: Has the program been implemented with high fidelity? Moderate. TIS+ followed most of its intended CPO steps but deviated substantially from its original design when it came to the establishment, training, and use of the POCs. While it is true that POCs form only part of the overarching CPO process, it is nevertheless a crucial part to ensure regular dialogue with authorities, community monitoring and oversight, and for the building of a sense of ownership and responsibility for the sustainability of the activities within the community. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? Moderate. Inconsistency in results across locations means that cross￾cluster conclusions on the effectiveness of TIS+'s approach cannot be determined. Identification and examination of other influencing factors: To what extent have other influencing factors been identified? Identification of alternative explanations: To what extent do alternative explanations explain results? High. As demonstrated by the inconsistency in the results, a number of other influencing factors and alternative explanations contributed to the results seen during the midline. These factors include (1) Activities of other development partners; (2) carry-over effects from TIS; (3) unique characteristics of settlements; and (4) historical and contextual factors. 34 IBTCI. “Final Evaluation of ‘Transition Initiatives for Stabilization’ (TIS)” 413 Summary Assessment of Contribution Claim: Weak. At present, there is limited evidence that TIS+’s Community Planning and Oversight approach is consistently contributing to higher-order, outcome-level results related to improved citizen engagement and confidence in government, as well as reduced conflict over resources or support for violent extremism. While the midline assessment found evidence of TIS+ achieving notable results in terms of community awareness and participation, as well as local and state government service provision and management of shared assets, these results were usually only at the individual settlement level and were not consistent across clusters. TIS+'s contribution claim is also limited by the lack of a consistent implementation approach across the program. While the community consensus workshops look to have been consistently implemented, the form and timing of other key parts of the CPO process varied across settlements. This includes a difference in the way that community contracting took place across the settlements, and most notably, significant deviations on how POCs were formed, trained, and utilized throughout the process. TIS+'s own program design documents clearly highlight the central role that POCs play in the overarching CPO process; however, evidence indicates that TIS+ did not prioritize their role until late in the PIKA process, almost always only after the construction of the different structures were already complete. 414 CONTRIBUTION STORY 2: TECHNICAL COORDINATORS TIS+ is a complex program with different interventions that allow it to adopt a flexible approach to programming in Somalia. In 2017, the TIS+ program and assessment teams developed a program-wide Theory of Change (ToC). Nested within this larger ToC were three sub-ToCs which specifically focused on key approaches utilized by the program to achieve stabilization results. One of these approaches was the use of Technical Coordinators (TCs) embedded within host-country government ministries at federal and state level. This “contribution story” will focus on critically examining the role of the TCs and any influence they may have had on the achievement of higher-level stabilization, governance, and community cohesion results. In the original TIS+ documents on coordination and engagement, TIS+ TCs, as TIS+ funded positions embedded within Ministries, were designed to support TIS+ Objectives 1, 2, and 3. These were explicitly seen as roles at least partly designed to enhance the efficacy of TIS+ interventions, for example, adding value to the projects delivered under our “core” contribution story on community planning and oversight of infrastructure. The TCs were also intended to deliver wider work in supporting communication between levels of government more broadly. These two functions are summarized in the TIS+ Approach to Engagement document from 2016: “To support continued coordination and collaboration of TIS+ plans and activities with federal and state government plans, TIS+-funded Technical Coordinators are proposed to sit within government ministries. These Technical Coordinators will facilitate not only planning coordination but will support better communication across levels of government and with other stabilization programs in support of more effective stabilization activities.”35 In assessing the TC contribution claim, we will therefore be looking for evidence that TCs have demonstrably improved TIS+ or other interventions, and that they have improved communications between levels of government, as an early step in the results chain, and for evidence that they have contributed to results under the three TIS+ objectives below. 1. Increase confidence in governance based on equitable participation in decision-making and management of community assets; 2. Empower community and government representatives to engage with the private sector and development actors in a collaborative process for community growth; 3. Increase Somali engagement in creating a more stable future (Federal-level Ministry TCs only). The contribution story is structured according to the five criteria to analyze the extent of the contribution claim between the TIS+ activities and results witnessed during the midline assessment. These include: 6. Plausibility: Is the theory of change plausible? 7. Implementation per plan: Has the program been implemented with high fidelity? 8. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? 9. Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? 10. Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? 35 TIS+ (March 2016) ‘TIS+ Process of Engagement’, p.4 415 PLAUSIBILITY OF THE TIS+ TCS THEORY OF CHANGE The practice of embedding TCs and advisors in Ministries and government agencies has been long utilized in Somalia. Against the background of nascent institutions and limited capacity in the civil service, it is seen as an effective way of facilitating donor engagement with government institutions while also building the capacity of the local staff in those institutions. This approach has not been without challenges which have forced a rethink of the interventions to make them more effective. UNDP Somalia36 identified obstacles including: • Radically weakened institutions with little legitimacy. • Dramatic variation in the political landscape from region to region with shifting power and allegiances over time. • Security threats. • The challenge of maintaining neutrality in a complex situation. The approaches to technical assistance delivered in Somalia has varied from utilizing diaspora experts (IOM/UNDP’s “Migration for Development in Africa”), project implementation units (Aid Coordination Units), Technical Advisors/Coordinators (UN, Care, Concern International, GIZ, USAID) and policy development. TIS+ approach is in line with this thinking to provide embedded capacity within the government ministries at the state and federal level. TIS+ Approach Across TIS+’ three objectives, TCs are expected to: • To support coordination and collaboration of TIS+ activities with government activities. • To support better communication with government at all levels and with other stabilization programs. • Expected to increase knowledge and gain support for locally-based Objective 3 activities at all levels of government. Figure 1 on the next page shows the causal steps through which the TCs were expected to contribute to TIS+ objectives and the assumptions on which these steps are based. 36 UNDP, “Supporting capacity development in conflict and fragile contexts,” https://www.undp.org/content/dam/undp/library/crisis%20prevention/2012SupportingCapacityDevelopmentinConflictFragileSetti ngs.pdf 416 Figure 6: Technical Coordinators Mini Results Chain 417 Since its inception, TIS+ has provided 27 different TCs to different government institutions37 including the: • Banadir Regional Administration • Ministries of Interior in Mogadishu, Baidoa, and Kismayo • Ministries of Gender, Human Rights and Family Affairs in Mogadishu and Kismayo • Ministry of Public Works and Reconstruction • Ministry of Youth and Sports • Ministry of Information Some of these TC positions have already been scrapped either due to funding or strategic realignment of the program. The work and output of the TCs will be highlighted in the later section of this document. Assessment of Plausibility: Moderate. The approach of utilizing embedded experts in government institutions across Somalia is common. That said, there are challenges associated with implementing this sort of assistance that constrains the overall prospects for institution and state building. For example, an approach paper commissioned by the World Bank Group and authored by the Overseas Development Institute warns that “in fragile contexts when institutional capacity has been eroded, the strategic positioning and short-term calculations of leadership figures and elite groups carry disproportionate weight in shaping policy directions and outcomes in what are frequently fluid and rapidly changing circumstances.”38 In the absence of strong institutions, policy is often dictated by strong personalities and interests. The success of TCs placed in government institutions depends more on the personality and network of the individuals than on their skills. High turnover in these ministries due to dynamic political changes further affects the influence and progress made on delivering key interventions. Interviews with TIS+ TCs confirmed the challenge of dealing with high turnover often preceded by a political impasse, which often disrupts the project timelines. "We often deal with new Ministers who stay between 6 months and a 1 year and it takes time to bring them up to speed. Sometimes, these are political appointments and focusing too much on the work without cozying up politically can threaten the buy-in of a project. As a Technical Coordinator, you’re seen as having divided loyalties between the project and the institution.”39 There is also the challenge of sustaining positive results beyond the short-term and managing expectations in order to achieve long-term capacity and effective coordination as envisioned in the TIS+ Theory of Change. In interviews, TIS+ TCs highlighted that they constantly worried about their own tenure and funding priorities, which affected their ability to commit to long-term visions by the government institutions they were supporting. “As a Technical Coordinator, my biggest challenge is the uncertainty of the work. We have temporary contracts. Mine runs out next year in February. The Ministry relies on us and we are developing plans going beyond our contract term, which is a bit difficult to do if you know you may not be there to implement them.”40 The challenge of divided loyalties and inherent capacity substitution effects has been noted by other partners supporting technical assistance interventions. In its “Guidelines for Technical Assistance” document, the Danish International Development Agency, the agency notes that “…program coordination and supervision must be vested in the partner institution. The partner institution may be supported in the overall planning, 37 AECOM, “FY2018 TIS+ Annual Report,” (2018). 38 Overseas Development Institute, “An Approach Paper for Capacity Development in Fragile States,” (2009). 39 Interview with TIS+ Federal TC. (2019) 40 Interview with TIS+ South West TC. (2018) 418 coordination, monitoring and reporting tasks related to program management by technical assistance on the basis of the partner organization's own efforts to build up such systems.”41 Both the TIS+ program and TCs noted the challenge of subsuming the authority of the government institutions in some instances, with TCs acting as representatives of the government, rather than externally injected capacity. What is evident, however, is that the choice of institutions receiving the TIS+ technical assistance is in line with the priorities of the government. This was confirmed in government interviews, as well as a review of government strategies. For example, the Federal Government of Somalia’s National Stabilization Strategy identifies the TIS+ supported Radio Wadajir platform, designed and produced by a TIS+ TC, as “the main strategic communication initiative supporting the stabilization strategy.” The validation in the government’s plans is an indicator for the expected buy-in and political will from the government for the work of the specific TC in this instance. “…the Technical Coordinators are one of the main conduits through which the government engages TIS+. They provided much-needed capacity in the Ministries. At the national level, the institutions are struggling due to lack of capacity. It is even worse at the state level because, for the first time in the history of the country, we are trying to have fully functioning ministries in the states. Previously, we had local administrations managed centrally from Mogadishu. To establish the functioning institutions in Kismayo, we need to make double the effort in a short time to make this a reality.”42 FIDELITY OF IMPLEMENTATION The below summary of some of the output by the TCs highlights their contribution: Jubaland State: At the Ministry of Interior and Security, the two TCs listed their deliverables as follows: • Drafting of the Jubaland Stabilization Plan • Drafting of the Ministry of Interior’s Briefing Note on Jubaland Public Expenditure Management Toolkit • Organizing a Counter Violent Extremism Workshop • Drafting of the Jubaland State Public Expenditure Management and Planning of the District PEM Cycle • Organizing a training on the Public Expenditure Management Toolkit • Supporting the Sanguuni District Community Consultation organized by SSI/OTI • Delivering a Presentation on the Jubaland Stabilization Plan at the bi-monthly Stabilization Meeting in Mogadishu • Organizing the Jubaland Quarterly Stabilization Meeting • Developing the Organogram for the Ministry • Drafting the Local Government Financial Management Manual At the Ministry of Youth and Sports in Jubaland, the TC’s activities and deliverables were listed as follows: • Developing the Ministry’s Strategic Plan • Supporting the development of the National Sports for Development Framework • Supporting the Kismayo Bridging Plan Consultation Workshop • Presenting on the Sports for Development Framework at the Coordination Meeting in Mogadishu • Supporting the Kismayo Book Fair • Supporting the Jubaland Youth Conference in Doolow • Organizing tournaments for Girls’ Basketball and Boys Soccer. The TC at the Ministry of Women, Family Affairs and Human Rights in Jubaland listed the following: 41 DANIDA, “Guidelines for Technical Assistance,” (2009) 42 Interview with former Director General, Ministry of Interior, Jubaland State of Somalia. (2018) 419 • Development of the Ministry’s three-year strategic plan • Contribution to the drafting of the Sexual Offenses Bill • Drafting of Standard Operating Procedures for Prevention and Response to Gender-Based Violence • Delivering a Training on Leadership and Management for Women Leaders • Organizing a Workshop for Civic Engagement on Women’s Constitutional Rights • Supporting the Opening Ceremony of the Garbaharey Women’s Center The TC at the Ministry of Public Works and Housing was relatively new but is working on a capacity building plan for the Ministry as well as a strategic plan in addition to supervising the construction of public buildings currently undergoing rehabilitation through the support of various donors. South West State: In Baidoa, the two TCs at the Ministry of Interior and Local Governance listed the following among their deliverables: • Development of the Stabilization Strategic Plan for South West State • Development of the Local Government’s Employment Regulations and the Ministry’s Organogram • Development of TORs for the Ministry’s departments • Development of the Ministry’s Human Resources Policy • Development of the Finance Policy • Drafting of the Local Government Financial Manual for Trainers • Delivering Leadership Training in Hudur and Berdale Districts • Delivering Induction Training for the new District Council Members in Hudur At the Federal level in Mogadishu, TCs contributed to: • Development of the Wadajir Radio Program • Development and launch of the Wadajir Framework for Local Governance • The Mogadishu Book Forum • The National Women’s Basketball Tournament • Gedo Reconciliation Meeting • Supporting the District Council Formation Processes • Marka Corridor Stabilization Plan In addition to the individual documentary output and coordination roles, TCs and government partners noted their contributions beyond the individual Terms of Reference (ToRs). Noting the inherent low capacity in the institutions they were embedded in, TCs often played larger than life roles in the day-to-day running of the Ministries. Whilst this had a positive short-term impact on the capacity of these institutions, it presents the risk of capacity substitution and sustainability challenges in the long-term. This is particularly of concern for stabilization programs that have to deal with the frequent alignment of priorities because of changing dynamics. As mentioned above, TCs regularly highlighted their concerns over their tenure in the long-term. “When I arrived, there were about four staff members in the entire Ministry, so the items on my job description are minuscule compared to the work I ended up doing. My formal title is Strategic Governance Advisor and Technical Program Coordinator. My real work has ended up being the de facto Chief of Staff at the Ministry because I facilitated discussions between the Minister and the international partners, and I also coordinated stabilization activities with partner government agencies.”43 Despite their contributions in the institutions they were embedded in, TCs expressed frustration that their strategic role was not adequately utilized by TIS+ and USAID. Many of the TCs felt that USAID did not 43 Interview with TIS+ JSS TC. (2018) 420 properly understand their role and also added that the lack of an efficient feedback loop with the TIS+ program team sometimes led to missed opportunities, especially in situations where the governments’ unmet expectations had to be managed. In such instances, TCs felt that TIS+ could have relied on their understanding of the local dynamics and individual relationships to navigate these challenges. “There is still some work to be done on improving the relationship between the Nairobi office and the technical coordinators. Because of this, sometimes we do not have visibility on how the decision-making process takes place. We are at the Ministries and have managed to build a solid relationship in the last two years and sometimes we feel there is lost opportunity to take advantage of the goodwill and sort out any potential misunderstanding or managing expectations.”44 It is important to note that some of these challenges and missed opportunities have been acknowledged by TIS+. One of the main outcomes from the 2018 annual Strategic Review Session (SRS) held by TIS+ was to "define the best possible way as a best practice-to engage the TCs and Advisors in the FGS and Emerging Federal States (EFS) ministries.”45 As part of the realignment, TIS+ has implemented systematic reporting from the TCs and incorporated their participation as part of TIS+ progress on improving links to the host government. TIS+ lists the contribution and participation of the TCs as a core approach to improving the program’s linkage with the government institutions it works with. Assessment of Fidelity of Implementation. Moderate. It is evident from the review of the program documents, the output from the TCs and interviews with government partners that the work of the TCs contributed to both improved coordination between the government and TIS+ and increased capacity by the government to engage with stabilization actors and donors and better planning by the government. Evidence Support Intended Results. Moderate. The next step is to examine the extent to which key elements of the Theory of Change are confirmed by evidence. TCs were designed to improve existing TIS+ interventions, and to improve coordination in government in order to contribute to program objectives one and two: 1. Increase confidence in governance based on equitable participation in decision-making and management of community assets; 2. Empower community and government representatives to engage with the private sector and development actors in a collaborative process for community growth; Ultimately, the TCs are aiming to contribute, alongside other TIS+ interventions to increased confidence in state government. We have a mixed picture of confidence in the government from qualitative and quantitative surveys. Moreover, the results chain connecting the TCs inputs to these results is so long and is decisively influenced by so many other variables, that it would be unfair to expect any contribution at this level in any survey data. Our results chain for the TCs work has therefore concentrated on intermediate steps, such as improving TIS+ interventions, improving coordination, and, crucially, injecting sustainable capacities in the relevant ministries. TIS+ TCs themselves certainly confirmed that their activities had been focused both on supporting TIS+ interventions and on promoting coordination. This is supported by the evidence of their produced documents, interviews with government officials and other partners as well as the program documents. TCs have also worked on processes of vital importance to the state government, such as district council formation processes. 44 Interview with TIS+ JSS TC. (2018) 45 AECOM, “FY2018 TIS+ Annual Report,” (2018). 421 There was evidence that government counterparts appreciated the work of TCs and saw them both as providing a point of contact with TIS+ programming, and promoting government ownership of TIS+ interventions, as well as building capacities in their ministries, but the clear emphasis was on the need for, and importance of, capacity building. “At national level, the institutions are struggling due to lack of capacity. It is even worse at the state level because, for the first time in the history of the country, we are trying to have fully functioning ministries in the states… The Technical Coordinators provide their own skills and experiences as required. In addition, they also train and build the capacity of the few available civil servants in the Ministries.”46 Broader CA interviews revealed that many other programs that work with government generally heard good feedback on TIS+ TCs from government partners, and some who had encountered TCs in Kismayo or been involved in events organized by TCs in Jubaland, were positive about these experiences. Some of these respondents did, however, suggest that following up on these events and deliverables was important. Others suggested that TIS+ could benefit from a more explicit focus on capacity building in the work of its TCs. This was confirmed by other respondents who said TIS+ advisers had made a positive impact to the development of stabilization strategies, but this should be matched with an assessment of the real capacities but also motivations and leverage of the political actors in the ministries to implement these. An approach more focused on improving capacities beyond the lifetime of supported TC roles would likely include such an assessment. Other donors working on capacity injection in Somalia have noted that there has often been a failure of capacity injection interventions in Somalia to be tailored to specific needs, which are different in different regions.47 TCs themselves did feel that they were building capacities in their ministries but felt that the needs in this respect were very large. “We provide a lot of support especially in terms of capacity for the Ministry because the skillset is not available because the South West State and the system of administration is less than five years old and when you think of a civil service in a country, you’re talking about building up capacity over fifty years or more.”48 Assessment of Influence of Alternative Explanations: High. The next stage in our analysis is to ask what other influencing factors could be accounting for the changes observed. These include other externally funded advisers as well as positive and negative factors affecting coordination and capacities in the relevant ministries. Qualitative research showed at in most of the Ministries where TIS+ TCs are deployed, they were often the only ones at the Ministry with the exception of the Ministries of Interior in both states where there were a few other TCs deployed by IDLO and UN JPLG. TIS+ coordinates very closely with programs such as SSF, which deliberately does not duplicate advisers in the same ministries where TIS+ TCs are placed. As a result, the TCs contribution to the state level Ministries is visible but there is a mixed picture at the Federal level. Levels of existing capacity within the Ministries where TCs are based has a clear impact on the contribution TCs will make. One of the main challenges that the Emerging Federal States have to contend with is the capacity gaps in the civil service. The advent of federalism has meant that the powers and services have been devolved to the regions, but this did not come with any transfer of capacity or resources from the center. Mogadishu itself struggles with lack of capacity in the federal civil service. However, there has been heavy investment through programs such as the Capacity Injection Program by the World Bank and the deployment of short term and long-term technical assistance by a variety of donors. 46 Interview with former Director General, Ministry of Interior, Jubaland State of Somalia. (2018) 47 World Bank (2015) Somalia- Capacity Injection Project (English). Washington, D.C.: World Bank Group 48 Interview with TIS+ South West TC. (2018) 422 Jubaland was the first of the Emerging Federal States to be established and as a result of the relative stability, it has seen a steady influx of professionals from Mogadishu and abroad moving to Kismayo. However, there has been no major donor investment in the capacity of the civil service beyond programs focused on public financial management. Short term technical assistance has come through IOM's MIDA especially in public healthcare but there remains a serious lack of capacity especially at ministerial level as a result of the limited available wage bill and the lack of donor investment when compared to Mogadishu and Puntland. The South West administration remains in a condition of some uncertainty especially with plans to move its capital to Baraawe as well as the inability of the government including the Presidency to move further away from Baidoa airport. As a result of this, the capacity of the ministries remains fairly limited and is further compounded by the lack of resources from the federal government or international community to develop the capacity of the local governments. It is, however, encouraging to note that as part of the Public Financial Management reforms, a civil service commission has been established and this may improve investment in the civil service. The overwhelming investment in technical assistance and capacity injection has been focused at the Federal level in Mogadishu. Programs including IOM MIDA, UNDP's Multi-Sectoral Capacity Development Program, the World Bank's Capacity Injection Project, SSF as well as USAID funded programs including Strengthening Somali Governance (SSG) and Growth, Employment, Economy and Livelihood (GEEL) have provided TCs and advisors to Federal level institutions. In addition to the contribution of other development partners, the assessment also acknowledges the carry￾over effect of TIS, particularly at the Federal level in Mogadishu, where the program was first piloted. The evaluation of the program concluded that the program had “contributed significantly toward increasing collaboration between government and stakeholders.”49 TIS utilized the ‘Dan Guud’ model, where it embedded professionals into the Banadir Regional Authority (BRA). This group effectively became the nascent cadre of civil servants that improved the capacity of the local government’s functions. The political and security environment had a strong influence on the contribution of the TCs. In particular, the frequent political crises which contributed to the high turnover in the leadership of the Ministries often constrained the momentum built by the TCs. This is further compounded by an unpredictable security situation, particularly in Mogadishu, where the Federal level TCs are based. ASSESSMENT OF CONTRIBUTION CLAIM Summary of Assessment Criteria Criteria Assessment Plausibility: Is the theory of change plausible? Moderate. The approach of utilizing embedded experts in government institutions across Somalia is common. However, there are challenges associated with the approach including the potential for elite capture, frequent turnover of government leadership and the challenge of sustaining positive results beyond the short term. Implementation per plan: Has the program been implemented with high fidelity? Moderate. It is evident from the review of the program documents, interviews and the output of the TCs that they implemented as per the plan. However, there was also evidence that TCs were not adequately utilized in situations where they would have greatly contributed to. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? Moderate. There is evidence that TC work contributed to improving TIS+ interventions, improved coordination and injected capacity in their host Ministries. However, the high-level expected contributions such as increasing confidence in government presents a mixed picture. 49 IBTCI. “Final Evaluation of ‘Transition Initiatives for Stabilization’ (TIS)” 423 Moreover, the results chain connecting the TCs inputs to these results is so long and is decisively influenced by so many other variables, that it would be unfair to expect any contribution at this level Identification and examination of other influencing factors: To what extent have other influencing factors been identified? Identification of alternative explanations: To what extent do alternative explanations explain results? High. A number of other influencing factors and alternative explanations contributed to the results seen. These included the activities of other actors, the carryover effects from the previous TIS program as well as the highly dynamic political and security situation. Summary of Assessment of Contribution Claim: Moderate. Despite missed opportunities and the impact of external factors, there was evidence that TIS+ TCs: • Supported continued coordination and collaboration of TIS+ plans and activities with federal and state government plans. • Supported coordination with other stabilization programs, and; • Improved communication especially on the government’s stabilization plans. The contribution claim is also limited by the fact that other actors, and in particular, stabilization programs also embedded TCs and Advisors to carry out similar functions such as supporting state and federal level coordination of stabilization programs. There was also limited evidence that the TIS+ TCs contributed as expected to higher level results such as increasing confidence in government. 424 Contribution Story 3: Sports for Development and Peace (SDP) The use of sports is a key feature of TIS+’s youth engagement strategy. The program has made sizable investments in either building or rehabilitating football stadiums and basketball courts as a shared community asset and to provide a venue for the community, and youth, in particular, to gather and interact. TIS+ has also undertaken efforts to develop and roll out a national “Sport for Development and Peace” (SDP) strategy through its Technical Coordinators and an outside SDP consultant. According to its 2018 Annual Report, TIS+’s investments in “sports and youth facilities reaches thousands of youth across Somalia and provides them with opportunities to meet, break down barriers of clan and geography, develop a national identity, participate in peaceful social interactions and learn to negotiate losses, compromise and non-violently resolve conflict-all of which directly counter the divisive narrative extremists propagate.”50 The following “contribution story” will critically examine the extent to which TIS+ has contributed to the positive engagement of youth and reduced support for violent extremism through the use of SDP. It is structured according to the five criteria identified by John Mayne, the creator of the Contribution Analysis methodology, to analyze the extent of a “contribution claim” (or “plausible association”) between TIS+ activities and the results witnessed during the midline assessment. These criteria include the following: 1. Plausibility: Is the theory of change (ToC) plausible? 2. Implementation per plan: Has the program been implemented with high fidelity? 3. Evidentiary confirmation of key elements: To what extent are the key elements of the ToC confirmed by new or existing evidence? 4. Identification and examination of other influencing factors: To what extent have other influencing factors been identified and accounted for? 5. Disproof of alternative explanations: To what extent have the most relevant alternative explanations been disproved? The contribution story will then conclude with an overarching assessment of the strength of TIS’s contribution claim to the results relevant to increased youth engagement and reduced support for violent extremism identified in the midline assessment. PLAUSIBILITY OF TIS+’S CPO THEORY OF CHANGE Theory of Change and Underlying Rationale TIS+ did not have a defined ToC specifically focused on SDP at the outset of the program. Rather, the renovation of sports stadiums was covered under its rationale for supporting quick-impact, small-scale infrastructure projects to promote the use and management of shared community assets. Likewise, the hosting of tournaments and other larger-scale sports activities was also covered under TIS+ more general community engagement approach and strategic communication plans. However, in order to enable a Contribution Analysis of TIS+’s results, the assessment team worked with the TIS+ team to develop an SDP ToC in June 2018 (see Figure 5 below). The resulting ToC highlighted some of the key intended results, causal logic, and underlying assumptions guiding TIS+’s SDP-related activities. However, the ToC was rather weak and included substantial leaps in the causal logic and several unrealistic assumptions. Identifying an opportunity to be more strategic in its approach to SDP, as well as wanting to support the Ministry of Youth and Sports (MOYS) develop a national sports policy, TIS+ contracted an outside SDP expert to work with the MOYS to develop a national SDP policy. A final draft of the “Somali Ministry of Youth & Sport (MOYS) Sport for Development and Peace (SDP) Strategy & Implementation Framework” was produced in August 2018. 50 AECOM, “Transition Initiatives for Stabilization Plus: Fy 2018 Annual Progress and Financial Report,” (2019). Pg. 78. 425 Figure 7: TIS+’s SDP Theory of Change 426 The SDP Strategy draws on international good practice and global experience on positive youth development (PYD) through the use of sports and highlights several factors that make it particularly appropriate for the Somali context. The Strategy posits that SDP can help reduce youth propensity toward violent extremism by offering an “attractive alternative” and using sports as a “hook” to attract at-risk youth.51 SDP was also assumed to bring additional benefits, such as physical and mental benefits of physical exercise; a mechanism for developing life and social skills such as teamwork, sportsmanship, and comradery; the possibility to link with educational and vocational opportunities to both attract youth and also strength their capacity and potentially employability; and even to promote community-wide civic engagement through the use of Community Planning and Oversight approaches to the selection, implementation, and oversight of SDP projects, as well as an opportunity to promote shared-asset management between the community and local authorities. The Strategy even identified opportunities to use SDP to promote other cross-sector approaches supporting gender equity, social inclusion, and trauma healing. Figure 6 below provides an overview of the Strategy’s conceptual framework. The Strategy also included an updated ToC behind the intended SDP activities (see textbox below). 51 Paraphrased from Stephen Reynard, “Somali Ministry of Youth & Sport (MOYS) Sport for Development and Peace (SDP) Strategy & Implementation Framework,“ ed. AECOM/TIS+ (August 2018). Figure 8: SDP Conceptual Framework Revised SDP Theory of Change “If the federal and state MOYS are able to implement the Holistic SDP Conceptual Framework, then youth will develop relevant professional, life, and social skills and feel more empowered and connected to local development issues, which will enable them (youth) to counter the negative physical, psychological, and emotional effects of their environments and will help them (youth) to contribute to improved social cohesion, social inclusion, and positive peace.” ~ “Sport for Development and Peace (SDP) Strategy and Implementation Framework” 427 Specific to Preventing and Countering Violent Extremism (P/CVE), SDP was seen as providing a “structure that young people detached from their communities and schools may need. It can also act as a ‘diversionary activity from violent and illicit activities’. Building resilience to radicalization can be viewed in a similar light as crime prevention in that there is ‘crossover in the potential influencing factors that may contribute to an individual becoming radicalized, or engaging in crime.’”52 Relevance with Somali Context and National Efforts Somalia has a long history of using sports as a mechanism for community-mobilization and developing a shared identity (see textbox). Sports have also been used a tool to promote peace by several Somali government ministries, such as MOYS, the Somali Olympic Committee (SOC), the Somali Football Federation (SFF), as well as by a variety of national and international NGOs. Private companies—such as Hormuud Telecom, NationLink Telecom, Premier Bank, and Dahabshiil Money Transfer—have also hosted sports tournaments or funded renovations to community sports fields and stadiums as Corporate Social Responsibility (CSR) and community outreach activities. The use of sports is also widely regarded as an effective way to engage youth, a key demographic with over 70% of Somalia’s population under the age of 30 according to UNICEF Somalia.53 This is reflected in Somalia’s “National Youth Policy” which states that “Sports and recreation are very important factors for youth since this provides opportunities to socialize, build social connections, counter boredom, spend their time productively, develop character and learn new skills and teamwork. They also shape the mental and physical wellbeing of the youth.”54 The Policy recommends several sports-focused interventions, which strongly align with those utilized by TIS+, including (1) “strengthening sports and restarting the interregional, regional and national tournaments for both male and female athletes;” (2) “increase the opportunities for young women and men to engage in a diverse range of sports and sporting competitions including soccer, basketball, volleyball, and athletics;” (3) “provide greater support and direction to youth and sports centers to enable them to become more friendly and conducive to young people with special needs; and” (4) “instigate media awareness programs to raise awareness amongst parents and community and religious leaders of the importance of recreational and sporting programs to the personal development of both young men and women.” Additionally, the TIS+ SDP approach is also consistent with other Somali policy frameworks, including the Somali National Development Plan and the Somalia Youth for Peace Pact. 52 Ibid. 53 UNICEF Somalia, “Education in Somalia.” 54 Ministry of Youth and Sports of the Federal Government of Somalia, “The National Youth Policy of the Federal Government of Somalia.” History of Sport in Somalia 1969–1991: Sport was used as a “mobilization and orientation” tool by the Siad Barre regime, mostly focused in Mogadishu and in a few major cities. All sports clubs were government funded; private sports clubs not encouraged. 1982–1990: Following the war with Ethiopia and subsequent political and economic effects, government-funded sport deteriorated. 1991–2009: Despite civil war and destruction of sports infrastructure, the private sector initiated sports programs to promote peace. Somalia was represented at various international sports competitions. 2010–2016: Banadir Stadium and Aden Yabarow Wish Sports Center were rehabilitated by the Fédération Internationale de Football Association (FIFA) and the International Olympic Committee (IOC); Sports for Peace was included in SOC’s strategic plan. 2017 – present: Development of 1st National Sports Policy (pending Cabinet approval). ~ “Sport for Development and Peace (SDP) Strategy and Implementation Framework” 428 Evidence from Other SDP Projects There are a number of studies that have examined the use of SDP both in Somalia and across the globe. An independent review of the International Labor Organization’s (ILO’s) “Youth for Change” project, which worked with over 6,000 youths determined to be a risk of Al-Shabaab recruitment, found that the project helped “ facilitated healthy competition and cohesion amongst the groups” and helped “reduce aggression among participants.”55 Further, an independent evaluation of the USAID-funded “Somalis Harmonizing Inter￾and Intra-Communal Relationships” (SHIIR) program implemented by PACT found that the program contributed to positive changes regarding the beliefs and attitudes of the 480 male youth participants towards other clans” but that “to have a meaningful impact on youth, the soccer drills should be accompanied by an in-depth educational program with a long-term perspective.”56 The MOYS SDP Strategy and Implementation Framework also cited a number of other international studies that demonstrates SDP’s potential to building social cohesion and promote “positive peace,” mend “broken social relationships,” promote psychosocial healing, address social exclusion, and challenge negative gender stereotypes and mitigate potential gender￾based violence. Assessment of Plausibility: Plausible The updated SDP ToC developed by TIS+ presents a convincing argument supporting its intended results, causal linkages, and underlying assumptions. The ToC is aligned to the Somali context and follows a deep and widespread history of using sports to promote community engagement, cohesion, and outreach. It is also aligned with host-government strategies such as the National Youth Policy and obviously also the SDP Strategy within which it is presented. Finally, the ToC is also supported by evidence from independent evaluations and secondary studies of SDP-related programs in both Somalia and across the globe. FIDELITY OF IMPLEMENTATION TIS+ has implemented a number SDP-related activities, including the construction or renovation of football and basketball stadiums, organization of tournaments and other sports events, the establishment of sport facility Project Oversight Committees, and also the distribution of sports equipment to both community and government stakeholders. Figure 7 below provides an overview of all SDP-related PIKAs implemented at the time of the midline assessment. Figure 9: List of SDP-related PIKAs Location PIKA No. and Title Description Status at Midline Abdiaziz (BRA004) BRA Districts Sports Facilities and Youth Engagement 59-member District Sports Facilities & Programming Bridging Plan Task Force (BRA, district consensus groups, technical experts, donor/partner organizations and civil society representatives) formed Completed Abdiaziz (BRA012) BRA Abdiaziz District Basketball Mini Stadia Rehabilitation Rehabilitation of basketball stadium, setting up and training IFOC (formerly POC) Ongoing Hawlwadag (BRA013) Mogadishu Hawlwadag District Basketball Court Rehabilitation Rehabilitation of basketball stadium, setting up and training IFOC Ongoing Warta Nabada (BRA011) Mogadishu Warta Nabada District Basketball Court Rehabilitation Rehabilitation of basketball stadium, setting up and training IFOC Ongoing Kismayo (JSS013) JSS Kismayo National Youth Day Youth Engagement National Youth Day support with football, basketball, bicycle race, peace march in Kismayo Closed Kismayo (JSS026) JSS Ministry of Youth and Sports Technical Coordinator/Sports Technical Coordinator support Ongoing 55 Lilla Schumicky-Logan, “Addressing Violent Extremism with a Different Approach: The Empirical Case of at-risk and Vulnerable Youth in Somalia,” Journal of Peacebuilding & Development (August 2017). 56 Reynard. 429 Garowe (PSS002) PSS Garowe Town Basketball Court Upgrading Construction of basketball stadium Ongoing Baidoa (SWS008) SWS Ministry of Youth and Sports Baidoa Youth Day Supported National Youth Day celebration Closed Dinsoor (SWS016) SWS Dinsoor District Football Stadium Construction Construction of football stadium; setting up and training IFOC Completed Baraawe (SWS015) SWS Ministry of Youth and Sports Baraawe District Football Stadium Construction Development of engineering documents Ongoing National (SOM014) Somalia Somali Women’s Basketball Support Regional Training 8 training sessions for already established women/girls’ basketball teams; providing new equipment, uniforms, and training in preparation for their participation in a national tournament (96 girls trained) Closed National (SOM015) Somalia Somali National Women’s Basketball Tournament A national women’s basketball tournament, with 8 regional teams (96 young women) participating, and the selection of a women’s national team Closed National (SOM028) Somalia Sports for Development and Peace Framework STTA STTA for FGS MOYS Sport for Development and Peace Framework and a rollout plan for Somalia in consultation with the FMSs MOYS and Sports Federations; SDP approach for TIS+ produced Ongoing In 2016, TIS+ supported preparatory basketball training camps for 119 young women (including in Kismayo) and then supported the Somali National Olympic Committee to organize a national women’s basketball tournament, which took place in the court rehabilitated by TIS+ in Garowe. Working through federal and regional MOYS, TIS+ supported logistics, transport, and accommodation for the second national men’s soccer tournament in December 2017. Both these tournaments generated significant interest and impact on new social media. Respondents interviewed during the midline assessment confirmed that the men’s soccer tournament had been a major event that the nation had stopped to watch. However, as evident in the PIKA list above, the majority of TIS+’s SDP-related activities have focused on the construction or renovation of simple community sports facilities. Only one of these facilities has finished construction at the time of the midline and unfortunately, that facility, a sports stadium in Dinsoor, was blown up by AS militants and almost completely destroyed. As noted in the ToC section, TIS+ did not have a defined strategy to guide its SDP work until mid-2018. This has resulted in a trend for most SDP activities to be stand-alone with few follow-on or complementary activities to support the achievement of higher-level community cohesion and youth engagement results. That said, the wide range of actors currently supporting SPD activities or interested in support SDP activities in the future, combined with capacity challenges in the MOYS, suggest that the TIS+ approach of working on a strategic framework for SDP for MOYS, as well as providing technical support to MOYS, was appropriate. Assessment of Fidelity: Weak. TIS+’s implementation of the SDP strategy and ToC has to date been limited mostly due to the recent development of these strategies. Interviews with TIS+ staff as well as evidence recent periodic program reports suggest that the program is committed to its new SDP strategy and endeavor to undertake a more systemic approach to implementing activities following this strategy. EVIDENCE SUPPORTING INTENDED RESULTS According to TIS+’s 2018 Annual Report, the program has achieved the following SDP-related results: • Facilitate the development and rollout of the Somalia Sports for Development and Peace (SDP) Framework while utilizing TIS+ investments in sports facilities and sectors. • Complete 20 infrastructure projects: 5 sports stadia. 430 • Hold 4 special celebrations including Somali National Youth Day and the International Day of Sport for Development and Peace in Baidoa (SWS), Galkayo (PSS) and Kismayo (JSS), with 1 celebration held in Q4. • Launch one National Sports for Development Framework in Mogadishu. • 1 Men’s National Soccer Tournament in Mogadishu attended by over 30,000 people. • Capacity-building and technical support provided by TIS+ Technical Coordinator to the Jubbaland Ministry of Youth and Sports. However, evidence from the midline assessment indicates that these output-level results have not yet translated into clear outcome-level results. There was a consensus among key informants working on sports￾related activities that conducting or facilitating sports events alone was not enough to deliver desired results. Physical education does deliver positive health benefits by itself, but the SDP framework is not a health intervention first and foremost. There may be some inherent benefits for social cohesion of providing sports venues and events. As one business leader in Kismayo noted: “People from different clans meet and interact with each other in places such as the sports stadiums, restaurants, businesses, and mosques. These venues provide an opportunity for people with different views to exchange ideas, learn from each other, reconcile and bridge differences.” Additionally, simply having sports venues without additional supporting activities was regarded by interviewees as insufficient for achieving higher-level results. “Awareness creation is important as well as having stadiums and youth centers.” This message was reinforced by several interviewees who had worked on, or had an interest in, sports. As one specialist in youth programming put it, “It is important not just to support events—it has to be more than kicking a football around.” It was seen as very important for these interventions to be conducted in a purposive way. As much as bringing people together in events and tournaments could support a sense of shared identity, these events needed to be more intentional in their design to be fully inclusive, as well as to avoid potential perverse outcomes from interstate games that could exacerbate regional tensions. The recent SDP approach developed by TIS+ focuses on using sport to deliver or to boost the delivery of other skills and messages. These can be through side-by-side programming in which educational, civic, or vocational training is provided alongside sport, or through “pedagogical sports drills” in which sport training sessions are used as a platform to diffuse educational and life skills messages. This approach appears to have been used by TIS+ in its support to the men’s football tournament in December 2017: “The Sports for Development and Peace approach uses sports to create social change, to facilitate community development, to increase cohesion, and to support peace and conflict mitigation efforts. . . Four SDP drills were conducted to show how working together can facilitate healthy competition that will extend off the playing field. . . . Before today we were just playing football for health and personal skills, but today we have learned that sports are a great tool for mitigating conflict and fostering peace.” The main challenge to the suggested mechanism in the SDP results chain is the focus on individual youth choices and decisions. This assumption was seen as crucial by some of our expert, peer donor, and program interviewees. As one such interviewee noted: “As for sports for development and peace, I don’t know if this would work according to my understanding of how recruitment happens. If we are saying sports will keep youth distracted and keep them away from AS – perhaps this could work when we have heard some cases of individual recruitment. But these are not many. In many cases, recruitment to AS is a collective decision, for example of a sub-clan who decides to franchise with Al-Shabaab because they think that in their particular circumstances, they best represent their interests. When that happens, it is the whole clan militia that is franchised to Al-Shabaab. They have to undergo training, most of which is ideological indoctrination, and out of this process a good proportion become hardcore jihadi extremists.” Related to gender, midline interviews suggested that Somali society is generally accepting of men’s participation in sports events. However, the participation of women and girls is much more controversial. 431 AS prohibits women and girls from playing sports, but disapproval of women and girls’ participation is much wider than AS. For example, before the TIS+-supported basketball tournament in Garowe in December 2016, the Somali Religious Council, a group of influential Somali clerics, issued a warning to women not to engage in sports: “We warn that the women basketball violates Islamic law, culture, and its values, and it is a place where women can be easily corrupted.”57 The tournament was conducted with high-security costs. From a “do no harm” perspective, this has clear implications for the protection of women and girls’ safety during sports activities and their protection from subsequent stigma or threats because of their involvement in such activities. The TIS+ 2016 basketball tournament and the training camps that preceded them are some examples of interventions targeted specifically at women and girls. TIS+ supported basketball training camps that involved 119 young women between the ages of 16 and 25 who came from seven different regions (states) in Somalia. TIS+ supported the Somali National Olympic Committee to organize a national women’s basketball tournament, the first such event in over two decades. The tournament was held on a TIS+ rehabilitated basketball court in Garowe and was watched by an estimated 900 viewers per day, many of whom were women. In addition to the direct participants and spectators, the tournament reached over one million people through social media was broadcast live on national and regional television, and sparked significant debate about women and sports across Somalia. Our respondents confirmed that these interventions had had a sustainable impact on women’s’ basketball in Somalia, which was progressing at a federal level, as well as at the Jubbaland level, where women’s basketball teams were still playing. Other respondents confirmed the huge positive impact that access to sport could have on women and girls who previously had no opportunity to engage in physical activity. Assessment of Supporting Evidence: Weak. TIS+’s SPD-related results to date have been primarily at the output level and there is little evidence of TIS+ implementing an integrated and intentional strategy that explores and utilizes potential complementarities between activities. That said, the development of the SDP Strategy and Implementation Framework is a key recent result and is expected to help systematize TIS+’s SDP approach going forward and increase the prospects of achieving outcome-level results. INFLUENCING FACTORS AND ALTERNATIVE EXPLANATIONS Our assessment indicates that although many stabilization and development actors have supported sports activities often in an ad hoc manner, efforts to develop a more consolidated programming approach to sport interventions are nascent. TIS+’s own stakeholder mapping, conducted by the SDP consultant in 2018, revealed a range of actors working on sports in Somalia and identified a wide range of actors: ● UNSOM, UNFPA, and UN-Habitat, which use sport as a tool to engage women and youth ● The EU, which is planning to build cultural centers with sport as a component ● Somalia Olympic Committee, which is planning to develop a Sports for Peace program (2017– 2020) ● The Pact SHIIR program, which piloted pedagogical sports drills in Jubbaland ● IOM, which has used sports interventions in its programs for rehabilitating AS defectors ● CARE Somalia, Mercy Corps, and Concern Worldwide, which were also identified as working or having worked on sports58 ● Private sector companies such as Hormuud Telecom, NationLink Telecom, and Premier Bank, and Dahabshiil Money Transfer, which have sponsored the Somali Olympic Committee, and 57 https://www.voanews.com/a/somali-clerics-warn-women-aginst-playing-basketball/3646679.html 58 TIS+. (2018). Sport for Development and Peace Strategy and Implementation Framework stakeholder mapping PowerPoint presentation. 432 Somali Football Federation, and SSF, as well as conducted a CSR program focused on education and health in the Horn of Africa ● Other donor, program, or implementing partner actors such as US OTI, GIZ, SSF, and the NISF, who have worked on sports interventions or have plans to do so. This range of actors who have worked/are working on sports interventions are certain to affect TIS+ future results and there was some evidence that these initiatives have already influenced perceptions of TIS+’s activities. The Kismayo sports stadium, mentioned by several Kismayo interviewees as a positive asset, was reconstructed by NISF in 2014 with UK funding and has more recently been upgraded in 2017 by the American Refugee Committee and the United Nations High Commission for Refugees. Sports tournaments were also mentioned by respondents as positive developments. TIS+ will certainly have contributed to these through its basketball tournament (which was in 2016) and the men’s soccer tournament in 2017.59 However, there are many other actors supporting tournaments, which may have had a larger effect on responses because they are more recent. For example, in May 2018, Save the Children Somalia, in partnership with MOYS, co-organized an interschool sports tournament for 14 public schools. The football and athletics tournament were funded by the German Federal Foreign Office.60 Results are also likely to be substantially affected by factors in the political economic and security context that affect sports events specifically. There are many supporting factors in the Somali environment that would enhance the effectiveness of sports interventions for youth. The fact that Somalis are big sports fans was mentioned by a number of expert, peer donor, and program interviewees. A number of respondents noted that there were positive Somali role models on the international sporting arena, such as Mo Farah (athletics), Abdi Bile (athletics), Maryam Nah Muse (athletics), Abdisalam Ibrahim (soccer), and Mukhtar Ali (soccer), and other Muslim sporting heroes such as Mo Salah who were wildly popular in Somalia. A further positive factor, also made by TIS+, is the existence of several private sector sports football stadiums and investors in sports events, including banks and telecom companies. This means that sports facilities and events would take place without donor intervention in Somalia. These interventions are sustainable after the initial investment. There are also threatening and challenging factors in the Somali environment that negatively affect sporting interventions and pose serious risks to them. These are related to AS threats and attacks and to the broader acceptance of women’s involvement in sport. This is a threat is exemplified by the AS attack on the TIS+ Dinsoor stadium in 2018. It is also demonstrated by the closure of 30 football fields in Mogadishu follow threats by AS. AS also handed down terms in AS prisons for some of those defying these orders.61 These orders are an effective way for AS to demonstrate and project its symbolic power into areas not formally under its control. There have also been AS attacks on sporting events, such as the bombing of a friendly match in a Baraawe football stadium in April 2018, which killed five people.62 This does not indicate that AS has an inherent opposition to football. There have been stories reporting that AS allows and encourages the playing of “halal” football between AS fighters and local teams in areas under its control.63 However, it does illustrate the security risks of public events and venues, especially those connected to the government, and the need to protect participants. Assessment of Influence of Alternative Explanations: Moderate. There are several other actors who are engaged in sports activities in Somalia, including donors and development programs, but much of this work has been ad hoc in nature. TIS+ could maximize its impact in this area by playing a coordination role among donors and programs, working with the MOYS. The popularity of the sport among Somalis and the proliferation of private sector venues suggest that sport would be positively regarded by Somalis with or without interventions by donors and programs, and are likely to 59 https://www.kawowo.com/2017/12/24/jubbaland-wins-somalia-inter-states-championship/ 60 https://somalia.savethechildren.net/news/enhancing-integration-through-inter-school-football-athletics-tournament-kismayo 61 https://www.aljazeera.com/indepth/features/somalia-al-shabab-shuts-football-pitches-mogadishu-180607045144830.html 62 https://www.aljazeera.com/news/2018/04/blast-somalia-stadium-kills-football-fans-180413070232458.html 63 https://www.aljazeera.com/blogs/africa/2014/06/99086.html 433 enhance the impact of interventions. However, AS attacks pose a risk to sports venues and events, and social and pervasive resistance to the participation of women in sports threatens to undermine the objectives of sports interventions, especially those engaging women. ASSESSMENT OF CONTRIBUTION CLAIM Summary of Assessment Criteria Criteria Assessment Plausibility: Is the theory of change plausible? Plausible. The updated SDP ToC developed by TIS+ presents a convincing argument supporting its intended results, causal linkages, and underlying assumptions. It is aligned to the Somali context and government strategies such as the National Youth Policy. It is also supported by evidence from independent evaluations and secondary studies of SDP-related programs in both Somalia and across the globe. Implementation per plan: Has the program been implemented with high fidelity? Weak. TIS+’s implementation of the SDP strategy and ToC has to date been limited mostly due to the recent development of these strategies. Evidentiary confirmation of key elements: To what extent are the key elements of the theory of change confirmed by new or existing evidence? Weak. TIS+’s SPD-related results to date have been primarily at the output level and there is little evidence of TIS+ implementing an integrated and intentional strategy that explores and utilizes potential complementarities between activities. That said, the development of the SDP Strategy and Implementation Framework is a key recent result and is expected to help systematize TIS+’s SDP approach going forward and increase the prospects of achieving outcome-level results. Identification and examination of other influencing factors: To what extent have other influencing factors been identified? Identification of alternative explanations: To what extent do alternative explanations explain results? Moderate. There are several other actors who are engaged in sports activities in Somalia, including donors and development programs, but much of this work has been ad hoc in nature. TIS+ could maximize its impact in this area by playing a coordination role among donors and programs, working with the MOYS. However, AS attacks pose a risk to sports venues and events, and social and pervasive resistance to the participation of women in sports threatens to undermine the objectives of sports interventions, especially those engaging women. Summary Assessment of Contribution Claim: Weak. The midline assessment found that any potential claim for TIS+’s SPD activities to have influenced higher￾order outcomes related to community cohesion or positive youth development to date are not well substantiated by current evidence. However, there is room for optimism as TIS+’s recent SDP Strategy and Implementation Framework demonstrated not only a commitment to more systematic link SDP activities to achieve higher-level results but also provides a concrete road map for how to do so. It is expected that TIS+’s contribution claim around its SDP results is likely to be strengthened in the coming years. 434 ANNEX G – Bibliography of Critical Background Documents AECOM. (2016). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-01, October to December 2015). AECOM. (2016). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-02, January to March 2016). AECOM. (2016). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-03, April to June 2016). AECOM. (2016). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-04, July to September 2016). AECOM. (2016). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Annual Progress and Financial Report, October 2015 to September 2016). AECOM. (2017). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-01, October to December 2016). AECOM. (2017). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-02, January to March 2017). AECOM. (2017). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-03, April to June 2017). AECOM. (2017). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-04, July to September 2017). AECOM. (2017). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Annual Progress and Financial Report, October 2016 to September 2017). AECOM. (2018). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-01, October to December 2018). AECOM. (2018). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-02, January to March 2018). AECOM. (2018). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-03, April to June 2018). AECOM. (2018). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Quarterly Progress and Financial Report- Q-04, July to September 2018). AECOM. (2018). “Transition Initiatives for Stabilization Plus (TIS+) Project” (USAID Annual Progress and Financial Report, October 2016 to September 2018). AECOM. Transition Initiatives for Stabilization Plus (TIS+ Program). Gender Equality and Women’s Empowerment Strategy (September 2015). AECOM. Transition Initiatives for Stabilization Plus (TIS+ Program). Process of Engagement (September 2015). 435 AECOM. Transition Initiatives for Stabilization Plus (TIS+ Program). Coordination Plan (February 2016). AECOM. Transition Initiatives for Stabilization Plus (TIS+) Program. Year- 01, Annual Work Plan. (May 2015 - September 2016). AECOM (March 2016) Community Consensus Building in Barawe, South West State (SWS), Workshop Report. AECOM Jubaland State of Somalia District and Regional Stability Assessment Report (December 2015). AECOM (2016) Monitoring and Evaluation Plan. AECOM (2017) Monitoring and Evaluation Plan. AECOM (2018) Monitoring and Evaluation Plan. Conflict Early Warning Response Unit, with support from Conflict Dynamics (2014) ‘From the bottom up: Southern Regions - Perspectives through conflict analysis and key political actors’ mapping of Gedo, Middle Juba, Lower Juba, and Lower Shabelle’, One-off paper re-released in 2014. Goldwyn, Rachel & Chigas, Diana (2013) ‘Monitoring and Evaluating Conflict Sensitivity: Methodological Challenges and Practical Solutions’ DFID, CARE and CDE Collaborative Learning Projects Paper. Heritage Institute (2014) ‘Perceptions of Security and Justice in Mogadishu’ IOM/Altai (2016) Youth, employment and migration in Mogadishu, Kismayo and Baidoa. Itad/Altai Consulting (May 2017) ‘Mapping of 2016-2017 Stabilization Projects in Somalia’ Report produced by the the UK Conflict, Security and Stabilization (CSSF) Somalia Program M&E Unit. Lemire, Sebastien et al. (2012) ‘Making Contribution Analysis Work: A Practical Framework for Handling Influencing Factors and Alternative Explanations’ Evaluation 18(3) pp.294-309. Quinn Patton, Michael (2008) Utilization-Focused Evaluation: The New Century. 4th edition. Thousand Oaks: Sage Publications. Rogers, Mark (2012) ‘Selecting Evaluation Approaches: Options in Peacebuilding Evaluation’ Care International Paper. Rift Valley Institute and Heritage Institute for Policy Studies (2017) Land Matters in Mogadishu: Settlement Ownership, and Displacement in a Contested City. Saferworld (2016) ‘Forging Jubaland: Community Perspectives on Federalism, Governance and Reconciliation.’ Somalia Conflict Early Warning Early Response Unit Final Report (September 2013). TIS+ (February 2016) Coordination Plan Document. USAID Strategic Rationales for Afmadow, Barawe, Baardheere and Dinsoor. 436 World Bank Group (2016) Aid Flows in Somalia – Analysis of Aid Flow Data February 2016. Vogel, Isabel (2012) ‘Review of the use of Theory of Change in International Development,’ Paper for DFID. Van Es, Marian et al. (2015)’ Theory of Change Thinking in Practice’ Hivos paper UN OCHA Somalia Drought Situation Reports – various. UNOCHA (April 2017) Map of 289 UN agencies and NGOs implementing activities in Somalia. USAID | Somalia. (2016). Fact Sheet: Transition Initiatives for Stabilization (TIS), As of February 2010- March 2016. WASH cluster, WASH Vulnerability Analysis (May 2017). Wimbush, Erica et al. (2012) ‘Applications of contribution analysis to outcome planning and impact evaluation’ Evaluation 2012 18: 310.