APRIL 2017 This publication was produced for review by the United States Agency for International Development. It was prepared by for Management Systems International. ASSESSMENT OF USAID/ SOUTH SUDAN’S VIABLE SUPPORT TO TRANSITION AND STABILITY (VISTAS) REPORT ii ASSESSMENT OF USAID/ SOUTH SUDAN’S VIABLE SUPPORT TO TRANSITION AND STABILITY (VISTAS) Contracted under IDIQ No: AID-668-I-13-00001 / Task Order No: AID-TO-13-00001 South Sudan Monitoring and Evaluation Support Project DISCLAIMER This Task Order Plan is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of the MSI and do not necessarily reflect the views of USAID or the United States Government. 3 CONTENTS ACRONYMS............................................................................................................ 5 EXECUTIVE SUMMARY ....................................................................................... 6 Recommendations for USAID .................................................................................................................................11 Recommendations for VISTAS ................................................................................................................................12 INTRODUCTION................................................................................................. 12 Background...................................................................................................................................................................12 Assessment Purpose and Audience........................................................................................................................14 Assessment Questions (See Annex for Details)..................................................................................................14 CONTEXT ANALYSIS........................................................................................ 14 Political Trends............................................................................................................................................................14 Economic Trends........................................................................................................................................................16 THE VISTAS ASSESSMENT............................................................................... 17 Scope of the Assessment ..........................................................................................................................................17 Methodology ................................................................................................................................................................17 Sampling.........................................................................................................................................................................17 Data collection.............................................................................................................................................................17 Data analysis.................................................................................................................................................................18 Strengths and Limitations..........................................................................................................................................18 FINDINGS AND CONCLUSIONS..................................................................... 19 Question 1: Is the current program strategy relevant and appropriate in the South Sudan context and the context of USAID/South Sudan’s strategy? ...................................................................................................19 Question 2: Program Effectiveness.........................................................................................................................28 CONCLUSIONS ................................................................................................... 35 LESSONS LEARNED & RECOMMENDATIONS............................................. 37 Lessons learned ...........................................................................................................................................................37 Recommendations for USAID .................................................................................................................................38 Recommendations for VISTAS ................................................................................................................................39 Scenarios.......................................................................................................................................................................40 General ..........................................................................................................................................................................40 ANNEX I................................................................................................................ 42 Background...................................................................................................................................................................42 4 Assessment Purpose...................................................................................................................................................42 Existing performance information sources...........................................................................................................43 Assessment Questions...............................................................................................................................................43 Methodology ................................................................................................................................................................43 ANNEX II............................................................................................................... 45 ANNEX III ............................................................................................................. 49 ANNEX IV ............................................................................................................. 52 ANNEX V .............................................................................................................. 53 ANNEX VI ............................................................................................................. 54 ANNEX VII............................................................................................................ 55 5 ACRONYMS AECOM AECOM International Development ARCISS Agreement on Resolution of the Conflict in the Republic of South Sudan AU African Union CBOs Community-Based Organizations CPA Comprehensive Peace Agreement CSOs Civil Society Organizations CSV Comma-separated Values DPs Development Partners FAAs Fixed Amount Awards FAO Food and Agriculture Organization of the United Nations FGDs Focus Group Discussions GMs Group Meetings HOM Heads of Mission HOC Heads of Cooperation HQ Headquarters JBPDA Joint Border Peace Development Agency JICA Japan International Cooperation Agency KIIs Key Informant Interviews M&E Monitoring and Evaluation MESP Monitoring and Evaluation Support Project OF Operational Framework OTCM Office of Transition and Conflict Mitigation OTI Office of Transition Initiatives POC Protection of Civilians RAs Rolling Assessments RPM Regional Program Manager SPLM Sudan People's Liberation Movement SRS Strategic Review Session TO Task Order TOC Theory of Change USAID United States Agency for International Development USG US Government VISTAS Viable Support to Transition and Stability WFP World Food Program 6 EXECUTIVE SUMMARY Implemented by AECOM International Development (AECOM) since July 2013, the goal of the Viable Support to Transition and Stability Project (VISTAS) was initially “to strengthen South Sudanese confidence and capacity to address the causes and consequences of political conflict, violence, and instability.” In 2013, a power struggle originated at the pinnacle of state power and reverberated across South Sudan — changing the field for VISTAS activities as USAID shifted focus from building capacity of state institutions to community building. The VISTAS goal was changed and became “to prevent the further spread of communal violence and tensions in critical areas where conflict may have national implications.” In the states, VISTAS pivoted from supporting local government to mitigating communal violence and rising tensions through non-state actors. Managed by USAID’s Office of Transition Initiatives (OTI) as of June 2014, the Project returned to USAID/South Sudan management under its Office of Transition and Conflict Mitigation (OTCM) in January 2016. This assessment documents the Project’s s strategic relevance, effectiveness, and lessons learned to make recommendations about VISTAS’s direction during the final two years of the Project and beyond. The study focused on VISTAS at the headquarters (HQ) and field offices. Interviews were also conducted with USAID and AECOM HQ. Primary data collection involved key informant interviews (KIIs), group meetings (GMs), and focus group discussions (FGDs) with staff of USAID, VISTAS, and grantees as well as beneficiaries and community members. Secondary data collection involved review of the activity databases and document review. The assessment found that VISTAS’s strategic relevance and effectiveness depend primarily on: having a smart Theory of Change, careful layering of Project activities, solid partnerships, operational capabilities, purposeful geographic targeting, and a solid approach to including gender. The study found that VISTAS performs well in these areas and—with minor adjustments and support—will continue to improve performance. The recommendations in this study highlight areas where strong performance can be maintained and improved. Regarding the first assessment question on the strategic relevance of the VISTAS Project to South Sudan and to US Government (USG) policies for humanitarian and development assistance, the study produced concrete findings based in evidence. Although the question was revised to focus more on the USAID Operational Framework (OF) than on USG policy writ large, the Team found no way to look at the OF without also considering USG policy. RELEVANCE OF VISTAS TO COMMUNITIES VISTAS grantees and beneficiaries appreciate the Project for its meaningful impact within their communities and cite tangible benefits that they associate directly with the VISTAS Project. Moreover, community members request that USAID expand and deepen its presence. VISTAS is a strategic resource for the USG in South Sudan that has maintained goodwill and an operational footprint in some of the most remote areas of the country, even at the height of the recent conflict, with national and 7 international staff working in solidarity. This point concerns both strategic relevance and Project effectiveness. IMPACT OF USG POLICY SHIFT TO VISTAS PROGRAMMING The USG 2014 policy shift away from state institutions and towards communities has been interpreted and implemented differently across VISTAS activities, but is generally understood to restrict VISTAS cooperation with the Government of South Sudan (GOSS), even at the local level. If possible VISTAS would opt to continue supporting local governments to accomplish more durable results and believe their managers can follow a nuanced approach in identifying partnerships. To resume support for local governments, VISTAS PROGRAMMING WOULD BENEFIT FROM USAID’S GUIDANCE REGARDING THE NUANCES OF APPLYING THE OPERATIONAL FRAMEWORK AT THE LOCAL LEVEL. RELEVANCE OF VISTAS TO USAID THEORY OF CHANGE (TOC) The VISTAS Theory of Change (TOC) is aligned to the Operational Framework (OF), particularly TOs 2.2 (Strengthen inter- and intra-communal relations and reconciliation); 2.3 (Increase space for conflict mediation); and 1.1 (Facilitate community-led response), in that order of significance; and, to a lesser extent to 1.3 (Strengthen Livelihoods and Resilience to Shock); with further possible alignment with 2.1 (Support inclusive peace process), clearly establishing the strategic value of VISTAS. However, there is little understanding of the OF within the VISTAS teams. VISTAS’ community work has the potential to serve as an incubator for pilot activities, which other USAID implementing partners could upscale successful activities over time, particularly in the areas of livelihoods and youth work. To increase understanding and collaboration in this regard, USAID could host occasional VISTAS presentations at quarterly meetings of Implementing Partners, Heads of Mission, Heads of Cooperation, Donor Working Groups, and USAID team meetings. The results of such collaboration could benefit both VISTAS and the Projects of other USAID Implementing Partners. RELEVANCE OF VISTAS TO THE CONTEXT OF CIVIL WAR Many VISTAS staff question the continued relevance of VISTAS’ strategy since the country descended into civil war in 2013. VISTAS’ dynamic innovation platform currently aggregates activities into regional plans, which are then intended to cohere into the overall Project plan. The continued relevance of the VISTAS Project to the context of civil war requires examining the connection between the Project goal (the starting point to strategy) and how it connects to or reflects regional objectives and activity sets. It also requires maintaining technical capacity with strong knowledge of South Sudan’s conflict dynamics at the Project level to ensure continued success. RELEVANCE OF LOCAL GOVERNMENT TO VISTAS STRATEGY As mentioned, most respondents reported that cooperation with local government is necessary for Project effectiveness because of the influence of local government officials and due to the military orientation and kinship structures of South Sudan. Even if activities are not directly supporting the 8 strengthening of local government managers cannot ignore the local government and still accomplish project objectives. It is important for USAID to clarify the parameters under which VISTAS should and should not engage with local government institutions and/or officials. RELEVANCE OF INFORMATION MANAGEMENT TO VISTAS STRATEGY EXECUTION There is more of a bottom-up flow of information within VISTAS in lessons, and ideas, but with less emphasis on processing information (analytics) or to support learning and adaptive management. VISTAS has planning, management, and review mechanisms such as Deep Dives, Strategy Review Sessions (SRSs), and Rolling Assessments (RAs) that are valuable programmatically and operationally for aligning strategy and operations. These provide platforms for learning and adaptive management, which support strategy planning and execution to align strategy and operations. With some adjustments, executive direction, and technical and management support, the Project has the potential to consolidate a more strategic approach to conflict mitigation, with greater coherence across regional, national, and local levels. “VISTAS should define its information needs at different levels of the program and improve the analytics capacity of the database to meet the identified information needs. Currently, the database is mainly used for activity approvals. This database can be redesigned to improve its analytics capabilities to generate monitoring information that supports organizational learning and spur performance improvements.” VISTAS may consider a mapping of stakeholder information needs and designing/improving the reporting capacity of the database to produce reports that respond to those information needs with predefined “push button” reports and “a query” facility to produce reports that respond to the unforeseen information needs. RELEVANCE OF VISTAS GEOGRAPHIC FOCUS The overall geographic focus is a remnant of the period of stabilization under the Comprehensive Peace Agreement (CPA); however, now there is an evolving orientation towards Equatoria and an increased national focus. Based on the above findings, the Team concluded that overall the USG is succeeding in building solidarity with communities on behalf of the American people (although USAID branding could be improved) through VISTAS, which is the product of a generally successful geographic focus. The geographic shifts represent an expansion rather than a strategic re-orientation and include the Protection of Civilian camps (POCs). On occasion, activities targeting the national level conflict have been undertaken directly, but these are comparatively few. VISTAS are considering to what extent VISTAS needs to leave areas where the Project has worked for a long time with some success to address new hotspots. Respondents had differences of opinion, but it was obvious that it is hard to determine geographic focus without a more comprehensive mapping of how the national conflict intersects with local conflicts, and whether (and where) that nexus is conducive to VISTAS intervention. For example, if VISTAS was to prioritize areas where two tribes intersect over water and grazing, and where these tribes have leaders in Juba who are competing for political advantage, then we could assess a conflict map from that 9 standpoint, even identifying how patronage may operate and what the chances are for building peace coalitions through dialogue. These are issues that could be teased out in a strategic planning process, which would include more concerted conflict analysis. USAID guidance is required to assist with these decisions. EFFECTIVENESS OF DIVISION OF ROLES AND RESPONSIBILITIES In line with the OTI division of roles and responsibilities, VISTAS senior managers are generally strong on operations and look to USAID for overall strategic direction and oversight. The highly decentralized and small grant-making system provides significant space for local decision￾making by Regional Program Managers (RPMs) and allows VISTAS to be both sensitive to the local context and responsive to emergent issues. The challenge remains for VISTAS to fit more seamlessly into the Mission’s Operational Framework (OF) without losing the strength of its bottom-up character. USAID could use VISTAS to fully carry out the OF, but this requires a more systematic approach with deeper engagement of USAID in Juba regarding policy, operational guidance, and coordination. In response to operational challenges, VISTAS pivots and adapts based on local developments, drawing upon the flexibility of the decentralized, in-kind grant-making mechanism, the determination of field staff, and the trust they have built with local partners. High turnover in key management positions within both VISTAS and USAID as well as the 2013 and 2016 evacuations have made it difficult to operate with a consistent vision and oversight. VISTAS, however, have strived to do their best under these conditions and in the face of short durations of assignment. EFFECTIVENESS OF ACTIVITY SELECTION AND LAYERING VISTAS most commonly cited activities with regards to effectiveness deal with traditional chiefs, women, and/or cattle-related issues (especially migrations) of both moral (compensation under customary law/restorative justice) and market (trade) natures. However, even activities focusing on women and traditional chiefs need the support or at least the acquiescence of local officials. VISTAS is just beginning to layer activities; however, the concept and its application are only partially understood.1 For example, Northern Bahr el Ghazal is regarded as a flagship program that evidenced “layering.” In many areas activities can be related in principle, but are not designed with the intent to build synergies. Hence, layering appears to be both de facto and intentional--VISTAS can always become more intentional in this regard. Based on these findings, the Team concluded that VISTAS does best when it supports community initiatives aimed at peace, recovery, and stability, supporting collective actions that unite communities. Successful VISTAS activity examples include pre-migration and post-migration peace conferences, dissemination of agreements on grazing rights, cattle vaccinations and dipping pens, livestock markets, and hand-dug well construction. These activities are best evidenced in Northern Bahr el Ghazal where the VISTAS team has used layering to create innovative synergies around migration-centered issues. Part 1 This study has interpreted “layering” to mean, “when separate activities contribute to common outputs and outcomes and/or build other synergies that multiply or increase impact.” 10 of the success seems to be the centrality of cattle to livelihoods and culture, given that key stakeholders in VISTAS activities are agro-pastoral communities. EFFECTIVENESS OF THE VISTAS APPROACH TO PARTNERSHIPS Respondents generally indicated that they believe there would be benefits to a robust partnership strategy within VISTAS, particularly with other USAID implementing partners, which would be led and encouraged by the USAID mission. Northern Bahr el Ghazal is a model to be replicated and improved upon. VISTAS has explained the model in generic terms with a Power Point presentation to the Mission. Even within the NBEG model, VISTAS’ main challenge is identifying reliable partners, building trusting relationships with them, and getting resources to them efficiently while preserving accountability. VISTAS performs best when it builds relationships spanning at least two years, evidenced by VISTAS along the North-South border. Resources, however, are spread thin across loosely connected activities and regions. As of March 31, 2017, VISTAS had cleared 378 grants (completed and closed).These figures indicate that VISTAS has undertaken many small activities, and when triangulated against data from the respondents, seems to indicate that the Project has spread resources thin. However, it is also true that small projects used strategically can have big impact. The assessment Team believes that this calls for more strategic targeting of resources into the activities with the greatest potential to contribute to local conflict mitigation, and with consideration to layering activities to achieve greater coherence. It is always possible that VISTAS support can be misinterpreted or politicized in South Sudan because dialogue may address political issues and involve actors with known political positions and interests. In an environment of civil war and extreme distrust, security personnel, whether working for government or opposition, are always looking for people and organizations that may be considered politically disagreeable. Therefore, all staff should remain vigilant and engaged as they build partnerships in the local community, maintaining a careful approach that is mindful of political sensitivities. EFFECTIVENESS OF VISTAS GRANT-MAKING MECHANISM The small in-kind grant mechanism is fast and effective for work with local and even informal organizations. However, the mechanism should not limit VISTAS from working with other innovative and capable organizations through other grant mechanisms such as simplified grants and Fixed Amount Awards (FAAs) to achieve the desired strategic impacts. EFFECTIVENESS OF MORNING STAR Morning Star is a unique initiative within VISTAS that seems to be appreciated at all levels, but, according to respondents, its tangible results are yet to be evidenced. VISTAS is preparing to launch trauma awareness and resilience training and support as a crosscutting element of the Project. This is timely as trauma is a compounding dimension, not only of conflict but of other social issues, such as productivity, motivation, and problem-solving ability. For optimal learning and communication to take place, Morning Star should be mindful that there is a concern, expressed by one respondent that trauma work may 11 bring to the surface psycho-social pain in individuals and communities which would need to be managed professionally by personnel with deep trauma training which exceeds the capacity of Morning Star staff. RECOMMENDATIONS FOR USAID 1. Communicate the Operational Framework (OF) or the most current Mission strategy document to VISTAS and other implementing partners and train them regarding the substance of the OF for IP staff to better address the OF in implementation of their projects. 2. Clarify and disseminate the broader USG policy regarding South Sudan to VISTAS leadership and staff, particularly the USG position on: ● Support to local governments and local government officials ● Cooperation with and support to change agents2 The aim should be a nuanced do-no-harm approach with due diligence and a long-term view. This should also apply to other USAID implementing partners in South Sudan on an ongoing basis, as policies and operating environments are fluid. 3. Use a “less extensive or simplified strategic planning” process that optimizes the Project elements (RA’s, SRS’s, Deep Dives, etc.) to facilitate a strategic planning process for VISTAS that coordinates staff efforts by: ● Clarifying the Project’s goal and Theory of Change (TOC) ● Reviewing and harmonizing the Project’s execution mechanisms and how they are being used (including RA’s, SRS’s, Deep Dives, etc.) ● Creating a common understanding among stakeholders of layering and consistent implementation ● Codifying principles for activity selection, including due consideration to the role of: sequencing, clustering, scaling-up, and coordination ● Redefining “critical areas” and “flashpoints” based on an understanding of structural factors that cause conflict across South Sudan and yield cooperation, especially aimed at linking livelihoods with conflict mitigation and recovery ● Continue reviewing geographic targeting ● Defining the classification and coding system for activities that could render the database more conducive to analytics i.e., to provide feedback and make project adjustments 3. Pay close attention to the border areas among communities that have been supported by political actors with influence in Juba. In such cases, competition for vital resources may connect local areas to Juba-based power struggles. VISTAS should be able to identify the approach to layering that best addresses such local conflicts and the concerns of local communities, while also remaining conscious of how these local impacts interact with the actors in Juba who wield extraordinary influences. 4. Invite presentations to Heads of Mission (HOM) meetings, Heads of Cooperation meetings (HOCs), donor working groups, quarterly implementing partner meetings, USAID “Our Hour,” 2 Change agents are cited as important for Project effectiveness because of their influence on local communities and local government officials. 12 and USAID technical teams to build synergies and explore opportunities for VISTAS activities to fully support the OF. RECOMMENDATIONS FOR VISTAS 1. Use Deep Dives, Rolling Assessments (RAs), and Strategy Review Sessions (SRSs) to execute strategy while harnessing lessons learned from activity flow. VISTAS should further refine these mechanisms with a view toward applying new information and knowledge to current and future activities. Develop a management response to recommendations and monitor actions taken to ensure that these recommendations are utilized. 2. Develop guidance for partnerships with grantees with an emphasis on how VISTAS enters a community, the ongoing communication and learning system, and a do-no-harm approach. 3. Determine core competencies and comparative advantages around the activities that VISTAS does best and the capacity to implement them as a package approach, with attention to timing and layering dialogue, trade, and small-scale infrastructure with migration-related issues across borders. 4. Plan for and invest in long-term partnerships, especially in new flashpoint areas. Include local implementing partners to help develop capacities across regions, codify gains, and increase institutional memory at the national level while setting up partners for eventual handover of activities. 5. Consider activities that integrate vertically into conflict mitigation and nation-building programs such as the Olympics and the National Dialogue. 6. Continuously seek clarity from USAID on policy matters and ensure both the sharing of information and consistency in implementation of the strategy. 7. As appropriate and when applicable, VISTAS should consider other grant-making mechanisms (such as simplified grants and Fixed Amount Awards) to achieve desired strategic impacts and attract innovative and capable organizations that require cash assistance as well. The primary objective is for VISTAS to have as many available tools as possible to exercise its activities and strengthen its partnerships. 8. VISTAS should delineate its information needs at different levels of the Project and improve the analytic capacity of the database to meet the information needs. Currently, the database is mainly used for activity approvals. It can be adjusted, however, to improve analytics to generate monitoring information that supports organizational learning and encourage improvements in performance. INTRODUCTION BACKGROUND The U.S. Agency for International Development/South Sudan launched the Viable Support to Transition and Stability (VISTAS) Project in July 2013 as the U.S. Government’s flexible response to the needs in the world’s newest country. Implemented by AECOM International Development (AECOM), VISTAS is a five-year conflict mitigation program that seeks to prevent the further spread of communal violence 13 and tensions in critical areas where conflict may have national implications. The initial Project objectives were to: 1) build capacity of civilian state authority to prevent, respond to, and mitigate conflict in key flashpoint areas; 2) promote increased access to information and engagement of citizenry in support of political and peace processes at the local level; and 3) engage at-risk youth and other vulnerable groups in productive social and economic activities. As a result of the conflict in December 2013, USAID/South Sudan personnel were evacuated, and almost all USG programming was suspended. During this time, USAID’s Office of Transition Initiatives (OTI) assisted the Mission by providing management support on a temporary basis (from June 2014 to January 2016) and helping shape a strategic shift of the Project in light of the violence and new political realities. The VISTAS team returned to Juba in July 2014 and restarted programming with newly defined objectives under the co-management of OTI, the USAID/South Sudan Mission, and USAID’s Africa Bureau. The VISTAS Project shifted from predominantly working with the local government in the states to working through community-based and non-governmental actors to mitigate communal violence and rising tensions in critical areas where conflict could have national implications. VISTAS currently has four overarching Project objectives: 1) to increase space and tools to manage conflict and tensions; 2) to build cross-line interdependency to promote peaceful coexistence; 3) to promote a more informed community; and 4) to engage communities in trauma awareness so to lay the foundation for healing and reconciliation. Based on these objectives, each geographic region has its own strategy. These are revised annually through a series of in-depth discussions (or Deep Dives). VISTAS currently operates in six regions (Central Equatoria, Jonglei, Unity, Upper Nile, Northern Bahr el Ghazal, and Lakes) that are considered critical to the political stability and security of the country. The July 2016 Rolling Assessment notes that “previous interventions focused mostly on inter-communal dynamics but this may no longer be sufficient and therefore VISTAS is considering more engagement on national level issues including Track 2/2.5 level peace-building activities.” As of March 31, 2017, VISTAS had cleared 378 grants (includes completed and closed. These figures indicate that VISTAS has undertaken a significant number of small activities, which call for strategic targeting of resources to progress towards the goal of national stabilization through local conflict mitigation. FIGURE 1: VISTAS TIMELINE 14 ASSESSMENT PURPOSE AND AUDIENCE Due to violence and Project disruptions in the country and in lieu of the planned mid-term evaluation, USAID decided to undertake an assessment the Project’s ongoing strategic relevance and effectiveness, as well as its ability to respond to changes in a fluid context, and to document lessons learned. Assessment findings, conclusions, and recommendations will inform VISTAS’s direction during the remaining time of Project implementation (the contract is scheduled to end in July 2018) and provide recommendations for future programming. ORGANIZATION OF THE REPORT The executive summary presents an overview of key findings and recommendations. In the body of the report, findings are presented first, conclusions are presented second, and recommendations and lessons learned are presented last. ASSESSMENT QUESTIONS (SEE ANNEX FOR DETAILS) 1. Is the current program strategy relevant and appropriate in the South Sudan context and the context of USAID/South Sudan’s strategy? 2. What are the most effective elements of the program to date and what challenges has the program faced in meeting the stated objectives? How has the program responded to those challenges? 3. What key issues, trends, approaches, and Theories of Change were particularly successful and should be marked for other USAID programs operating in South Sudan? Which were not? What are the lessons learned and recommendations for USAID South Sudan related to VISTAS’ methodology and approach to programming? CONTEXT ANALYSIS POLITICAL TRENDS South Sudan signed the Comprehensive Peace Agreement (CPA) in 2005 after decades of war. The country then emerged into a period of stabilization supported by humanitarian and development assistance from the United States and other donors. The Government of South Sudan undertook a “big tent” stabilization strategy through which other armed groups (OAGs) that had fought against the SPLM for years joined the army but remained loyal to their tribal commanders. Stabilization proceeded, and Juba went from being Khartoum’s southern garrison to a boom town in the lead up to independence of the country in 2011. Rapid, urban-centered growth continued alongside a general neglect of the rural areas. USAID designed the predecessor project to VISTAS in recognition that stabilization required support to institutions of local government that had been marginalized for years. The current VISTAS Project began in 2013 with a concerted focus on supporting local government. 15 Hope for stabilization were shattered in December 2013 when the country relapsed into civil war due to a power struggle in Juba for succession to the presidency. Violence erupted in Juba after months of worsening conditions in the SPLM and spread through regimented pastoral age-sets (traditional age groups that are initiated together through rites of passage) to remote areas on the Ethiopian border from whose cattle camps tens of thousands of Nuer herders—the White Army—launched an insurrection. Fighting spread and threatened Juba, which was defended by Ugandan forces and a state-led counter-insurgency of Dinka cattle herders (Gelweng), mostly from Aweil and Warrap. Poorly trained and largely uneducated, they became part and parcel of the army (Pendle 2014) as other ethnic groups steadily fell in numbers. What had started as an elite conflict devolved into a tribal war. As military factions divided along political lines, civil war enveloped the country. Millions have since been displaced or become refugees. A massive challenge of national integration faces the nation, no less severe than during the times of the North-South civil war (1983-2005). The African Union (AU) established that the conflict was triggered by a political dispute in the center (AUCISS 2014). The international community disengaged from direct support to the state and maintained development activities through non-governmental channels, including substantial humanitarian assistance and continued support to the peace process. Despite the Agreement on Resolution of the Conflict in the Republic of South Sudan (ARCISS) signed in August 2015, civil war erupted again in July 2016 starting in Juba and spreading with terrible consequences for the civilian population. Federalism has been the dominant political trend since at least 1947 (Johnson 2014). Constitutionalism was born with the CPA but now focuses on how power is shared at the center and decentralized down to local government. The creation of the 28 states introduced boundary disputes along ethnic lines. Most of these have national implications. The call for an inclusive constitutionalism leading to federalism continues, but war makes such aspirations far-fetched. President Kiir announced a National Dialogue to begin in March 2017. His stated concept is that independent, inclusive local dialogues to address community grievances will feed into regional dialogues and then cascade upward into national dialogue to inform the future of the nation. Many see this as the only solution, but wonder whether a truly inclusive and transparent process is possible under current conditions. Others warn it could become tribal monologue. The benefit of feeding into this process would be if VISTAS provides an independent channel through which well-organized local dialogues could inform national processes in a systematic way. The downside would be if the National Dialogue lacks the necessary political will to support it, and therefore does not benefit from the VISTAS initiative. Government is isolated from many people, civil society is constrained, and institutions linking communities to the government are weak. Hundreds of thousands of war-affected South Sudanese are now bunkered in Protection of Civilians (POCs) sites, scared to return home. Many displaced peoples have sought refuge in neighboring countries. A divided region operates largely on the basis of national self-interest, yet maintains nominal cooperation behind the ARCISS. A point of political focus is needed. It can only come through dialogue in the common search for peace locally, nationally, and internationally. 16 ECONOMIC TRENDS As the first quarter of 2017 ended, almost half of South Sudan’s population was food insecure with 4.9 million (about 42% of population) estimated to be severely food insecure (IPC Phases 3, 4, and 5) from February to April 2017. This is projected to increase to 5.5 million people (47% of the national population) at the height of the 2017 lean season in July. The magnitude of these food insecure populations is unprecedented across all periods. Hunger walked hand-in-hand with conflict. The primary cause of local conflicts was access to water, grazing, and agriculture lands. However, these conflicts were embroiled in the political and military dynamics of the civil war. South Sudan’s extensive river network is the principal integrating factor across all agro-pastoral groups where VISTAS operates. Its ebbs and flows modulate the seasonal cattle migrations around which patterns of life are organized. Communities contest and share these waters, mostly for their cows, which are considered sacred, used primarily for marriage and compensation, and often raided. Cattle raiding evidences a martial capacity that has long been instrumentalized in war in South Sudan. Oil and cattle continue to bind the two Sudans in an uneasy alliance, while the same resources create conflicts locally and nationally. Trade connects communities in South Sudan and maintains relationships across identity lines where potential conflicts may threaten communities. Some resist conflict due to the mutual benefit of trade, which preserves a connection between communities, including those across borders. From 2014, the role of livestock started to receive increasing recognition by the development partners (DPs), with recent studies establishing the role of livestock in resilience (WFP), recovery (FAO), development (USAID), and conflict mitigation (JICA). South Sudan has the highest per capita livestock in Africa (FAO 2015). Regional cattle trade remains robust, while raiding increases. High dowries in South Sudan has increased social pressures on men to raid in an environment where there are a few other options for acquisition of cattle. Militias increasingly provide avenues towards marriage. South Sudan's expansive informal economy consists mostly of subsistence farmers, including 950,000 livestock farmers, 350,000 herders (mostly children below 18 years), some 4,500 live animal traders, 1,500-2,000 slaughterers, between 2,000 and 4,000 butchery owners, and some 500 holding ground owners, which are facilities for temporarily housing cattle that are to be sold or slaughtered (FAO 2016: p. 38). Despite promises, oil revenues have not been used to fuel agriculture. Shifting cattle migrations cause conflict, while negotiating these routes has been the source of cooperation. Urban areas remain poor and without employment opportunities, while the large rural subsistence economy is unable to feed the people, although it is resilient enough that they still survive – except in the most severe conditions of food insecurity – famine, which was evidenced in 2017, and threatens to affect more and more people if the crisis continues. In their hunger, communities are prone to fight in the belief that their benefit is being withheld from them in the capital. The young men comprise the most difficult demographic to track, not least because they stay on the move with their families’ cattle. It is in this context that the profound connection between cattle and war on the one hand, and dialogues and trade on the other should be understood. 17 THE VISTAS ASSESSMENT SCOPE OF THE ASSESSMENT Geographically, data collection gathering and learning sessions were held in Nairobi, Kenya; Washington, D.C.; and in the VISTAS field offices of Aweil, Yirol, Bor, Bentiu, and Akobo, providing information on the VISTAS effectiveness to its catchment population. The study also examined the spread VISTAS activities in different geographic areas. Technically, the assessment covered the current Project strategy’s relevance and appropriateness in the South Sudan context and the context of USAID/South Sudan’s strategy. It examined the Project’s ability to pivot and adapt to changes in the political and security context; the continued overall correctness of the geographic focus; and whether VISTAS is working in areas where communal violence has national implications. It also assessed the Project’s focus on key strategic issues in South Sudan from the perspective of U.S. foreign policy and USAID strategy. And it assessed the most effective elements of the Project to date and what challenges the Project faced in meeting the stated objectives. It also documented key issues, trends, lessons learned, approaches, and Theories of Change (TOCs) that were particularly successful and should be marked for other USAID Projects operating in South Sudan. METHODOLOGY MSI assigned a four-person team of experts to carry out this USAID assessment. The study analyzed Project performance with an emphasis on shifts in strategy and operations due to impacts of the 2013 conflict and its aftermath. The MSI Team considered the Congressional Appropriations Act, the USAID South Sudan Operational Framework (OF), and the VISTAS design as key points of reference regarding U.S. foreign policy. SAMPLING MSI employed multistage-purposive sampling to select activity clusters and geographic priorities in collaboration with VISTAS, OTCM, and respondents (Yin, 1994). Due to being information rich, the field sites of Aweil, Yirol, Bor, Bentiu, and Akobo were purposively selected (Creswell, 2007). The bulk of primary data collection was conducted through interviews with USAID and AECOM staff as well as with key informants familiar with VISTAS’s work. A wealth of practical information and strategic wisdom was tapped from USAID and AECOM staff to help understand the past and plan for the future. Thus, the qualitative study targeted VISTAS headquarters (HQ) staff and field offices. Purposive sampling was also used to select respondents from USAID, VISTAS, and grantees as well as beneficiaries/community members to assess the project’s effectiveness and lessons learned. DATA COLLECTION The conduct of this assessment was qualitative in nature and included Key Informant Interviews (KII), Group Meetings (GM), and Focus Group Discussions (FGD). This approach was directly aligned with the evaluation questions as was the assessment team’s document review. The desk review findings helped 18 shape questionnaires for KIIs and focus groups. The assessment team triangulated data from all sources to generate evidence that addressed the assessment questions. The assessment utilized a mixed method approach, relying on quantitative and qualitative data from both primary and secondary sources. Data stemmed from Project documents (VISTAS Award Agreement AID-668-LA-12-00001 and relevant modifications, VISTAS Activity Database, VISTAS Work Plans, VISTAS Monitoring Plans, VISTAS Quarterly and Annual Reports, VISTAS internal portfolio and cluster reviews, reports from Deep Dives, Rolling Assessments and SRSs). The qualitative study targeted VISTAS headquarters (HQ) staff in Juba (at the time of the study were based in Nairobi) and field offices. It utilized the activity database, KIIs, group meetings (GMs), and focus group discussions (FGD). It targeted staff from USAID, VISTAS, and grantees as well as beneficiaries/community members. Secondary data was based on VISTAS performance monitoring and reporting. DATA ANALYSIS The assessment team triangulated data from these sources to generate evidence that addressed the assessment questions. Data analysis methods depended upon the nature of the data to be analyzed and included some of the following: ● Secondary data: Document review and activity database mining ● Key informant interviews and pattern analysis of content produced by focus groups that were compared ● Comparison of case studies ● Analysis of reports to produce a narrative that identifies recurrent themes and activities ● Interviews enriched the narrative and data at hand. Post-interview discussions within the team helped integrate hand-written notes. The processing of information gave rise to themes. . The interpretation was based on recurrence analysis, triangulation, and pattern identification in transcripts. The analyzed data from all sources was triangulated to provide a thorough answer to each study question. STRENGTHS AND LIMITATIONS The main strength of the assessment’s approach was the use of multiple data sources and methods to develop findings for each question. The triangulation strengthened the validity and reliability of the findings. The overall approach is flexible and was tailored to the needs and operating conditions at the time of the assessment. The assessment was conducted under challenging political, security and economic conditions with data collectors working from South Sudan states and the MESP office in Juba and international evaluators working from Nairobi and Washington, DC. Communication with the data collection team in South Sudan and Juba was primarily by phone and electronic means, including daily Skype calls. Debriefing of data collectors took place in Juba. South Sudan continues to face serious security challenges in several 19 regions of the country where data collection took place and at the national level. The security situation affected sampling of communities and data collection areas as well as access to respondents. The findings may have also been restricted to certain areas. There may have been an incentive for certain informants to disclose limited or false information. Thus, triangulation of data was a central tactic for obtaining valid findings. The local assessment team members have the appropriate language skills and were people who originated from the states where they collected data. The overall approach was flexible and adaptable to the conditions of the operating environment. The assessment team remained flexible and open to last-minute changes whenever security or weather conditions precluded a visit to one or more selected communities. The assessment was not designed to be statistically significant, and therefore some findings may not be generalizable. It demonstrates how important the presence of VISTAS could be in community life. FINDINGS AND CONCLUSIONS Question 1: IS THE CURRENT PROGRAM STRATEGY RELEVANT AND APPROPRIATE IN THE SOUTH SUDAN CONTEXT AND THE CONTEXT OF USAID/SOUTH SUDAN’S STRATEGY? In addressing this question the assessment team conducted analysis of USG priorities identified/reflected in the OF, the South Sudan political, security, social, and economic context, the VISTAS Project Theory of Change (TOC), small and in-kind grants, and Project execution mechanisms. The Assessment Team’s overall conclusion is the VISTAS Project has adapted well to the changing political and security context and demonstrates strategic relevance and appropriateness with the context of both South Sudan and USAID/South Sudan’s strategy. Its continued relevance could be further strengthened with some adjustments. STRATEGIC RELEVANCE 1. RELEVANCE OF VISTAS TO COMMUNITIES Available evidence suggests that VISTAS grantees and beneficiaries appreciate the Project for its meaningful impact within their communities and they can cite tangible benefits that they associate directly with the VISTAS Project. Community members request USAID to expand and deepen its presence through VISTAS. Generally, national staff, grantees, and beneficiaries share a positive view of VISTAS’s impact on the everyday lives of common people. For example, one respondent said, “we are bordering three communities…we are supporting inter-communal markets (Grant # WUN005 Amokpiny Border Market Office Construction and WUN0008 Amokpiny Peace Market Office PV and Furnishing) so that these communities can reinforce what has already been there, like the intermarriages” which have built bridges between communities. Another respondent said, “we have empowered the Chief with reconstructing the traditional laws and put it in a booklet which is helping them work effectively and apply laws that the community respects and have been used to settle several cases that could have resulted in revenge killings; so it has had an impact (Grant #JON057 Customary Law Refresher Training 20 for Traditional Chiefs in the Greater Bor Area and JON068 Akobo East Chiefs Training).” Another respondent said “we give training to Akobo women association, which has produced impact and signaled that their voice is stronger than the men’s voice in promoting peace.” For example, the women could respond to youth involved in revenge killings, who were unwilling to talk to the elder men, but, after the inclusion of women, agreed to the dialogue. In Lakes, revenge killings are reportedly reduced. There are many examples of positive localized impacts such as these from across all regions and areas where VISTAS supports activities. In one case, a beneficiary said, “VISTAS is more trusted than the local government.” This shows how important the presence of VISTAS could be in community life. 2. IMPACT OF USG POLICY ON VISTAS The 2013 USG policy shift away from state institutions and towards communities has unfolded in two levels of policy dialogue: the general level and the operational level. At the operational level, this dialogue has focused on whether or not VISTAS can work with local governments. The policy has been interpreted and implemented differently across VISTAS activities, but is generally understood to restrict VISTAS cooperation with government at all levels. This change represented a paradigm shift for VISTAS, which was designed to work with the government, building offices for local administrations during the CPA Interim period, and only more recently diversified into a variety of different activity sets. While respondents were under a strict interpretation of the presumed policy, there is no written USAID South Sudan policy restricting VISTAS engagement with local government. The policy ensures that no USG funding is used to subsidize war or reward human rights abusers, and out of due caution, the Mission instructed VISTAS to restrict all work with local governments. This decision may not promote the relevance and impact of Project activities and should be revisited. VISTAS would rather support local government through a nuanced approach that engages entire communities in identifying partnership opportunities. USAID’s intention in creating VISTAS was to help extend the reach of local governments at a time when the USG policy prioritized support to CPA implementation by building the capacity of state institutions. It targeted “flashpoints” to mitigate local conflicts that could escalate with national significance. After the 2013 conflict, the USG policy shifted from building the capacity of state institutions towards assisting communities through non-state actors (community-based organizations, traditional authorities, civil society and private sector). This had a major impact on VISTAS, and the types of activities the Project could support; mostly it stopped material support to local government, but it also formed an impetus for other forms of partnership. VISTAS originally had a “hardware” component, and one respondent noted, “The relationships with local authorities and communities was predicated on VISTAS giving stuff.” It also had a software component, “the material provisions were not the point, although created openings…mostly local government officials were the counterpart and convened communities.” Before December 2013, 72 percent of the funds went towards activities including infrastructure components. After December 2013, funding decreased to 31 percent, a proxy indicator for decreased support to local government 21 considering building and equipping county administrative offices was the nature of the infrastructure support. This shows that VISTAS diversified its activity design because of the shift described. After the 2013 conflict, the U.S. and other donors determined that “business as usual is no longer possible” and closed key Projects. They adopted a common position articulated in a joint-donor document called “Working in the Current Context: Donor Principles for Engagement, Priorities.” According to the outlined donor principles, “present conditions make channeling aid through government systems unfeasible for donors.” The donor principles committed to engaging “in dialogue with all parties, including the national and state governments.” It further committed to activities that involve South Sudanese people and their communities in the planning, management and implementation of Projects that “freely expressed wishes of local communities” and develop “joint actions to mitigate or react to risks, and optimize effectiveness.” Recognizing that “South Sudan’s human, financial and natural resources have been diverted from social and economic development to military purposes,” the donor principles note that “present conditions make channeling aid through government systems unfeasible for donors,” and assert that “We do not accept our assistance being used for any military, political or sectarian purposes.” The donor principles are the earliest written evidence of the “policy” of which its interpretation would seriously test the Project by limiting support to local government and forcing a reconsideration of strategy. Findings from this assessment showed that the USAID interpretation of the policy is more nuanced than it appears to some of the respondents. Many respondents perceived the policy in cut-and-dry terms: as one respondent put it, the policy is “don’t work with the government.” FIGURE 2: Levels of Grants and Disbursements Relative to Evacuation This policy discussion within VISTAS regarding support to local government caused a severe cut back in VISTAS activities, with grants falling precipitously. This was also in part due to the evacuation. When 22 activities resumed, there was a high level of scrutiny on all levels of management, as is expected of OTI￾like projects. There was also a high degree of political sensitivity associated with the VISTAS project for good reason. Confusion as to what was allowed and what was not allowed set in among senior managers, followed by a culture of self-censorship so as not to risk potentially serious consequences of a misstep. The policy shift was then clarified in the Congressional Appropriations Act (2015-2016), which addresses Projects that “respond to humanitarian needs and the delivery of basic services and to mitigate conflict and promote stability.” The Act states: “None of the funds appropriated by this Act that are available for assistance for the central Government of South Sudan may be made available until the Secretary of State certifies and reports to the Committee on Appropriations that such government is taking effective steps in accordance with a list of conditions” (SEE ANNEX). The assessment found that a majority of respondents (i.e., VISTAS and beneficiaries) were not familiar either with the donor principles or the Congressional Appropriations Act. Without clear guidance on the evolving policy, there was confusion regarding interpretation and implementation of the policy. One respondent summed up the general sentiment: “people didn’t really understand what the Project would do because they did not understand the U.S. foreign policy on South Sudan.” Another said, “People had been used to business as usual - work with communities, including their local governments. Relationships and expectations had been developed.” Another respondent added, “There was a sense that the Project had identified change agents within local government who should not be abandoned. This became tricky post-July 2014. The Operational Framework was established and the Bureau served as a filter for grants.” One respondent saw Washington’s active engagement “as micro-management,” but another explained, “The Bureau was trying to protect the Project knowing Congress was angry and could cut funding.” These concerns continue to be a reality. 3. RELEVANCE OF VISTAS TO THE OPERATIONAL FRAMEWORK (OF) The VISTAS’ Theory of Change (TOC) is aligned with the Operational Framework (OF), particularly TOs 2.2 (Strengthen inter- and intra-communal relations and reconciliation), 2.3 (Increase space for conflict mediation), and 1.1. (Facilitate community-led response) in that order of significance. It also makes a lesser contribution to 1.3 (Strengthen Livelihoods and Resilience to Shock), and might also further 2.1 (Support inclusive peace process), although in practice some staff pursue these objectives. The OF has the following goal: to “build the foundation for a more stable and socially cohesive South Sudan.” The USAID South Sudan OF guides the Mission’s programming until conditions are appropriate for a new Transition Strategy. The USAID South Sudan TOC states: 23 If the psychological, social, political, economic, and security dimensions of current and past crises are effectively addressed and institutional systems ensure greater inclusion, THEN foundations for a more stable and socially cohesive South Sudan would be established, because the risks for continued crises in South Sudan would be mitigated. The overarching VISTAS goal is to mitigate the further spread of communal violence and rising tensions in critical areas where conflict may have national implications. VISTAS Project’s Theory of Change states: If VISTAS supports vulnerable communities in key targeted areas, communities will be more resilient to political pressure to fight and conflict will be mitigated because they will view peace as a viable alternative to conflict. The VISTAS strategy framework is directly aligned with OF sub TO 2.2 (Strengthen inter- and intra￾communal relations and reconciliation) and sub TO 2.3 (Improve mechanisms for conflict management). VISTAS has institutionalized the tools to maintain a robust strategic analysis of programming objectives, and this also allows the Project to respond swiftly and appropriately when the facts on the ground change. This ensures that the Project remains relevant to the local context and VISTAS’s goals, more so than would be possible in a traditional project that is less adaptable. Assessment findings indicated a limited understanding of the OF among VISTAS. One respondent said, “Every single time I have been presented with USAID strategy, (it) has no indication of what they want to do, what is the TOC? We fall within it but only because it is so wide. To my knowledge, there has been no conversation about link to OF but maybe that is happening at a different level.” The respondent also said, “We are operating under a TOC [at the country level] that very few of us think is legitimate.” The respondent meant that some VISTAS questioned whether their activities could have national implications, as the scope of the challenge of national conflict mitigation is huge. One respondent said, “Hypothesis is that the more interaction people have, the less likely they are to engage in conflict. If people routinely interact they will increase familiarity; build friendships and ties.” Another argued there are “local spaces that are important, (such as): traditional authority courts because this is where people sit and resolve conflicts; markets because people are coming with merchandise; livestock complex due to the importance of cattle and access to vaccines; also the market as a place for sale of vegetables.” The emphasis is on the social space itself as well as the interactions that take place. One respondent said, “Supporting local initiative that already exists by far is the best approach – problem is there is not a lot of initiative.” Another said VISTAS should “prioritize projects that can be handed over and scaled up.” Another said, “Cattle camps – we will find a gazillion things useful to the whole development assistance…we have to watch out that we don't get caught up in flavor of the month, but look at structural things – look at cattle camps.” 24 4. RELEVANCE OF VISTAS TO THE NEW CONTEXT OF CIVIL WAR SINCE DECEMBER 2013. Many VISTAS questioned the continued relevance of VISTAS’s strategy since December 2013 when the country descended into civil war. VISTAS’s dynamic innovation platform currently aggregates activities into regional plans which should then coalesce into an overall Project plan. The crisis in 2013 resulted in forced migrations, military battles, new borders, food insecurity, and the spread of diseases. With millions living as refugees outside of the country and others as internally displaced persons (IDPs) within Protection of Civilian (POC) sites, the context for VISTAS dramatically changed. While there is a Project plan, there remain outstanding questions delineated in this report that need to be clarified. Foremost is the question of the national objective, since the existing objective is not convincing to the implementers of the Project. Some respondents defend the goal of “mitigating local conflicts with national implications,” while others question its viability and/or relevance. Others think it should be changed since national conflict is in part the cause of local conflict than vice versa and reference the power of political and military actors in Juba. One respondent concurred, “At the highest level and at its inception, we were looking at local conflicts with national level implications and this was before the 2013 conflict broke out… The reverse was actually true - the highly elite competition influences where conflict breaks out in the periphery…they should reverse the strategy - national conflicts with local conflict implications.” All agreed that the major driver of local conflict is national power struggles. One respondent took the extreme view that VISTAS should prioritize the national level since that is where the major conflict drivers are rooted. The respondent pointed out that local conflicts take all the project focus, and yet are not the source of conflict. Most respondents who knew the strategy considered it unrealistic under current circumstances of a civil war driven by disagreements in the center. Some suggested the goal should focus only on local conflict mitigation without necessarily having national implications. One respondent qualified this by saying, “If all communities were to say ‘I am following VISTAS way of doing things’ then it will affect national level,” but this is not possible at the current scale. Most beneficiaries reported that VISTAS activities help resolve local conflicts and were less concerned with national-level impacts. However, some drew connections between certain local and regional activities and the national level. For example, trade activities between southern Unity state, Lakes, and Jonglei state were perceived as preventing army attacks across these territories. Peace dialogues among Lou Nuer and Dinka Bor chiefs dissuaded the warring armies from using these areas. The reduction in revenge killings in Rumbek was seen to have national economic impacts because it opened national roads for trade that had been blocked due to deteriorating security. However, local conflict mitigation could only reasonably be argued to have national implications in a minority of cases. It is unlikely that serious attention was given to the local-national connection during the process of idea generation and activity development, despite it being explicitly and repeatedly stated 25 in all strategy documents. This is not because VISTAS are not concerned. Rather, it is not clear that such a strategy makes sense given the overriding influence of Juba as the root of most serious, national-level conflicts. Also, staff recognize the value of local conflict mitigation activities even if they do not have national implications. VISTAS undertook some activities at the national level; however, most activities continued to target local conflicts. The July 2016 Rolling Assessment notes, “Previous interventions focused mostly on inter￾communal dynamics but this may no longer be sufficient and therefore VISTAS is considering more engagement on national level issues including ‘Track 2/2.5’ level peace-building activities. One respondent said, “One of the original theories of change that still lingers is that local level focus on critical flashpoints where conflicts could spiral out of control can positively affect the stability of the country as a whole.” The respondent explained, “it was all about political actors with unbelievably powerful warlords running amok, and this little ‘piddly’ project is trying to stabilize the country…it’s just a drop in the bucket.” The respondent also noted, however, that “no activities are a waste – they are all important in the moment we do them for the people we do them for.” In all, it should be noted that some activities are clearly more strategic than others. 5. RELEVANCE OF LOCAL GOVERNMENT TO VISTAS STRATEGY Evidence suggests that cooperation with government, particularly local government, is necessary for Project effectiveness and the partnerships with local actors are indispensable to VISTAS’ success. When the shift away from support to local governments was interpreted strictly, it had the benefit of pushing VISTAS to look for other partnerships in communities. Respondents believe this aspect has contributed positively to the success of the Project, which generally appears to innovate well. One respondent said, “The Government was upset. The Policy was not to support Government institutions…in practice the question was where the line between state and society was to be drawn. The major murky question was how to interpret the local government including Traditional Chiefs and customary leaders.” This indecision is evident in the database, where many chiefs were targeted but often through local NGOs. 26 FIGURE 3: The Numbers of Grantees BEFORE and AFTER January 2014 Respondents made a variety of similar points: ● “As for working with government, it is unavoidable;” ● “The fact is that we need working relations with the government while we have a policy that we should not work with the government;” ● “If I were to be given a chance to do something different, it would be building relationship with the government;” ● “If there is exclusion of one group (in this case, local government), it means there will not be a viable peace.” A minority of respondents focused on the positive side of the policy, although with qualifications, “the good side of the policy at a local level was opening eyes to other opportunities beyond building local government…” Another said VISTAS “did manage to change and shifted things in ways that were positive…to build up skills in conflict resolution techniques and mediation techniques.” In practice, the Project’s pivot to avoid directly working with local government was meant to increase assistance to traditional leaders, but instead, as one respondent pointed out, “One thing is very clear - the traditional authorities definitely have an influence. When VISTAS says we’re working with traditional authorities, there is a contradiction because they are appointed by local government.” It should be noted here that, as one respondent said, “Many if not most (traditional authorities) aren’t appointed by local government and they have an important role in accountability.” 27 6. RELEVANCE OF THE VISTAS INFORMATION MANAGEMENT SYSTEM Information flows efficiently from the bottom-up. However, it was not clear to the respondents how information was being used. The database is being used mainly for project approval processes. Information is also generated from Strategic Review Session (SRS), Deep Dives, Ad Hoc Third-Party Monitoring, Rolling Assessments, and quick studies. However, evidence suggests that there has been little use of performance information for learning and adaptive management. Rather, information has been used significantly towards accountability for results purposes and for operational and administrative improvements. In this sense, the feedback appears to be working, but less so at a strategic level. This is also because 378 grants take time to process. 7. RELEVANCE OF THE VISTAS GEOGRAPHIC FOCUS Overall geographic focus is a remnant of the original Project design and has been slow to respond to the simmering conflict(s) in Equatoria, a fact over which staff are conflicted with some wanting to hold current geographic positions and others to expand and/or open new offices. A minority would contract operations, although all would target Equatoria. Originally, targeting was focused on Northern Bahr el Ghazal, the Sobat Triangle, the Wunlit Triangle, and Southern Unity as critical flashpoint areas. One respondent said, “The Project’s current geographic targeting is in some ways a legacy issue, having to do with the initial targeting in 2010; they got comfortable and built up an infrastructure.” However, the same respondent concedes, “those areas were and continue to be relevant”. Respondents understood that cross-border issues could embroil South Sudan in conflicts with Sudan over CPA-related issues (in Northern Bahr el Ghazal and Southern Unity) and that certain internal conflicts in South Sudan could escalate and assume national proportions (Wunlit and Sobat triangles). Since then, “the major tweak to geographic targeting was to add POC sites.” Maintenance of focus on the border in Northern Bahr el Ghazal is consistent with USG policy as spelled out in the Congressional Appropriation Act to support activities related to implementation of the CPA’s outstanding issues. Before the events of December 2013, nearly all VISTAS activities were at the community or state level. The Activity Database shows one national level activity for this period. Since the Project returned to Juba in July 2014, VISTAS has developed 20 national level activities worth $3,619,378. This may seem like a large increase, but it is still a small part of the VISTAS programming. As of March 31, 2017 VISTAS had cleared 378 grants (includes completed and closed) worth $33,951,274 (average grant size $89,818), with national activities counting for less than 2% of activities. One area where all staff recognize local conflicts that may have national implications are boiling is Equatoria. There has been recent violence in Yambio, Maridi, Yei, Kajo Keji, Wau, and Magwi. One staff member said, “I think – it’s hard to say - but the Equatorias are a place where there is work to be done and they are looking at that…” Also, several respondents explained in depth why more work is needed 28 in Malakal, where a land dispute between Shilluk and Dinka has already assumed national significance with threat of further escalation. One respondent said, “It comes down now and then to what does USAID wants us to do.” QUESTION 2: PROGRAM EFFECTIVENESS The assessment team examined those project accomplishments at output level with the greatest potential to effect change at beneficiaries and systems levels. The analysis also examined the interaction between context and the VISTAS Project 1. EFFECTIVENESS OF ACTIVITY SELECTION AND EXECUTION Absent support to local government, evidence indicated that VISTAS’s most effective activities are those that deal with Traditional Chiefs, women, and/or cattle-related issues of both a moral (compensation) and market (trade) nature. The analysis of the respondents’ priorities indicates that VISTAS has experience and comparative advantage in dialogues, trade, traditional courts, media and cattle-related activities. Another said, “we are good at deliberating and facilitating; we can design the dialogue, mobilize community, push and prod, and arrive at resolutions and outcomes.” In terms of dialogue, one respondent said, peace dialogues have been most successful because it is easy to implement - if there is a need and you can easily bring communities together;” however, “the problem a lot of time we face is we don’t do follow on activities.” In Likuangole, “the youth of the pastoralist communities of South Sudan are particularly vulnerable to being mobilized for fighting between communities,” and VISTAS helped to engage “youth in activities that benefit their communities.” The community felt that these activities reduced conflict. One respondent said, “In Lakes, we are not dealing with migration but are mitigating conflict on livestock, including raids. In 2015, we established spaces and facilities based on lessons learned in Northern Bahr el Ghazal - these are livestock registry centers, which include also vaccination. The reason why we did this was in response to a request from local communities in these flashpoint areas where there were a lot of movement of stolen cattle being traded in other areas - Nyang and Luak Luak. The reason why they wanted these facilities was because the local chiefs have the idea to register the animals to minimize conflict.” VISTAS has undertaken 86 activities (37% of the funds committed towards activities) that involve pastoralists and land issues. One respondent reflected on the importance of harmonizing customary law in the modern constitutional system, when inside the Bentiu POC, where there were 100,000 people under an array of traditional courts, “VISTAS pulled all of the traditional justice actors together…and marched them through a whole series of workshops on customary law, international law and most importantly, a presentation by the South Sudanese constitutional lawyer (which everyone raved about)”. The respondent continued, saying the impact was that “there was an agreement where the different courts 29 would recognize everyone’s ruling – so we worked with courts outside, so the legal ruling would be respected across the whole area,” inside and outside the POC. One respondent said, “Trade is a dormant strength of AECOM – it draws on two strengths of VISTAS: operationally, VISTAS is really strong on the one hand, and, on the other hand, VISTAS can creatively access and afford a range of technical experts, more so than some of the more traditional UN agencies.” Another said with respect to livestock registration, “It is not easy, but they want to also tie this to trade and commercial activities; to be able to have oversight over the commercial aspect. They (the chiefs) will confiscate any animal that is for trade if they don’t have a letter from the chief where the cattle is coming from.” Another respondent observed that, “Cattle is an unaddressed realm of opportunity.” Several respondents focused on the importance of migration routes, including establishing “vaccination pens, introduction of hygienic practices through slaughter slabs, and skins and hide section as a way of adding value and capturing an asset which is lost.” In terms of timeframes, respondents emphasized sequencing as exemplified in the Pre & Post Migration conferences in Northern Bahr el Ghazal. In terms of geographic targets, VISTAS has had the most success with border communities. For example, one respondent said, “the goal is still relevant [as] some activities are localized, but spill across state borders.” Another noted “border agreements – trusted by the community and allow for crossing of walls and barriers” are an example of excellent work. Another respondent said: We have a good example of the success of traditional leaders. There were attempts by SPLA to gain access into IO territory, but they were push back from the local community in Likuangole (shared border between Akobo and Pibor). VISTAS had supported the construction of traditional authority offices and it gave them a say on what happens to engage with county commissioner and people. Given past fighting the community undertook collective action to prevent collateral damage. Another said, “On the border between Bor and Pibor and Bor and Lou, there are communities that are bi-lingual where there have been intermarriages and are usually instrumental in resolving conflicts and relaying information.” Another pointed out that VISTAS seeks to “enhance water access so that they (border communities) do not have to move across borders” One respondent summed up the views of many across the regions by saying, “VISTAS has made significant contribution in reduction of inter￾communal mistrust and the spreading of conflicts across borders.” In terms of beneficiary targets, traditional leaders and women’s groups have spearheaded the most effective activities, while youth groups are often cited as an important target because they engage in violence. However, they are seldom reached in the cattle camps where trouble is rooted. Women’s activities in almost all regions were cited as powerful because women can influence the youth with peace messaging. Women also do not generally demand incentives, but rather, display a deep sense of initiative. 30 Evidence suggests that partnerships are the true power of the VISTAS Project because all activities are dependent upon them. In some areas, VISTAS has built meaningful community relations based on mutual respect and a long-term view of partnership. In others, the community relations are more activity-based and short-lived. Those where relationships have been built through time correlate more closely to perceptions of VISTAS success and impact. All regions include gender in their strategies and are committed to it in principle. Of the 161 partners awarded grants, only 11 are women’s organizations. Most consider it a matter of female participation in activities. They all find it difficult to meet the quotas due to division of roles and responsibilities in society. Internally, VISTAS has not employed many women. All respondents recognize the value of a purposeful approach to gender in formulating activities and can cite examples from their experience of programming that included solid gender analysis. One respondent noted, “[She] had done studies on singing and social process… She found that the women were singing songs telling the warriors to go over and kick some (ass)…” In Akobo, a beneficiary said, “As women, we traveled to various locations to preach peace. VISTAS connected us with Murle and community of Nasir for peace dialogue.” Another respondent explained, “The main actors (in conflict mitigation) are the youth and women group…because women have an impact, like when a person is killed in revenge killings, women cry that I do not have a child or a husband to take revenge, their cry alone will have a voice and since the youth are the ones who go and raid, and are the ones who go and kill.” For that reason, “VISTAS has provided women with skills of peace building after we supported them in disseminating the peace messages to the communities, and now will provide them with tents, mats and mosquito nets so that they can go outside to other locations to disseminate peace.” Another respondent said because of VISTAS activities, “Women now also have voice, and we see them as very important partners; if we involve them they can mobilize their fellow women, like what they did going up to cattle camps,” from where the youth organize raids, revenge killings and militias. Another grantee explained, “In the beginning women feel shy; they cannot easily discuss issues in front of men…So, there was a need to build the women capacity alone, so that they support equally in the peace building. We do that for five days, after which the women decided to travel to payams to disseminate the peace messages. We gave them transport, they went to payams and preached and now things have seriously changed.” 2. EFFECTIVENESS OF DIVISION OF ROLES AND RESPONSIBILITIES In line with the OTI division of roles and responsibilities, VISTAS senior managers are generally strong on operations and expect USAID to oversee overall strategic direction and oversight. Several respondents noted that senior VISTAS managers were strong on operations and weak on strategy. For example, one respondent said their “job is to burn money,” which should be understood in the context of the extremely difficult operating environment, where disbursement is a greater challenge than under normal circumstances. Another saw their job as “an implementer…not concerned about strategy, quality assurance, outcomes being meant.” One respondent said the VISTAS is “very output oriented” and that, generally, VISTAS is concerned more with “operational realism than with strategic relevance.” This is an ethic that has enabled VISTAS to program 378 grants, many of which were developed and funded under conditions of civil war in the country. 31 3. EFFECTIVENESS OF LAYERING OF ACTIVITIES VISTAS is just beginning to layer activities. The concept and its application is only partially understood. Northern Bahr el Ghazal is a flagship of activity layering. In many areas, activities can be related in principle because they all seek to mitigate the same local conflicts, but these are not approached strategically at a design stage with the intent to build synergies. Evidence suggests that some VISTAS activities in some areas appear to be layered. However, the exact meaning of the concept is not well understood. Several respondents said respectively, “I don’t understand the term layered,” “what does it mean?”, and “I don’t know the meaning.” Another respondent said, “The layering approach is a USAID question.” One respondent said activities “Didn’t roll up into something bigger” and noted that “Everyone had their pet project and their geographic connections and attachments.” One respondent called for “more targeted and layered programming with enough operational and programmatic resources to be able to impact from different sides, as opposed to one off activities.” Another respondent said VISTAS “probably does too much” and is “spread across too many types of activities. The respondent continued, “if you focus on Traditional Authorities, and we drive all activities in that direction…chiefs, elders, community level leaders… and target them…instead of working with radio stations, media, building trade structures, auction centers, civil society organizations…” The respondent concluded by saying, “I don't think I am smart enough to say what exact activity or target group we should focus on, but the principle should be FOCUS and it is currently lacking.” On clustering, one respondent said, “there has been thought [about it], but I am not sure it is yet operational,” and regarding sequencing, considered to be a question of “technical rigor that VISTAS does not have.” One respondent noted that approaching clustering through M&E “helps focus the discussion on common threads: do [activities] relate to each other and build on each other? I think we are ready for the discussion.” Another considered layering and clustering a need that should be driven by the Chief of Party (COP). The respondent gave the following example: “If we do peace dialogue, it must be on a regular basis – it should be a continued effort…These efforts need to be constantly reinforced – layering – for example, by giving traditional authority refresher trainings…Layer it with follow up activity and sequencing, reinforcing and building upon it so that we stay present and continue working with them (the partners) over a long period of time.” One respondent noted that in Unity, VISTAS “followed dispersed methodology; no cohesion among grants; very little layering – but it was an incredibly difficult operating environment during that period, with chaos in POC, movement was difficult, and so only a few grants were issued; but now we have explicitly decided to pursue a more layered approach.” Another said, “We will not move the bar in one direction because we are moving in 30 directions.” One respondent said, “There is nothing in South 32 Sudan that is sustainable right now – the basic strategy is to throw it all against the wall and see what sticks for a few months.” From document review, analysis, and researchers’ observations of Northern Bahr el Ghazal as substantiated by the database, activities and layering mostly centered on cattle-related issues. A respondent said that in Northern Bahr el Ghazal VISTAS “built livestock and trade support structures because these are pastoralist communities and the livestock infrastructures are where people interact; in Aweil East and Aweil West along migration routes, butchers, vet clinics, and vaccination pens, cow skin and hides, and slaughter slabs; now this is where communities come and interact.” 4. EFFECTIVENESS OF VISTAS APPROACH TO PARTNERSHIP Respondents generally perceive a benefit to a robust partnership strategy, which would be led, strengthened, and encouraged from within the USAID mission. They called for greater coordination with the political, development, and humanitarian wings of the USG in South Sudan, as well as other partners on the ground. One way to view VISTAS is as an incubator to pilot activities, which other projects then upscale as full-blown development Projects (for example, in the areas of livestock and the cattle economy). Opportunities equally exist for coordination with humanitarian programs such as Food for Peace (FFP) or the World Food Program’s (WFP) Food for Work (FFW) programs. The coordination strategy needs to be strengthened and encouraged from the USAID level for programming to build synergies that VISTAS potentially provides. USAID can factor VISTAS into its donor coordination. On the political level, it was noted that there is a need for a “balancing act of the bottom-up and top￾down approach; the Embassy could be more involved if it could plan for the engagement.” One respondent said, “We need to have that national level addressed, and while it is not [up to] VISTAS to do that, it is the Embassy…and that requires some degree of coordination and interaction between VISTAS at the local level and national level – there has to be some relationship…and we don’t have that…” Another respondent said, “Clearly, it will not be for VISTAS to take on elite politics, but there needs [to be] greater synergy, like with OTI where there is close linkage and coordination between development and Embassy.” The same respondent suggested that a collaboration on a national chiefs conference could have proceeded if they had the “Embassy to generate political cover,” SUCCESS to provide “technical expertise,” and VISTAS to do the screening. In some cases, respondents were concerned about duplicating activities or doing things in areas where VISTAS was not strong. One respondent said, “the UN is coordinating POCs, so what do they need from us?” Another respondent focused on the strength of the VISTAS mechanism in terms of filling gaps: We can feed in to fill gaps - the Non-Violent Peace Force (NVPF) for example, where VISTAS provided transport to move participants. There were a lot of ambushes, cattle raiding, river attacks, so traders could not cross; all the governors wanted to have an interstate conference, and a quite wonderful discussion took place. The Governor of Bor and former SPLA spokesperson is a very good person; his initiative, along with those of the neighboring Governors, was instrumental, and VISTAS supported the gaps. 33 Several respondents pointed out that incentives are lacking within the in-kind grant mechanism, because some of the most capable partners expect some level of remuneration for their work. Respondents noted an exception in the cases of women and chiefs, who seem more comfortable working on a volunteer basis. Others also noted sustainability issues in terms of maintenance of equipment and continuity of processes. However, the in-kind mechanism was also recognized to deter creation of dependency. 5. EFFECTIVENESS OF THE VISTAS GRANT-MAKING MECHANISM Respondents appreciated the small grants and decentralized decision-making. The in-kind grant mechanism is fast and effective for working with informal organizations, but should not limit VISTAS from working with innovative and capable organizations through other grant mechanisms—such as simplified grants and FAAs—to achieve the desired strategic impacts. Several respondents with institutional memory cited the personality of the first COP as having established the organizational culture of “getting the job done,” regardless of difficult operational circumstances. The VISTAS operational mechanism provides for rapid response capabilities to grant opportunities and support to local implementing partners that may or may not be officially registered. One respondent said, “What works in the OTI model is that we are not wedded in the written-in-stone five-year contracts. Plus, our OTI model is based on small grants that don’t require registration. I think the rest of USAID should operate this way.” VISTAS relies heavily on trusted people in geographic pockets for high performance. The most cited example of successful partnership was in Northern Bahr el Ghazal where “one of the things VISTAS has done is to empower local CSOs, even though we know the civil society is not very active.” VISTAS helped to register a hybrid institution, the Joint Border Peace Development Agency (JBPDA) in Northern Bahr el Ghazal, “and then continued to build their capacity” and layer their programs. VISTAS planned and recently approved a roving team to address East, West, and Central Equatoria in a manner that maintains a small administrative footprint but with the capacity to get resources to Partners. One respondent said, “the key is to have local partners with light footprint so we don't have heavy operational burdens; more targeted and layered programming with enough operational and programmatic resources, but to be able to impact from different sides as opposed to one-off activities.” In terms of tools, respondents appreciated the Deep Dives as an effective planning mechanism. One respondent said, “the Deep Dives are most substantive and useful; …the difference between a tweet and a book; for the Deep Dive we close our lap top and close the door and really think it through.” The CSV is a key data-capture stage in idea development. The activity database is being updated regularly, but the information is not fully utilized because of heavy workload and higher-level strategy misaligned with operations. In other words, there are no guidelines for activity selection and layering based on national level analysis or lessons learned. Several respondents recognized that the grant mechanism provided fast processing of grant requests through the yellow light and green light processes, flexibility in selecting implementing partners since 34 local CSOs, CBOs, and traditional authorities did not have to be registered, and significant authority to RPMs whose recommendations regarding grant activities were usually supported. However, VISTAS could improve how it codes activities and related information. For example, before the events of December 2013, VISTAS did not need to specifically track activities that were in government- or opposition-controlled areas. Afterwards, VISTAS loosely coded some activities as such. When looking at the 110 grants that were coded, of the total 310, 26% are coded to government￾controlled and 8% are coded to opposition-controlled. Many grants are not being coded to either. (To date, approximately 81% of TAP funding has been implemented in government controlled territories; with approximately 19% in IO controlled territories and POCs, coded or not.) VISTAS is generally considered to be a flexible and context-driven operation. VISTAS maintains a highly qualified mix of international and national staff that constitute one of its core resources. VISTAS is creative and pioneering. It has contributed to knowledge creation and is learning-focused. Staff members are risk-taking and see their presence and relationships as important. 6. EFFECTIVENESS OF MORNING STAR Morning Star is a unique initiative within VISTAS that seems to be appreciated at all levels, although its tangible results are yet to be seen. Morning Star has carried out 47 activities. One respondent said, “A really dynamic and important part of the program is trauma work and awareness. Master trainers have been assigned to all regional offices…their job is to look at conflict mitigation through a trauma awareness lens, and putting trauma awareness in as many activities as they can.” Another respondent said, “We had only touched a limited number and now we can go a little wider and that was more important than the sustainability aspect right now.” Another reflected on achievements and noted, “Morning Star and where it is going is already an achievement and will continue to be an achievement.” Respondents said respectively, “Morning Star is rolled out in support of other regional objectives,” and, “the direction we are taking with Morning Star was great.” According to one respondent, “Morning Star is addressing the trauma locally of people who have brought trauma back; they begin displaying traits that they bring back - like wandering aimlessly in the streets. While the trauma is not from what is happening within the community, it is brought back to the community from areas they had gone and where they experienced conflict.” However, there was no consensus on how to approach trauma or Morning Star as a program. One senior staff “wanted the rest of the Mission to sub to Morning Star,” while another idea was to integrate it into a livelihood framework. Part of the complication is that “Morning Star is a special case that came from the top…it was going to be a stand-alone program as an NGO with a board, but when it came time to funding them…the board kind of imploded and we realized this is not a viable solution, but the need was still there.” So, Morning Star was re-conceived from being “so big that it was unrealistic” to something that is, hopefully, more manageable within the VISTAS framework. According to VISTAS 2016 Q1 Quarterly Report, in the last quarter, Morning Star community sessions resulted in 110 dialogue discussions. The MS team conducted a total of 18 field visits to provide technical support and monitor some of the sessions. Morning Star is a tool to be used and a good example of adaptability and growth of the VISTAS Project. One concern raised is that Morning Star needs greater expertise in psycho-social support given 35 trauma work also carries risks. It should note that bi-annual Debriefing Sessions and In-Service Trainings have been organized that will provide psycho-social support, as well as a contract for ongoing reach back support from trained counselors for the Master Trainers. CONCLUSIONS From the report’s findings of the document review, interviews, focus groups, database analysis, and field observations, the following conclusions and lessons learned are presented. 1. VISTAS is a strategic resource for the USG in South Sudan. It has built and maintained good will and an operational footprint in remote parts of South Sudan at the height of the war through national and international staff working together as teams. Its operational capabilities may prove beneficial to USAID in the future. 2. The USG policy shift from state-building to community-building represents a new paradigm, but there is no evidence of a specific policy guidance that restricts work with local government. VISTAS want to continue supporting local government and believe managers can follow a nuanced approach for successful results. To move in this direction, they require authorization from USAID. 3. The highly decentralized, small grant-making system provides significant space for local participation in decisions, making VISTAS culturally sensitive and highly responsive. VISTAS can seamlessly fit into the OF without losing its bottom-up character if a national program strategy is designed for that purpose (see Recommendations). 4. OF disconnect from the activity level reflects a highly decentralized system that provides significant space for local decision-making, particularly by the RPMs. However, there is little or no understanding of the OF at all levels of the program. 5. USG must determine more specifically what they want VISTAS to do in South Sudan given the current political context, starting with re-articulation of the goal and a refocusing of the Project’s strategy. 6. VISTAS’ dynamic platform for innovation in a difficult and shifting operating environment currently aggregates activities at a regional level, which can further advance to a national integrated strategy, assuming the necessary facilitation and oversight from USAID. 7. Currently, there is a bottom-up flow of information and lessons learned. Top-down feedback loops should be strengthened, with an emphasis on how to use such information to feed into existing strategy execution. In this respect, the Assessment Team notes that all tools are in place to strengthen the organizational culture around information, including by redesigning the database to serve an analytical function based on coding. 8. Healthy partnerships with a diversity of actors cultivated over time start with entry into community and evolve into a shared sense of ownership and responsibility. VISTAS’s mechanisms are well suited to adjust strategy and operations and, with effort, can consolidate a more strategic approach to management at national level. 9. VISTAS can respond with agility, but resources are spread across vast territories and loosely connected activities. VISTAS faces important decisions regarding geographic targeting. The Project should consider leaving areas where it has worked for a long time to address new 36 hotspots or decide consciously and with articulated reasons to remain where it currently operates, consolidating results and broadening activity slightly if resources permit. 10. VISTAS succeeds by going with the grain of society, supporting collective actions and hybrid institutions that bring all corners of community together in dialogue. This is particularly notable around facilitating cattle migrations and layering around those core activities that focus on cattle. Cattle is both a source of conflict and cooperation. Strengthening the cattle economy in line with these suggestions and other best practices is an important area for VISTAS. 11. Cattle-related activities appear most directly related to OF TO 2.2 (Strengthen inter- and intra￾communal relations and reconciliation); and to achieve it, VISTAS can launch other supportive activities directly pursuant to TO 1.1 (Facilitate community-led response); TO 1.3 (Strengthen Livelihoods and Resilience to Shock); and TO 2.3 (Increase space for conflict mediation). These can be layered to support TO 2.2, which is the model in Northern Bahr el Ghazal. Sustained mechanisms of dialogue and interaction may in turn support TO 2.1 (Support inclusive peace process), perhaps by feeding into national dialogue/guidelines on mitigating cattle-related conflicts. 12. VISTAS’s main challenge is identifying reliable partners, building relationships and trust with partners, and getting resources to partners in a manner that is efficient, accountable, and supports the OF. The dangers of misinterpretation of VISTAS support, given the political nature of conflict and its mitigation, calls for staff to remain vigilant in this regard. 13. A robust partnership strategy is necessary for USAID programming to build synergies. VISTAS can serve as an incubator to pilot activities, which other USAID projects can upscale over time. 14. In response to operational challenges, VISTAS pivots and adapts, drawing on the in-kind grant making mechanism and a determined, resilient staff. 15. Morning Star’s human resources are in place to achieve the objectives. However, learning and communication requires focus and information-sharing. Since Morning Star is an example of integrating and harmonizing at national and local levels, it should be carefully observed as a priority for learning. Based on concerns raised by VISTAS regarding potential unintended psycho-social effects of the trauma work, requisite precautions are necessary to minimize risk and ensure that the VISTAS Project does not exacerbate conflict, but instead helps mitigate it. 16. The small grant mechanism of VISTAS has the potential to support unity through community￾based engagements such as preparations for the next Olympic Games. An Olympic debut was a once in a lifetime opportunity for citizens to mobilize the nation under one flag, “South Sudan.” An Olympic process should organize athletic competitions in all the states and host a national tournament to fairly select the best talent in the country, celebrate them as the South Sudanese national team, and send them to compete under one flag with the support of the whole country. Citizens can then watch with pride as South Sudan competes with the athletes of other countries. While donors supported the team’s participation internationally, no process supports the nation-building agenda domestically through athletics. Yet VISTAS is designed to support local activities with national implications. Other donor countries supported the Olympics team to get to Rio, but may not have had the resources to support local level competitions that would feed into a national process. 17. The opportunity regarding the Olympics, however, illustrates a broader point. There are other nation-building processes from the local level towards a national or international impact, and many of these target the youth. For example, there are beauty contests, talent shows, spelling 37 bees, and academic competitions. Each can promote healthy inter-tribal relations in a framework of constructive competition from local activities towards national effects. VISTAS could encourage these layered activities in areas where VISTAS is engaged in dialogues, including areas where VISTAS is building a new presence. National level partners could help build activities over time, supplementing the efforts of the VISTAS team. LESSONS LEARNED & RECOMMENDATIONS LESSONS LEARNED ISSUES, TRENDS, APPROACHES, AND THEORIES OF CHANGE THAT WERE SUCCESSFUL • Awareness among USAID and VISTAS of: the importance of understanding the relationship between the local and national levels of conflicts, the role and potential value of traditional authorities and women as partners in peace, and the recognition that long-term operational relationships based on mutual respect, open communication, and joint decision-making produce better results is critical for the Project’s success. • Well-layered projects are successful. Layering should be approached purposefully with a careful understanding of community dynamics. The VISTAS layered activities that have been effective deal with traditional chiefs, women, and/or cattle-related issues (especially migrations) and markets (trade). VISTAS is just beginning to layer activities. The concept and its application remain partially understood. Northern Bahr el Ghazal embodies successful layering. However, in many areas, activities can be related in principle, but are not designed with the intent to build synergies. • Accomplishments occur where there is a strong and trusting relationship with local authority. Limiting support to local government may have created cleavages in communities and missed program opportunities to contribute to stability and recovery. RPMs may be well positioned to make responsible, informed decisions about programming using a do-no-harm approach. However, the politically sensitive issues in a fluid policy environment require “a strong, ongoing (daily if necessary) two-way communication between VISTAS and USAID.” ISSUES, TRENDS, APPROACHES, AND THEORIES OF CHANGE THAT WERE NOT SUCCESSFUL • Oftentimes, information was channeled from the bottom-up, and not enough information was shared from the top-down in terms of analysis, feedback, and guidance across regional portfolios. • In some cases, too many activities were exercised in certain areas without proper layering or focus. • In Equatoria USAID and VISTAS were slow to pivot towards new flashpoint areas. • There was limited engagement on national issues with insufficient analysis of the political context. • There was staff turnover in key management positions, especially at USAID, which raised concerns over institutional memory and made it more difficult to maintain and manage a strategic approach. 38 • There was confusion regarding the policy as to whether VISTAS could engage with local governments. • Sustainability issues have yet to be fully addressed, thus promising that the effects of the Project will remain even after it has closed. This involves building and incorporating sustainability elements into programming at all levels such as exit and handover plans. • The question of how to incentivize local partners remains a challenge especially with the in-kind grants models where grantees are not paid and hence may not have full incentive to work or stay committed to activities, a concern raised by respondents. This is even more important among poverty-stricken communities as result of prolonged war. However, at the same time, the principles of community self-help, volunteer work, and community service are important for VISTAS success and should be maintained. This tension between professionalism and volunteering may require further consideration. RECOMMENDATIONS FOR USAID 1. Communicate regarding the operational framework (OF) or the most current Mission strategy document to VISTAS and other implementing partners and train them in their capacity as the vehicles by which the Mission implements its strategy. 2. Clarify and disseminate the broader USG policy regarding South Sudan to VISTAS leadership and staff, taking care to clarify the USG position on: a. Support to local governments and local government officials b. Cooperation with and support to change agents3 c. A nuanced do-no-harm approach with due diligence and a long-term view. This should also apply to other USAID implementing partners in South Sudan on an ongoing basis considering policies and operating environments are fluid. 3. Use a “less extensive or simplified strategic planning” process that optimizes all Project elements (RA’s, SRS’s, Deep Dives, etc.) facilitate a Strategic Planning process for the VISTAS Project that gets all staff on the same page by: a. Clarifying the goal and Theory of Change (TOC) b. Reviewing and harmonizing the Project’s execution mechanisms and how they are being used (including RA’s, SRS’s, Deep Dives, etc.) c. Creating a common understanding among stakeholders of layering, and guiding consistent implementation d. Codifying principles for activity selection including due consideration to the role of: sequencing, clustering, scaling-up, and coordination e. Redefining “critical areas” and “flashpoints” based on an understanding of structural factors that cause conflict across South Sudan and yield cooperation, especially aimed at linking livelihoods with conflict mitigation and recovery; f. Reviewing geographic targeting g. Defining the classification and coding system for activities that could make the database more conducive to analytics; i.e., to provide feedback and making project adjustments. 3 Change agents are cited as important for Project effectiveness because of their influence on local communities and local government officials. 39 4. Pay close attention to the border areas among communities that have been supported by political actors with influence in Juba. In such cases, competition for vital resources may connect local areas to Juba-based power struggles. VISTAS should be able to say what approach to layering best addresses such local conflicts and the concerns of local communities, while also remaining cognizant of how these local impacts interact with the actors in Juba who wield extraordinary influences. 5. Invite presentations to Heads of Mission (HOM) meetings, Heads of Cooperation meetings (HOCs), donor working groups, quarterly implementing partner meetings, USAID “Our Hour,” and USAID technical teams to build synergies and explore opportunities for VISTAS activities to more fully support the OF. RECOMMENDATIONS FOR VISTAS 1. Vistas should refine use of Deep Dives, Rolling Assessments (RAs), and Strategy Review Sessions (SRSs) to execute strategy while harnessing lessons learned from activity flow. 2. Management should develop procedures to monitor and ensure that the recommendations are acted upon and utilized. 3. Develop guidance for partnerships with grantees emphasizing how VISTAS enters a community, the ongoing communication and learning process, and a do-no-harm approach. 4. Determine core competencies and comparative advantages of the activities VISTAS does best and the capacity to implement them as a package approach, with regard to timing and layering dialogue, trade, and small-scale infrastructure with migration-related issues across borders where communities interact. 5. Plan for and invest in long-term partnerships, especially in new flashpoint areas. Include local implementing partners to develop capacities across regions, codify gains, and increase institutional memory at the national level while setting up partners for eventual handover. 6. Consider activities that integrate vertically into existing conflict mitigation and nation-building programs such as the Olympics and the National Dialogue. 7. Consistently seek clarity from USAID on policy matters and ensure consistency in implementation of the strategy. 8. As appropriate and when applicable, VISTAS should consider other grant making mechanisms (such as simplified grants and Fixed Amount Awards) to achieve desired strategic impacts and attract innovative and capable organizations that require cash assistance as well. The primary objective is for VISTAS to have as many available tools as possible to exercise its activities and strengthen its partnerships. 9. VISTAS should define its information needs at different levels of the Project and improve the analytics capacity of the database to meet the identified information needs. Currently, the database is mainly used for activity approvals. It can be adjusted to improve its analytics capabilities to generate monitoring information that supports organizational learning and spur performance improvements. 40 SCENARIOS The following five scenarios illustrate possible articulations of some of the strategic pathways VISTAS may consider during a strategic planning process and are suggested to spark dialogue. SCENARIO 1 – VISTAS incorporates “change agents,” locally and nationally, within coalitions for peace, which increases space for meaningful interactions across state-society lines, with impacts that may trigger system-wide improvements and/or transformations over time. SCENARIO 2 –VISTAS tries to stabilize local areas by building community resilience through well￾layered activities that drive value to local systems. SCENARIO 3 – VISTAS supports local activities that cascade upwards to the national level – such as in the case of sports (Olympics), dialogue (National Dialogue), trauma awareness (Morning Star), and a range of international competitions, such as Taekwondo (an activity VISTAS supported). SCENARIO 4 – VISTAS targets national level activities that directly support implementation of the peace agreement through Track II diplomacy, support to media, and citizen participation. SCENARIO 5 – VISTAS launches a nationwide strategy to mitigate cattle-related conflicts and normalize migration routes, targeting borders where groups interact in the cattle economy by layering dialogue, trade, and other synergistic activities. SCENARIO 6 – VISTAS develops a hybrid whereby local activities that integrate gender and are strategically layered assist critical target communities to become resilient to conflict and its impacts. The activities will connect to broader coalitions for peace across state-society lines, emphasizing the change agents as the center of a partnership strategy. There will be efforts to empower change agents not only in the peripheries where militias originate, but also in the center at the target of their political aims. GENERAL VISTAS has made important contributions to lessening local conflicts and is valued and appreciated by grantees and beneficiaries alike for the meaningful impact it is having on communities. However, the original VISTAS strategy was conceived at a time of stabilization when the major fear in South Sudan was local conflicts. The context shifted and the power struggle in Juba within the ruling party became the epicenter of conflict that destabilized the whole country, leading to local conflicts, some of which were a continuation and/or expansion of existing conflicts, and some of which were totally new. VISTAS recognized this and tried to adjust its strategy to be relevant to the new context while also responding to shifting USG policy, which at times appeared unclear. VISTAS has begun to engage nationally and to respond concurrently to local conflict. Despite the successful view of VISTAS in the field, the Project tends to emphasize differences across regions, contributing to incoherence in national strategy—most acutely felt by the disconnect between goals and activities, demonstrating how problem analysis at a national level is incompletely shared across regions for activity selection and layering. Despite differences, the common features that cut across all target areas include the following: 41 ● All areas are bounded and defined by rivers and water resources ● Areas share common political and administrative features (bomas, payam, and counties) ● Areas are interconnected by a cattle economy that is increasingly militarized and criminalized, despite continued interdependence in trade and marriage ● Most communities (particularly pastoral) pay dowries and practice polygamy ● All communities are organized by age sets that are used for community defense, dialogue, and social inculcation ● All possess traditional authorities that are weakened yet remain a front line for conflict mitigation; ● All have women who compose songs encouraging young men to either pursue war or peace ● All possess small arms that are readily available. VISTAS has provided substantial information for understanding common variables and their local and national impact. Militias are seemingly born out of traditional “defenders of cattle” who serve as proxy for political power struggles in the center share characteristics and face similar conditions in target areas. They are the conduit for national conflict to take root locally and for local conflict to escalate and expand, often with national implications. Both bottom-up and top-down approaches are valid, and the relationship between the two is a matter of strategy. VISTAS should be complimented and appreciated for its presence, flexibility, responsiveness on the ground, and speed of service delivery. Moving forward it should not compromise on its strengths, which include excellent human resources recruited locally, regionally and internationally. VISTAS should continue to build long-term, mutually respectful community relations. It should also, however, harness the power of innovation from its creativity and better define its National Strategic Plan, Guiding Principles, and Information Feedback Mechanisms centered on the Deep Divers, SRS, and Rolling Assessments, all of which can be supported by a re-calibrated M&E system that codifies and analyzes key information into metrics that will enhance organizational learning and strategic decision making. 42 ANNEX I Scope of Work Assessment of USAID South Sudan’s Viable Support to Transition and Stability (VISTAS) Project BACKGROUND On July 2013, USAID/South Sudan launched the Viable Support to Transition and Stability (VISTAS)4 Project as the U.S. Government’s flexible response to the needs in the world’s newest country. Implemented by AECOM International Development (AECOM), the goal of VISTAS was to strengthen South Sudanese confidence and capacity to address the causes and consequences of political conflict, violence, and instability. The initial Project objectives were to: 1) build capacity of civilian state authority to prevent, respond to and mitigate conflict in key flashpoint areas; 2) promote increased access to information and engagement of citizenry in support of political and peace processes at the local level; and 3) engage at-risk youth and other vulnerable groups in productive social and economic activities. As a result of the conflict in December 2013, USAID/South Sudan personnel were evacuated, and almost all USG programming was suspended. During this time, USAID’s Office of Transition Initiatives (OTI) assisted the Mission by managing VISTAS on a temporary basis and helping shape a strategic shift of the Project in light of new political realities. The VISTAS team returned to Juba in July 2014 and restarted programming with newly-defined objectives, under the co-management of OTI and USAID’s Africa Bureau. The VISTAS Project shifted from predominantly working with the local government in the states to mitigating communal violence and rising tensions in critical areas where conflict could have national implications. The Project returned to USAID South Sudan management under the Office of Transition and Conflict Mitigation (OTCM) in January 2016. ASSESSMENT PURPOSE The purpose of the assessment will be to document the Project’s effectiveness, challenges, and progress against the stated objectives to date and assess the Project’s approach and ability to respond to changes in a fluid context. Assessment results and findings will inform VISTAS’s direction during the final two years of the Project and provide recommendations for future programming based on lessons learned. The assessment will also provide an opportunity to share lessons learned with USAID South Sudan and other interested groups. The assessment should maximize the learning opportunities of the VISTAS team. 4 The Project was originally titled South Sudan Transition and Conflict Mitigation Program II (SSTCM II) but a name change took place before the Project launched in July 2013. 43 EXISTING PERFORMANCE INFORMATION SOURCES Initially, the MESP assessment team will review the VISTAS original project documents, cluster assessment reports, work plans and other performance reports as well as quarterly and annual reports. Any portfolio review materials conducted by the Mission as well as site visit observation reports will also be reviewed. ASSESSMENT QUESTIONS The evaluation report will answer the following assessment questions: ● Is the current program strategy relevant and appropriate in the South Sudan context and the context of USAID/South Sudan’s strategy? To answer this question, the assessment team will consider the following: o Is the program able to pivot to adapt to changes in the context? If so, how well? o Within the strategy, is the overall geographic focus still correct? Is VISTAS working in areas where communal violence has national implications? If not, what geographic areas should VISTAS focus on? o Is the program focused on key strategic issues in South Sudan from the perspective of U.S. foreign policy and USAID strategy? ● What are the most effective elements of the program to date and what challenges has the program faced in meeting the stated objectives? How has the program responded to those challenges? ● What key issues, trends, approaches, and Theories of Change were particularly successful and should be flagged for other USAID programs operating in South Sudan? Which were not? To answer this question, the assessment team will consider the following: o Which tools and approaches, or combinations thereof, have been most or least effective at facilitating conflict mitigation? o Was VISTAS team able to develop critical relationships within the community and relevant stakeholders to advance program/activity goals? o Has VISTAS remained in a community for an appropriate period of time to lay the groundwork for longer term peace building and development? o Has VISTAS’s “layered” appropriate objectives and activities to add up to effective conflict mitigation? o How has VISTAS taken gender dynamics in conflict and peace building into consideration in activity design and implementation? o What are the lessons learned and recommendations for USAID South Sudan related to VISTAS’ methodology and approach to programming? METHODOLOGY The assessment will utilize a mixed method approach, relying on quantitative and qualitative data from both primary and secondary sources. Data will stem from program documents, other secondary source data (like from the projects cluster studies and field visits) and will include key informant interviews, 44 group discussions and observations. The e assessment team will triangulate data from these sources to generate evidence to address the assessment questions. The assessment will generate findings, conclusions and recommendations based on the analysis.” Secondary data will be based on VISTAS reporting and monitoring as well as review of political/economic information by the assessment team. Primary data collection will be difficult to achieve in field sites but will be gathered through targeted third-party monitoring by MESP and a possible assessment team for data collection inside South Sudan. The bulk of primary data collection will be through interviews of USAID and AECOM staff, and key informants familiar with VISTAS work, but not primarily AECOM staff. AECOM staff will participate and be asked to help MESP gain telephone/Skype access to some beneficiaries and stakeholders in the field. Given the above, the amount of fully objective data available (that is, not dependent on implementing partner sources) are limited. However, a wealth of practical information and strategic wisdom will be tapped from USAID and AECOM staff to help understand the past and plan for the future. The team will have to guard against the risk that focusing on only internal perspectives may suffer from biases and “group think.” ASSESSMENT LOE The evaluation consultants will consist of four main team members; a Team Leader and two technical experts. In addition, a representative of USAID will also join the team. However, USAID will join the exercise on part-time basis, and in selected trips. The Team Leader will take full responsibility for managing the team, organizing its work, and ensuring quality control and delivery of a final report acceptable to USAID standards. TEAM LEADER: Should be a senior Evaluation Specialist, and a postgraduate degree holder in International development, Evaluation, Political Science, or any other related Social Science. S/he must have at least 10 years’ experience – 5 of which should be working in a developing country context especially in the field of political transformations, engagements of similar nature. The candidate should also have analytical and good report writing skills. S/he must have experience of leading large scale studies. A sound knowledge of understanding USAID programming approaches and methodologies will be an added advantage. TEAM MEMBERS: Three technical experts with extensive experience of 8 + years. The technical experts should have postgraduate degree in Peace-building, Conflict management and resolution, Political science, Community development or any other related Social Science. S/he should also have expertise in one or combination of the following technical areas: building capacity of civilian state authority to prevent, respond to and mitigate conflict; promoting increased access to information and engagement of citizenry in support of political and peace processes at the local level; and engaging at-risk youth and other vulnerable groups in productive social and economic activities. Previous experience in Office of Transition Initiatives (OTI) in other countries or engagements of similar nature is an added advantage. The individuals should have experience in research and demonstrated knowledge of conducting qualitative studies. Local experience as well as experience in Africa or/and other similar settings will be an added advantage. 45 ANNEX II MESP Approach to VISTAS Assessment (Mid-October/Mid-December, 2016) Guiding Principles In the context of the deteriorating peace and security environment of South Sudan, USAID/SS seeks to undertake a “Learning Assessment of VISTAS Experience for Strategic Adaptation (LAVESA)” for these purposes: ● More to support learning than to judge performance; ● It will be called an assessment, in as it will involve rigorous inquiry but not fully align with USAID’s Evaluation Policy; ● To learn from the experience of VISTAS and to understand how that mechanism can be revised and supported in the evolving context of South Sudan; ● It will be grounded in the experience of VISTAS based on reports, data, and insights from VISTAS, stakeholders, and USAID; ● VISTAS is already considering strategic adjustments to the location, activities, and operational aspects of the Project due to the changing environment and the recent Rolling Assessment in preparation for a Strategic Review Session (SRS) in November; and ● The results of the assessment will inform activity adaptation by including AECOM and USAID in the inquiry, learning, and developing recommendations to ensure full ownership and rapid application of what is learned. Operational Considerations ● Security conditions in SS do not currently permit MESP to conduct a field-based evaluation. ● AECOM and MESP staff are geographically dispersed: 1. USAID and AECOM: a. Expat staff are outside of Juba in either Nairobi or D.C. b. USAID South Sudanese staff are split between Juba, Kampala, and Nairobi c. VISTAS South Sudanese staff continue to work across South Sudan with some in Nairobi. 2. MESP a. Expat staff are outside of South Sudan, generally in Uganda, USA, Kenya and Australia b. South Sudanese staff are working from Juba with a few in Kampala 46 c. The MESP offices in Juba are operational to support meetings and third party monitoring. Conclusions: Secondary data will be largely based on VISTAS reporting and monitoring as well as review of political/economic information by the assessment team. Primary data collection will be difficult to achieve in field sites but will be gathered through targeted third-party monitoring by MSI and a possible subcontract for data collection support inside South Sudan through Forcier, a South Sudan subcontractor approved within the MESP contract. The bulk of primary data collection will be through interviews of USAID and AECOM staff, and key informants familiar with VISTAS work, but not primarily AECOM staff. AECOM staff will participate and be asked to help MESP gain telephone/skype access to some beneficiaries and stakeholders in the field. Given the above, the amount of fully objective data available (that is, not dependent on implementing partner sources) are limited. Fortunately, there is a wealth of practical information and strategic wisdom to be tapped from USAID and AECOM staff to help understand the past and plan for the future. The team will have to guard against the risk that focusing on only internal perspectives may suffer from biases and “group think.” Recommendations: Since field learning will be constrained, LAVESA will strive to tap as much as possible the experience and wisdom within AECOM, USAID, and key beneficiaries and stakeholders. To maximize utilization and speed data collection and analysis of project experience, participatory workshop events will engage key players from AECOM and USAID in learning from experience and planning the future. The Assessment Team will structure these discussions based on analysis of VISTAS experience (from interviews, data review, and limited field monitor visits) and strategic perspectives (see below). To maximize learning and ownership of the results – while containing costs and fully utilizing assets – it may be advisable to hold facilitated learning sessions in Juba, Nairobi, and Washington, D.C. MESP’s Approach As discussed MSI will undertake a three-part effort: I. External analysis of USG priorities, the South Sudan context, and VISTAS mechanisms to inform and shape a way forward. This is to begin as soon as possible and feed into Parts II and III: a. An initial analysis (to inform Assessment Questions 1, 1B, and 1 C – see Annex) by credible South Sudan experts and practitioners regarding: i. A summary of current USG/USAID/State interests and priorities in South Sudan ii. A description of the evolving political/economic/social/security conditions to provide a context to understand what can accomplish in South Sudan. iii. Approaches and lessons learned from other conflict/post-conflict countries that could be applied to South Sudan5 . iv. A brief compilation of the work of other USAID programs (e.g., PROPEL, SUCCESS, peace initiatives, etc.) and donors in these areas to understand complementarities and leverage resources. 47 PRODUCTs: (1) Summary of USG priorities, the South Sudan context, and a description of the VISTAS mechanism. (2) Brief preliminary conclusions relevant to LAVESA. b. The Assessment Team will develop a subsequent analysis, informed by preliminary conclusions derived from the secondary data and interview portions of Part II, brainstorming how strategic adjustments to the location, activities, and how operational aspects of the Project could be adopted to optimize results. NOTE: this is meant only to generate a starting point for Part III. Since Conclusions will be modified, and recommendations generated, this is only an interim contribution to the final project. The intent is to ensure external experience and perspectives are considered in Part III. PRODUCT: Brief findings and conclusions, and recommendations to provide a framework for discussions in Part III. II. External analysis of VISTAS experience to date (to inform Assessment Questions 1B and 2): a. Systematic desk review of VISTAS monitoring, evaluative, and reporting documentation. This should contribute to a preliminary understanding of EQs # 1A and 2. b. Targeted interviews with USAID, AECOM expat and South Sudanese staff, as well as Key Informant Interviews with people familiar with VISTAS work but not AECOM staff, and beneficiary interviews (in person or on the phone/skype) to inform understanding of the experience and brainstorm the way forward. The assessment team would need to work with AECOM to develop a realistic logistical approach for this. c. Consider targeted field verification visits. This, too, will be dependent on AECOM recommendations regarding security and logistics and/or ability to procure a sub￾contract with a local organization. PRODUCT: Findings and Conclusions to be verified in Phase III. No recommendations will be made. Recommendations will emerge from Phase III. III. Participatory workshops with AECOM and USAID staff to validate preliminary conclusions and recommendations and develop a way forward for consideration by USAID. This will finalize analysis of all assessment questions and provide a path forward, based on those collaboratively￾determined results. Further discussions with USAID, and probably AECOM will be needed to flesh this out. Ideally, there would be a series of small workshops, beginning in Juba, followed by discussions in Nairobi and possibly Washington, D.C. The idea would be build up from field understanding to higher-level decision makers so that the way forward has full ownership, based on thorough understanding of what has been learned. These events must be thoughtfully facilitated. PRODUCTS: (1) Final Findings, Conclusions and Recommendations. (2) A time-frame for implementing the recommendations, possibly with alternative scenarios. Implementation MSI will need to hire expert consultants in the South Sudan Context and OTI programming and operations. MSI should, however, engage permanent MESP expat and South Sudanese staff in Juba, USA, 48 Kenya and Uganda in the planning, logistics, management, analysis, and training and oversight of any subcontractors related to the VISTAS assessment. 49 ANNEX III VISTAS Learning Assessment Data Collection Tool for USAID Conflict 1. How has VISTAS Pivoted with the Changing Context of Conflict What is the USG foreign policy? ● What RFPs and RFAs have been awarded by USAID in South Sudan from 2005 to 2011 (CPA Interim Period) and from 2011 - 2013 (Post-Independence Peace Period) and from 2013 - 2016 (Post-Independence Civil War)? ● What was the balance of investment in State-building and Nation-building respectively in each period? ● How do the USAID strategies reflect the direction of investment with respect to State￾building and Nation-building? Were there any changes in the USG policy on assistance to South Sudan after the 2013 conflict? If any, what was the rationale? ● Reference doc - Donor Principles Document Operational Framework Advisory Council Reports In your view, did VISTAS pivot in response to the USG policy changes? ● What were the specific changes to which the pivots responded? ● How did VISTAS design connects conceive of National/Local dynamics? Did this change based on the new Policy? ● What is USAID view of VISTAS Project strategy in terms of national/local conflict dynamics? Is it more accurate to say that National level dynamics impact local level or vice versa? 2. How has the security situation affected space for local action? Provide examples? ● How do the impacts of VISTAS activities affect the national level? ● How do decisions at a national level affect VISTAS? Are there incidents where national action and/or local action protect “democratic” space? ● How has the role of the Government changed since the CPA? Since the 2013 crisis? Since the 2016 crisis? 50 3. How does VISTAS Manage Policy-Relevant Information that can interconnect these Levels How does VISTAS collect, consolidate and report information? ● What kind of information does VISTAS collect? ● How has USAID used information collected by VISTAS at Policy-level? Provide examples. ● Are there any improvements you would want to see with regards to the information VISTAS collects, consolidate and report, including its use by either USAID or VISTAS? Coordination at Program Level (Design phase) ● What type of coordination took place during VISTAS design? Are there areas that could have been done differently and how? 4. How did VISTAS engage in the Cattle economy? Cattle raiding ● How did VISTAS respond to cattle related conflicts in the states they operate? Is there something VISTAS should you do differently? Migrations ● How has VISTAS helped to improve harmony among communities that share seasonal access to grazing? ● Who is the most important actor in addressing migration related conflicts? Government? Traditional leaders? Civil Society? Churches? Has VISTAS engaged with them? ● Is there something VISTAS should do differently? 5. Geographic Targeting ● What criteria does VISTAS use for geographic targeting? ● How has the current VISTAS geographic targeting taken into consideration inclusivity (in terms of tribe, gender, region, political orientation, other)? ● Do you have any recommendations for VISTAS regarding geographic targeting? 6. What is the Role of Key Stakeholder Groups? 51 Role of Civil Society ● How has the change in context since 2013 impacted on civil society’s role in South Sudan? What role should the VISTAS project play with regards to civil society? Does VISTAS work with Traditional Leaders, Churches and CSOs? ● How effective was the cooperation? ● Should VISTAS continue working with traditional leaders? Probe what could be done differently How has VISTAS supported trading between communities? ● What would you do differently? (Probe if something can be done differently to expand)? 52 ANNEX IV List of Interviewees 53 ANNEX V List of KEY DOCUMENTS 54 ANNEX VI Key informants (KIs) 55 ANNEX VII VISTAS Assessment Data Collection Tools (a) National Staff 1. Layering ⁃ What activities has VISTAS supported in this community? ⁃ Have they been layered? ⁃ Have they been sequenced? ⁃ What do these terms mean to you? ⁃ How does VISTAS balance between the number of activities supported and the quality of activities? ⁃ Is this balance effective? ⁃ Does it produce impact? ⁃ What would you do differently? 2. Targeting ⁃ How does VISTAS choose activities? ⁃ How did VISTAS choose the target area/community? ⁃ What criteria did you use? 3. Theory of Change (TOC) ⁃ How do you perceive the shift from working with Local Governments to Local Community Organization and Civil Society (including Traditional Leaders)? ⁃ Is it more accurate to say local conflicts lead to national conflicts or that national conflicts lead to local conflicts? ⁃ What is your understanding of VISTAS strategy? ⁃ Does the strategy of targeting local conflicts with national implications work in your experience? ⁃ Is it more accurate to say local conflicts impact national conflicts or national conflicts impact local conflicts? ⁃ In either case, give some examples. 56 4. Partnership ⁃ Who are the main actors in this community that influence conflict and its mitigation? ⁃ Who did VISTAS engage in the Activity? ⁃ Where did the idea come from? ⁃ How was the idea developed leading to funding? ⁃ How did you feel about the whole process including implementation? ⁃ What were some of the main challenges in the Partnership? ⁃ How can the relationship be improved? ⁃ What are the strengths and weakness of VISTAS grant making process? ⁃ How can it be improved? ⁃ What kind of information has VISTAS requested and what did you give them? ⁃ Was there any feedback from them after the activity was complete? ⁃ How long has VISTAS been operating in this community? ⁃ Did they build the kind of relationships for long-term success? 5. Gender ⁃ Do you have a gender strategy? ⁃ Do you think it is sufficient? ⁃ How could it be improved? ⁃ What have you achieved in terms of gender? ⁃ Give some examples. 6. Cross-cutting ⁃ Can people easily discuss issues in this community? ⁃ Has VISTAS helped people to discuss issues? ⁃ What issues were discussed? ⁃ Were there specific difficulties when discussing issues? ⁃ Where did these discussions take place? ⁃ What did VISTAS provide? 57 ⁃ Who are the key partners that need to be engaged in this community to resolved conflicts? ⁃ What are the roles of different stakeholders/partners (local gov’t, chiefs, women, youth, politicians, military)? ⁃ How have the VISTAS activities benefitted from these partnerships? ⁃ How effective are Traditional Leaders at solving conflicts in this community? ⁃ How can they be supported to become more effective? ⁃ How have they worked with VISTAS? ⁃ What impact has this markets in this place had on conflicts? ⁃ What contribution did VISTAS make? ⁃ Could it be improved? (b) Grantees 1. Layering ⁃ What activities has VISTAS supported in this community? ⁃ Is this balance effective? ⁃ Does it produce impact? ⁃ What would you do differently? 2. Targeting ⁃ How does VISTAS choose activities? ⁃ How did VISTAS choose the target area/community? ⁃ What criteria did you use? 3. TOC ⁃ Is it more accurate to say local conflicts lead to national conflicts or that national conflicts lead to local conflicts? ⁃ What is your understanding of VISTAS strategy? ⁃ Is it more accurate to say local conflicts impact national conflicts or national conflicts impact local conflicts? ⁃ In either case, give some examples. 4. Partnership ⁃ Who are the main actors in this community that influence conflict and its mitigation? ⁃ Who did VISTAS engage in the Activity? ⁃ Where did the idea come from? 58 ⁃ How was the idea developed leading to funding? ⁃ How did you feel about the whole process including implementation? ⁃ What were some of the main challenges in the Partnership? ⁃ How can the relationship be improved? ⁃ What are the strengths and weakness of VISTAS grant making process? ⁃ How can it be improved? ⁃ What kind of information has VISTAS requested and what did you give them? ⁃ Was there any feedback from them after the activity was complete? ⁃ How long has VISTAS been operating in this community? ⁃ Did they build the kind of relationships for longterm success? 5. Gender ⁃ How was gender taken into account in your activity? ⁃ Could it have been done better? 6. Cross-cutting ⁃ Can people easily discuss issues in this community? ⁃ Has VISTAS helped people to discuss such issues? ⁃ What issues were discussed? ⁃ Were there specific difficulties when discussing issues? ⁃ Where did these discussions take place? ⁃ What did VISTAS provide? ⁃ Who are the key partners that need to be engaged in this community to resolved conflicts? ⁃ What are the roles of different stakeholders/partners (local gov’t, chiefs, women, youth, politicians, military)? ⁃ How have the VISTAS activities benefitted from these partnerships? ⁃ How effective are Traditional Leaders at solving conflicts in this community? ⁃ How can they be supported to become more effective? ⁃ How have they worked with VISTAS? ⁃ What impact has this market had on conflict? ⁃ What contribution did VISTAS make? ⁃ Could it be improved? (c) Beneficiaries 59 1. Layering ⁃ What activities has VISTAS supported in this community? ⁃ Were they the right activities to address conflict? ⁃ Did they produce impact? ⁃ What would you advise VISTAS to differently to help resolve conflict in this community? 2. Targeting ⁃ How did VISTAS choose the target area/community for their activities? ⁃ Did VISTAS choose the right areas? ⁃ What would you advise VISTAS? 3. TOC (Theory of Change) ⁃ What is your understanding of what VISTAS activities are trying to achieve? ⁃ Is it more accurate to say local conflicts impact national conflicts or national conflicts impact local conflicts? ⁃ In either case, can you give some examples? 4. Partnership ⁃ Who are the main actors in this community that influence conflict and its mitigation? ⁃ What are the roles of different stakeholders/partners (local gov’t, chiefs, women, youth, politicians, military)? ⁃ How have the VISTAS activities benefitted from these partnerships? ⁃ Who did VISTAS engage in the Activity? ⁃ How long has VISTAS been operating in this community? ⁃ Did they build the kind of relationships for longterm success? 5. Gender ⁃ Does VISTAS engage women in their activities? ⁃ If so, how? ⁃ How would you advise VISTAS to engage women to resolve conflict? 5. Cross-cutting ⁃ Can people easily discuss issues in this community? ⁃ Has VISTAS helped people to discuss such issues? ⁃ What issues were discussed? ⁃ Were there specific difficulties when discussing issues? 60 ⁃ Where did these discussions take place? ⁃ What did VISTAS provide? ⁃ How effective are Traditional Leaders at solving conflicts in this community? ⁃ How can they be supported to become more effective? ⁃ How have they worked with VISTAS? ⁃ What impact has this market had on conflict? ⁃ What contribution did VISTAS make? ⁃ Could it be improved?