This document is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of a and do not necessarily reflect the views of USAID or the United States Government. REBUILDING INTERETHNIC TRUST IN BOSNIA AND HERZEGOVINA: IMPACT AND PERFORMANCE EVALUATION OF USAID/BiH TRUST, UNDERSTANDING, AND RESPONSIBILITY FOR THE FUTURE ACTIVITY (PRO-FUTURE) SEPTEMBER 2017 PRO-FUTURE LITERARY AND ART COMPETITION 2 MONITORING AND EVALUATION SUPPORT ACTIVITY (MEASURE-BiH) REBUILDING INTERETHNIC TRUST IN BOSNIA AND HERZEGOVINA: IMPACT AND PERFORMANCE EVALUATION OF USAID/BiH TRUST, UNDERSTANDING, AND RESPONSIBILITY FOR THE FUTURE (PRO-FUTURE) ACTIVITY September 2017 Prepared under the USAID Monitoring and Evaluation Support Activity (MEASURE-BiH), Contract Number AID-168-C-14-00003 Submitted to: USAID/Bosnia and Herzegovina Submitted by: IMPAQ International, LLC IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 3 ABSTRACT This evaluation is commissioned by the U.S. Agency for International Development in Bosnia and Herzegovina (USAID/BiH), in line with USAID’s Evaluation Policy, to examine the USAID/BiH 2013-2017 reconciliation intervention Trust, Understanding, and Responsibility for the Future Activity (PRO-Future). PRO-Future’s goal is to increase interethnic trust in BiH by helping citizens understand and accept each other’s narratives of the past (expected outcome 1), and by engaging key influencers in supporting local reconciliation initiatives (expected outcome 2). The intervention’s underlying development hypothesis is: if BiH citizens accept alternative political and historical narratives, supported by key influencers, they will change their individual attitudes and behaviors toward one another, resulting in increased social trust within communities, enabling citizens to work together to overcome societal divisions and constructively advocate for institutional changes, which in turn will lead to social well-being and economic prosperity. This impact and performance evaluation answers four questions, informed by the main elements of PRO-Future’s development hypothesis (evaluation questions 1 and 2) and two main expected outcomes (evaluation questions 3 and 4). The first two questions examine PRO-Future’s impact on interethnic attitudes and trust in participating municipalities/cities, while the third and fourth question examine progress toward achieving the two expected outcomes. The evaluation team employed a mixed-methods approach. For the impact portion of the evaluation (evaluation questions 1 and 2), three different household surveys were combined: the PRO-Future baseline survey and the Social Cohesion and Reconciliation Index for BiH (both conducted in early 2014, with sample sizes of 3,000 and 2,000 households respectively), and the PRO-Future end-line survey (conducted in the spring of 2017 with a sample size of 4,200 households). The impact analysis was conducted by using the difference-in-differences method to examine the impact of PRO-Future on interethnic attitudes and trust within the 30 municipalities in which PRO-Future has been implemented, compared to the regions in which PRO-Future has not been implemented. The study of implementation and perceived performance (evaluation questions 3 and 4) used the PRO-Future documentation and database, 65 key informant semi-structured interviews, four focus groups with 22 participants, and an online survey of 431 PRO-Future beneficiaries. Our evaluation does not find any significant overall impact on either attitudes or trust when examining all ethnic groups together, while calculations for the three ethnicities separately found different impacts for each, including different significance levels, direction, and magnitude of impact. This indicates that further research should be done on an approach to reconciliation interventions and its potential customization based on differences between ethnicities. Despite the inconclusive/ambiguous conclusions about PRO-Future’s overall impact, all stakeholders directly included in PRO-Future who spoke to the evaluation team perceived PRO-Future as useful and having a positive influence. This seeming dichotomy indicates that PRO-Future has had an important influence but on narrow groups and types of citizens. This is further confirmed in interviews/focus group discussions, in which most note that PRO-Future’s reach was mostly toward a narrow group of the most active and open citizens, such as CSO members and activists. Our evaluation concludes that reconciliation initiatives should be continued in BiH to build on and expand the reach of the considerable achievements of PRO-Future. The evaluation conclusions related to needs for specific adjustments of any potential future reconciliation interventions based on lessons learned from PRO-Future include: i) grounding interventions more extensively in psychological theory and in a clearer and simpler logical framework; ii) considering a more systematic way of targeting beneficiaries/participants aiming to reach less receptive groups iii) focusing on fewer localities with more intensity and with a further customized and sequenced approach considering locality specificities, iv) continuing with small grant allocation, v) focusing on activities that combine a reconciliation/peace-building component with interethnic interaction based on common local/target group interest to attract broader participation; vi) continuing with the speaking-out events as the most effective type of activity in influencing participants’ individual, personal transformation, vii) strengthening key influencers’ engagement with a focus on high-level and local politicians, religious leaders, and mainstream media; and viii) working on designing specific and actionable tasks to be implemented by the government based on the Platform for Peace signed within PRO-Future. 4 TABLE OF CONTENTS ABSTRACT ............................................................................................................................................... 3 TABLE OF CONTENTS .......................................................................................................................... 4 LIST OF EXHIBITS................................................................................................................................... 5 ACKNOWLEDGMENTS .......................................................................................................................... 6 ACRONYMS .............................................................................................................................................. 7 EXECUTIVE SUMMARY.......................................................................................................................... 8 EVALUATION PURPOSE AND QUESTIONS ..................................................................................... 8 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS .................................................. 13 1.1. EVALUATION PURPOSE .............................................................................................................................................. 13 1.2. EVALUATION QUESTIONS ........................................................................................................................................ 14 2. PRO-FUTURE BACKGROUND ................................................................................................. 16 3. EVALUATION METHODS AND LIMITATIONS .................................................................... 19 3.1. METHODOLOGY AND DATA COLLECTION ..................................................................................................... 19 3.2. EVALUATION LIMITATIONS ...................................................................................................................................... 24 4. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ................................................ 27 4.1. FINDINGS .......................................................................................................................................................................... 27 4.2. CONCLUSIONS .............................................................................................................................................................. 52 4.3. RECOMMENDATIONS ................................................................................................................................................. 56 ANNEXES ................................................................................................................................................ 59 ANNEX I: IMPLEMENTING PARTNER’S COMMENTS ON MEASURE-BIH PRO-FUTURE IMPACT AND PERFORMANCE EVALUATION ................................................................................ 60 ANNEX II: USAID DEMOCRACY OFFICE COMMENTS ON PRO-FUTURE IMPACT AND PERFORMANCE EVALUATION .......................................................................................................... 67 ANNEX III: EVALUATION STATEMENT OF WORK ...................................................................... 72 PROGRAM INFORMATION .................................................................................................................. 72 BACKGROUND ...................................................................................................................................... 72 EVALUATION QUESTIONS ................................................................................................................. 76 EVALUATION DESIGN AND METHODOLOGY............................................................................... 77 DELIVERABLES, SCHEDULE, AND REPORTING REQUIREMENTS .............................................. 79 ANNEX IV: PRO-FUTURE DOCUMENTS/DATABASES REVIEWED ............................................ 81 ANNEX V: ADDITIONAL INFORMATION ON PRO-FUTURE ...................................................... 83 ANNEX VI: DATA COLLECTION INSTRUMENTS .......................................................................... 91 END-LINE HOUSEHOLD SURVEY QUESTIONNAIRE ..................................................................................................... 91 IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 5 PRO-FUTURE EVALUATION INTERVIEW GUIDES ........................................................................................................ 105 ANNEX VIII: DISCLOSURE OF ANY CONFLICT OF INTEREST ................................................ 126 REFERENCES ........................................................................................................................................ 132 LIST OF EXHIBITS Exhibit 1: Interviewees/FG Participants per Beneficiary/Stakeholder Group .................................................................. 22 Exhibit 2: Evaluation Matrix ......................................................................................................................................................... 23 Exhibit 3: Group Averages for All Ethnicity Combinations, in % of Surveyed Citizens Expressing Any Level of Positive Attitude ............................................................................................................................................................................. 28 Exhibit 4: Group Averages for All Ethnicity Combinations, in % of Surveyed Citizens Expressing Any Level of Trust .................................................................................................................................................................................................. 29 Exhibit 5: PRO-Future Impacts on Inter-Ethnic Attitudes and Trust ................................................................................ 30 Exhibit 6: Citizens’ and PRO-Future Beneficiaries’ Opinions on the Role of Talking about the Past in Reconciliation (in % that somewhat/completely agree) ........................................................................................................ 34 Exhibit 7: Importance of the Nine Categories of PRO-Future Activities for Reconciliation, in % of Respondents Who Find Them Very or Somewhat Important ..................................................................................................................... 37 Exhibit 8: Requirements for BiH to Advance in Inter-Ethnic Reconciliation, in % of Surveyed Citizens in End-Line Survey ................................................................................................................................................................................................ 47 Exhibit 9: Trust in Different Categories of Key Reconciliation Influencers, in % of Surveyed Citizens Expressing any Level of Trust .......................................................................................................................................................................... 50 Exhibit 10: Mechanisms Planned in PRO-Future Design (per June 2014 Award Modification) ................................... 83 Exhibit 11: Role of Implementers in PRO-Future (per June 2014 Award Modification) .............................................. 85 Exhibit 12: PRO-Future Activities and Small Grants per Location, based on PRO-Future Database ....................... 86 Exhibit 13: PRO-Future Small Grants per Purpose, Based on PRO-Future Database .................................................. 87 Exhibit 14: PRO-Future Small Grants in the 30 Target Municipalities, Based on PRO-Future Database ................. 88 Exhibit 15: PRO-Future Activities per Purpose, based on PRO-Future Database ......................................................... 89 Exhibit 16: PRO-Future Direct Beneficiaries in Target and Other Municipalities, Based on PRO-Future Database ............................................................................................................................................................................................................ 90 6 ACKNOWLEDGMENTS Guided by the U.S. Agency for International Development (USAID) Evaluation Policy, USAID/BiH commissioned IMPAQ International, through USAID/BiH Monitoring and Evaluation Support Activity (MEASURE-BiH), to design and conduct an impact and performance evaluation of USAID/BiH’s Trust, Understanding, and Responsibility for the Future Activity (PRO-Future). The evaluation adopts a rigorous design and methodological approach to address the evaluation questions and to make links between findings, conclusions, and recommendations related to the extent of PRO-Future’s achievements in increasing interethnic trust in Bosnia and Herzegovina. MEASURE-BiH’s Evaluation Team members who worked on this evaluation and prepared this report include Naida Čaršimamović Vukotić, Evaluation Lead and MEASURE-BiH Chief of Party; Sabina Ćehajić Clancy, Evaluation Team Member, MEASURE-BiH external consultant and MEASURE-BiH Research Fellow; Emina Ćosić-Puljić, Evaluation Team Member and MEASURE-BiH Senior Research Analyst; Siniša Lakić, Evaluation Team Member, MEASURE-BiH external consultant, and MEASURE-BiH Research Fellow; Salminka Fazlić, Evaluation Team Member and MEASURE-BiH Research Analyst; and Sarah Moore, Evaluation Team Member and MEASURE-BiH external consultant. The authors thank those who contributed to the implementation of this evaluation and the completion of the analysis for this final report. At USAID/Bosnia and Herzegovina (USAID/BiH), Shanley Pinchotti, Democracy Office Director, and Elma Bukvic Jusic, MEASURE-BiH Contracting Officer’s Representative, have been instrumental in the design of the evaluation. Mirjana Popović Valjevac, PRO-Future Agreement Officer’s Representative from USAID/BiH, contributed to the report by helping us understand the design of the PRO-Future Activity and its management. We also thank numerous PRO-Future implementation team members for their prompt assistance with the PRO-Future documentation, database, and information requests. Ye Zhang, MEASURE-BiH Technical Director from IMPAQ, provided unparalleled technical leadership and guidance, lead impact analysis, and made significant contributions to this report. Finally, we thank USAID/W, USAID-BiH, and PRO-Future implementers for providing comment to our draft report. All of these organizations and individuals have contributed tremendously to this report and to the MEASURE-BiH Activity more generally, but any remaining errors are the fault of the authors alone. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 7 ACRONYMS AB BiH CIM CPT CRS CSO DG DID EU FBiH FG FTC HCA IRC KI KII M&E MEASURE-BiH MEL MoU MWG NBP NDC NGO NSC NYBB PRM PRO-Future RS SCORE SOW USAID USAID/BiH USG YIF YIHR Advisory Board Bosnia and Herzegovina Center for Peacebuilding Sanski Most (Centar za izgradnju mira) Choosing Peace Together Catholic Relief Service Civil society organization Democracy and Governance Difference-in-differences method European Union Federation of Bosnia and Herzegovina Focus group Forum of Tuzla Citizens Helsinki Citizens Assembly Inter-Religious Council Key informant Key informant interview Monitoring and evaluation USAID/BiH Monitoring and Evaluation Support Activity Monitoring, evaluation, and learning Memorandum of understanding Municipal working groups Network for Building Peace Nansen Dialogue Center Non-governmental organization National Steering Committee National Youth Bank Board U.S. Department of State Bureau of Population, Refugees, and Migration USAID/BiH’s Trust, Understanding, and Responsibility for the Future Activity Republika Srpska Social Cohesion and Reconciliation Index Statement of work U.S. Agency for International Development U.S. Agency for International Development in Bosnia and Herzegovina U.S. government Youth Initiative Funds Youth Initiative for Human Rights 8 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY EXECUTIVE SUMMARY Trust, Understanding, and Responsibility for the Future Activity (PRO-Future) began in September 2013 and is expected to last until September 2017. It is funded by the U.S. Agency for International Development in Bosnia and Herzegovina (USAID/BiH) and implemented by the Catholic Relief Service (CRS) and its four implementing partners—the Mozaik Foundation, Caritas of the Bishops’ Conference of BiH, the Forum of Tuzla Citizens, and Helsinki Citizens Assembly Banja Luka—and in cooperation with the Inter-Religious Council of BiH (IRC), Mediacentar Foundation, Vive Zene, and Youth Initiative for Human Rights. PRO-Future’s main development hypothesis can be described as follows: If BiH citizens accept alternative political and historical narratives, supported by key influencers, they will change their individual attitudes and behaviors toward one another, resulting in increased social trust within communities, enabling citizens to work together to overcome societal divisions and constructively advocate for institutional changes, which in turn will lead to improved social well-being and economic prosperity. PRO-Future’s goal is to rebuild trust and confidence among citizens of all ethnic backgrounds in BiH by helping them address the past in their communities and reach an understanding and acceptance of each other’s narratives of the past (expected outcome 1), as well as to engage key influencers in supporting local reconciliation initiatives toward attaining those objectives (expected outcome 2). EVALUATION PURPOSE AND QUESTIONS Between April and June 2017, IMPAQ International and USAID/BiH Monitoring and Evaluation Support Activity (MEASURE-BiH) designed and conducted an impact and performance evaluation of PRO-Future, as commissioned by the U.S. Agency for International Development (USAID). The purpose of the evaluation was to assess progress toward expected results and the Activity’s impact on those results; to describe the Activity’s implementation and identify key challenges and lessons learned; and to provide recommendations for any potential future interventions in rebuilding trust and reconciliation in BiH. This evaluation answers the following four research questions, informed by the main elements of PRO-Future’s development hypothesis (evaluation questions 1 and 2) and two main expected outcomes (evaluation questions 3 and 4): 1. What impacts did the PRO-Future Activity have on interethnic attitudes in participating municipalities/cities? 2. What impacts did the PRO-Future Activity have on interethnic trust in participating municipalities/cities? 3. How effective has PRO-Future been in changing participating citizens’ behaviors that contribute to increased stability and consensus, as demonstrated by stakeholders’ perceptions of the Activity, and whether it has made changes in terms of increased capacity and demand for improved relationships? 4. How effective has PRO-Future been in getting key influencers in targeted municipalities/cities to support local reconciliation initiatives, as demonstrated by stakeholders’ perceptions of the Activity and whether it has made changes in terms of key influencers’ openness to dialogue and acting for reconciliation? IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 9 EVALUATION METHODS AND LIMITATIONS The evaluation team employed a mixed-methods approach for the design of this impact and performance evaluation. For the impact portion of the evaluation, three different household surveys were combined: the PRO￾Future baseline survey (2014, N=3,000), the Social Cohesion and Reconciliation Index for BiH (2014, N=2,000), and the PRO-Future end-line survey (2017, N=4,200). The impact analysis was conducted by using the difference￾in-differences method. Additionally, the implementation study and performance assessment portion of the evaluation used the PRO-Future documentation and database, 65 key informant semi-structured interviews, four focus groups with 22 participants, and an online survey of 431 PRO-Future beneficiaries/participants administered by CRS. The main limitations of this evaluation are related to the limitations of the baseline measurement instruments and methodology, selection bias, response bias, recall bias, socially desirable responses, and limitations of PRO-Future’s monitoring and evaluation (M&E) reporting and database. FINDINGS AND CONCLUSIONS An analysis of interethnic attitudes and trust levels from household survey data show that, although group averages for interethnic attitudes and trust improved for most ethnicity combinations between 2014 and 2017, there was no consistent trend when comparing target and comparison regions. There were no statistically significant impacts on either interethnic attitudes or trust in the entire sample. When examining each of the three ethnicities separately, there was a positive and significant impact on Croats’ and Serbs’ attitudes and a negative and significant impact on Bosniaks’ attitudes of Bosniaks, as well as a positive and significant impact on Serbs’ trust and a negative and significant impact on Bosniaks’ trust. On the other hand, interviews/FG discussions showed that PRO-Future has made considerable strides in encouraging interethnic communication and trust of PRO-Future beneficiaries/participants. All interviewees/FG participants state that PRO-Future was generally beneficial, that it changed participants’ attitudes and behavior, and that reconciliation initiatives are needed in BiH. There was consensus among interviewees/FG participants that reconciliation is a long-term process, indicating that PRO-Future’s final expected results were too ambitious to have been achieved by PRO-Future implementation only. All of the interviewees/FG participants also believe that the momentum resulting from Choosing Peace Together (USAID/BiH’s predecessor reconciliation intervention from 2010 to 2014) and PRO-Future should continue. All PRO-Future activities were positively perceived by the audience and participants. However, PRO-Future’s causal chain of designed expected results was both imprecise and unclear in regards to psychological theory and research on reconciliation and peace studies. It is also not confirmed in the impact estimates within this evaluation. Neither implementers nor beneficiaries/participants were familiar with designed expected results and did not have a shared understanding of the meaning of the main terms used in designed expected results (e.g., the meaning of reconciliation, alternative narratives, and the role of discussing the past in trust building). Moreover, although PRO-Future was perceived by interviewees/FG participants as having resulted in personal transformation of participants and their acknowledgment of victims from other ethnic groups (most notably as a result of speaking￾out events), it did not address the need to establish a shared truth (or elements of thereof) in either its design or implementation. At the same time, eight out of ten citizens in the household survey believed that a shared truth about the past is a pre-requisite for reconciliation, which was also brought up in interviews/FG participants. 10 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY PRO-Future participant/beneficiary interviewees described PRO-Future staff as professional, respectful, and passionate in their work with PRO-Future. However, some brought up examples of poor coordination among the implementers and/or inadequate expertise/experience by some staff members, especially during the first stages of PRO-Future implementation, after which the implementers improved their coordination and trained their staff. PRO-Future’s implementation mechanisms planned in the design were too detailed with specific tasks and in many cases numerical targeted outputs. Such a level of detail and prescribed tasks within the design are in contrast with the implementation practices which were flexible. Several designed mechanisms were not implemented to the planned extent, including the Network for Peace Building, Association of Parent Councils, Media Metar, Women’s Alternative Government, Online Peace Schools, and legislation related to torture and missing people. At the same time, the activities that have been implemented by PRO-Future were perceived by many as too broad in type and geographic coverage given the available resources. Thus, they were not deep enough in terms of content and/or frequency. In addition, the intensity of PRO-Future activities varied greatly among the 30 originally targeted municipalities, sometimes due to differences in their “maturity,” the level of CSO capacities, and MWG members’ engagement. Events held in specific localities were at times insufficiently tailored to local specificities, with rare presence of key influencers from those localities, although the activities were generally well-accepted and supported by local governments. Small grants were perceived as useful by all interviewees/focus group participants, although some interviewees noted that selection criteria were not always clear or consistent. When analyzing the effectiveness of PRO-Future activities/events, it is important to distinguish between two types of desired influence: i) activities/events that generate the largest interest and can thus attract a wider audience; and ii) activities/events that have the strongest influence on personal transformation and long-term change of participant attitudes. The activities/events (within both small grants and general PRO-Future activities) that were perceived as generating the largest interest, and thus, being most effective in targeting the broader population, were those that had more than one focus. They not only included a direct reconciliation/peace-building component, but also included active interethnic interaction based on common local/target group interests and encouraged active participation by local key influencers. Interviewees gave examples that included youth camps and other events in which topics of leadership, teamwork, and other skill-building or sports/recreation activities, were combined with reconciliation components (e.g., street workout and reconciliation workshop). They also included events that had economic development elements (e.g., women’s entrepreneurship and reconciliation workshops); those that had a local environment element (e.g., municipality landscaping and peace-building activity); and arts/culture events that had a direct peace building element (e.g. theater plays). Youth participants/beneficiaries, in particular, believed that the most effective reconciliation activities for youth need to include activities of interest/use for young people, specifically, in addition to a peace/reconciliation component and be based on a participatory action approach with interaction between the participants. On the other hand, activities/events perceived as the most effective type of activity in influencing participants’ personal transformation and long-term change of interethnic attitudes were those that included a more direct and comprehensive reconciliation/peace-building approach. These were speaking-out events, youth peace camps with comprehensive reconciliation/peace-building activities, and open-door days. Nevertheless, several interviewees/FG participants noted that the effectiveness of speaking-out events could have been increased with more interaction and the addition of some kind of follow-up. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 11 Our impact analysis and interviews/focus groups indicated that PRO-Future has had an important influence, but only on narrow groups and types of citizens. In other words, the extent to which PRO-Future reached wider groups of citizens and those less receptive to interethnic reconciliation and peace-building activities was perceived as not being large enough to influence citizens in a broader sense, as envisaged by PRO-Future design. PRO-Future beneficiaries/participants were mostly CSO members and activists. Our comparison of the results from household survey of general population and the results from the CRS’s survey of PRO-Future participants/beneficiaries also indicate that PRO-Future beneficiaries/participants differ from general population in several aspects important for reconciliation, including their opinion on role of talking about the past in reconciliation, prioritization of the overall challenges in BiH, participation in peacebuilding activities, and significance they assign to different types of peace building activities. An analysis of the end-line household survey data indicated that the following factors may be the most effective in reconciliation initiatives: high-quality contact with members of other ethnicities (i.e. contact perceived as positive contact as opposed to any type of contact); activities that target reversing the subtle dehumanization processes of other groups; a high frequency of any interethnic contact; and targeting ethnically homogenous localities. Work with politicians as key influencers is perceived as essential for reconciliation. Within its work with key influencers, PRO-Future focused mostly on politicians. It broadened the scope of this work in the last stage of PRO-Future implementation to include politicians and government officials from higher government levels, in addition to municipality/city mayors and officials as envisaged originally. Most interviewees/FG participants perceived this as the right approach. PRO-Future’s Platform for Peace was endorsed by 60 mayors in late 2016 and by the Council of Ministers of BiH in 2017. Its signing and the comprehensive process that led to it, was considered a major achievement by PRO-Future’s work with politicians. However, as the Platform was written in quite broad terms, there is a need to design and adopt concrete measures and policies, presumably within the Strategy of Sustainable Long-Term Peace for which the Platform calls. PRO-Future’s work with key influencers was perceived by most stakeholder interviewees as being less focused on religious leaders and the media than on politicians. However, the work that has been done with both the IRC and the media was perceived as laying the grounds for intensified work in the future. Not much work was done with business sector representatives or educators as key influencer categories in comparison to the role envisaged in the design (described later within the chapter on PRO-Future background). Stakeholders perceived the mainstream media as the second most important key influencer after politicians, followed by religious leaders. Although all interviewees/FG participants perceived PRO-Future’s work with key influencers as important, most of interviewees, focus group participants, and citizens that were surveyed with the end-line household survey did not believe that cooperation among key influencers improved. PRO-Future’s main implementation mechanism for working in targeted areas was through municipal working groups (MWGs), which was generally perceived as a good approach necessary to ensure local ownership and appropriateness of local activities for the specific localities. However, the levels of MWGs’ and MWG members’ engagement in PRO-Future activities varied widely and was in some cases perceived as not frequent enough and not coordinated well by PRO-Future. MAIN RECOMMENDATIONS Based on this evaluation, it is notable that reconciliation initiatives are still needed in BiH, since the process of changing citizens’ behaviors and building interethnic trust in this country will be long term, given its unique complexities. Furthermore, reconciliation initiatives should be implemented to build on the considerable achievements of PRO-Future. They should be grounded more extensively in psychological theory and research 12 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY findings related to peace building and have a simple, clear, and consistent monitoring and evaluation logical framework. The overall conclusions from our estimates on PRO-Future’s impact on the general population are inconclusive in the whole sample and ambiguous when analyzed at an ethnicity-combination level. However, all stakeholders directly included in PRO-Future and its events who spoke to the evaluation team believed that PRO-Future was useful, that it positively influenced those who participated, and that it, in many cases, resulted in their individual and personal transformation, which would not have been possible without PRO-Future. The apparent dichotomy between participants’ mostly positive perceptions of PRO-Future’s effectiveness, and its ambiguous impact on the broader population indicate that PRO-Future has had an important influence, but on narrow groups and types of citizens. Building on the foundations laid by PRO-Future, future interventions should consider a more systematic way of targeting beneficiaries/participants, aiming to reach less-receptive citizens in addition to activists. It is recommended that they consider focusing on fewer localities with more intensity and with a further customized and sequenced approach considering locality specificities, as well as to continue with small-grant allocation, speaking-out events, youth camps, and open-door days. In both small grants and activities outside the small grants, to attract broader participation and frame activities in the least threatening form (i.e. not as events that are entirely conflict- and past-related), the focus should be on activities that combine a reconciliation/peace-building component with interethnic interaction based on common local/target group interest. At the same time, speaking-out events should continue, both as stand-alone events and components within other activities and small grants, as it is the most effective type of activity in influencing participants’ personal transformation. Key influencers’ engagement should be strengthened with more joint activities, focusing on high-level and local politicians, religious leaders, and mainstream media. In working with politicians, continue with a two-pronged approach focusing on municipality government institutions as well as higher-level government officials/institutions. Future interventions should work on designing specific and actionable tasks to be implemented by the government based on the Platform for Peace signed within PRO-Future. Finally, potential future interventions should coordinate closely with the USAID/BiH media intervention (given the importance of media as a key influencer in reconciliation) and with other USAID/BiH interventions dealing with reconciliation in formal education and youth ambassadors for reconciliation. They should also consider coordinating with other donors’ interventions, most notably the UN’s expected follow-up to Dialogue for the Future (in order to avoid possible duplications and explore synergies in reconciliation activities of all of these interventions). IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 13 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS 1.1. EVALUATION PURPOSE The specific purpose of this impact and performance evaluation is threefold: I. to assess progress toward expected results and the Activity’s impact on those results; II. to describe how the Activity was implemented and the key challenges and lessons learned from implementation; and III. to provide recommendations for any potential future donor/government interventions in rebuilding trust and reconciliation in BiH. A key component of the evaluation design was the inclusion of a diverse set of PRO-Future stakeholders in the evaluation process. These stakeholders include PRO-Future funders; implementers and partners; participating mayors/members of the municipal working groups (MWGs); small grant recipients; participants in various events, including youth, victim association representatives, speaking-out event participants; lecturers and facilitators engaged by PRO-Future, and other reconciliation experts; and religious leaders. In addition, through household surveys, data from a large sample of 4,200 citizens was collected, in both targeted communities and in a comparison group of municipalities in which PRO-Future was not implemented. Findings, conclusions, and recommendations of the PRO-Future impact and performance evaluation can contribute to the knowledge and learning of several groups of stakeholders: 1. USAID/BiH can reassess the role of PRO-Future in strengthening reconciliation in BiH. The evaluation helps USAID/BiH staff achieve a better understanding of Activity implementation, lessons learned, and best practices, which can inform any potential future funding decisions and program designs in this area. 2. Implementing partners can learn about their strengths and areas for improvement. 3. Other USG stakeholders (including USAID/Washington and the U.S. Embassy in BiH) can better understand the USAID-funded reconciliation interventions in BiH. 4. Other stakeholders, including BiH governing institutions, religious leaders and CSOs working on reconciliation issues, and other international development donors and partners, may also benefit from USAID’s contribution to public knowledge on the most recent development efforts in reconciliation in BiH. The PRO-Future Activity began in September 2013 and is expected to last until September 2017. This evaluation covers the implementation period between September 2013 and March 2017. The MEASURE-BiH team conducted this evaluation between April and June 2017. The household survey data collection took place between April 25 and May 19. Key informant interviews (KIIs) and focus group (FG) discussions were conducted between May 15 and June 21. The evaluation team traveled to 18 municipalities across BiH (Istocno Novo Sarajevo, Sokolac, Prijedor, Sanski Most, Drvar, Glamoc, Gornji Vakuf/Uskoplje, Prozor-Rama, Gračanica, Doboj, Petrovo, Teslić, Tešanj, Maglaj, Mostar, Nevesinje, Srebrenica i Bratunac), in addition to holding KIIs in Sarajevo and holding telephone interviews with KIs from Banja Luka and Tuzla. 14 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY This evaluation focuses only on the original 30 municipalities in which the PRO-Future intervention was implemented in all years over the life of the Activity (an additional 30 have been added: two in year two, 21 in year three, and seven in year four). The Evaluation Statement of Work (SOW) is given in Annex 1. 1.2. EVALUATION QUESTIONS This evaluation addresses four questions: 1. What impacts did the PRO-Future Activity have on interethnic attitudes in participating municipalities/cities? To what extent did they vary across ethnicities? 2. What impacts did the PRO-Future Activity have on interethnic trust in participating municipalities/cities? To what extent did they vary across ethnicities? 3. How effective has PRO-Future been in changing participating citizens’ behaviors that contribute to increased stability and consensus, as demonstrated by stakeholders’ perceptions of the Activity, and whether it has made changes in terms of increased capacity and demand for improved relationships? How have PRO-Future interventions targeting change in citizens’ behaviors been implemented; how relevant were they to social, political and cultural context, needs, and capacities of target communities; and what were the main challenges and lessons learned from stakeholders’ perspective for future interventions in rebuilding trust and reconciliation? 4. How effective has PRO-Future been in getting key influencers in targeted municipalities/cities to support local reconciliation initiatives, as demonstrated by stakeholders’ perceptions of the Activity and whether it has made changes in terms of key influencers’ openness to dialogue and acting for reconciliation? How have PRO-Future interventions targeting key influencers been implemented, and what were the main challenges and lessons learned from stakeholders’ perspectives for future interventions in rebuilding trust and reconciliation? To answer questions 1 and 2, we examined whether interethnic attitudes and level of trust in the targeted and comparison groups had changed and what the difference was from 2013 to 2017. To conduct these analyses, we combined PRO-Future baseline (2014), SCORE Index (2014), and PRO-Future end-line survey data. The difference-in-differences (DID) method through multivariate regression models was used, controlling for demographic variables to isolate the changes in treatment group that could be attributed to the intervention. Question 3 relates to the PRO-Future Goal (BiH citizens, encouraged by key influencers, advocate for institutional changes in society), and particularly Outcome 1: Citizens in 30 targeted municipalities accept alternative political and historical narratives. We examined how the related interventions were implemented. We sought to better understand the implementation process of the PRO-Future Activity as perceived by the various stakeholders that interacted with it. The PRO-Future quarterly and annual reports suggested that output and outcome targets were primarily met or surpassed, particularly in numbers of beneficiaries reached by PRO-Future events (speaking-out events, community initiatives, forums, number of communities, and participants/beneficiaries). In particular, PRO￾Future reports described increasing enthusiasm of communities to join the pairs of municipalities and re-establish pre-war relationships. This, according to the implementers, maximized and extended the exposure of mono-ethnic IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 15 communities to all three narratives. Furthermore, PRO-Future reported a continuous increase in citizens’ critical thinking skills, resulting in a higher understanding of people with different experiences and a willingness to hear and respect others’ opinions, as well as a more critical interpretation of political/ethnic/religious leaders’ statements. Beyond the numbers and types of activities that PRO-Future implemented, it was important to us to understand how these interventions were perceived by stakeholders, how PRO-Future responded to the specific communities’ political and cultural context, and what lessons were learned through this process. Hence, this evaluation focused on: (1) examining achievements and challenges of the interventions that aimed to improve the citizens’ behaviors and acceptance of others’ narratives as key drivers for establishing renewed relationships between the communities across ethnic divides; (2) examining in more detail whether the Activity responded to the nature of communities’ political and cultural context, and how these have been addressed in order to better understand their relative vibrancy and potential for rebuilding trust and sustainable reconciliation. To answer these questions, a review of PRO-Future documentation and databases provided a description about the type and frequency of activities in 30 targeted communities. This was accompanied by information from KIIs and FG discussions with selected stakeholders, who testified about their experiences and perceptions about PRO￾Future implementation, effectiveness, and lessons learned. Moreover, these sources were supplemented by relevant data from the end-line household survey and the online survey of PRO-Future beneficiaries carried out by CRS. Question 4 relates to PRO-Future Outcome 2: Key influencers in 30 targeted municipalities support local reconciliation initiatives. According to PRO-Future, key influencers — defined as political and religious leaders, the media, business owners, civil society, and education officials — are critically important role models that encourage citizens to work toward reconciliation. Their encouragement is demonstrated by working together, engaging in dialogue, promoting reconciliation, and supporting local initiatives. Key influencers are perceived as acting on both national and community levels. In addition, small grants were awarded to encourage the local governments to support the local initiatives by involving them in the MWGs and requiring a financial contribution from them. Furthermore, key influencers are seen as important drivers for positive media content through their engagement in the media and sharing positive stories. This question sought to identify how effective PRO-Future has been in engaging the key influencers in the targeted communities in order to work together and support the local and national initiatives. It also asked to what extent the Activity engaged the less supportive key influencers and helped increase their mutual trust. The PRO-Future Award targeted key actors who are both inclined and resistant to reconciliation efforts, while the ultimate goal was to reach as many citizens as possible. We also sought to understand what type of key influencers’ engagement has been perceived as most effective for reconciliation, and to identify specific types of messages that have been perceived as building interethnic trust. To answer these questions, we used the same sources as described for Question 3. 16 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY 2. PRO-FUTURE BACKGROUND PRO-Future is a $4.1 million (with an additional $670,000 of cost sharing), four-year Activity implemented by Catholic Relief Service (CRS) and its four partners (Mozaik Foundation, Caritas of the Bishops’ Conference of BiH, Forum of Tuzla Citizens, and Helsinki Citizens Assembly) that began in September 2013. CRS also cooperated with the Inter-Religious Council of BiH (IRC), Mediacentar Foundation, Vive Zene, and the Youth Initiative for Human Rights. This intervention is built on USAID/BiH’s previous reconciliation intervention, Choosing Peace Together (CPT), also implemented by CRS between January 2010 and September 2014. CPT focused on working with war victims, allowing for processing of their war experiences and adoption of new narratives to support reconciliation. PRO-Future’s goal is to rebuild trust and confidence among citizens of all ethnic backgrounds, by helping them address the past in their communities and reach an understanding and acceptance of each other’s narratives of the pas. It also seeks to engage key influencers in supporting local reconciliation initiatives toward attaining those objectives. PRO-Future is based on the premise that effective peace-building efforts start by opening dialogue, understanding people with opposing viewpoints, changing attitudes, and building skills and relationships, at the individual/personal level. The main development hypothesis used in PRO-Future’s can be described as the following: If BiH citizens accept alternative political and historical narratives, supported by key influencers, they will change their individual attitudes and behaviors toward one another, resulting in increased social trust within communities, enabling citizens to work together to overcome societal divisions and constructively advocate for institutional changes, which in turn will lead to social well-being and economic prosperity. Two main expected outcomes in the PRO-Future design were: 1. Citizens in 30 targeted municipalities accept alternative political and historical narratives. The design envisaged more than 100,000 citizens in 30 target municipalities (15 pairs of communities that have been divided along ethnic lines since the war) engaged to challenge and transform their own beliefs and to begin transforming their communities. Activities to change citizens’ attitudes and behaviors were to include trainings, peace camps, speaking-out events, roundtable discussions, cultural events, online competitions and discussions, small grants, and community initiatives. Targeted groups were to include youth, war victims, women, minority returnees, and civil society (war victim associations, local grassroots initiatives, NGOs). 2. Key influencers in 30 targeted municipalities support local reconciliation initiatives. Key influencers were defined as political and religious leaders, the media, business owners, civil society, and education officials. They were expected to engage to support local reconciliation initiatives by, for example, leading or engaging in inter-religious events, promoting reconciliation in speeches and through the media, and providing sponsorship or co-funding of initiatives that work across ethnic or party lines to address local concerns. Furthermore, key influencers were to engage in a national steering committee1 1 According to implementers, the National Steering Committee was transformed into the Advisory Board (AB) by the end of Y2, so this term will be used across future sections of this report. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 17 and municipal working groups to build a stronger foundation of support for reconciliation and consensus. PRO-Future was planned to culminate in the Platform for Building Peace, a citizen-influenced roadmap that will outline policies to help attain a country where citizens of all ethnicities will work in solidarity to advocate for social change. The PRO-Future Award lays out 36 implementation mechanisms planned for achieving the two outcomes, each with very specific planned tasks, as shown in Exhibit 10 in Annex III. Some specific elements of CRS’s implementation design included in the PRO-Future Award (as modified in June 2014) include: 1. Targeting key actors who are both inclined and resistant to reconciliation efforts and using those already inclined to reconciliation to exert a positive influence on their resistant peers. The ultimate goal was to reach as many people as possible. 2. Identifying “key influencer sectors” as: a. war victim associations; b. religious leaders (IRC); c. Network for Building Peace (NBP); d. National Youth Bank Board; e. schools (Association of Parent School Councils, education ministers, and school principals); and f. media (engaging five local media, Mediacentar’s online portal, NBP website, Srđan Aleksić Award). 3. Replicating experiences from CPT, in which relationships across key leaders of war victims’ associations from all three ethnic groups were built to influence diverse religious leaders through partnership with the Inter￾Religious Council (IRC). 4. Giving small grants to CSOs as an incentive to work with local government and business leaders, and adopt participatory approaches, with key influencers in target communities directly engaging with citizens. 5. Replicating experiences from CRS’s projects supporting minority returnees to form municipality working groups (MWGs) composed of different stakeholders, selecting at least five religious or political leaders, education officials, NGO leaders, or other relevant community leaders to serve in a MWG. The MWG was envisaged to play a role in planning PRO-Future activities, defining criteria for participant selection, mobilizing participants, identifying linkages within the communities, interacting with citizens, participating in community dialogues and roundtables, and contributing to the Platform for Building Peace. 6. Convening a national steering committee (NSC) with state and entity government officials, religious leaders, NGOs, and the like, which will encourage countrywide advocacy campaigns, and advise on selecting municipalities and priorities, and mobilizing resources. 7. Engaging the state-level Ministry of Human Rights and Refugees of BiH and the two entity-level ministries for displaced persons and refugees in NSC, including securing matching funds to support joint community initiatives. 18 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY 8. Forming an expert group consisting of three members of the NSC to play an advisory role for strategic planning for PRO-Future, including support to develop strategies, recommendations, and technical advice. 9. Ensuring a strong media presence through formal and informal media outlets. 10. Using NBP as a critical element of PRO-Future, including coordinating the Platform and leading national-level advocacy efforts. More specifically: a. taking advantage of NBP’s 89 fee-paying members, representing a broad range of civil society, including women’s organizations, human rights NGOs, youth, and war victim associations, that share a vision for reconciliation; b. expanding NBP further to include MWGs and organizations in municipalities to become affiliate network chapters; c. using NBP as a structure for the growing “constituency for peace” that emerges in target communities; d. using NBP to build CSOs’ capacities to help and secure funds and cooperate on broader awareness and advocacy, as well as using the NBP website as a focal point for promoting and disseminating PRO-Future information; and e. creating NBP’s sub-networks on torture and youth. 11. Including 10 pairs of municipalities that were included in previous reconciliation initiatives and have had progress, we well as five pairs of more challenging municipalities among the 15 targeted municipality pairs. 12. Avoiding double counting of beneficiaries/participants and determining key assets, challenges, and community needs, which will, in turn, inform the specific type, sequencing, and frequency of specific activities. Exhibit 11 in Annex III shows the role of each implementer in PRO-Future, as specified in the Award. This evaluation focuses only on the original 30 municipalities in which the PRO-Future intervention has been implemented in all years over the life of the Activity (an additional 30 have been added: two in year two, 21 in year three, and seven in year four). Exhibits 12 through 16 show overviews of data as calculated/mapped by the evaluation team based on the PRO￾Future internal database on small grants and activities per location and purpose defined by PRO-Future. They include the number of direct and indirect beneficiaries for both small grants and activities, and amounts and contributions for small grants. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 19 3. EVALUATION METHODS AND LIMITATIONS 3.1. METHODOLOGY AND DATA COLLECTION The evaluation team employed a mixed-methods approach for the design of this impact and performance evaluation. Although a performance evaluation, as defined in the USAID Evaluation Policy, often identifies results and lessons learned in Activity implementation with a focus on the programmatic, technical, managerial, and organizational elements, it lacks a rigorously defined counterfactual and is unable to attribute results to interventions conducted by the USAID activities. In order to build the evidence base for the USAID/BiH Democracy and Governance (DG) portfolio and provide empirical, scientific results for the Mission’s learning and accountability in the important area of social change and reconciliation, it is not enough for us to know whether the Activity has achieved its targets or how the Activity is being implemented; it is critical for the Mission to examine how much, if any, the observed changes in Activity outcomes can be attributable to the Activity interventions—that is, what are the impacts of the interventions implemented by PRO-Future. A nationally representative household survey was conducted in early 2014 to collect data for the Social Cohesion and Reconciliation (SCORE) Index for BiH. Soon after, PRO-Future conducted baseline household surveys in target municipalities using several of the same or similar questions. Through these surveys, the evaluation team was able to construct baseline values for both target municipalities and the comparison region in which PRO￾Future had not been implemented. MEASURE-BiH administered an end-line survey in April and May 2017, structured as similarly as possible to the PRO-Future baseline survey using a representative sample of BiH citizens in both the targeted and comparison municipalities of the SCORE survey. This enabled us to identify the causal effects of PRO-Future interventions on intergroup attitudes and trust. The evaluation team employed the difference-in-differences (DID) method to identify and parametrically estimate PRO-Future impacts through multivariate regressions. The DID method compared the before-after changes in outcomes between citizens in target communities (treatment group) and citizens in the comparison region (comparison group). It is important to note that the DID does not require baseline (pre-intervention) conditions to be the same in treatment and comparison groups. For the DID to be valid, however, the comparison group must accurately represent the change in outcomes that would have been experienced by the treatment group in absence of the intervention. In other words, the key identifying assumption behind the DID is that trends (changes) in outcomes between the treatment and comparison groups should be similar. The common trend assumption cannot be directly tested (especially in data with only two points in time). A potential concern is that, if pre-intervention observable characteristics of the treatment and comparison group citizens are very different, they might experience dissimilar changes in their outcomes, independent of the intervention (i.e. a violation of the common trend assumption). If comparison regions have been chosen to be as similar as possible to the treatment municipalities, this will help mitigate the risk, thereby giving a greater chance of having a comparison pool of citizens that is similar to citizens living in the municipalities affected by the intervention. Thus, they would be more likely to experience similar behavior over time in the absence of PRO-Future. This is a potential limitation of our DID method. 20 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY The first two evaluation questions examined the impact of the PRO-Future intervention on interethnic attitudes and trust, based on the Activity’s designed causal chain/development hypothesis. By this process, PRO-Future was to improve interethnic attitudes, which was to result in an increase in interethnic trust. We combined household survey data for both the baseline (2014) and the end line (2017) to estimate the DID model by using the following regression approach: Outcome = +  +  +   ∙  + + . (1) The left-hand side of the equation is the outcome variable of interest. The variables on the right-hand side include:  A dummy variable  that is equal to 1 if the observation is in the treatment group and zero if otherwise. The estimate of  captures the group effect. In other words,  controls for any differences in the outcome variable that are associated with being in the treatment group.  A dummy variable  that is equal to 1 in the end line and zero in the baseline year. The estimate of  captures the time effect. In other words,  controls for any changes in the outcome variable that occur over time and are common for treatment and comparison group citizens.  An interaction term  ∙  that is equal to 1 if the observation is in the treatment group and in the end line year, and zero if otherwise (i.e., for comparison group members in both the baseline and end line years, and for the treatment group in the baseline year). The estimate of  captures the impact of the PRO-Future interventions on the outcome variable—this is the parameter of interest.  A vector of other relevant explanatory variables that may be related to the outcome of interest and will help control for baseline characteristics. includes variables like the age, gender, ethnicities, education, employment status, rural/urban status, ethnicity mix of the municipality/region. These explanatory variables reduce the amount of unexplained variation in the outcome variable, thereby increasing the accuracy of our parameter estimates. For outcomes that are binary variables, such as many of the variables describing individuals’ attitudes and trust, a linear regression model has two theoretical problems. A linear regression model applied to a binary outcome measure is called a linear probability model (LPM). The first potential problem with a linear probability model is that predicted probabilities may be less than 0 or greater than 1. The second problem is that the error terms in the model violate homoscedasticity distributional assumptions, in which case standard statistical inference could be incorrect. This problem can be easily solved by applying heteroscedasticity robust standard errors. To overcome these problems, policy analysts often use probit or logit models when the outcome measure is a binary variable. However, if the main purpose is to estimate the partial effect of the independent variable on the response probability such as in this evaluation, averaged across the distribution of the independent variable, then the fact that some predicted values are outside the unit interval may not be very important. We tested the validity of the LPM against a probit model in the present context using two key binary attitude outcome measures. For both of these outcome measures, the estimated impacts were very similar and the p-values were similar, suggesting that statistical inference based on LPM was still valid. Moreover, across all of the binary outcomes examined in this evaluation, we did not find any regression-adjusted treatment or comparison means below 0 or above 1 when using the linear probability model. For these reasons, we used linear probability IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 21 models to estimate impacts of the PRO-Future interventions on all binary outcomes included in the present report. The third and fourth questions examined implementation and perceived progress toward expected results, modeled on two outcomes of the PRO-Future intervention. To answer these questions, we used a review of the PRO-Future Activity implementation documentation and database, KIIs, and FG discussions.2 In addition, we used relevant questions from the end-line household survey for these evaluation questions. Finally, in June 2017, CRS conducted its own online survey of PRO-Future beneficiaries and stakeholders based on the questionnaire we used for the end-line survey. CRS sent the results of this survey to the evaluation team, and these data were used as an additional data source for this evaluation. All data sources we used in this evaluation are as follows: 1. household surveys: a. baseline data for impact analysis—data collected through two household surveys in 2014: PRO-Future baseline survey of 3,000 citizens in 30 targeted municipalities, and SCORE Index Survey conducted on nationally representative sample of 2,000 citizens, and b. end-line data for impact analysis—data collected through a household survey of 4,200 citizens in 30 targeted municipalities and eight regions in which PRO-Future was not implemented; 2. PRO-Future implementation documentation and database; 3. sixty-five key informant semi-structured interviews; 4. four focus groups with a total of 22 participants (two youth focus groups, one for small grant recipients, and one for speakers/participants in speaking-out events); and 5. an online survey of PRO-Future beneficiaries conducted by CRS based on the end-line household survey questionnaire (437 respondents out of 3,592 invited beneficiaries). The evaluation team spoke with 87 interviewees/FG participants for this evaluation, including stakeholders from 10 groups, as shown in Exhibit 1. The team first randomly selected interviewees and FG participants from the PRO-Future database. Three issues arose, however. First, the database did not sufficiently distinguish the intensity of participation/assistance—that is, the level of participation—and, in some cases, not all of the activities attended by a participant were recorded (according to several of the interviewees/FG participants). Second, there were many cases of multiple counting within and across beneficiary type. Third, when the evaluation team attempted to schedule meetings/FGs with the randomly selected participants, the acceptance rate was low. On average, one out of three or four contacted beneficiaries/participants agreed to an interview/FG discussion; most of those beneficiaries/participants had engaged in several activities and were very familiar with PRO-Future. 2 Information we received from the key informant interviews is paraphrased in text boxes throughout this report. 22 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY The data from the above-listed sources were triangulated to address the same evaluation questions from multiple perspectives whenever possible. Comparing and contrasting data helped us gain a more complete understanding of the issues and provided more confidence in the findings. The information obtained from the semi-structured interviews and focus groups was consolidated for a thematic analysis using information categories related to the evaluation research questions by transcript coding. We conducted a qualitative analysis of interview transcripts, using a process in which we consolidated multiple responses related to similar themes that were mentioned by different categories of respondents, and analyzed them for general findings. In this manner, we were able to determine the common themes. We also conducted desk research of the available PRO-Future M&E data where possible, to complement the findings and provide a further description of the PRO-Future implementation process and achieved results. Exhibit 1: Interviewees/FG Participants per Beneficiary/Stakeholder Group Beneficiaries/Stakeholders in the Database Sent by PRO-Future for Interviewee/FG Selection Interviewees/ FG Participants Based on Selection from PRO￾Future Database Interviewees/ FG Participants Based on Their Actual Roles in PRO-Future (as per their explanation in interviews/FGs) Donors/Implementers/Partners 19 19 Mayors 30 3 3 MWG Members 104 11 16 Advisory Board Members 10 1 1 Facilitators/Lecturers 25 4 9 Public Speakers 29 4 5 IRC speakers/Religious Leaders 38 2 6 Participants in Speaking-out Events 1,620 (of which 1,585 unique) 8 31 Participants in Youth Events 4,910 (of which 3,530 unique) 20 48 Small Grant Recipients 306 15 21 TOTAL: 7,072 (of which 5,657 were unique within categories, but additional duplication across categories) TOTAL PERSONS: 87 TOTAL ROLES: 159 AVERAGE ROLES PER PERSON: 2 Annex II provides a detailed list of the documents we reviewed during this evaluation. Data collection instruments are given in Annex IV, and interviewees/FG participants are listed in Annex V. The Evaluation matrix is outlined in Exhibit 2. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 23 Exhibit 2: Evaluation Matrix Evaluation Questions Data Sources Research Design 1. What impacts did the PRO-Future Activity have on inter-ethnic attitudes in participating municipalities/cities? Sub-question: To what extent did the impacts vary across ethnicities? Surveys: Baseline data: SCORE and PRO-Future Baseline Survey End-line data: PRO-Future End-Line Survey Impact evaluation: Quasi￾experimental design with difference-in-differences method, implemented through multivariate regression models 2. What impacts did the PRO-Future Activity have on inter-ethnic trust in participating municipalities/cities? Sub-question: To what extent did the impacts vary across ethnicities? Surveys: Baseline data: SCORE and PRO-Future Baseline Survey End-line data: PRO-Future End-Line Survey Impact evaluation: Quasi￾experimental design with difference-in-differences method, implemented through multivariate regression models 3. How effective has PRO-Future been in changing participating citizens’ behaviors that contribute to increased stability and consensus, as demonstrated by stakeholders’ perceptions of the Activity and whether it has made changes in terms of increased capacity and demand for improved relationships? Sub-question: How have PRO-Future interventions targeting change in citizens’ behaviors been implemented; how relevant were they to social, political and cultural context, needs, and capacities of target communities; and what were the main challenges and lessons learned from stakeholders’ perspective for future interventions in rebuilding trust and reconciliation? PRO-Future Activity implementation documentation and database; key informant interviews; focus groups; PRO-Future End-Line Survey; and CRS’s survey of PRO-Future beneficiaries and stakeholders Performance evaluation: Mixed methods, with a focus on qualitative data 4. How effective has PRO-Future been in getting key influencers in targeted municipalities/cities to support local reconciliation initiatives, as demonstrated by stakeholders’ perceptions of the Activity and whether it has made changes in terms of key influencers’ openness to dialogue and acting for reconciliation? Sub-question: How have PRO-Future interventions targeting key influencers been implemented, and what were the main challenges and lessons learned, from stakeholders’ perspective, for future interventions in rebuilding trust and reconciliation? PRO-Future Activity implementation documentation and database; key informant interviews; focus groups; PRO-Future End-Line Survey; and CRS’s survey of PRO-Future beneficiaries and stakeholders Performance evaluation: Mixed methods, with a focus on qualitative data 24 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY 3.2. EVALUATION LIMITATIONS We encountered nine major limitations during our evaluation. Limitation 1: Limitation related to comparison region sample Because a rigorous impact evaluation of PRO-Future was not planned during the Activity’s design stage, the selection of comparison areas was limited by the SCORE Index data. Given that conflict in BiH differed in different regions and that ethnic composition and inter-ethnic relations differ in different regions, it is not possible to ensure full comparability between or within targeted and comparison areas. Limitation 2: Limitations related to the formulation of the end-line household survey data collection instrument To preserve comparability with the baseline PRO-Future survey and the SCORE survey, the evaluation team had to use formulations of questions in the end-line household survey to match the baseline surveys, despite observed measurement issues in the baseline surveys, most notably related to capturing trust; understanding of reconciliation (question not being specific enough); absence of more implicit items measuring people’s perceptions and behavioral tendencies (for example, dehumanization of others as measured through emotion attribution); absence of more emotion-oriented measures (for example, intergroup anxiety); and absence of some usual social identification measures on the dimensions/complexities of social identity. Limitation 3: Different answer scales for the outcome variable on trust in the baseline survey for the comparison group, compared to baseline and end-line surveys The harmonization of the outcome variables for trust across the SCORE Index data and PRO-Future baseline data required some approximation. Namely, the SCORE questionnaire included only a yes/no answer, whereas the PRO-Future baseline and end-line surveys include a five-point scale.3 Limitation 4: Selection bias in agreeing to be interviewed/participate in focus groups The team first randomly selected interviewees and FG participants from the PRO-Future database; however, the acceptance rate was low, with only one in three contacted beneficiaries/participants agreeing to an interview/FG discussion. Almost all of the beneficiaries/participants that were interviewed or participated in focus groups had engaged in several PRO-Future activities and were very familiar with PRO-Future. Limitation 5: Relatively low response rate for the online survey of PRO-Future beneficiaries/participants conducted by PRO-Future implementer Out of 3,592 invitations, 431 beneficiaries/participants accessed the online survey, with around 350 of them completing the survey (10% response rate). 3 When it comes to measuring trust, the SCORE Index used the following question: Of the various groups on this card, which do you believe are trustworthy and which do you believe are not trustworthy? If for some groups you do not have an opinion, please do not select them. Ethnic groups: Bosniaks, Serbs of BiH, Croats of BiH. Answer options: 1. Trustworthy, 2. Not Trustworthy, 9. (Do not read) Don’t Know/Refusal. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 25 Limitation 6: Social desirability bias in surveys and in interviews/FGs While social desirability bias is a limitation in all research examining self-assessed opinion/values/behaviors based on surveys/interviews/focus groups, it is particularly strong in research related to reconciliation. Because ethnicity matters and war-related topics are still a very sensitive issue in the BiH context, one can assume that the respondents provided more socially desirable answers compared to reality, both in household surveys and in KIIs/FGs. This may have been exacerbated by the fact that it was not possible to ensure that the interviewers/surveyors were of the same ethnicity as the interviewee/survey respondents in all cases. One in four end-line household survey respondents was perceived by surveyors as being upset when answering questions on interethnic attitudes and trust. Limitation 7: Response bias Most of the key informants had direct interaction with the Activity and may have overstated the positive effects of the PRO-Future intervention and understated the negative effects. The team attempted to mitigate the response bias as much as possible by drawing on multiple sources of information for each evaluation question, by carefully designing interview guides (with the specific examples required), by guaranteeing the interviewees’ confidentiality, and by broad coverage of the PRO-Future stakeholders in KIIs/FG discussions/surveys. Limitation 8: Recall bias Given multiple donor interventions in the reconciliation sector (including consecutive USAID/BiH interventions implemented by PRO-Future implementers), the responses of interviewees/FG participants and online survey respondents are likely to have been somewhat influenced by their experiences with non–PRO-Future interventions. The team attempted to mitigate the recall bias as much as possible by providing information on specific PRO-Future activities within the interviews and FG discussions and by asking respondents to back up their feedback with PRO-Future–specific examples. Limitation 9: Limitations of PRO-Future’s own M&E reporting and database There have been several revisions related to PRO-Future’s M&E reporting since the beginning of the intervention, including indicator definitions. Implementation databases received by the evaluation team from PRO-Future implementers included the following issues: a. at times unclear, overlapping, and/or changing categorization of activities/grant purposes (including events vs. meetings vs. initiatives vs. presentations vs. campaigns), as shown in Exhibit 13; b. double counting within and among assistance types; c. unclear definition and inconsistent recording of indirect beneficiaries; d. implementers’ internal meetings and tasks related to implementation management included with participant/beneficiary data; e. unclear explanation of the nature of the local contribution per grant; f. unclear recoding of the formal versus the informal group among grant recipients; g. missing information on all activities in which a particular organization/individual participated was listed, as discovered within KIIs/FG discussions; h. incorrect or missing contact information; and i. beneficiaries/participants not being recorded based on intensity of assistance/participation. 26 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Although it is clear from the PRO-Future’s implementation databases that significant efforts were invested and a large amount of data were collected and implementers state that the databases were sufficient for their internal use, the evaluation team finds that the databases are conducive for effective and efficient monitoring and they represented an important limitation for our evaluation, including the difficulty to distinguish between the PRO￾Future staff and beneficiaries, lack of details regarding the events description, double entries within and across the data files, and definitions of direct and indirect beneficiaries. In addition, a few of the interviewees mentioned that the PRO-Future post-event feedback sheets were complicated and long, focused more on feedback on logistics instead of content, and did not offer an exhaustive list of answer options on key questions. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 27 4. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 4.1. FINDINGS EVALUATION QUESTIONS 1 AND 2: WHAT IMPACTS DID THE PRO-FUTURE ACTIVITY HAVE ON INTERETHNIC ATTITUDES AND TRUST IN PARTICIPATING MUNICIPALITIES? Finding 1: Although group averages for interethnic attitudes and trust improved for most ethnicity combinations between 2014 and 2017, there is no consistent trend when comparing target and comparison regions. As shown in Exhibits 3 and 44, when looking at the group averages for each ethnicity combination, attitudes improved for all combinations, with the exception of Croats’ attitudes toward both Bosniaks and Serbs in the comparison region. Similarly, trust increased for all ethnicity combinations, with the exception of Croats’ trust in Bosniaks in the comparison region. When comparing the target group before and after change to the comparison group change, the target group outperformed the comparison group for all combinations for Croats, and for all combinations for Serbs, except for Serbs’ trust in Bosniaks. On the other hand, the target group did not outperform the comparison group for any combination for Bosniaks. 4 In the target group, interethnic trust was measured using the following question: To what degree do you trust people from following ethnic groups? Ethnic groups: Bosniaks/Serbs/Croats; Answer options: scale 1–5, 1 being “No trust at all“ and 5 “Completely trust.“ The same question was used to measure interethnic trust in the comparison group reached through the end-line survey. If a respondent stated 4 or 5, he/she was treated as trusting other ethnic groups. The SCORE Index data, which was used as the comparison for the baseline, used a different question and the measurement scale: Of the various groups on this card, which do you believe are trustworthy and which do you believe are not trustworthy? Ethnic groups: Bosniaks/Serbs/Croats. Answer options: 1. Trustworthy, 2. Not Trustworthy, 9. (Do not read) Don’t Know/Refusal. It was considered that a respondent trusts other ethnic groups if he or she responded with answer option 1 to this question. Inter-ethnic attitudes were measured using the following question for the target group, both for the baseline and end￾line: What is your opinion toward people from the following ethnic groups? Ethnic groups: Bosniaks/Serbs/Croats. Measurement scale: 1– 4, 1 being Very Negative, and 4 being Very Positive. Answer options 3 and 4 were taken as positive attitudes toward other ethnic groups. Another question was used for the comparison group, both for the baseline and end-line: We would like to know your general feelings toward the following groups. On a scale from 0 to 10, where 0 means “very negative“ and 10 means “very positive,“ please circle the number that best represents how you feel toward each group. Ethnic groups: Bosniaks/Serbs/Croats. Measurement scale: 0–10. In the analysis, answer options 6–10 were treated as positive attitudes toward other ethnicities. 28 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Exhibit 3: Group Averages for All Ethnicity Combinations, in % of Surveyed Citizens Expressing Any Level of Positive Attitude ATTITUDE OF BOSNIAKS TOWARD: SERBS: Comparison Target Baseline 47 75 End-line 61 77 -14 -2 CROATS: Comparison Target Baseline 49 78 End-line 62 80 -13 -2 ATTITUDE OF SERBS TOWARD: BOSNIAKS: Comparison Target Baseline 52 72 End-line 60 84 -8 -12 CROATS: Comparison Target Baseline 56 72 End-line 59 82 -3 -10 ATTITUDE OF CROATS TOWARD: BOSNIAKS: Comparison Target Baseline 60 82 End-line 51 88 9 -6 SERBS: Comparison Target Baseline 53 85 End-line 50 85 IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 29 Exhibit 4: Group Averages for All Ethnicity Combinations, in % of Surveyed Citizens Expressing Any Level of Trust TRUST OF BOSNIAKS TOWARD: SERBS: Comparison Target Baseline 30 51 End-line 52 60 -22 -9 CROATS: Comparison Target Baseline 32 52 End-line 51 61 -19 -9 TRUST OF SERBS TOWARD: BOSNIAKS: Comparison Target Baseline 34 47 End-line 58 67 -24 -20 CROATS: Comparison Target Baseline 41 46 End-line 56 66 -15 -20 TRUST OF CROATS TOWARD: BOSNIAKS: Comparison Target Baseline 74 59 End-line 69 75 5 -16 SERBS: Comparison Target Baseline 64 58 End-line 68 69 Finding 2: There are no statistically significant impacts of PRO-Future intervention on either interethnic attitudes or trust in the whole sample. Analyzing each of the three ethnicities separately, we find a positive and significant impact on attitudes of Croats and Serbs, and a negative and significant impact on attitudes of Bosniaks, as well as a positive and significant impact on trust of Croats and negative and significant impact on trust of Bosniaks. Exhibit 5 shows baseline group averages and impact estimates. We did not find a statistically significant impact on interethnic attitudes in the whole sample. When analyzing each of the three ethnicities separately, taking into account positive attitudes toward one of the two groups, we found a positive and significant impact on attitudes of Croats (+10%) and Serbs (+7%) toward the out-groups, and negative and significant impact on attitudes of Bosniaks (–9%) toward the out-groups. 30 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY We also did not find a statistically significant impact on interethnic trust in the whole sample. When analyzing each of the three ethnicities separately, taking into account trust in one of the two groups, we found no significant impact on Serbs, a negative significant impact on Bosniaks (–9%), and a positive significant impact on Croats (+16%). Exhibit 5: PRO-Future Impacts on Inter-Ethnic Attitudes and Trust BASELINE GROUP AVERAGES IMPACT ANALYSIS Attitude Attitude WHOLE SAMPLE 73.2% –0.3% ATTITUDE OF CROATS 80.9% +9.6%* ATTITUDE OF SERBS 72.3% +6.6** ATTITUDE OF BOSNIAKS 72.7% –9.4%*** Trust Trust WHOLE SAMPLE 50.1% –2.2% TRUST OF CROATS 65.8% +15.5%** TRUST OF SERBS 48.6% +4.3% TRUST OF BOSNIAKS 48.5% –9.2%** *p-value between 5% and 10%,, **p-value between 1% and 5%, ***p-value of 1% or below. Figures without stars denote p-values of above 10%. Controlling for demographic variables: age, sex, education, employment status, urban/rural, and multiethnic municipality or not. We also conducted impact analyses taking into account attitudes and trust toward both out-groups and found weaker impacts. EVALUATION QUESTION 3: HOW EFFECTIVE HAS PRO-FUTURE BEEN IN CHANGING PARTICIPATING CITIZENS’ BEHAVIORS? IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 31 Finding 3: All interviewees/FG participants state that PRO-Future was generally beneficial, that it changed participants’ attitudes and behavior, and that reconciliation initiatives are needed in BiH. Overall, all stakeholders with whom the evaluation team spoke expressed their belief that interventions such as PRO-Future are needed. Their opinion was that PRO-Future was generally useful and beneficial and that it positively affected participants’ interethnic attitudes and behaviors. Finding 4: All the interviewees/FG participants also believe that the process of changing citizens’ behavior and building interethnic trust in BiH will be a long-term process and that the momentum resulting from CPT and PRO-Future needs to continue. Based on the KIIs and FG discussions, the highest expected results envisaged by PRO-Future (increased interethnic trust enabling citizens to work together to overcome societal divisions and constructively advocate for institutional changes, which in turn will lead to social well-being and economic prosperity) cannot be achieved in short-term or mid-term, given the complexities of the interethnic relations in BiH. Interviewees/FG participants overall stated that CPT and PRO-Future have made an important step and laid the foundations for further work needed in this area. Finding 5: Both implementers and participants/beneficiaries are insufficiently familiar with PRO￾Future’s designed high-level expected results and have a different understanding of the key terms (e.g., the meaning of reconciliation, alternative narratives, and the role of discussing the past in trust building), expected results, and causal chains. The causal chain of expected results was not supported by the impact analysis conducted within this evaluation. Stakeholder interviewees who have subject matter expertise note that PRO￾Future’s causal chain of expected results is complex and imprecise/unclear in terms of connection to psychological theory and research on reconciliation and peace studies. Moreover, in PRO-Future design (and therefore the implementation as well) there is no work toward establishing any shared narratives. “PRO-Future activities were very useful and beneficial. I think that each participant in the events I attended thought so. Any type of inter￾ethnic communication and building of relationships is extremely important for BiH.” —Participant in PRO-Future events “PRO-Future has done a wonderful job in giving us opportunities to meet, get to know each other, and work together on small concrete projects/events. This is very important. Without PRO-Future, this would not happen, as there are not such opportunities in ethnically homogenous communities.” —Small grant recipient “Inter-ethnic reconciliation here will come, I am convinced of that. However, such complex social processes cannot be artificially sped up anywhere in the world. It takes time and relentless work.” —Religious leader “There is definitely a need to continue with reconciliation interventions, because we have now scratched the surface. Local communities, both citizens and key influencers, have heard of PRO-Future, which is very important, and we now have no problems to contact them and get them engaged. This means that we have built a good infrastructure of people working together and trusting each other.” —Implementer/partner “Reconciliation is a very broad term and it is an individual and personal process that differs for each person. We can only talk about trust at a collective level that can be influenced by interventions such as PRO￾Future.” —Participant in PRO-Future event “Reconciliation is living and working together to ensure that we all have a better living standard.” —Small grant recipient 32 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY The fifteen individuals with whom we spoke who participate (or previously participated) in the PRO￾Future implementation as implementing partners or other partners were mostly not able to articulate the PRO-Future causal change and expected results as laid out in the PRO-Future design. In addition, only one of four mayors/AB member interviewees was able to cite the exact expected results. Similarly, PRO-Future beneficiaries/participants with whom the evaluation team spoke were not aware of PRO-Future specific expected results and causal change. Their general understanding was that PRO-Future aims for better interethnic relations with a focus on the future rather than the past. Furthermore, our impact analysis, as discussed under Finding 2 does not confirm PRO￾Future’s assumed causal chain between interethnic attitudes and interethnic trust. According to MEASURE-BiH’s “Overview of Reconciliation Conceptual Framework in International Literature and Research on Reconciliation in BiH” (2017), the available research papers on reconciliation conducted in BiH are mostly focused on concepts like acknowledgement of responsibility, forgiveness, in-group identification, and interethnic contacts/exposure. The research indicates that BiH citizens strongly identify themselves with their ethnic groups, which is generally viewed as a factor that has negative effects on interethnic interactions and trust (Leonard et al, 2016; O’Loughlin, 2010; Hakansson & Sjoholm, 2017; Whitt, 2010). Considering geography, there is some evidence that interethnic interactions are less frequent in the Republika Srpska than in the Federation of BiH (Leonard et al, 2016). Moreover, it is suggested that heterogeneity of area of residence affects interethnic trust, but the results are ambiguous when it comes to direction of the effect (Hakasson & Sjoholm, 2007; Whitt, 2010). Additionally, ethnic minorities have a higher level of interethnic trust compared to the rest of the BiH population (Whitt, 2010). IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 33 When it comes to understanding the term, “reconciliation,” in both the end-line household survey and CRS survey of PRO-Future beneficiaries/participants, more than half cited vague positive associations about reconciliation (better future, harmony, unity, or tolerance), while almost 20% in the end-line survey and 6% in the CRS survey answered that they did not know what reconciliation is. When asked to assess prerequisites for reconciliation5 , around 77% of citizens surveyed in the end-line survey stated that the truth about what happened must be told in order for individuals and ethnic groups to move on, and that acknowledging suffering of others and prosecution of war criminals are needed for reconciliation, as shown in Exhibit 13. These percentages are slightly higher in the CRS’s survey of PRO-Future beneficiaries/participants, indicating that CRS beneficiaries/participants differ from the general population. However, Exhibit 6 shows that a similar percentage of people (only slightly lower) also affirm that shared truth about the past is a prerequisite for reconciliation. When speaking to the evaluation team, some interviewees/FG participants specifically brought up the issue of a lack of any shared narrative established within PRO-Future. They note that although PRO-Future was successful in transforming some individuals and helping them acknowledge the victims from other ethnic groups (most notably as a result of speaking￾out events), it did not establish any kind of shared narrative or elements thereof. In four cases, interviewees specifically voiced concerns that the terminology used by PRO-Future in terms of acceptance of alternative narratives/acceptance of the existence of alternative narratives of the past may be understood differently by different ethnic groups or different individuals. For example, some people may associate the acceptance of alternative narratives with equating all war roles and losses; for some (and perhaps some ethnic groups more than others), this terminology initially evokes defensive attitudes. 5 This analysis is related to Question 37 of the end-line survey questionnaire. There is essentially one unambiguously common attribute found in existing literature both in terms of definition of reconciliation and the variable which affects trust – and that is inter-group contact/exposure. However, the type of exposure/contact important for reconciliation varies or is left unspecified (e.g. daily contact through work versus contact in which past is specifically discussed). The empirical literature generally confirms strong correlations between inter-group exposure/contact and inter-group trust, however there is no consensus on the causal relationship. In terms of the role of dealing with the past in achieving reconciliation, while traditional theoretical approach is based on the premise that establishing shared truth about the past and acknowledging suffering is a precondition for reconciliation (traditional justice discourse), some of the most recent literature focusing on ordinary citizens’ take on reconciliation in their daily lives contradicts the scholarly theoretical model of transitional justice, arguing that dealing with the past can be counterproductive. In terms of factors influencing trust, general theoretical framework and empirical research indicate that stronger in-group identification and more exclusive identification category influence the out-group trust and interaction negatively, while inter-group contact has a positive effect on trust, although there are differences in how contact/exposure is defined and measured and in the magnitude of found effect of contact on trust (excerpt from USAID/BiH MEASURE-BiH, 2017). Despite ambiguous empirical evidence, existing theoretical literature on reconciliation in large part asserts that for reconciliation to take place each party must truly acknowledge what happened in the past (Čehajić & Brown, 2008; 1997; Chirwa, 1997; Gardner Feldman, 1999; Hayes, 1998; Hayner, 1999; Lederach, 1998; Norval, 1998). It is further asserted that the acknowledgment of the past implies at least recognizing that there are two or more, narratives of the conflict (Hayner, 1999; Kopstein, 1997; Norval, 1999; Salomon, 2004). This is believed to be an important factor in reconciliation, as the collective memories of each party about its own past underpin the continuation of the conflict and obstruct sustainable restoration of intergroup trust (Bar-Tal, 2007). In this approach, reconciliation necessitates changing societal beliefs about the past by learning about the rival group’s collective memory. Through this process, in which one’s own past and understanding are critically revised, a new narrative emerges (Asmal et al., 1997; Hayes, 1998; Norval, 1998). With time, this new narrative should be acknowledged by all. Achievement of stable and lasting peace, as well as reconciled intergroup relations, requires both structural and psychological changes (Čehajić-Clancy, Goldenberg, Gross & Halperin, 2016). Focusing only on the psychological changes, it requires not only a change of perceptions, feelings, beliefs, attitudes and behavioral intentions, but also changes in societal, cultural and educational products such as information in mass media, ceremonies, leaders’ speeches, books, films, or school textbooks. Finally, the fundamental requirement is that the new socio-psychological repertoire be shared by the majority of society members through taking deep root in societal institutions, organizations, and channels of communication (Asmal et al., 1997; Bar-Tal, 2000; Kriesberg, 1998; Lederach, 1997). 34 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Finally, it is also important to note that, while the overwhelming majority of citizens(77%) and PRO-Future beneficiaries/participants (61 out of 876 ) surveyed believe that the truth about past must be told for reconciliation to take place in BiH. As mentioned above, the same percentage of citizens (76%) believe that focus on economic and social development issues is more important than talking about the past, as shown in Exhibit 6. Similarly, when asked what needs to be worked on for everyday life in BiH to get better, 73% of citizens in the end-line household survey said it should be improving the economy, 13% improving interethnic relations, and 11% improving the political/institutional system. Within the CRS’s survey of PRO-Future beneficiaries/participants, 48% said it should be improving the economy, 19% improving the political/institutional system, and 16% improving the interethnic relations. This difference between the overall population and PRO￾Future beneficiaries/participants indicates that PRO-Future beneficiaries/participants may not be representative of the general population in BiH, as the general population prioritized economy improvement more than PRO-Future beneficiaries/participants. Exhibit 6: Citizens’ and PRO-Future Beneficiaries’ Opinions on the Role of Talking about the Past in Reconciliation (in % that somewhat/completely agree) 6 Overall, 16 beneficiaries/participants expressed an opinion that conversations about the past are not necessary for the achievement of reconciliation, whereas 76 beneficiaries/participants did not provide a clear response to this question. “There can only be different interpretations of truth, there cannot be different truths. All ethnic groups must accept one joint truth.” —Mayor “I do not think we can have a solid base for reconciliation unless we have objective data which all ethnic groups recognize as objective. Reconciliation is a never-ending process, but guilt and responsibility must be distinguished from one another.” —Facilitator/lecturer IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 35 Finding 6: Participants in the PRO-Future events and grant beneficiaries who spoke to the evaluation team generally praised the PRO-Future staff’s work. However, several interviewees/FG participants also brought up examples of poor coordination among the implementers and/or inadequate expertise/experience by some staff members, particularly during the early stages, with noticeable improvements made by implementers toward the end of PRO-Future. The majority of interviewees/FG participants (38 out of 76 interviewees/FG participants, excluding donors/implementers/partners) perceived the PRO￾Future implementers and partners as professional, respectful, and passionate in their work on PRO-Future. Particularly high satisfaction was expressed about CIM staff’s work within the youth camps, and therapy work with the speaking-out event speakers, both for expertise in event/activity delivery, and for inciting strong positive reactions of participants through interesting delivery approaches. However, several (11) of the stakeholders that were interviewed noted some problems, including a large number dealing with implementation and other partners. Other problems were seemingly unclear implementation roles and weak coordination, especially in the early stages of implementation. Multiple stakeholders (including by those that were part of the implementation team) shared some examples of poor coordination with the evaluation team. These included: unclear division of roles among implementers with insufficient collaboration at early stages of implementation, issues with small grant management, poor organization of the 2014 peace event, poor planning of the process of MoU signing with the mayors, and an incident in moderating an event in Srebrenica/Bratunac. Also, several interviewees/focus group participants noted that some of the implementing partners/staff members did not have sufficient subject matter expertise and/or enough work experience when they joined the Activity. PRO-Future implementers confirmed this and noted that they organized a number of peace-building education sessions for staff and have intensified internal communication among implementers to mitigate these issues. Finding 7: PRO-Future’s implementation mechanisms planned in Award 1 are numerous and lay out specific tasks. In some cases, they outlined numerical targeted outputs. However, several of these implementation mechanisms have not been used to the planned extent7. Moreover, many interviewers/FG participants described the activities that have been implemented by PRO-Future as too broad in terms of their type and geographic coverage given the available resources, and not deep enough in terms of content and/or frequency. 7 According to the implementing partner, the fact that several implementation mechanisms were not implemented was the result of PRO￾Future’s constant analysis of influence of different implementation mechanisms. Implementing partner also notes that several external political or social events that occurred in 2013-2017 influenced PRO-Future results, including the 2016 local elections in which the winning political party(ies) changed in a large portion of PRO-Future targeted municipalities, inflammatory pre-election rhetoric, disputed celebration of day of Republika Srpska, and application to the International Court of Justice for revision of the judgment of 26 February 2007 in the case concerning the application of the Convention on the Prevention and Punishment of the Crime of Genocide (Bosnia and Herzegovina vs. Serbia). “I have to commend the PRO-Future staff; they are very professional, dedicated, and passionate in their work and that is very important.” —MWG member “For quite some time in the beginning of implementation, it was not clear what the expectation are from each partner. This was the first time that we work with as many as 5-6 other organizations on the same project, in addition to working with local communities.” —Implementer/partner 36 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Exhibit 1 shows 36 detailed implementation mechanisms planned in the PRO-Future Award. Several of them were not implemented as envisaged, including the scope and/or level of engagement of the following: ▪ Network for Building Peace. Based on design, the role of the NBP was to mobilize its members to work with government institutions and the civil society on developing the Platform for Building Peace, and to coordinate the implementation of the Platform’s action plan. On the other hand, the Platform was created late during implementation, without an action plan or inclusion on the side of the NBP. ▪ Association of Parent Councils. According to the Award, activities related to the Association of Parent Councils were to involve capacity building, followed by participation of parents, school principals, and ministers of education, in MWGs and roundtables. During the implementation, engagement of the association was abolished due to their poor capacities, as stated by the implementers. ▪ Media Metar. The designed role of Media Metar was to gather together media critics, analysts and columnists, to annually publish 20 media products on the role of media in reconciliation. They were also to organize two media fora for representatives of media, civil society, and other stakeholders, to discuss the role of media in reconciliation in BiH. However, during year 3 of implementation, Media Metar was replaced with the activity “Journalists for Peace”, although both activities were part of the Award. ▪ Women’s Alternative Government (WAG). The activities were to include collaboration with HCA’s Women’s Alternative Government project, as well as mobilization of women from target municipalities and WAG representatives to engage in local reconciliation initiatives, and in MWGs and Media Metar initiative, respectively. As stated by an implementing partner, the WAG project did not prove effective, so it was substituted with “Living Library.” ▪ Online Peace Schools. The Award envisaged the online peace schools as 6-week courses accessed through the NBP website, with topics selected by network members and website visitors through an online voting system. At the end of each 6-week period, participants were supposed to submit an essay. Also, a learning blog was to be created to regularly post materials and hold discussions. Courses were delivered in the fourth year of implementation through a different website, with a total of five five-day courses Thirty-nine participants completed all five courses and 16 submitted an essay. ▪ Legislation related to torture and missing people. As described in the Award, PRO-Future planned to work on expanding the Network Against Torture’s membership base, delivering advocacy training, and developing an advocacy action plan to amend the BiH Law on Torture and Law on Missing People and other advocacy campaigns. According to PRO-Future reports, the Law on Torture Victims was submitted to the BiH Parliament during year one and was rejected. A two-day training on advocacy and lobbying was given to 15 members of the Network Against Torture in Sarajevo. No further activities were reported by PRO-Future on these issues after the first year of implementation. Looking at PRO-Future reports, it is difficult to group activities into meaningful categories that do not overlap. When asked by the evaluation team to further consolidate types of implemented activities, PRO-Future’s most consolidated list of activities included the following nine categories: 1. activities for youth belonging to different ethnic groups (youth camps, small grants for formal and informal youth groups, youth advocacy, and peace-building campaigns); 2. discussion events (public speaking by war victims, roundtables on the pressing issues in the community, public forums on peace and reconciliation); IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 37 3. inter-religious events (open-door days in religious institutions, public forums on faith and peace building); 4. educational activities on peace building and reconciliation (trainings, online schools, seminars, conferences); 5. activities for women (women’s peace initiatives, events related to women’s rights, small grants for women’s associations); 6. small grants for connection of various ethnic and faith groups through joint work on peace and reconciliation promotion; 7. peace-building campaigns and media products (articles, movies, video competitions on peace-building topics); 8. art events motivated by peace building and reconciliation (e.g., concerts, plays, exhibitions); and 9. sports events with the purpose of promoting peace building and reconciliation. These categories of PRO-Future activities were included in the end-line survey; however, there are some potential overlaps within them as well, as some categories are based on target audience (e.g., youth), whereas others are more based on the content. For example, youth activities were defined as a separate category, although the youth participated in all other activities as well. Exhibit 7 shows that surveyed citizens assigned a similar high level of importance to all nine types of activities, ranking them all as important (range 79%–86%) with little difference by ethnicity and by comparison against the target region. In CRS’s online survey of PRO-Future beneficiaries/participants, there is also little variation among the importance levels assigned to these nine categories, with beneficiaries/participants assigning higher importance to all categories compared to the general population. Exhibit 7: Importance of the Nine Categories of PRO-Future Activities for Reconciliation, in % of Respondents Who Find Them Very or Somewhat Important 38 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY A majority of interviewees/FG participants who were familiar with PRO-Future specificities and implementation stated that PRO-Future was spread too thin, given its available resources, especially after covering 30 new municipalities after the originally targeted 30 municipalities in years two through four. According to them, PRO-Future should have deepened the activities in a smaller number of localities as opposed to widening geographic coverage. Finding 8: PRO-Future events perceived as most effective in attracting the largest interest combined a direct reconciliation/peace-building component with issues based on common local/target group interests and including local key influencers engaging in practical local issues, together with citizens from different ethnic backgrounds. However, interviewees/focus group participants note only occasional presence of key influencers in PRO-Future activities in their localities. According to Activity documentation and interviews/focus groups, the intensity of PRO￾Future activities varied greatly among the 30 originally targeted municipalities. Additionally, although PRO-Future reported close collaboration with communities to make sure that the activities were adjusted to local specificities, specificities, stakeholders perceive that these attempts were not always successful. 86% 86% 84% 84% 84% 84% 83% 82% 79% 89% 88% 90% 87% 86% 88% 86% 86% 82% 93% 93% 92% 90% 89% 93% 84% 87% 91% 70% 75% 80% 85% 90% 95% Educational activities Sports events Youth activities Activities for women Campaigns and media products Art events Discussion events Inter-religious events Small grants CRS’s Online Survey of PRO-Future Beneficiaries/ Participants End-Line Household Survey in Comparison Region End-Line Household Survey in PRO-Future Municipalities “The workshops that were implemented were focused on quantity, not on quality. With one or two short events and no follow-up, we cannot expect any real impact, given the complexity of inter-ethnic relations in this country.” —Participant in PRO-Future events IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 39 Events implemented in localities were perceived as generally interesting by everyone, but at times not specific enough to that locality (e.g., roundtables, which were delivered on general topics, such as the role of culture in peace building) and without enough participation of local key influencers. Events perceived as most effective in attracting the largest interest included local key influencers engaging in issues of practical importance to citizens and the local context in interactive formats that bring citizens from different ethnic backgrounds to interact with each other to do something practical or useful for the community and/or their specific group (e.g. women or youth), while simultaneously also discussing/addressing interethnic perspectives and/or reconciliation. However, only a few interviewees/focus group participants noted presence of key influencers in PRO-Future activities in their localities. Based on interviewees’ examples, such events included a religious leaders’ discussion about the environment in Gorazde, religious communities’ cemetery repair in Maglaj, and the repair of the road connecting two villages in Srebrenica, supported by key influencers. As shown in Exhibits 12, 14, and 16, the number of PRO-Future events and small grants ranged from 16 events and four grants in the Drvar-Glamoč municipality pair, to 52 events in the Mostar-Nevesinje municipality pair, and 57 grants in the Doboj-Maglaj municipality pair. The largest municipalities (by population and territory) absorbed the most grants and activities. According to interviews with implementers, this was due in part to a custom-made approach implemented by PRO-Future taking into account the different level of the “maturity of municipalities” in terms of local interethnic relations and progress in reconciliation and due in part to higher interest expressed in PRO-Future activities and grants in those larger municipalities. PRO-Future implementers noted that their implementation approach has been constantly adjusted based on each community’s/ethnicity’s specificities, as MWGs were involved in decision-making, related to activities that would be implemented in given localities. However, it is unclear how systematic this was (which criteria were taken into account) and to what extent it was implemented. One example of a custom-made implementation approach in communities was speaking-out events not being organized in some municipalities, because it was assessed that the reception would not be positive. Although the custom-made approach was acknowledged as necessary by most interviewees/FG participants, only a few interviewees/focus group participants noted the presence of key influencers in PRO-Future activities in their localities and ten interviewees/FG participants noted that not enough was done in less-developed target municipalities, mostly due to lack of local CSO capacity, and/or weak engagement by the MWG members, but not because of a lack of will in those localities. Finding 9: PRO-Future small grants were perceived as useful by all interviewees/focus group participants, with activities establishing common ground for active interethnic cooperation being perceived as the most effective. However, small grant recipients/applicants noted that selection criteria were not sufficiently clear or consistent. “We were at a disadvantage from the start, as we do not have CSOs that have enough capacity to be able to apply for and implement projects with PRO-Future. Our problem was not that we and our pair municipality did not want to work together—we really did, but we had nothing to work on.” —MWG member “For our municipalities here, we should have started with the most basic events to bring us together, as we have no chance whatsoever for inter-ethnic interaction here. Every community has a different issue and starting position, that must be taken into account more”. —Participant in PRO-Future event “Each activity and event that is not narrowly related to a specific locality and their specific interests is not effective enough. Bringing speakers from somewhere else instead of having the political leaders or religious leaders or any leaders from that specific community is not effective enough.” —Participant in PRO-Future event 40 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Small grants were perceived as useful and as a motivation for many to participate in PRO-Future. As is the case with examples of non-grant local events, explained under the previous findings, the most frequent examples of small grant initiatives that generated the largest interest included activities in which, in addition to direct discussion related to peace building/reconciliation, an active inter-ethic interaction was established based on common local/target group interests. According to interviewees and the PRO-Future database, these included youth camps and other events in which topics of leadership, teamwork, and other skill-building or sports/recreation activities, were combined with reconciliation components (e.g., street workout and reconciliation workshop); initiatives that included economic development elements (e.g., women’s entrepreneurship and reconciliation workshop); and local environmental initiatives (e.g., municipality landscaping and peace-building activity). The majority of interviewed grant recipients (11; most notably, the smaller CSOs and informal groups) noted that they faced some difficulties in the process, such as complicated online application procedures, complications with payment procedures, delays in payments, and unclear rules when it came to visuals and branding. Additionally, 10 other stakeholders who did not receive small grants themselves stated they heard that small grant recipients faced such difficulties. Based on the perceptions of the majority (10) of small grant applicants, the criteria for small grants were not clear or consistent in terms of the peace component definition and requirements, without sufficiently taking into account specificities of the localities at times, and without enough explanation and communication of the process and goals up front. Some interviewees (six) perceived selection committee members as not having sufficient expertise to judge the peace component of the proposals. In some cases, grants were perceived as being awarded for activities in which the reconciliation/peace component was too marginal or did not exist at all (e.g. an infrastructural project that in practice does not bring people from different ethnic groups together or events that may include some interethnic contact but without any explicit reconciliation/peace building element or discussion). Grant procedures were generally perceived as too complex, due both to USAID rules and additional implementers’ rules. In a couple of cases, potential applicants gave up applying and/or implementing activities because the process was too complicated. Finding 10: Most of the KIs and FG participants perceived speaking-out events, youth peace camps, and open-door days as the most effective types of activities in influencing participants’ personal transformation and long-term change of attitudes. “The most effective activities are something tangible, something productive that people work on together in direct contact to bring about joint benefits. This is a pre-condition for increased trust to be achieved subsequently, as people can then turn around and reflect on the positive outcome of their joint work in a non-threating way. Because of this, any kind of reconciliation or peace-building activity that is subsequently done with them is more effective, as they are better prepared and more receptive to it.” —MWG member “Some of the approved grants were simply not peace or reconciliation projects at all. Because they did not have good applications, they approved just anything. And in many cases, those were infrastructure projects that municipal officials wanted, without any or sufficient inter-ethnic contact or use or any kind of reconciliation discussions/event.” —Small grant recipient and participant in PRO-Future events “Small grants are so small that one cannot expect the activities implemented with them to be effective. And procedure-wise, they were too complicated, they returned my application multiple times. I have prepared applications and implemented many projects, including the US Embassy projects, but I never had as many problems with grant applications as I had with PRO-Future.”. —Small grant recipient and participant in PRO-Future events IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 41 Almost all interviewees/FG participants testified that PRO-Future’s speaking-out events and youth camps were the most effective in sparking transformation in accepting the existence of alternative narratives of the past in participants. During speaking-out events, Bosniak, Serb, and Croat war victims, who previously underwent group therapy as well as training in public speaking and presentation skills, told personal stories about their war experiences. These individual stories were followed by the common message that every ethnic group suffered during the war. By demonstrating directly to the audience that all ethnicities suffered, the speakers motivated the audience to rethink their narratives on the war. Youth peace camps gathered youth from different parts of BiH, with the goal of meeting each other and improving interethnic relationships. Typically, the camps included sports activities, peace-building workshops, and/or workshops on stereotypes and prejudices, as well as some other activities of interest, such as teamwork or leadership skills. All interviewees involved in peace camps agreed that participation in these camps motivated some young people to use the knowledge and skills acquired through the camp to become active in the peace-building movement, apply for small grants, and organize their own peace camps or other peace-building activities. All participants in youth camps with whom the evaluation team spoke testified to connections being made with other youth camp participants from other ethnicities after the camp. Open-door days, in which youth visited each other’s place of worship and entered into dialogues with participants and religious leaders from different faiths, have also been cited by almost all interviewees/FG participants who attended them as effective, enlightening, interesting to youth, and having the potential to spur long-term personal transformation. “The most influential are speaking-out events. I participated in several of them and regardless of who the participants are, these are very influential. Audience always has strong reactions, in some cases very emotional. Speaking-out events are what sets PRO-Future apart from other interventions and makes it more effective.” —MWG member “The best events were youth camps implemented by CIM. I think they did the best job overall in the whole PRO-Future, the most effective.” —PRO-Future event participant “I personally transformed as a result of PRO-Future youth camps. I listened to the stories of other young people and I understood they had prejudice against my ethnic group in the same way I had prejudice against their ethnic group. I completely changed and realized how similar we all are.” —PRO-Future event participant “We visited Catholic and Orthodox churches and a mosque. In each of them, religious leaders welcomed us warmly and talked about their religion and view of the world. It was beautiful and we were so interested and ended up staying longer than planned to interact and ask questions and make parallels among our different religions. They are much more similar than we thought before” —PRO-Future event participant 42 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Finding 11: Several interviewees/FG participants noted that the effectiveness of speaking-out events could have been further increased with the addition of some kind of follow-up and more interaction. In addition, interviewed participants of speaking-out events, stated that these events are mostly attended by those who are already active in reconciliation/peace-building activities. Although everyone agreed that the speakers were generally well prepared and felt empowered when telling their stories, a few KIs mentioned some fatigue among the speakers from repeatedly telling the same story. Some thought that PRO-Future needed to provide additional psychological support. All interviewees/FG participants who attended speaking￾out events described the events as positive, touching, and cathartic. However, around half of them (15) shared a concern that these events may trigger painful emotional responses in the audience and leave them without appropriate closure, follow-up, or debriefing. Speaking-out events were held in front of various audiences (e.g., members of war victims’ associations and students) whose personal experiences in the war or the ways they are dealing with the past were not assessed. Moreover, the participants of speaking-out events, as stated by all interviewed participants and several other stakeholders, were mostly activists/CSO members who have already been active in other reconciliation /peace activities, as it was difficult to attract wider groups of citizens who in many cases are not willing to talk about the past. According to speakers’ and speaker trainers’ perceptions, speakers generally received adequate psychological preparation and felt empowered when telling their stories. However, some concerns were also voiced, in particular that some speakers may have started to feel detached from their stories as a result of sharing them many times, and some interviewees noted that PRO-Future needed to provide additional psychological support to speakers as it was done in CPT. Finding 12: Youth participants/beneficiaries, in particular, believed that the most effective reconciliation activities for youth included activities of interest/use for young people specifically in addition to a peace/reconciliation component, with interaction among the participants, ensuring their sense of agency in the activities. “I think everybody should attend a speaking-out event; they are very powerful and important. However, I think some kind of closure is needed after these events. Even those of us who knew things about different concentration camps were very shaken after hearing the stories, so I think that audience needs to be prepared for these events and also provided with some type of follow-up.” —Participant in PRO-Future event “I personally do not want to even remember the war, I would like to erase those four years from my memory. It is traumatic for me to think about it.” —MWG member “I do not think that we should be digging out the past, because usually one side accepts the truth, the other does not, and it all can end up being counter-productive. The gap between different versions of the stories is simply too large.” —Participant in PRO-Future event “I think all of us working on organizing or supporting events had issues with not being able to get participants for the events, because when we use terms such as “reconciliation” and “accountability for future” they somehow close down and do not want to participate”. - Implementer/partner “Speaking at the speaking-out events means a lot to me. Each time I tell my story I feel better, more relieved, as I see that people are interested and empathize with me. And I somehow feel that they heal with us too.” —Speaker IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 43 Youth perceived youth camps as the most effective in creating interethnic friendships and continued contacts and collaboration. They highlighted that youth camp components such as leadership, team building skills, arts, or sports components are essential, as they both prepare the participants for the reconciliation/peace￾building component by making them more relaxed, and also provide the opportunity for one-on-one direct interaction with participants from other ethnic groups. Youth interviewees/FG participants shared particular examples that highlighted the importance of having ownership and a sense of agency in the activity consistent with a participatory action approach. Participants gave suggestions, such as: placing more youth in leadership roles, offering mentorship for youth in areas like writing proposals, and giving youth more agency and overall responsibility in activities they are participating in. Although PRO-Future implementers in many cases recognized youth who have potential to be future leaders and gave them roles of responsibility within the project (as a part of youth camps, roundtables, and even in designing future activities for youth), the few young persons who were interviewed were very active and included in the activities. They brought up (without being explicitly asked) that they felt they should have been given an even larger role in designing and delivering all youth activities. All of the youth with whom the evaluation team spoke who participated in the camps where the so-called council method was applied (implemented by CIM, in which participants sit in a circle, everyone speaks, and everyone is equal) were thrilled by the experience they had. In general, youth interviewees/FG participants believe that future interventions should target youth exclusively, whereas most adult interviewees/FG participants believe that both youth and adults should be targeted. Finding 13: In most cases, participants in PRO-Future arts/culture peace-building events found them interesting, attractive to broader audiences, and most effective when combined with a direct peace￾building activity. Interviewees/FG participants who participated in PRO￾Future cultural peace building events (10) perceived these as very effective when combined with a component focusing directly on addressing peace building and allowing for follow-up discussions and opportunities to draw parallels to reconciliation experiences. This is in contrast to including only indirect forms of peace building—i.e., bringing individuals together only to view/watch an event (such as film screenings, plays, concerts, exhibitions) or organizing arts-based media competitions, without direct interaction between participants. A participatory action approach gives participants ownership and agency over setting and meeting goals and engaging in the intervention from planning, to implementation, and through the evaluation (Minkler and Wallerstein, 2003; Grbich, 1999; Baum, MacDougall, & Smith, 2006). The purpose of a participatory action approach is to enable action; deliberately share power between the implementers and participants; and grant participants opportunities to become partners in the entire process. Literature shows that using a participatory action approach ensures that interventions make a deeper impact on the participants involved and lead to longer term effectiveness in meeting goals compared to a traditional implementation approach. “Young people have same problems wherever they live in BiH and whatever ethnic group they belong to, their main problems having to do with unemployment and lack of opportunities to develop their skills and interests. So bringing them together around these issues, for example to volunteer together or get some job training, while at the same time having them interact with each other and also hear and discuss about reconciliation issues is the best way to influence them.” —MWG member In literature, the theoretical framework for using the arts as a tool in peacebuilding is based on the premise that reconciliation and acceptance requires modes of expression that give voice to thoughts and feelings where rationale falls short (Brandeis University, 2017; Zelizer, 2005). Research highlights several themes and benefits that result from utilizing the arts in peacebuilding including how the arts offer opportunities to: address conflicts non-violently; transform relationships through safe and expressive processing; facilitate processes that foster the abilities required for sustained peace (e.g. compassion, receptivity, imagination, creativity, and the ability to embrace paradox); utilize creative connection to counteract dehumanization of the other; and bear witness to suffering; and create safe spaces for storytelling. 44 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY According to a number of KIs (20), these events are interesting and appealing to the audience. One KI stated that these activities could be more successful in localities that are not yet ready for speaking-out events or in smaller communities with no regular access to cultural events. Among the PRO-Future events of this type, Living Library seems to have left the strongest impression on the audience. These libraries function like normal libraries, except the books are the people — readers can borrow a “book” (a person) for a limited period of time, and “read” it. The PRO-Future library includes women victims of domestic violence and war, women with disabilities, youth returnees, parents who lost their children, a priest who lost relatives in the war, peace activists, war veterans, children from mixed marriages, and others. Finding 14: The extent to which PRO-Future reached wider groups of citizens and those less receptive to interethnic reconciliation and peace-building activities was perceived as not being large enough, as most participants were those already active in their communities. Within the end-line household survey, 12% of citizens in PRO-Future municipalities reported that they participated in some type of peace-building events, while 3% reported participating in PRO-Future activities. On the other hand, within CRS’s survey of PRO-Future beneficiaries/participants, 78% reported that they participated in some type of peace-building events. This stark difference indicates once again that the PRO-Future beneficiaries/participants differed significantly from the general population, with nine of ten citizens never participating in any peace-building activities. According to PRO-Future staff and other KIs, there was no particular systematic way of targeting participants for main PRO-Future events (speaking-out events, roundtables, and other events open to the public). PRO￾Future generally reached out to the active population (activities targeted people who were already active in communities and more likely to be receptive), particularly among youth and CSO members. According to most interviewed event participants (28), less receptive audiences (ordinary citizens as compared with activists) were not sufficiently included in PRO-Future activities, although all agreed that it is very difficult to do so. Suggestions provided by KIs and FG participants included: 1. framing reconciliation and peace-building activities in a way that is more interesting and less threatening8 to the broader audience than only conflict- or past-related events (such as speaking out events), by including them within activities that have components such as economic cooperation, training in life skills, sports, and arts/culture; 2. using local contacts specifically to actively target the less receptive audience; 3. specifically targeting war veterans’ associations and political parties as one of the primary “spoilers;” 4. working more with mainstream media; 5. working through formal education institutions and extracurricular activities to target broader youth; and 6. using diplomatic efforts in working with high-level politicians. 8Activities that could be perceived as threatening are those that include talking about the past only. Less threatening activities are those that, in addition to talking about the past, involve some other activities, such as skills trainings, sports, arts, etc. “Art exhibits and theater plays were excellent, reactions of the audience were great and it open up discussions about inter-ethnic reconciliation among them.” —Participant in PRO-Future event “It is always same organizations whose members are invited to events. Participants are from different NGOs, but ordinary citizens are never there. This is the greatest shortcoming of PRO-Future, in my opinion.” —Participant in PRO-Future event IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 45 The finding that PRO-Future reached only a narrow group of active citizens was confirmed by the results of the impact analysis, which did not show an overall impact of PRO-Future in targeted municipalities. Finding 15: Analysis of the end-line household survey data indicated that the following factors may be most effective in reconciliation initiatives: high-quality contact with members of other ethnicities9; activities that target reversing the subtle dehumanization processes of other groups; high frequency of interethnic contact; and targeting ethnically homogenous localities10. End-line household survey data indicated that frequent and positive intergroup contact is associated with less negative intergroup emotions and higher levels of acceptance of others. All interviewees/FG participants highlighted the importance of interethnic exposure11. Household survey data also included data on primary and secondary emotions. All respondents were asked to indicate the degree to which members of ethnic groups were capable of feeling six different emotions, both primary (happiness, fear, sadness) and secondary (love, guilt, compassion). Overall, citizens, irrespective of their ethnic/national self-categorization, tended to express in￾group preference by attributing more secondary emotions (emotions considered to be uniquely human) to their own group in comparison to the other two groups. Results indicated that participants tended to engage in a subtle dehumanization process of other groups by attributing fewer secondary emotions to them12. 9 High quality contacts are those described as somewhat or very positive by the end-line survey respondents, as shown in the household survey questionnaire in annex. 10 The evaluation team assessed whether a municipality is ethnically homogenous using two different measures: 1. using the assessments of the end-line survey respondents whether they live in municipality where their ethnic groups is majority, minority, or balanced; 2. using Census 2013 data, and marking municipalities that have at least 30% of citizens belonging to an ethnicity other than majority, as well as ethnicities that have an approximately balanced number of at least two ethnic groups. The results were comparable using both measures in a regression analysis. 11 Similarly, MEASURE-BiH’s National Survey of Citizens’ Perceptions (NSCP-BiH 2016) indicated that more frequent contacts lead to a higher level of trust. 12 Such subconscious process of others’ dehumanization (infrahumanization as an implicit dehumanization process) can stand in contrast to more explicit psychological processes. It is not uncommon that peoples’ explicit attitudes are more in line with expected social norms and under the influence of social desirability processes. In contrast, more subconscious processes might be a more valid reflection of true attitudes and beliefs. This indicates that future reconciliation interventions should consider attempting to capture attitudes and beliefs in a more implicit way (e.g., through attribution of emotions). These findings are in line with theoretical and empirical evidence found in the literature, which has consistently re￾confirmed intergroup contact to be associated with improving intergroup relations. For example, quantitative (Pettigrew & Tropp, 2006, 2008) and narrative (Brown & Hewstone, 2005; Pettigrew, 1998) reviews have provided support for the contact hypothesis (Allport, 1954), reaffirming intergroup contact as a key variable in improving intergroup attitudes. Brown and Hewstone (2005) noted that the beneficial effects of intergroup contact have now been observed on a wide range of outcome measures. However, research shows that it is not the mere contact frequency that is associated with es positive changes, but rather the quality of contact, as also indicated by findings of this evaluation. As Allport (1954) stipulated, the quality of intergroup contact rather than its mere quantity (i.e., simply the number of out-group people known) is most predictive of prejudice reduction (e.g., Binder et al., 2009; Brown, Maras, Masser, Vivian, & Hewstone, 2001; Eller & Abrams, 2003; Stephan, Diaz-Loving, & Duran, 2000). The underlying assumption of the beneficial effects of contact on intergroup attitudes is that getting to know members of other groups, with time, decreases negative emotions (e.g., anxiety) and thus produces more positive out group perceptions and behavioral tendencies. 46 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY The evaluation team used a multivariate regression analysis of end-line household survey data to examine personal￾level correlates of the ethnic out-group attitudes. We found that the strongest predictor showing a significant correlation with overall readiness for close interethnic contact13 was the perceived quality of the contact with out-group members. Following were: a willingness to accept out-group perceptions on war events,14 frequency of interethnic contacts, and not belonging to a clear ethnic majority in the area. The frequency of religious attendance is slightly negatively correlated with out-group attitudes. Sex, age, and urban/rural settlement do not have statistically significant predicting power. EVALUATION QUESTION 4: HOW EFFECTIVE HAS PRO-FUTURE BEEN IN GETTING KEY INFLUENCERS IN TARGETED MUNICIPALITIES/CITIES TO SUPPORT LOCAL RECONCILIATION INITIATIVES? Finding 16: All KIs and FG participants agreed that work with politicians as key influencers should continue, as it is perceived as essential for reconciliation. Within its work with key influencers, PRO￾Future focused mostly on work with politicians, and broadened the scope of its work within this category in the last stage of PRO-Future implementation to include politicians/government officials from higher government levels, in addition to municipality/city mayors and officials. Most interviewees/FG participants perceived this as the right approach. Among the categories originally defined as PRO￾Future key influencers—political and religious leaders, media, business owners, civil society, and education officials—PRO-Future focused mostly on working with politicians. Almost all of the interviewees/FG participants perceived that work with key influencers was essential, singling out politicians as the most important key influencer category, albeit the most difficult and challenging to work with. Consequently, a majority of the interviewees/FG participants (22 out of 42) first spoke about politicians as key influencers, even though they were presented with a definition that included religious leaders, the media, teachers, and so on. Similarly, when it came to their opinion on what is needed for BiH to advance in reconciliation, citizens’ open￾ended answers in the end-line household survey ranked better politics above better relations among citizens, as presented in Exhibit 8. 13 Measured using factor analysis of end-line survey question 26A-H. 14 Measured using factor analysis of end-line survey question 25A-D. “I think that people from different ethnic groups simply do not communicate frequently enough and deeply enough through everyday work and private life. Mutual respect can be built only at such a personal level.” —Participant in PRO-Future event “The main influencers are politicians, there is no doubt about that. This means that we have to work with them, regardless of how challenging that is, they cannot be avoided.” —Participant in PRO-Future event IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 47 Exhibit 8: Requirements for BiH to Advance in Inter-Ethnic Reconciliation, in % of Surveyed Citizens in End-Line Survey In 2016, the AB, originally intending to focus on local politicians, redirected its focus to higher-level government officials/institutions including state parliaments, the council of the ministers of BiH, and the presidency of BiH. This two-pronged approach in working with the politicians at both the local and higher levels (bottom-up and top-down approaches) was perceived by most stakeholders as necessary, given that they believe that the political influence comes from high-level political party politicians who mostly obstruct the processes of reconciliation with inflammatory rhetoric. All speakers at the speaking-out events, with whom the evaluation team spoke, stated that these high-level political party politicians and government officials should attend speaking-out events, as they have the most influence on the country’s path in reconciliation. Finding 17: PRO-Future’s Platform for Peace, endorsed by 60 mayors in late 2016 and by the Council of Ministers of BiH in 2017, was considered a major achievement by PRO-Future’s work with politicians, as stated by KIs and FG participants. However, the Platform does not propose any concrete measures or policies to be designed and adopted, except from calling for the Strategy of Sustainable Long-Term Peace to be developed and adopted. 13% 6% 0% 1% 1% 1% 1% 1% 10% 15% 23% 32% 14% 3% 0% 1% 1% 2% 2% 3% 6% 17% 25% 35% 14% 2% 1% 1% 1% 0% 2% 3% 4% 12% 20% 47% 14% 3% 1% 1% 1% 1% 2% 3% 5% 14% 23% 41% 0% 10% 20% 30% 40% 50% DK/NA Other Better education People have already reached reconciliation Reconciliation is hardly achievable Foreigners should leave the country Youth should take over the government Truth should be told Forgetting about past, forgiveness Better economy/employment Better relations among people Better politics Whole Sample Bosniak Serb Croat 48 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Almost all of interviewees/FG participants who were familiar with PRO-Future’s Platform for Peace commended PRO-Future for this accomplishment, as this was the first document signed by politicians from a broad spectrum of political parties and ethnic groups. PRO-Future implemented a comprehensive and complex process that led to politicians’ support of the Platform. The Platform for Peace was envisaged in the Award as a citizen-influenced roadmap that would outline policies. However, the endorsed Platform was written in broad terms and did not outline specific measures/policies, except to “call for the creation of a strategy of sustainable long-term peace.” It is not clear how much of a collaborative approach was used in drafting the Platform, as only one KI (from the Facilitator/Expert Group) reported being given an opportunity to comment on the Platform.15 In addition, a couple of KIs expressed concern with the fact that the Platform is not binding, and that political rhetoric did not change although political representatives supported the Platform. Finding 18: Stakeholders perceived the mainstream media as the second most important key reconciliation influencer after politicians, followed by religious leaders. According to the KIs and FG participants, PRO-Future’s work with key influencers was perceived as less focused on religious leaders and the media in comparison to its work with politicians. Nevertheless, the work that has been done with both the IRC and the media was perceived as laying the grounds for necessary intensified work in the future. Not much work was done with business sector representatives or educators as key influencer categories. PRO-Future staff explained that they intensified work with the media toward the end of the intervention. According to their reports, PRO-Future reached more than 1.6 million people through the media, and almost half of that was in the first quarter of the last implementation year. Overall, one in five citizens had heard of PRO-Future, 20% in target and 16% in comparison regions, based on the end-line survey. There are several examples of journalists whose capacities were built in this subject matter through PRO-Future. However, overall KIs reported that the media participated mostly in providing coverage of events, as opposed to being an active partaker as a key influencer. Generally, local media has been responsive to covering PRO-Future activities, as well as some online portals and social media. Although the Activity included a number of large media outlets to some extent (including Al Jazeera Balkans, N1, Bosnian-Herzegovinian RTV, Federal TV, Radio and Television of Republika Srpska, Croatian National TV, Deutsche Welle, Radio Free Europe and Liberty TV, Klix, Bljesak, Federal News Agency, Radio Sarajevo, Anadolu Agency, Dnevni avaz, Nezavisne novine, and Večernji list), overall coverage in the mainstream media, which mostly shapes public opinion, was perceived by almost all interviewees/FG participants as not frequent or deep enough to have a country-wide reach and to affect the awareness of a wider audience. Interviewed implementers and other stakeholders note that a perception by the mainstream media that positive stories were uninteresting presented an implementation challenge. The experts 15 According to a KI, PRO-Future also ensured support by presidency members, presidents of both entities, and Ministry of Human Rights toward the end of project. “Media, especially public TV broadcasters, have a very strong influence on citizens and because they are not politically neutral, they exacerbate interethnic tensions.” —Participant in PRO-Future event “Public broadcasting service, their TV channels, are key, because of the influence they have on most citizens. I believe that serious work needs to be done with them, at the highest levels with management and editors. What we need is prime time devoted to promoting reconciliation, whole TV shows devoted to it, rather than one minute at the end of the news.” —Facilitator/lecturer IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 49 who were interviewed noted that BiH the media need capacity building and financial assistance when it comes to peace journalism, and providing effective stories instead of just covering events. Similarly, PRO-Future’s established relationship with the IRC is perceived as laying the grounds for more intensive future work. The IRC’s participation is perceived as necessary and positive overall. The PRO￾Future team noted that religious leaders were engaged through public forums, open door days, inter-religious chapters, small grants, an online peace school, and other activities. However, 15% of interviewed stakeholders who were familiar with PRO-Future’s overall activities perceived the involvement of religious leaders, especially at the local level, as rare. Religious leaders received only a few small grants within PRO￾Future, mostly due to lack of their applications. Several reasons for this were mentioned by the implementers and the partners, including: lack of technical skills to write project proposals; lack of free time aside from their main duties; a strictly centralized decision-making protocol; and a lack of understanding of the financial reporting concept. Moreover, a few KIs believed that the IRC speakers should be better trained to speak about peace to the wider audience. The fact that they are respectable in their communities and that citizens are listening to their messages highlights the importance of their communication style, language, and rhetoric. Most PRO-Future beneficiaries/participants (22) believe that working with religious leaders and media is essential for effective reconciliation initiatives, given their influence on citizens and the close connection between ethnicity and religion in BiH, notwithstanding the fact that working with these key influencer categories is very challenging. All of PRO-Future’s activities involving religious leaders from different religions participating jointly, such as open￾door days or visits to the places of suffering, were perceived as positive and effective in bringing about interethnic understanding. According to the Award, prominent business owners were considered important key influencers in design and were to be encouraged to take a more active role in reconciliation. However, none of the KIs/FG participants mentioned involvement of business sector representatives as key influencers within PRO-Future. Representatives of this sector were mentioned only in the context of providing local contributions for the small grants or within small grants initiatives that combined entrepreneurship/economic development with reconciliation/peace-building workshops and components. Although PRO-Future did some work in the education sector through EUROCLIO HIP and the Nansen Dialogue Center, educators/schools’ involvement and work with the Association of Parent Student Councils was not implemented as planned in the Award, which envisaged engaging and influencing ministers of education and principals to support and participate in reconciliation projects. They would facilitate dialogues about challenging reconciliation topics related to education (such as two schools under one roof and developing a unified “Our imam here has a radio talk show in which he encourages young people to socialize together, including through joint iftars or though the youth center. Our friar here also frequently holds workshops or events so that these kinds of inter-ethnic interactions can take place. Whenever youth would like to visit the monastery of the church or the mosque, the doors are open to everyone. Our imam here gives rides to church for Catholics every Sunday. He is an example how we should behave, how to build our life together. So, when it comes to our religious leaders, they are very influential and they can help the reconciliation processes.” —PRO-Future small grant recipient “In our community, religious leaders never do anything together. They are influential in communities and getting them together and discussing issues of local importance would send a strong reconciliation message.” —Participant in PRO-Future event “Our two mayors had not even met once in these four years and present something to the public. As a working group, we were meeting, but we never met with mayors. The only time the two mayors met was during the large ceremony in which the Platform was signed.” —Member of a municipality working group 50 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY curriculum), and participate in MWGs and roundtables. Many stakeholders (especially emphasized by facilitators and educators themselves) stated that a systematic institutional approach is necessary, in general, when it comes to greatly needed education reform in BiH. However, they noted that such comprehensive reform is out of the scope of PRO-Future and would be extremely difficult to implement in the current political environment in BiH. Exhibit 9 shows BiH citizens’ and PRO-Future beneficiaries’ trust levels toward different categories of key influencers. Based on the household survey, among politicians, journalists, religious leaders, civil society leaders, education officials/educators, and business owners, citizens trusted education officials/educators and religious leaders the most, and politicians and journalists the least. Results for PRO-Future beneficiaries/participants are similar. Exhibit 9: Trust in Different Categories of Key Reconciliation Influencers, in % of Surveyed Citizens Expressing any Level of Trust Finding 19: Although all perceived PRO-Future’s work with key influencers as important, most did not perceive that cooperation among key influencers increased. Neither interviewees/FG participants (including both key influencers and other participants/beneficiaries of PRO-Future) nor the general population in the end-line survey perceived key influencers to have increased cooperation (in terms of frequency) in the past four years in BiH. Interviewees/FG participants noted that most IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 51 examples of good cooperation in pairs of municipalities were established prior to PRO-Future, along the lines of economic or other community infrastructural joint interest, and not many new partnerships were built during PRO-Future. Apart from MWG members, only eight other KIIs recounted activity where mayors of different municipalities interacted. Half of those were related to the event for signing the Platform for Peace. In the end￾line household survey, 16% of citizens stated that cooperation had increased, while in the CRS’s survey of PRO￾Future beneficiaries/participants, 40% of respondents stated that cooperation had increased, again showing the difference between the general population and PRO-Future beneficiaries/participants. This possibly indicates that PRO-Future events with key influencers have not been visible. Finding 20: PRO-Future’s main implementation mechanism for working in targeted areas was through MWGs, which was generally perceived as a necessary approach to ensure local ownership and appropriateness of local activities for those specific localities. However, the levels of MWGs’ and MWG members’ engagement in PRO-Future activities varied widely and was in some cases perceived as not frequent enough and not well coordinated by PRO-Future. The Award envisaged that MWGs would be composed of different stakeholders, with at least 5 religious or political leaders, education officials, NGO leaders, or other relevant community leaders serving on an MWG. These community leaders were then envisaged to play a role in planning activities, defining criteria for participant selection, mobilizing participants, identifying linkages within the communities, interacting with citizens, participating in community dialogues and roundtables, and contributing to the Platform for Peace. The creation of MWGs was perceived as a good idea by all stakeholders. However, in practice, as described by the KIs, the implementation differed somewhat from the envisaged plan. MWG composition, direct contact of mayors/municipal officials and their visibility in the community and participation in PRO-Future activities, and MWGs’ contribution to PRO-Future activities implemented in the municipalities are examples of this. Based on interviews/FGs (in particular, the interviews with the MWG members), there was large variation in the level of MWG members’ engagement in implemented activities. MWG members were selected mostly by the mayors and usually consisted of three or four members from municipal offices and NGOs. The majority (eight) of the MWG members interviewed noted that only one or two people were active in the MWG (usually from NGOs), and that the expectations were too high regarding the large scope of MWG members’ responsibilities (particularly in communities where CSO sector capacities are weak), given the voluntary nature of their engagement. Some very active MWG members explained that they invested much of their personal time and resources (as they were holding meetings, spot checks, etc.) in these activities and that some MWG members were not ready to take over so many responsibilities. Other MWG members confirmed this by saying that they were not much involved in the MWG work, usually due to other duties and responsibilities However, PRO￾Future tried to address this problem and strengthen the MWGs by bringing new members. A few stakeholders specifically brought up the issue of not enough contact being established between the paired mayors, who mostly did not participate in local events. Additionally, the wider community was not aware of existing cooperation. Moreover, a few of them “In our community, we had good support from both municipality officials and religious leaders. The mayor opened several events and it helped with attracting media attention. The MWG managed to get them interested and engaged, that was very important.” —PRO-Future small grant recipient “The two mayors never got together and did not attend MWG meetings. They should hold joint events in their locality—for example, get together with young people in an open discussion on how inter￾ethnic relations in our community can be improved.” —MWG member 52 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY noted that there were not enough opportunities for cooperation between MWGs in paired municipalities, especially among all of the MWGs. A couple of MWG members also noted that PRO-Future did not provide a timely schedule for planned activities, and that there was no feedback on the implemented activities. Finding 21: The Expert Group was formed in first year of implementation consisting of three experts, and during implementation was significantly expanded. The extent of experts’ technical advice and guidance on strategic planning and implementation was not clear, with some thought that it was limited. PRO-Future’s Award envisaged an Expert Group consisting of three members of the AB to be formed in order to play an advisory role for strategic planning for the project, including support to develop strategies, recommendations, and technical advice. The Expert Group, formed in year one of PRO-Future implementation, included 3 experts as planned, but was later expanded to include 25 experts from different profiles. It is not clear to what extent, or whether, the Expert Group was involved in the strategic planning of the Activity. According to some KIs, the extent of their strategic engagement was limited. 4.2. CONCLUSIONS Estimates of the impact of PRO-Future interventions on citizens’ interethnic attitudes and trust in the 30 municipalities in which PRO-Future has been implemented in 2013-2017 compared to the municipalities/regions in which PRO-Future has not been implemented provide important implications for future reconciliation interventions. While the impact estimates do not show any significant overall impact on either attitudes or trust, estimates for the three ethnicities separately found a notably different impact for different ethnicities, including different significance levels, direction, and magnitude of impact. This indicates that an approach to reconciliation interventions may need to be further researched and possibly customized to some extent for different ethnicities. Existing differences in impact for different ethnicities may be a reflection of PRO-Future’s vaguely formulated expected results and terminology regarding the acceptance of alternative narratives/acceptance of the existence of alternative narratives, which may have resonated differently with different ethnic groups or individuals, as well PRO-Future’s lack of work in establishing any shared narrative, which was missing from PRO-Future’s design and implementation. Moreover, differences in impact for different ethnicities possibly reflect different external environments and political rhetoric that is specific not only to each ethnicity in their relations with out-groups, but also for each of the six ethnicity combinations among the three main ethnic groups in BiH. PRO-Future implementers seem to have taken this into account to some extent during implementation, as they noted that their implementation approach had been increasingly adjusted based on each community’s/ethnicity’s specificities. However, it is unclear how systematic this was, and to what extent or how it was implemented — i.e. what criteria were used, were the same criteria used in each municipality, etc. Although the overall estimates of PRO-Future’s impact on the general population are inconclusive in the whole sample and have mixed results when analyzed at an ethnicity-combination level, for beneficiaries/participants PRO￾Future was beneficial and changed their attitudes and behavior. It positively influenced those who participated, and in many cases, resulted in their individual and personal transformation, which would not have been possible without PRO-Future. There was consensus among interviewees/FG participants that reconciliation initiatives are needed in BiH, given the interethnic tensions, and that it is a long-term process. This indicates that PRO-Future’s IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 53 final expected results were too ambitious to have been achieved by PRO-Future alone. This seeming dichotomy between mostly positive perceptions of PRO-Future’s effectiveness by direct participants and ambiguous impact on the broader population indicate that PRO-Future has had an important influence, but on a narrow groups and types of citizens. In other words, the extent to which PRO-Future reached wider groups of citizens and those less receptive to interethnic reconciliation and peace-building activities was perceived as not being large enough to influence citizens in a broader sense as envisaged by its design. PRO-Future beneficiaries/participants were typically activists/CSO members who have already been active in other reconciliation /peace activities. Our comparison of the results from household survey of general population and the results from the CRS’s survey of PRO-Future participants/beneficiaries also indicate that PRO-Future beneficiaries/participants differ from general population in several aspects important for reconciliation, including their opinion on role of talking about the past in reconciliation, prioritization of the overall challenges in BiH, participation in peacebuilding activities, and significance they assign to different types of peace building activities. In general, the greatest challenge in changing citizens’ behaviors and progressing in interethnic reconciliation in BiH is how to go beyond typical small groups of citizens that are already active in their community and attend multiple events and reach those who are less receptive. The difficulties in reaching the less receptive audience may be due to several reasons. These include: lack of openness to hearing any type of alternative narratives due to their disbelief in their existence; avoidance of speaking about the past due to painful traumatic experiences; fear of stirring up negative emotions; and fear of causing further divisions. Thus, although there is a clear need for continued reconciliation initiatives and a great opportunity to continue positive continued momentum resulting from CPT and PRO-Future, the design of any future initiative should directly address the desired reach of the intervention to the broader population that includes less receptive citizens/groups. One option is continuing to focus mostly on the minority of activists who are inclined to reconciliation efforts, with the hope that they will grow, and in the long term, be able to exert their influence on less receptive citizens. Another is actively targeting the less receptive broader population. Third, one could implement a careful balance of these two options. Whatever the design decision is, it needs to be clearly reflected in the expected results of future interventions, to be clearer and more realistic as compared to PRO-Future, and to further build on the reconciliation foundations laid out by PRO-Future., given unique complexities and political environment. This issue of scope of desired reach of intervention is also reflected in the analyses of n analyzing the effectiveness of PRO-Future activities/events, as it is important to distinguish between two types of desired influence: i) activities/events that generate the largest interest and can thus attract wider audience and ii) activities/events that have the strongest influence on personal transformation and long-term change of attitudes of participants. In terms of activities/events (within both small grants and general PRO-Future activities) that are generating the largest interest and thus being most effective in targeting the broader population, these included activities/events that focused not only on direct reconciliation/peace-building component but also on establishing an active inter￾ethic interaction based on common local/target group interests and/or including active participation by local key influencers. Examples given by interviewees include youth camps and other events in which topics of leadership, teamwork, and other skill-building or sports/recreation activities are combined with reconciliation components (e.g., street workout and reconciliation workshop); events that included economic development elements (e.g., women’s entrepreneurship and reconciliation workshops); events that included local environment element (e.g., municipality landscaping and peace-building activity); and arts/culture events that included direct peace building element (e.g. theater plays). Youth participants/beneficiaries, in particular, believe that the most effective 54 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY reconciliation activities for youth need to include activities of interest/use for young people specifically in addition to a peace/reconciliation component and be based on a participatory action approach with interaction among the participants. On the other hand, activities/events perceived as the most effective type of activity in influencing participants’ personal transformation and long-term change of inter-ethnic attitudes were those that included a more direct and comprehensive reconciliation/peace building - speaking-out events, youth peace camps with comprehensive reconciliation/peace building activities, and open-doors days. Nevertheless, the effectiveness of speaking-out events could have been increased further with the addition of some kind of follow-up and more interaction. PRO-Future’s implementation appears disconnected from its design, as neither implementers nor beneficiaries/participants are familiar with the designed high-level expected results and have a different understanding of its main terms (e.g., the meaning of reconciliation, alternative narratives, and the role of discussing the past in trust building), expected results, and expected causal chains laid out in Pro-Future design. The assumed causal chains among different elements of the PRO-Future development hypothesis (which assumes that confronting the past will improve inter-ethnic attitudes, which will in turn lead to increased inter-ethnic trust and subsequently to joint work to overcome societal divisions and make institutional changes, and finally lead to social and economic prosperity) is not directly grounded in psychological and peace studies theory and empirical research. Social and behavioral mechanisms and meanings behind peace and reconciliation are complex, and there is no consensus in research and empirical evidence on how reconciliation is defined (ranging from more comprehensive and ambitious perspectives that include forgiveness and target overall happiness, welfare, and productivity, to more narrow perspectives of reaching peaceful coexistence), what type of trust is needed for reconciliation, what approach to building trust is the most effective for reconciliation, and what the role of confronting the past is in trust building (USAID/BiH MEASURE-BiH, 2017). The complexity of the field becomes evident as soon as one tries to define the concept of reconciliation, as there is no general agreement on what reconciliation is or what would indicate that a society has achieved this goal. This evaluation clearly confirmed that both interviewees/FG participants and the broader population have a vague and varying understanding of the term. Moreover, although PRO-Future was perceived by interviewees/FG participants as having resulted in participants’ personal transformation and their acknowledgment of victims from other ethnic groups (most notably as a result of speaking-out events), it did not address the need to establish a shared truth (or elements thereof) in either its design or implementation. At the same time, eight out of ten citizens in the household survey believed that a shared truth about the past is a prerequisite for reconciliation, which was also brought up in interviews/FGs. These issues with the complexity and imprecision of PRO-Future’s development hypothesis/causal change and terminology were reflected in the previously mentioned different, unclear, and vague understanding of its expected results and causal mechanisms even among the implementers. This was, in turn, reflected in the similarly unclear and vague picture of PRO-Future’s expected results. Vague and inconclusive associations in terms of reconciliation terminology, the causal chain, and ways to advance reconciliation indicated a lack of specific and widely recognized objectives of PRO-Future. PRO-Future’s implementation is also to some extent disconnected from its design in terms of implementation mechanisms. PRO-Future’s 36 designed implementation mechanisms were too numerous, too ambitious, too specific in terms of actual task, and included too many organizations in charge of implementation. If implemented, this would have left little flexibility during implementation. In practice, several of these mechanisms have not been used either as planned or to the extent envisaged. PRO-Future implementers/partners seemed to have different understandings of the planned implementation mechanisms. Despite several designed implementation mechanisms IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 55 that ended up not being implemented, PRO-Future’s implemented mechanisms and activities are still deemed as too broad in their type and geographical coverage by the participants/beneficiaries and not deep enough (especially given the fact that PRO-Future in years 2–4 expanded their activities to 30 new municipalities in addition to the 30 originally planned). Although all recognize that it would have been beneficial to implement such interventions country-wide, given the resource constraints, it is clearly not feasible to implement an approach that is both broad and deep. In such circumstances the quality may be more important than quantity. This evaluation also concludes that activities implemented in the 30 originally targeted municipalities were perceived as generally useful and interesting, but in some cases insufficiently tailored to local specificities, with insufficient presence of key influencers from those localities. Another important conclusion related to PRO￾Future’s influence on citizens was that the small grant scheme was perceived as a great motivation for broader participation and the implemented grant initiatives were perceived as useful, albeit with some implementation issues. In terms of speaking-out events, another important conclusion is that despite fatigue and the potential need for additional psychological support for some speakers (war victims), which may have manifested from speaking frequently at such events, overall, war victims were very satisfied with the psychological support provided, appreciated the opportunity to work through their personal trauma, experienced relief, and affected personal change, which gave new meaning to their lives. Our final conclusions related to influence on citizens are based on our empirical analysis of household survey data. An analysis of socio-psychological processes and relationships underlying interethnic openness and attitudes showed that frequent interethnic exposure is an essential precondition for reconciliation. However, the quality of interethnic contacts matters more than the frequency. Ethnic groups tended to engage in the subtle dehumanization processes of other ethnic groups by attributing fewer secondary emotions to them. This may imply that interaction targeting more superordinate levels of categorization (such as humanity) to promote perceptions of similarity and consequently humanization of others may be needed, as well as activities directed toward developing empathy and perspective-taking. Ethnically homogenous localities are most in need for reconciliation interventions. The overall conclusion related to PRO-Future’s work with key influencers is that work with politicians/government officials, religious leaders, and the media, may not have been as extensive or as effective as envisaged in the PRO￾Future design (partially due to the difficult external environment in which PRO-Future operated). It did not result in overall improved cooperation among key influencers in BiH, but laid the necessary grounds for continued and intensified work with these important key reconciliation influencer groups in future reconciliation interventions. PRO-Future’s approach of focusing not only on the bottom-up targeting of citizens, but also on the top-down targeting of politicians as key reconciliation influencers, was appropriate. Likewise, PRO-Future’s recent shift in its work with politicians to focus not only on local politicians from the municipality/city level but also to include politicians/government officials from higher government levels proved useful. PRO-Future’s main implementation mechanism to work in targeted areas through municipal working groups was considered a good approach, but the levels of MWGs’ and MWG members’ engagement in its activities varied widely and, in some cases, were insufficient. In addition, the extent of MWG engagement and participation in PRO-Future activities was limited and not visible enough. PRO-Future’s Platform for Peace was recently endorsed by 60 mayors and by the Council of Ministers of BiH. It (and the comprehensive and complex process leading to it) was a major achievement of PRO-Future’s work with politicians as key reconciliation influencers. The Platform was the first document signed by politicians from a broad 56 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY spectrum of political parties and ethnic groups. However, while it was envisaged in the PRO-Future Award as a citizen-influenced roadmap that would outline policies, the endorsed Platform lacked concrete measures and policies. According to the Platform, these will be included in the Strategy of Sustainable Long-Term Peace for which it calls. There is an obvious need to continue work on this. Aside from politicians, who were the main focus of PRO-Future’s work with key influencers, the intervention also worked with religious leaders and the media. The Inter-Religious Council’s participation in PRO-Future was important and useful, although the extent of the engagement of local religious leaders was limited, and only a couple of small grants were implemented by religious organizations. PRO-Future’s work with media focused mostly on media coverage of PRO-Future events, which was quite wide, especially toward the end of the intervention, resulting in relatively high awareness in the broader population (with 20% of citizens in the end-line survey having heard of PRO-Future), and an immense potential for future work. Moreover, in some cases, journalists’ capacities were built in this subject matter. Although many mainstream media, which are most influential in shaping public opinion, were at some point covering some PRO-Future events, their engagement as active key influencers was limited, as was their coverage of PRO-Future events. Additionally, although they were initially envisaged as key influencer categories, not much work was done with business sector representatives or educators. In terms of PRO-Future management, overall, its staff was appreciated as professional, respectful, and passionate in their work on the intervention. Although there were several instances of poor coordination among the implementers and/or inadequate expertise/experience at the beginning of PRO-Future, the implementers subsequently improved their coordination and trained their staff. The Expert Group that was envisaged to provide technical advice and play advisory role for project strategic planning was formed in year 1, but their role and engagement in the strategic planning is not clear and appears limited. Finally. PRO-Future’s own implementation databases, though voluminous, were overly complex, unclear, and inconsistent in some aspects, as laid out in evaluation limitations within this report, which hampered monitoring. 4.3. RECOMMENDATIONS Based on this evaluation of PRO-Future, we offer the following recommendations for any potential future donor/government/CSO interventions in rebuilding trust and reconciliation in BiH (the first recommendation is paramount, whereas the remaining recommendations give specific suggestions for adjusting a future intervention in comparison to PRO-Future): 1. Continue financing reconciliation initiatives in BiH to build on the considerable achievements of PRO￾Future, taking into account that the process of changing citizens’ behaviors and building interethnic trust in BiH will be a long-term process, given its unique complexities. 2. In potential future interventions, carefully construct a causal change/development hypothesis to include clearly defined and realistic expected results, grounded more extensively in psychological theory and research on reconciliation and peace studies. Consider including efforts to progress toward acknowledging the need for and establishing some type of shared narrative. Ensure that all implementers and partners have the same understanding and clearly communicate expected results to all stakeholders and the broader population. Leave greater implementation flexibility and focus on fewer and deeper implementation mechanisms, monitoring them to ensure implementation. Ensure that implementers have clear roles, strong and clear coordination by the lead implementer, and that the main staff involved in implementation have enough experience and expertise. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 57 3. Building on grounds laid by PRO-Future, consider employing a more systematic way of targeting event participants. Consider aiming to reach less receptive groups in addition to activists (e.g., specifically targeting war veterans’ associations and political parties, working more with mainstream media, targeting formal education institutions, and mobilizing local expertise to reach out to the less-receptive fractions). 4. Consider further analyses and differentiated approaches for each ethnicity and ethnicity combination, especially given varying external environments and political rhetoric, as the analysis of attitudes/trust levels shows large differences. 5. Ensure that implementers have a clear and consistent monitoring and evaluation logical framework, and a verifiable database in order to avoid double counting and/or note it carefully, categorize types of main activities clearly and consistently, and record beneficiaries/participants consistently noting the intensity of assistance/participation. Ensure that all surveys and participants’ feedback forms in future interventions are prepared and piloted with input by psychometricians. Ensure that a strong expert advisory group (e.g., including psychological expertise, peace-building expertise, government process expertise) is created and works on a continuous basis to provide strategic input for all interventions. 6. Consider focusing on implementing interventions in fewer localities but with more activities, rather than spreading to more localities but holding only a few activities in each place. Continue and strengthen the tailoring of local activities to local specificities and needs with increased participation of local key influencers. At the same time, consider working more intensely with the mainstream media and higher￾level government counterparts to spread positive effects across the country. Also, consider employing a more sequential approach in working with localities, taking into account specificities/“maturity” (e.g., first establishing interethnic contacts, followed by joint work for common interest, inserting more explicit peace components, speaking-out event participation, follow-up activities, and working on establishing a shared local narrative). Finally, consider devoting additional attention to homogenous localities, where insufficient intergroup contact exists. For example, implement events with more easily predictable positive outcomes, such as youth camps, and economic collaboration between officials in such homogenous communities. 7. Continue using small grant mechanisms to motivate participation and attract wider participation. Apply clear and consistent grant criteria and co-financing options, and ensure that the selection committee has the necessary expertise to make its decisions. 8. Continue with intergroup contact-based activities and interventions, and consider making them obligatory for all events and small grants. Consider designing contact-based activities so they reflect more frequent and positive intergroup encounters. Consider focusing on implementing activities aimed at increasing and promoting perspective-taking and empathy for others and interventions framed at more superordinate levels of categorization (such as humanity) to promote perceptions of similarity and consequently humanization of others. 9. Continue and intensify working with key influencers, but define them more narrowly and consistently, focusing on both high-level and local politicians, religious leaders, and the mainstream media. In working with politicians, continue a two-pronged approach focusing both on municipality government institutions and higher-level government officials/institutions. However, consider ensuring more active engagement of these officials and more joint activities/cooperation. Also, consider working on designing specific and actionable tasks to be implemented by the government, based on the Platform and using clear MoUs with 58 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY well-articulated expected results and specific activities and obligations (including engagement and visibility by the mayors) for all participating government institutions (at the local and highest levels). Consider broadening MWG membership to require members aside from CSO and government representatives, providing financial assistance for MWG management (e.g., part-time coordinator) and tightening their coordination. In working with religious leaders, consider strengthening the IRC’s participation in future interventions, focusing on engaging local religious leaders from different faiths in local events and interfaith activities. In working with the media, refocus to devote funds to educating journalists and securing media space. Include the media as active participants, as opposed to just coverage providers. Specifically target the mainstream media, while continuing to work with online portals and social media. 10. Consider carefully whether/how to include work with formal education institutions, depending on the size of future reconciliation interventions and education-related interventions, If included in the next reconciliation intervention devote more attention to this component within AB, Expert Group, and actionable measures, which should come out of the Platform for Peace. 11. Coordinate closely with the USAID/BiH media intervention and with other USAID/BiH interventions dealing with reconciliation in formal education and youth ambassadors for reconciliation, as well as other donors’ interventions, most notably the UN’s expected follow up to the Dialogue for the Future. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 59 ANNEXES 60 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY ANNEX I: IMPLEMENTING PARTNER’S COMMENTS ON MEASURE-BIH PRO-FUTURE IMPACT AND PERFORMANCE EVALUATION August 9, 2017 Mirjana Valjevac Project Management Specialist USAID Sarajevo, Bosnia and Herzegovina Via email: mpopovic@usaid.gov Subject: CRS Comments on MEASURE PRO-Future Impact Evaluation Dear Ms. Valjevac, We would like to express our appreciation both to you and the MEASURE team for sharing a draft copy of the Impact and Performance Evaluation of USAID/BiH Trust, Understanding, and Responsibility for the Future Activity (PRO-Future). In response to the Impact and Performance Evaluation Report and your letter of July 18, 2017, please find below CRS’ comments and suggestions. Please note that CRS has focused its comments and suggestions on what we feel are the most substantial issues that materially affect the reader’s ability to fully understand the nature of CRS’ implementation of PRO Future, its design and expected results, and the context in which it takes place. While there were a number of other minor issues identified, we have chosen not to dwell on them in order to keep our comments within a manageable length. We are ready to follow up with you or the MEASURE team at any point to clarify our comments or to provide additional information, if you would like. GENERAL COMMENTS a) While CRS appreciates that the findings of the draft report highlight areas for improvement, we also feel that presenting an overview of any successes or promising practices from the projects would also be valuable for us and for other stakeholders. Presumably, the MEASURE identified such successes/practices and as such, CRS would ask that they consider including them in the findings. b) Given that PRO Future’s results are closely intertwined with the broader BiH and regional context of social cohesion and reconciliation, CRS recommends that the report provide a more meaningful overview of the key political and social events that occurred during 2013-17, and how each of these events may have influenced project results (both positively and negatively) as it relates to changing attitudes/behaviors at the grassroots level, and motivating key influencers to publicly support local reconciliation efforts. From CRS’ perspective, some of these key events include: IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 61 • Highly inflammatory pre-election rhetoric, the Celebration of the January 9, and ICTY court revision that have happened during project lifetime and effected negatively on citizen’s attitudes toward others. • Local election strongly affected communities and therefore almost half of communities engaged in PRO-Future project experienced different political options to come to the power. Such occurrence directly influenced project implementation MWG Model, and required additional efforts to ensure support of all municipalities included regarding political clime and changes. PRO-FUTURE PROJECT METHODOLOGY a) CRS would like for MEASURE to reconsider part of the Finding 5, stating that “PRO-Future’s causal chain of expected results is ambitious, complex, and somewhat imprecise/unclear in terms of connection to a psychological theory and research on reconciliation and peace studies”. The causal chain of expected results was tailored in accordance with USAID’s Theory of change and indicator development in conflict management and mitigation16, focused on the listed aspect in USAID RFA-168-13-000003 (pg. 10-16). Number of theory of change families were used as rationale for change and adjusted to different target groups, level of participant’s engagement and local community context. b) CRS would like for MEASURE to reconsider part of the Finding 5, stating “Both implementers and stakeholders are insufficiently familiar with the high- level expected results, and seem to have a different understanding of main terms (e.g., meaning of reconciliation, alternative narratives, and shared narrative), expected results, and causal chains.”. Project and scientific terminology on peacebuilding is more discourse among scholars and practitioners, PRO-Future was focused on promoting and fostering principles and values of understanding and social trust. Concepts of trust, understanding and responsibility used within PRO-Future project, are more in accordance with BiH context, individuals and social groups understanding of societal dynamic, rather than notions and newly reconciliation and peacebuilding terminology. c) We are also a little confused by the part of Finding 5 that states: “There is not enough focus on working toward establishing any shared narratives.” The first outcome of PRO-Future (as articulated in the original USAID RFA itself) is to promote the acceptance of alternative narratives (later adapted to accepting the existence of alternative narratives), not to create shared narratives. Thus, to have a finding that there wasn’t enough focus on creating shared narratives does seem to be an appropriate finding. After all, the PRO-Future theory of change is that: “If BiH citizens accept alternative and historical narratives, supported by key influencers, they will change their individual attitudes and behaviors towards one another”. d) CRS believes that MEASURE’s phrasing of this pg. 39 survey question should not have used the word confronting: “When asked specific question on the role of the confronting past in reconciliation, around 75% of citizens surveyed in the…”. Recognizing the importance of understanding ethno-political trauma and narratives and seeking to interrupt new cycles of violence, CRS worked with individuals and groups to come to terms with the pain of the past, rather than searching for truth or using term “confronting narratives”, understanding as Wessells (2007) that “without dealing with this pain, there would be no bridge to a nonviolent future, and emotional and social wounds would continue to fuel cycles of violence”17. Taking into consideration other’s organizations performances in field of Dealing with Past and Transitional Justice, CRS’s strategic approach foster inclusiveness and focuses more on trauma￾healing and social-relationships building processes of understanding the pain of the others. We are not 16 https://pdf.usaid.gov/pdf_docs/Pnads460.pdf 17 Wessells, Michael G. 2007. “Post-Conflict Healing and Reconstruction for Peace: The Power of Social Mobilization.” In Fear of Persecution: Global Human Rights, International Law and Human Well-being”, edited by Timothy J. White and Anthony J. Marsella. New York: Lexington, 257-278. 62 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY surprised with the results of the question, but question their relevance to PRO Future results and impact given the phrasing of the question. PRO-FUTURE PROJECT IMPLEMENTATION a) As indicated in the general comments section above, CRS believes that this evaluation report would be more valuable if it reflected more deeply on the extent to which the working group methodology, the key influencer engagement strategy and the media strategy were effective mechanisms to achieving the key project outcomes. If so, then such findings should be added to the report. b) CRS would like for MEASURE to reconsider the wording and intent of two sections of Finding 8: • PRO-Future’s implementation mechanisms While CRS agrees that several of our originally planned mechanisms were not implemented as envisaged, we feel that this demonstrates a positive reflection of our constant assessment and reassessment of which mechanisms are turning out to be the most effective, and using that ongoing analysis to make project decisions accordingly that will have the greatest impact. For example: i. Network for Building Peace (NBP) involvement in project was revised and adjusted in discussions with USAID, after the Peace Event 2014 and subsequent interaction with Board members convinced CRS that this network was not ready to make the strategic changes needed to promote national-level peacebuilding work in a systematic, effective manner. Nevertheless, CRS maintained a certain level of cooperation with NBP in accordance with approved project design and work plans. ii. While CRS originally thought that the Association of Parent Councils would be able to play a key role (as they successfully had done in the past) of bridging parents with local school directors to promote reconciliation efforts, this plan turned out to not be as effective as we had envisaged, due to deteriorated capacity of this local partner. Instead of continuing fruitless down this past, we elected to shift our partnership support to other local partners with more of a recent track record, namely EUROCLIO HIP and NDC Sarajevo, to ensure participation of schools, teachers, parents and councils in accordance to project outcomes and expected impact. EUROCLIO HIP developed curricula for teachers about joint history in local communities, and NDC Sarajevo conducted over 15 initial peacebuilding trainings in school for students and teachers both in schools in FBiH and RS. iii. As it relates to the Law on Victims of Torture, CRS proposed to continue our support for this initiative, which had started during implementation of the Choosing Peace Together project. However, with the legislation stalling on its way to the State Parliament and the Network of War Victim Association / Against Victims of Torture becoming less active, CRS decided that we needed to change course. Instead, PRO-Future continued to work with War Victim Associations, increasing the network size of Public Speakers and engaging new associations in peace processes. The Platform for Peace now can serve as a potential mechanism for civil society to relaunch advocacy for such victims’ rights legislation. • Widening geographical coverage: The way this finding is summarized doesn’t seem to accurately represent the data shared in report narrative. The finding makes it sound as project is generally spread too thin, but what respondents said specifically is that PRO-Future should have stayed and deepened in the original municipalities. Inclusion of additional 30 municipalities was planned during first year of the project (please see indicator No.11: Number of communities engaged in interfaith dialogue, healing and reconciliation initiatives). In sum, we don’t quite understand a finding that suggests that PRO Future had too many types of activities in too many parts of the country. By design, the USAID RFA mandates a country- IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 63 wide, comprehensive approach to promoting reconciliation across a wide range of key influencers and demographics. Sure, the project was ambitious, but for CRS, a more effective use of this evaluation report would be to identify how we can implement a country-wide, comprehensive approach more effectively (and on the same budget!), instead of suggesting that activities should be scaled down and scaled back. Feedback from government and civil society partners across the country confirm that there is a growing demand to participate in reconciliation activities, so CRS’ attitude is that we should be encouraging, not discouraging, such demand. c) CRS would like for MEASURE to reconsider Finding 9. During the project lifetime, PRO-Future highly engaged communities in project decision-making. Working Group Members (WGM) have been focal point in strategizing activities flow in paired or cluster communities, to ensure that activities are adjusted to local community context and specificities. On the other hand, understanding limitation of working group model, PRO-Future continuously worked with its members to better understand processes of reconciliation and to advocate both towards stakeholders and community membership to welcome project activities such as Speaking-Out Events. We agree that the overall project challenge was to include as many key influencers as possible in regular activities, and yet we consider that key influencers involvement was higher that noted within the Report. Many local government officials, mayors, local religious leaders, and other influential community members were present at PRO-Future events, when possible and feasible. In addition, a list of all activities involving them (PfP, conferences, etc.) should be composed and show level of their support. Another shortcoming, recognized by PRO-Future team is that, not all of them filled in sign-up sheets and thus were not counted as present. Therefore, we consider that such explanation should be added to the report. d) CRS would like for MEASURE to reconsider Finding 13. CRS consider wording “need to” to indicate as it important elements were missing in finding, especially when PRO-Future opened range of opportunities for youth and used participatory approach through all project implementation. CRS consider participatory action based approach was used with youth component, and would ask MEASURE to consider and include that in findings. For example, after each Peace Camp most proactive youth members, who stated willingness to be involved in projects were included both in PRO-Future activities and other CSOs peacebuilding activities (CIM, NDC, Youth Centre Gornji Vakuf Uskoplje and others). Youth from PRO-Future Peace Camp 2015, participated the Workshop on Council Method and lead and implemented by US Centre for Council (http://www.centerforcouncil.org/#!team/c1zh4), organized by CIM, along with many others courses. In addition to networking of young peace activist and strengthening their connection with local peace organizations, PRO-Future ensure space from their engagement in different roles and actions: some youth become members of MWG, they were engaged as speakers at FGT roundtables, presenter and support for small grant application process in local communities (MOZAIK). In addition their joint and independent initiative - Campaigns “Our blood is the same” was supported by PRO-Future project. e) CRS would like for MEASURE to reconsider Finding 19. PRO-Future worked with different range of key influencers in accordance to the project scope and expected outcome. Therefore, CRS would like such acknowledgment to be part of the findings in Report. Detail explanation is: • Political leaders: PRO-Future ensured support of 60 mayors to project, such support was provide by all three presidency members, presidents of both entities, Ministry of human rights embraced PfP, advised and supported process of its adoption on the highest state levels. Support for new reconciliation activity was provided by many politicians, parliamentarians and key influencers, both position and opposition. 64 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY • Religious representatives were involved through Public Forums, Open Doors Days, within 12 interreligious chapters active in PRO-Future communities. Churches and other religious institutions received small grants. They were also involved through the campaign and film, on￾line peace school and activities at the local level. Highest religious representatives Bishop Grigorije, the Deputy Grand Mufti and Cardinal supported PRO-Future. The SOE was organized on all three Theological faculties in BiH, and etc. • Media: Some of the most important media outlets which were directly included and approached during the PRO-Future implementation - especially in the previous two years - are as follows: Al Jazeera Balkans, N1, Bosnian-Herzegovinian RTV (BHRT), Federal TV, Radio and Television of Republika Srpska (RTRS), Croatian National TV (HRT), Deutsche Welle (DW), Radio Free Europe and Liberty TV, Klix, Bljesak, Federal News Agency (FENA), Radio Sarajevo, Anadolu Agency, Dnevni Avaz, Nezavisne Novine, Večernji List. All of the mentioned are either national or regional mainstream media outlets. The project worked with these media outlets either directly (via individual work, involving media in project activities, continuously holding informative meetings) or via Journalist for Peace component conducted by Medaicentar. This resulted with over 240 articles encouraging empathy or positive messages, or are empathy￾driven, reaching the project indicator. Aside from that, the total average of published/broadcasted media content, related to the project, is almost three a day, which is presented in Media tracking tables with reports. By combining the project-designed activity Journalists for Peace with media relations carefully led directly by the project team, the general PRO-Future media approach was structured, strategic, professional and long-term oriented. According to the current data, the annual targets for reaching people through media reports were, combined, 120,000. The relevant available official data shows that the project has managed to reach the total of 911,748 people, excluding documentary film "Zemlja za nas" (A Land for Us), which solely reached more than 1,5 million people, according to the official data received from the relevant TV outlets. • Education – PRO-Future worked with EUROCLIO HIP and NDC (as described previously), in accordance with donor requirements. Such extensive work on education need to include political key influencers, where PfP could be used as support in a process. PRO-FUTURE PROJECT PERFORMANCE a) The management efforts to manage and mitigate better performance haven’t been taken into account under Finding 6; therefore, CRS would like for second part of the finding to be modified by (1) specifying examples referred to poor coordination during early stage of the project and (2) include recognition of the many mitigation measures and improvement made during the life of the project. All management issues regarding PRO-Future were reported to USAID by Quarterly and Annual reports and such findings/challenges were timely addressed. Better partners’ cooperation was established during Year 2 through constant communication and quarterly coordination meetings for all partners (including IRC although it is technically a sub-contractor), and many other internal meetings such as the quarterly CARITAS coordination meeting. A number of peacebuilding education sessions were provided to CRS and Partners staff: learning sessions with Nell Bolton (at the time CRS Regional Technical Advisor for Peacebuilding), lecturers with Nerzuk Curak and Small School for Peacebuilding, and many other seminars and workshops about reconciliation and peacebuilding. Furthermore, number M&E, gender and accountability trainings were provided both for CRS and Partners staff. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 65 b) CRS would like for MEASURE to reconsider Limitation 8. The PRO-Future database was designed for internal use, and categorizations were defined up to different partner’s organizations and types of events and activities under specific outcome. The database allows for PRO-Future staff to easily use any selection on 2-3 columns and pull out any needed information to avoid double counting. When speaking about the participant data base, the PRO-Future team had extensive discussion which participant list should be used as most activities were public events, and taking contact data raised ethical issues. It was decided to use this at some events due to USAID reporting and evaluation requirements, as well as for internal M&E purposes. CRS expressed concerns that the proposed empirical data collection could harm the process of building relationships with local community members and beneficiaries, which is considered to be one of the most important process when working on peacebuilding. c) CRS would like for MEASURE to reconsider Finding 7. CRS understands and limitation of internal M&E reporting and database system, and some of the issues stated in section Report Section 3.2. Evaluation limitations were addressed and mitigated during project implementation. Further, CRS considers this finding not significantly relevant for quality/efficiency of the project implementation, and suggests that it should be removed from Report section. 4. Findings, Conclusions, and Recommendations. TECHNICAL CORRECTIONS CRS suggest following technical changes in document: a) Pg.21. Exhibit 2: Role of Implementers in PRO-Future (per June 2014 Award Modification) From the table, first column: It is stated CARITAS/Vive Zene. Explanation: Vive žene should be removed as its contribution was limited only to 1 training for war victims during Y1. During the rest of the project, other sub-contractors, with experienced psychologists, were selected to work with victims through trainings, as well as psychological supervision and support. b) Pg. 13. The MEASURE-BiH team conducted this evaluation between April and June 2017. The household survey data collection took place between April 25 and May 19. Key informant interviews (KIIs) and focus group (FG) discussions were conducted between May 15 and June 21. The evaluation team travelled to 18 municipalities across BiH (Istocno Novo Sarajevo, Sokolac, Prijedor, Sanski Most, Drvar, Glamoc, Gornji Vakuf/Uskoplje, Prozor￾Rama, Gračanica, Doboj, Petrovo, Teslić, Tešanj, Maglaj, Mostar, Nevesinje, Srebrenica i Bratunac), in addition to holding KIIs in Sarajevo and holding telephone interviews with KIs from Banja Luka and Tuzla. Explanation: In this research, the interviewed were members of the working group from Gradačac, not Gračanica. The report should be corrected to reflect this. c) The National Steering Committee (NSC) was transformed into the Advisory Board (AB) to be more influential and it included parliamentarians and other significant persons. The Expert Group grew from 3 persons at the very beginning of the project to current composition of 25 members. The NCS term is being used throughout the document, even though it was no longer active by Year 3. d) Pg. 40 and pg. 58. Particularly high satisfaction was expressed about youth camps implemented by CIM and Vive Zene/Caritas therapy work with the speaking-out event speakers. Explanation: Please correct statement: Three youth camps were delivered during the PRO-Future project cycle. First Peace Camp was organised in 2015, where Vahidin Omanovic and Mevludin Rahmanovic (CIM members) and Ivana Stankovic (hCa Banja Luka) were engaged as trainers to deliver trainings in accordance to CPT methodology. Peace Camp 2016, following the same principle engaged 66 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY six trainers for three groups of participants: (1) youth age 15-18: Snjezana Filipovic and Tanja Damjanovic, (2) youth age 18-30: Vahidin Osmanovic and Mevludin Rahmanovic, (3) teachers: Ljuljjeta Goranci Brkic and Nebojsa Savija Valha. The most active youth age 18+ were offered to participate in Advance II Peace |Camp and Peace Caravan delivered by CIM financially supported by PRO-Future, with aim to furtherly strengthen their capacities and peace advocacy. Advance II Peace Camp and Peace Caravan. There is possibility that MEASURE team just spoke with most advanced participants who were engaged in range PRO-Future activities and the latest at Advance II Peace Camp. Vive Zene, were engaged within project during the first year of its implementation, conducting one training for ex-war victims and speakers. Cooperation with Vive Zene was not continued due to high level on un-satisfaction with training results expressed by participants. For others trainings, independent psychologists and consultants were engaged. Their previous experience and excellent work was shown through CPT project implementation. e) Pg.59. National Steering Committee (NSC), should be changed to Advisory Board. We hope that you find these comments useful. Thank you for giving CRS an opportunity to provide input. Sincerely, Sanela Imamovic Head of Programs CRS BiH IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 67 ANNEX II: USAID DEMOCRACY OFFICE COMMENTS ON PRO-FUTURE IMPACT AND PERFORMANCE EVALUATION Elma Bukvic Jusic RE: DEMO comments on Pro-Future Evaluation 1 message Naida Čaršimamović Tue, Sep 19, 2017 at 3:23 PM To: Elma Bukvic Jusic Cc: Ye Zhang Dear Elma, We thank you again for all the comments you sent and forwarded to us on our draft PRO-Future impact and performance evaluation report. We greatly appreciate the time invested into reading our report by everyone who provided comments. Please find attached revised report that takes into account the comments we received from USAID/W and you, CRS, and USAID/BiH DEMO. We would like to use this opportunity to also provide our response to USAID/BiH DEMO’s five comments outlined in their email: 1. Our findings were informed by data sources carefully outlined in our Work Plan, which was approved by USAID/BiH. Namely, the findings related to the first two evaluation questions were informed by the household survey data, while the findings related to the third and fourth questions were informed by key informant interviews and focus group discussions, but also by PRO-Future documentation review, household survey data, and PRO- Future’s own survey of their beneficiaries/participants wherever applicable. We would also like to stress that the third and fourth evaluation questions as specified in both SOW and Work Plan clearly indicate that stakeholders’ perceptions will be used. Moreover, our selection of key informants and focus group participants was careful and random to the extent possible, given limitation of PRO-Future databases and the limitation that very few of the contacted persons were willing to talk to evaluators, which actually resulted in interviewed key informants and focus group discussants being positively biased as these were in almost all cases extremely involved in PRO-Future, i.e. they were more involved than a typical PRO-Future beneficiary/participant. In no case were findings formulated based on perception of a single person and only significant findings and those consistently mentioned by a significant number of key informants and focus group discussants are in our report. The number of interviewed key informants and focus group discussants who were participants/beneficiaries of PRO-Future was sufficient, based on research criteria for reaching saturation point. Perceptions of typical program participants/beneficiaries are a credible source for any evaluation and in our case they were triangulated with other sources whenever possible, as explained above and in the report. 68 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY In terms of USAID/BiH DEMO’s comment about contradicting findings, please note that we distinguish between two types of effectiveness of PRO-Future activities: i) activities/events that generate the largest interest and can thus attract a wider audience; and ii) activities/events that have the strongest influence on personal transformation and long-term change of participant attitudes. We additionally elaborate on the difference in this version of the report. 2. Annex IV provides data collection instruments and shows what questions were asked. When talking to the evaluation team, key informants and focus group participants conveyed their perception of whether the participation of different types of key influencers in the events they participated was in their opinion frequent and sufficient enough compared to PRO-Future design and compared to what they believe is the level that would have the maximum effectiveness on participants. Thus, this cannot be quantified in percentages. 3. The data collection instruments are provided in Annex IV and the survey question related to cooperation among key influencers came straight from PRO-Future’s own survey questionnaire, capturing overall perception of extent of overall change in key influencers’ interaction from very decreased to very increased. The questions used in interviews/focus groups are given in the section on Role of Key Influencers and Their Influence on Reconciliation in the guides shown in Annex IV. Key informants and focus group participants mostly talked about frequency/quantity of interaction among politicians in their local community, while when it comes to interaction of politicians at higher levels (including political party leaders) they talked about quality (as they do interact) in terms of whether their interactions are encouraging or discouraging inter-ethnic reconciliation. In terms of percentages, as written in Finding 19, only 16% of citizens in the household survey noted that key influencers’ interaction increased. Even among the PRO-Future participants/beneficiaries, most do not believe that interaction increased - in CRS’s own survey of beneficiaries/participants only 40% of respondents stated that cooperation had increased. However, as the percentage believing that interaction increased is still higher among participants/beneficiaries, this possibly indicates that PRO-Future events with key influencers have not been visible enough. In key informant interviews/focus group discussions, there was not much difference by type of interviewee/focus group participants and most noted that interaction has not improved. 4. Thank you for your clarification on Expert Group, we have adjusted the report accordingly. 5. We have expanded the report to include CRS’s explanation on their work with media. We have also expanded the explanation on the difference between design and actual work of the Online Peace School (as well as for other implementation mechanisms mention in that finding). In terms of evaluation lead’s comment in the presentation that the results were much better than expected, this referred to the performance evaluation portion of the evaluation. Namely, as USAID/BiH may remember from the Work Plan stage, the whole work on key informant interviews and focus group discussions was timed so that it starts only after household data collection is completed (this is why we had to postpone the key informant interviews and focus group discussions compared to what we planned originally when CRS’s procurement of the endline household survey data collection was cancelled). This was done so that the evaluation team can have impact estimates before the start of key informant interviews and focus group discussions, so that we can compare those results to interviewees/focus group participants’ perceptions and their own experiences (which we did). Thus, what we stated in the presentation is what we also wrote in our report – given that impact estimates show no impact overall, perceptions of the participants/beneficiaries in key informant interviews and focus group discussions are surprisingly positive. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 69 USAID/W’s comments and your comments are incorporated in the revised report, including expanded section on econometrics, further improvement of the connection between findings and conclusions, and moving parts of the background chapter to Annex. Additionally, the section on conclusion was further strengthened. Please note that the USAID/DEMO and CRS’s comments that were not incorporated in the report can be grouped into the following main categories: 1. Comments that refer to design, as we evaluate both design and implementation 2. Comments that are out of scope of this evaluation (e.g. detailed assessment of external environment or how to implement a country-wide approach or how successful or unsuccessful was PRO-Future in engaging less receptive citizens in %) 3. Comments related to perceptions not being a relevant source of information (as explained above in this email) 4. Comments given in the early sections of our report (e.g. Executive Summary) on issues that we later addressed in subsequent section of our report, including questions related to survey questionnaire or key informant interview/focus group guides which are given in the Appendix 5. Unclear comments and comments for which we do not see the basis (e.g. that MEASURE failed to ensure that questions are tailored to fit individual stakeholders’ relationship with PRO-Future – this is not the case and in any case could not be deducted without reviewing our interview transcripts). Please also note that they were some opposing comments among those we received, and in such cases we used our best judgment for the final report attached. Let us know if you have any questions and if something should be included in the report as statements of difference. Best regards, Naida Čaršimamović Vukotić Chief of Party USAID/BiH Monitoring and Evaluation Support Activity (MEASURE-BiH) Fra Anđela Zvizdovića 1 UNITIC Tower B, Floor 21 71000 Sarajevo Bosnia and Herzegovina Office Telephone: 387 33 941 674 Mobile: 387 61 558 914 www.measurebih.com 70 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY From: Elma Bukvic Jusic [mailto:ebukvic@usaid.gov] Sent: Friday, August 11, 2017 2:17 PM To: Naida Čaršimamović Subject: Fwd: DEMO comments on Pro-Future Evaluation Sent from my iPhone Begin forwarded message: From: Shanley Pinchotti To: Elma Bukvic Jusic Cc: "Kent (CHISINAU/AID) Larson" Subject: DEMO comments on Pro-Future Evaluation Dear Elma, DEMO appreciates MEASURE's work to complete the impact and performance evaluation of Pro Future. The subject itself is very complex, and the research design was as well given the need to retroactively create baselines, control groups, as well as talk with participants about difficult subjects. We commend the team on the effort that it took to carry out this analysis and draft the report. It was useful to see the comparisons of target and control municipalities, discuss differences among ethnic groups and approaches to reconciliation, as well as hear more feedback from key interviewees. Our five overarching comments are below. The attached word document has our additional clarifying comments, but those are numerous and not all require response, the most important are highlighted below: 1. Findings and recommendations: The evaluation team was tasked to conduct both a performance and impact evaluation of PRO-Future. The findings and recommendations were supposed to be informed by many data sets - the citizen survey, individual/group interviews, and project progress reports - and the draft report suggests that they were. However, it seems that a majority of the evaluation findings are based exclusively on individual perceptions of people interviewed and very dependent on the level of their knowledge and understanding of the project and all the variety of its interventions. Because of that, some findings are incorrect as they are not based on facts and statistics. Other times, it seems that the findings contradict one another and those discrepancies are not discussed in the findings. For example, the findings suggest that it is most effective to bring people together on topics of common interest or areas where they can work together that are not related to peace building, but then through such activities, peace building topics should also be addressed. However, the most effective activities highlighted through interviewees were speaking out events, open door days, and youth peacebuilding camps. These three activities are the most explicit in terms of talking about reconciliation and do not bring people together based on other topics of mutual interest. 2. Engagement of key influencers: When explaining the level of impact the project had, or the level of IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 71 engagement of key influencers such as politicians, religious leaders and mainstream media, the draft report employs the terms “insufficient”, “not enough”, etc. throughout without adequately explaining what that means. Given the heavy reliance on interviewees' perceptions, were there follow-on questions in order to either validate the statement or dismiss it. Also, the evaluation report should give some definitions or at least elaborate how much is enough, how much is sufficient. Understanding the burden of responsibility these stakeholders have for the current level of trust between people and in institutions, or the absence of it in BiH, citizens and key informants might expect the level of their engagement in reconciliation work to be much higher than can realistically be expected. Also, we would be interested, in the context of engaging key stakeholders like politicians or religious officials in reconciliation work, or getting traditional media to report on project activities, or citizens on the community level independently taking actions to better understand where the “other side” is coming from; whether there was consideration of the change between how it was before and how it is now as a result of the project. 3. Improvement of cooperation among key influencers: According to the evaluation most (people interviewed) do not believe that cooperation among key influencers improved. Such statement could mean nothing and at the same time many things depending on the follow-on question was one posed to people interviewed. Were they referring to the quality of cooperation or the quantity. Was this perception dominant among citizens or key influencers interviewed, and in which percentage? Was there a difference in perceptions with these two groups? Was there a follow-on question as to why they didn’t think there was an improvement? More clarification would help. 4. Expert group: The evaluation states that the Expert Group was formed late (3rd project year) and that it was composed mostly of implementers/partners and experts, and that the extent of their technical advice and guidance on design and implementation was limited. This is incorrect. The expert group was formed in year one with professor Nerzuk Curak from Sarajevo, professor Miodrag Zivanovic from Banja Luka and Mr. Vlatko Kraljevic from Mostar. It was significantly expanded in years 2 and 3 to include 25 experts. The expert group was not supposed to be engaged in the design of the project and it is not mentioned in the program description. Other than Vehid Sehic who manages the Forum of Tuzla citizens association, one of four key partners, all other members of the expert group are independent experts who were approached and engaged in different phases of the project. All important, politically sensitive actions and activities are being communicated with the expert group. Please revise. 5. Presentation: DEMO would like to note that during the presentation, a number of statements were made that were not immediately visible in the report. The presentation mentioned that work with the media, more precisely with TV stations, was "rather poor with the exception of the documentary 'Zemlja za Nas' or 2-3 times when the Chief of Party was interviewed on a TV show." The team suggested that the Online Peace School did not happen and that many things from the workplan were not accomplished. Please provide evidence for these in the report. Additionally, the statement was made that the evaluation results were much better than they had expected and we wonder why there was a low expectation of the project from the evaluation team at the outset. PRO-Future Evaluation Report MEASURE-BiH 09192017.pdf 6051K 72 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY ANNEX III: EVALUATION STATEMENT OF WORK USAID Democracy and Governance Office STATEMENT OF WORK Impact and Performance Evaluation OF Trust, Understanding, Responsibility for the Future (PRO-Future) Activity PURPOSEOF THE EVALUATION The purpose of this evaluation of the PRO-Future Activity is to: i) assess progress toward designed expected results and Activity’s impact on those results, ii) describe how Activity is implemented and what were the key challenges and lessons learnt from implementation, iii) provide recommendations for any possible future donor/government interventions in rebuilding trust and reconciliation in BiH PROGRAM INFORMATION Activity/Project Name Trust, Understanding, Responsibility for the Future (PRO-Future) Contractor Catholic Relief Services (CRS) Cooperative Agreement/Contrac t # AID-168-A-13-00007 Total Estimated Cost (TEC) $4,100,000 + $670,000 cost share Life of Project/Activity September 17, 2013 – September 16, 2017 Active Geographic Regions 30 BiH municipalities where the Activity has been implemented over the entire life of Activity (additional 30 municipalities/cities were added at later stages of the Activity but will not be subject of this evaluation) Mission Development Objective (DO) IR 1.2 Increased citizen participation in governance BACKGROUND USAID’s PRO-Future Activity is implemented by Catholic Relief Services (CRS), in partnership with four core local partners: Mozaik Foundation, Caritas of the Bishops’ Conference of BiH, Forum of Tuzla Citizens and Helsinki Citizens Assembly. The Activity is also partnering with the Inter-Religious Council of Bosnia and Herzegovina, Mediacentar Foundation, Vive Zene, Youth Initiative for Human Rights and others. This Activity is designed to build trust and confidence among citizens of all ethnic backgrounds in 30 targeted municipalities, help them accept alternative political and historical IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 73 narratives, and allow for key influencers in these municipalities to support local reconciliation initiatives. USAID/BIH designed PRO-Future taking into account the following assumptions of the following significant drivers of conflict in BiH: 1. Nationalist discourse and narratives of victimization, justification, and rationalization that dehumanize and demonize ‘the others’. 2. The structures of governance established with the DPA institutionalized divisions along ethnic lines, which allowed for discrimination and violation of human rights. 3. Unresolved war-related psycho-trauma affects not only the primary victims of the war, but also the youth born after the war (secondary trauma). The PRO-Future Activity is intended to create neutral space for discussion between groups with opposing viewpoints and thus help them identify points of agreement, which in turn is expected to foster cooperation and collaboration, and, ultimately, generate demands for change from government institutions and leaders. The Activity was designed based on the following development hypothesis: If BiH citizens from all parts of the country and from all ethnic backgrounds change their individual attitudes and behavior toward one another, then social trust within communities will increase. This trust will enable citizens to work together to overcome societal divisions and constructively advocate for institutional changes. Changed attitudes and behaviors, and thus increased trust, are pre-conditions for reaching USAID’s long-term goal of institutional changes, which will in turn lead to improved social well-being and economic prosperity for BiH citizens of all ethnicities. Theories of Change: The following tested theories of change were consulted to produce a development hypothesis and indicators against the mission results framework and to establish criteria for impact and outcomes: Inside-Out Peace-building: Consciousness Shift: If key actors and/or enough individuals undergo constructive shifts in their consciousness, such as developing more universal identities or awareness of identity formation and if they resist the mobilization of conflicting identities, then their commitment and capacity for the peaceful resolution of conflicts will increase and influence social change in that area. Coming to Terms with the Past: Reconciliation: If individuals, elites, and groups from across the conflict divide are engaged in a process of reconciliation, then healthier relationships and shared futures can be generated to increase social resilience to mobilized violence. 74 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Attitudes toward Peace: Culture of Peace: If war-torn societies focus cultural, media, and education resources on changing people's attitudes and social norms to support the peaceful resolution of conflicts, then a culture of peace will emerge that promotes coexistence and resists mobilization to adopt violence. Healthy Relationships: Building Bridges: If key actors from belligerent groups are given the opportunity to interact, then they will better understand and appreciate one another, be better able to work with one another, and prefer to resolve conflicts peacefully. Behavior changes anticipated during the Activity include multi-ethnic civil society initiatives to address common concerns, creation of and attendance at social or cultural events (athletic tournaments, film or arts festivals, etc.), and reduced rhetoric about separation and victimization in targeted areas. As individuals and communities are not solely responsible for changing attitudes and accepting alternative narratives. the second key element of increasing social trust in BiH is that key influencers support local reconciliation initiatives. The Activity was designed to encourage key influencers to demonstrate this support by, for example, leading or engaging in inter-religious events, promoting reconciliation in speeches and through the media, and providing sponsorship or co-funding of initiatives that work across ethnic or party lines to address local concerns. Feeling encouraged by positive messages from their leaders, citizens will find value in participating in civic initiatives across ethnic lines to address/resolve local problems. The long-term goal to which this Activity is expected to contribute is to produce institutional changes which will improve the social well-being and economic prosperity of BiH citizens. In order to meet this goal, citizens must be encouraged to work together to advocate for change. The second level objective to which this Activity contributes is that BiH citizens, encouraged by key influencers, will advocate for institutional changes in society. To do so, PRO-Future was designed to rebuilt trust and confidence among citizens of BiH need to be rebuilt. Only then will citizens work together to achieve the common good. Mutual trust and confidence can be realized if citizens are supported to accept alternative political and historical narratives, work together to overcome divisions, build bridges, and create sustainable changes in society. This Activity is based on the premise that effective peace-building efforts start by opening dialogue, understanding people with opposing points of view, changing attitudes, and building skills and relationships at the individual/personal level. Main expected results of PRO-Future  Outcome 1: Citizens in 30 targeted municipalities accept alternative political and historical narratives.  Outcome 2: Key influencers in 30 targeted municipalities support local reconciliation initiatives. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 75 Primary tasks:  Establishing advisory bodies and engaging individuals who enjoy respect and credibility in the community (National Steering Committee, Municipal Working Groups)  Technical and advisory and financial support to formal and informal groups o Capacity building for Members of War Victims’ Associations o Online Peace School o Assistance to the Network Against Torture  Speaking Out events  Annual Competitions for Youth Sub-Network for Building Peace, MESS, SFF, Art Colony – Exhibition, Youth Initiative Funds, Activities in schools in targeted municipalities through Parent School Councils  Women’s Peace Initiatives (Alternative Women’s Government)  Association of Parent Student Councils  Production of positive media stories PRO-Future Newsletter, Educating for peace journalism; announcement and selection of Srdjan Aleksic Award  Municipal leadership roundtables  Inter Religious Council activities  Platform for Building Peace  Social media activities The illustrative hierarchy of the results is as follows: Narrative Summary Development Hypothesis If Purpose, then Goal Goal (The highest-order objective to which this Activity contributes): BiH Citizens encouraged by key influencers advocate for institutional changes in society Activity Purpose (Central result to be produced by and attributable to the project): Trust and confidence among citizens of all ethnic backgrounds rebuilt Manageable Interest If Outcomes, then Purpose Outcomes/Outputs: BiH Citizens accept alternative political and historical narratives. Community level influencers support local reconciliation initiatives to provide a basis for joint constructive engagement. PRO-Future Activity is implemented in following 30 BiH municipalities with whom we signed an MoU: Prijedor, Sanski Most, Goražde, Novo Goražde, Maglaj, Doboj, Modriča, Grač, Srebrenica, Bratunac, Sokolac, Tešanj, Teslić, Foča, Foča FBiH, Petrovo, Gradačac, Ugljevik, Teočak, Mostar, Nevesinje, Kladanj, Vlasenica, Drvar, Glamoč, Novi Grad, Bosanka Krupa, Istočno Novo Sarajevo, Gornji Vakuf- 76 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Uskoplje, Prozor, Rama. Activities are also implemented in other cities in BiH: Sarajevo, Banja Luka and Tuzla. Evaluators will have an access to PRO-Future’s Award documentation, work plans, quarterly reports, annual reports, all M&E documentation, and other relevant documentation. EVALUATION QUESTIONS The evaluation will be used by USAID/BiH to assess the impacts of PRO-Future interventions, help USAID/BiH staff achieve a better understanding of Activity implementation, lessons learned, and best practices; and provide USAID/BiH staff with evidence and information that could inform future funding decisions and program designs in rebuilding trust and reconciliation. The Evaluation Report will be also shared with relevant organizations and stakeholders who participated in the project activities. Evaluation should be also used as a source of learning for subsequent project interventions. Evaluation Objectives The objectives of this specific evaluation are the following: 1. Conduct project evaluation based on interviews with project’s beneficiaries, project documentary review and field visits to partner organizations; 2. Identify and examine the project impact among project stakeholders 3. Identify lessons learned and best practices to be used in future phases of this project or other projects’ of similar kind in a similar contexts. 4. Based on the input from stakeholders, provide recommendations for the possible next project phase or project intervention that will include information on: project relevance; project effectives; recommendation for project contribution the upcoming year. Evaluation Questions 1. What impacts did PRO-Future Activity have on interethnic attitudes (general opinion, empathy, and tolerance) in the participating municipalities/cities? To what extent did the impacts vary across ethnicities? 2. What impacts did PRO-Future Activity have on interethnic trust in the participating municipalities/cities? To what extent did the impacts vary across ethnicities? 3. How effective has PRO-Future been in changing participating citizens’ behaviors that contribute to increased stability and consensus, as demonstrated by stakeholders’ perceptions of the Activity and whether it has made changes in terms of increased capacity and demand for improved relationships? How have PRO-Future interventions targeting change in citizens’ behaviors been implemented, how relevant were they to social, IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 77 political and cultural context, needs and capacities of target communities, and what were the main challenges and lessons learned from stakeholders’ perspective for future interventions in rebuilding trust and reconciliation? 4. How effective has PRO-Future been in getting key influences in targeted municipalities/cities to support local reconciliation initiatives, as demonstrated by stakeholders’ perceptions of the Activity and whether it has made changes in terms of key influencers’ openness to dialogue and acting for reconciliation? How have PRO￾Future interventions targeting key influencers been implemented and what were the main challenges and lessons learned from stakeholders’ perspective for future interventions in rebuilding trust and reconciliation? EVALUATION DESIGN AND METHODOLOGY Questions Data Sources Research Design 1. What impacts did PRO-Future Activity have on interethnic attitudes (general opinion, empathy, and tolerance) in the participating municipalities/cities? To what extent did the impacts vary across ethnicities? Surveys (Base-line data: SCORE and PRO-Future Baseline Survey; End-line data: PRO-Future End-line Survey) Impact evaluation: Quasi-experimental design with difference-in-differences method, implemented through multivariate regression models 2. What impacts did PRO-Future Activity have on interethnic trust in the participating municipalities/cities? To what extent did the impacts vary across ethnicities? Surveys (Base-line data: SCORE and PRO-Future Baseline Survey; End-line data: PRO-Future End-line Survey) Impact evaluation: Quasi-experimental design with difference-in-differences method, implemented through multivariate regression models 78 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY 3. How effective has PRO-Future been in changing participating citizens’ behaviors that contribute to increased stability and consensus, as demonstrated by stakeholders’ perceptions of the Activity and whether it has made changes in terms of increased capacity and demand for improved relationships? How have PRO-Future interventions targeting change in citizens’ behaviors been implemented, how relevant were they to social, political and cultural context, needs and capacities of target communities, and what were the main challenges and lessons learned from stakeholders’ perspective for future interventions in rebuilding trust and reconciliation? PRO-Future Activity implementation documentation and database (including M&E documentation), Key Informant Interviews, and Focus Groups Performance evaluation: Mixed methods, with a focus on qualitative data 4. How effective has PRO-Future been in getting key influences in targeted municipalities/cities to support local reconciliation initiatives, as demonstrated by stakeholders’ perceptions of the Activity and whether it has made changes in terms of key influencers’ openness to dialogue and acting for reconciliation? How have PRO￾Future interventions targeting key influencers been implemented and what were the main challenges and lessons learned from stakeholders’ perspective for future interventions in rebuilding trust and reconciliation? PRO-Future Activity implementation documentation and database (including M&E documentation), Key Informant Interviews, and Focus Groups Performance evaluation: Mixed methods, with a focus on qualitative data IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 79 DELIVERABLES, SCHEDULE, AND REPORTING REQUIREMENTS 1. Evaluation Design and Work plan: A draft work plan and evaluation design document for the evaluation shall be submitted to USAID/BiH two weeks after SOW approval. The evaluation design will include: (1) a detailed evaluation design matrix (including the key questions, methods, and data sources used to address each question and the data analysis plan for each question); (2) draft questionnaires and other data collection instruments or their main features; (3) the list of potential interviewees and sites to be visited; (4) known limitations to the evaluation design; and (5) a dissemination plan. The workplan will include: (1) the anticipated schedule and logistical arrangements; and (2) a list of the members of the evaluation team, delineated by roles and responsibilities. USAID offices and relevant stakeholders are asked to take up to one week to review and consolidate comments through the AOR/COR. Once the evaluation team receives the consolidated comments on the initial evaluation design and workplan, they are expected to return with a revised evaluation design and work plan within 2 days. 2. Data collection: Key informant interviews will commence one week after receiving database from the PRO-Future end-line survey. 3. In-briefing: Prior to conducting key informant interviews with Activity beneficiaries, the evaluation team will have an in-briefing with the USAID/BiH Democracy Office to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and workplan. 4. Final Exit Briefing: After conducting key informant interviews with Activity beneficiaries, the evaluation team will hold a final briefing prior to report drafting for final clarifications needed from the Mission and to discuss the status of data collection. 5. Evaluation Presentation: The evaluation team is expected to hold a final presentation to USAID/BiH to discuss the summary of findings and recommendations to USAID. 6. Draft Evaluation Report: The draft evaluation report will be submitted no later than 7 weeks after the start of key informant interviews. The report shall be consistent with the USAID Evaluation Report Requirements provided in ADS REFERENCE 201MAH (USAID Evaluation Report Requirements https://www.usaid.gov/ads/policy/200/201mah) and take into account criteria to ensure the quality of the evaluation report specified in ADS REFERENCE 201MAA (https://www.usaid.gov/ads/policy/200/201maa). Once the initial draft evaluation report is submitted, USAID/BiH will have 10 calendar days in which to review and comment on the initial draft, and submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a revised final draft report in 10 calendar days hence, and again the 80 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID/BiH will review and send comments on this final draft report within 5 calendar days of its submission. 7. Final Evaluation Report: The evaluation team will be asked to take no more than 10 calendar days to respond/incorporate the final comments from USAID/BiH. The evaluation team leader will then submit the final report. All data and records will be submitted in full and should be in electronic form in easily readable format, organized and documented for use by those not fully familiar with the Activity or evaluation, and owned by USAID. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 81 ANNEX IV: PRO-FUTURE DOCUMENTS/DATABASES REVIEWED 1. Activity Award (AID-168-A-13-00007 AWARD) 2. PRO-Future Criteria for Selection of Target Municipalities (Kriteriji Pro-Buducnost 19-02-14) 3. Municipal Selection Process Explanation 4. Examples of PRO-Future MoU (03 BHS PRO-Future MoU Drvar - Glamoc april 2014, 04 PRO￾Future MoU Bratunac - Srebrenica April 2014) 5. List Municipalities in PRO F project per year when joined 6. PRO-Future Application (Aplikacija-za-PRO-Buducnost I ciklus) 7. PRO-Future Quarterly report for Y1Q1 (Y1Q1 PRO Future Narrative Report Q1, January 2014) 8. PRO-Future Quarterly report for Y1Q2 (Y1Q2 PRO Future Second Narrative Report, April 2014) 9. PRO-Future Quarterly report for Y1Q3 (Y1Q3 PRO Future Q3 Narrative Report, July 2014) 10. PRO-Future Annual report for Y1 (Y1Q4 PRO Future - Yr1 Annual Report, October 2014) 11. PRO-Future Quarterly report for Y2Q1 (Y2Q1 PRO Future - Yr2 Q1 Report A) 12. PRO-Future Quarterly report for Y2Q2 (Y2Q2 PRO-Future Report) 13. PRO-Future Quarterly report for Y2Q3 (Y2Q3 Summary Quarterly Report PRO-Future) 14. PRO-Future Annual report for Y2 (Y2Q4 Summary Quarterly Report Pro-Future 10.15.2015) 15. PRO-Future Quarterly report for Y3Q1 (Y3Q1 Summary Quarterly Report Pro-Future) 16. PRO-Future Quarterly report for Y3Q2 (Y3Q2 Summary Quarterly Report Pro-Future final) 17. PRO-Future Quarterly report for Y3Q3 (Y3Q3 Summary Quarterly Report Pro-Future) 18. PRO-Future Annual report for Y3 (Y3Q4 Summary Quarterly Report Pro-Future) 19. PRO-Future Quarterly report for Y4Q1 (Y4Q1 Summary Quarterly Report Pro-Future 12-01-17) 20. PRO-Future Quarterly report for Y4Q2 (Y4Q2 Summary Quarterly Report Pro-Future 17-04-17) 21. List of PRO-Future Public Speakers and War Vctims Associations (05 Lista kontakata javnih govornika i udruženja žrtava rata CARITAS BIH) 22. List of PRO-Future Experts (09 Kontakti expertna grupa - okrugli stolovi) 23. List of PRO-Future Municipal Working Groups Members (06 List of Working group members) 24. List of PRO-Future Steering Committe members (07 Steering committee members) 25. List of Mayors in PRO-Future Target Municipalities (08 Kontakt načelnici) 26. List of PRO-Future IRC Speakers (10 Lista govornika MRV) 27. PRO-Future Grants Manual (01 PRO-Future Grants Manual approved revised version FINAL) 28. PRO-Future Small Grants Recipients Database, Solidarity Fund (11 Kontakti mali grantovi - Fond solidarnosti) 29. PRO-Future Small Grants Recipients Database, Formal groups (12 Kontakti mali grantovi - Formalne grupe) 30. PRO-Future Small Grants Recipients Database, PRO-Active Fund (13 Kontakti mali grantovi - PRO￾Activ) 31. PRO-Future Database of Participants in Speaking Out Events (13 Participants of speaking out events) 32. PRO-Future Database of Participants in Youth Events (14 Youth targeting events - participants list) 33. PRO-Future Small Grants Database (Q15-MOZAIK-Small-Grants overview - May 2017) 34. List of PRO-Future Peace Camps (00 List of Peace camps - PRO-Future) 35. List of PRO-Future Round Tables (List of round tables in PRO-Future project) 36. List of PRO-Future Speaking Out Events (01 Speaking Out Events) 37. Platform for Peace, in English (Platform for Peace - General Version) 38. Platform for Peace, in local language (BOS-CRS-PROB-Platforma) 39. List of Disbursed Ad-hoc Grants 82 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY 40. List of Mayors and Organization that signed the PRO-Future Plaform for Peace (Lista načelnika i organizacija koje su potpisale Platformu za mir) 41. PRO-Future types of activities (PRO_F. Summary of Activities) 42. PRO-Future Tracking Tables (PRO-Future IPTT Y4Q3) 43. Invitation for Participation in the PRO-Future Activity (01 Poziv-za-ucesce-u-projektu-PRO￾Buducnost) 44. Letter of Interest for Participation in the PRO-Future Activity (02 Pismo-interesa-za-ucesce-u￾projektu-PRO-Buducnost) 45. PRO-Future Application (03 Aplikacija-za-PRO-Buducnost) IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 83 ANNEX V: ADDITIONAL INFORMATION ON PRO-FUTURE Exhibit 10: Mechanisms Planned in PRO-Future Design (per June 2014 Award Modification) Outcome 1: Citizens in 30 targeted municipalities accept alternative political and historical narratives Output 1.1: Members of war victims’ associations have the skills and confidence to hold speaking-out events in their communities 1.1. 1. Advanced training for members of war victims’ associations (Vive Zene to deliver training to 25 members who participated in CPT to speak at speaking-out events; capacity building for proposal development and $18K earmarked for small grants) 1.1.2. Speaking-out events (100 events in four years; core group of 40 speakers from CPT; audience to include high-school students and their teachers, religious seminarians, leadership and laypeople, academics and university student populations, BiH media, and other members of victims’ groups; participation of municipal officials) Output 1.2: Citizens of BiH from different ethnic communities, particularly youth, women, and minority returnees, have the skills and understanding to think critically about the narratives of their and other ethnic communities 1.2.1. Municipal youth events (at least 30 events per year in municipalities, such as documentary film screenings and discussions, after-school discussion groups, sporting events, and other activities to explore and challenge their assumptions about their peers from other ethnic groups; identify interested youth by engaging student councils at secondary schools, or through other formal or informal local youth groups) 1.2.2. Municipal youth training (deliver critical thinking, reconciliation, and peace-building training to youth in each of the target municipalities at least once per quarter) 1.2.3. Online peace schools (each year offer moderated reading/teaching online – on the NBP website – about specific topics related to peace building, forgiveness, and advocacy; 6-week courses on topics selected by network members and website visitors through an online voting system; a “learning blog” on NBP) 1.2.4. Annual competitions (using the NBP website as a platform, PRO-Future will issue an annual challenge to youth to propose creative solutions to peace and reconciliation challenges. Responses can take a variety of forms including videos, art, writing, music, or other creative expressions; key influencers to attend the events) 1.2.5. Youth Sub-Network for Building Peace (within NBP, to foster collaboration among youth and their engagement with other civil society sectors; securing youth input into the Platform for Building Peace) 1.2.6. June 2014 Peace Event (invite 250 Pro-Future project participants to attend these events to lead panel discussions, present essays or artwork related to their interpretation of peace or reconciliation, or serve as hosts to their peers from elsewhere in Europe; key influencers to attend and participate) 1.2.7. City Hall Opening (PRO-Future-coordinated art activities) 1.2.8. Making Peace, Sarajevo 2014 (support the International Peace Bureau with an outdoor photo exhibit, Making Peace) 1.2.9. Youth Peace and Advocacy Camp (in 2015 and 2016, identify the 30 most active youth to attend a week-long camp to further build their leadership, reconciliation, and advocacy skills; key influencers to participate) 1.2.10. Joint cultural events for youth (Sarajevo Film Festival and the MESS International Theatre Festival in Sarajevo hosting YIHR activists; Memory Walks; the best initiatives to be integrated into the platform; two of the most active youth leaders will be invited to sit on the NSC) 1.2.11. National Youth Bank Board (youth to have the opportunity to make joint decisions of importance and engage with key government and CSO influencers; support cultural initiatives and provide financial incentives for interested youth to mobilize community members to work on peace and community development) 1.2.12. Youth Initiative biannual conferences (in 2015 and 2017, Mozaik to gather 150 NYBB members, YIF implementers, and other relevant stakeholders from the 30 target municipalities to celebrate results, share lessons learned, and plan advocacy efforts, develop policy proposals and network with key influencers; government officials, religious leaders, and journalists to be invited; crowdsourcing to be used to encourage a broad base of youth to select the priorities via internet) 1.2.13. Women's Peace Initiatives (work with the Women’s Network, a network of 88 women NGOs to strengthen the intersection of women’s human rights and peace efforts; support at least three annual high profile events that promote women’s human rights) 1.2.14. Women’s Alternative Government (collaborate with HCA’s Women’s Alternative Government [WAG] project; mobilize women in the 30 targeted municipalities to actively engage in local reconciliation initiatives; selected members of WAG will be invited to join the municipal working groups; representatives will also be selected to contribute to the Media Metar initiative). 1.2.15. Association of Parent School Councils (capacity building; participating in MWGs and roundtables along with the ministers of education and principals) 1.2.16. Minority returnee social integration (cooperation with complementary projects funded by PRM) Output 1.3: Citizens of BiH, particularly youth, women and minority returnees, from different ethnic communities have skills and financial support to cooperate across ethnic lines 1.3.1. Youth Initiative Funds – YIF (provide approximately $233,000 to fund multi-ethnic youth initiatives in the 30 target municipalities; establish the National Youth Bank Board [NYBB] composed of 31 youth leaders from 31 municipalities who will select and mentor YIF recipients by engaging secondary school student councils, issue annual calls for proposals, select youth initiatives; the NYBB also to organize monitoring/evaluation of the approved projects; support at least 145 youth projects, at an average of $1,600 per initiative) 1.3.2. Capacity building (three sessions per year for the to ensure their sensitivity toward reconciliation issues, build their monitoring and evaluation skills, and generally build participants’ ability to work together in a team) 1.3.3. Joint Reconciliation and Flood Response Fund (20 projects at an average of $8,000) 84 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY 1.3.4. Women's Initiative Funds ($220,318 in small grants to fund multi-ethnic women’s reconciliation initiatives in the 30 target municipalities; support grassroots women’s networks to develop competitive applications with explicit outcomes in both development and reconciliation; support approximately 40 women’s initiatives over the four years, at an average of $5,500 per initiative) 1.3.5. Inter-Religious Council (establish regional branches of the council in four municipalities per year; set aside $18,000 of the small grants fund to support local religious communities working closely with the IRC on reconciliation initiatives) 1.3.6. Association of Parent Student Councils (earmark $33,333 in small grants funds to support local parent school council initiatives that simultaneously promote improved education and reconciliation in schools across the 30 targeted municipalities) Output 1.4: Citizens of BiH from different ethnic communities have an increased understanding of different media narratives and the role media have in reconciliation processes 1.4.1. Journalists for Peace (Mediacentar to engage five local/community media to support PRO-Future; each medium will delegate one journalist from local/community media based in the targeted municipalities or regions that will follow and produce a minimum of 2 features per month related to the PRO￾Future; the Journalists for Peace will produce at least 120 features per year; evaluation of the produced content and mentoring of journalists will be done by Mediacentar Sarajevo). Outcome 2: Key influencers in 30 targeted municipalities support local reconciliation initiatives Output 2.1: Local NGOs have the skills and resources to conduct local advocacy campaigns or reconciliation campaigns 2.1.1. Small grants to NGOs ($100,000 as traditional small grants, with special focus on organizations than work with/are led by women, minorities, and youth; require NGOs to mobilize local resources, relying on local leadership, local benefit, local solutions, and volunteerism; approximately 20 grants, at an average value of $5,000, to be disbursed during the project; place an emphasis on funding reconciliation efforts/advocacy activities that build on commonalities; some grants will also strengthen the capacity of the NBP and its members, will capitalize on existing community resources, and will ensure that local governments are actively engaged through co-funding or in-kind donations of at least 30%) 2.1.2. Assistance to the Network Against Torture (continue supporting the network created under CPT to advocate for a better policy framework to assist victims of torture and to prevent future use of torture; expand its membership base to additional victims’ associations, deliver advocacy training, and develop an advocacy action plan that will guide the network’s national-level efforts to amend the BiH Law on Torture and Law on Missing People; two trainings for 15 people will take place in years 1 and 2) Output 2.2: Formal and informal media outlets have greater capacity to communicate positive messages of reconciliation 2.2.1. Production of positive media stories (the Foundation for Creative Development to produce at least 40 videos of positive reconciliation stories; arrange screenings on national and local television stations and at public events) 2.2.2. Srđan Aleksić (recognize the top media promoters of positive messages for peace building in BiH through the Srđan Aleksić Award) 2.2.3. Engaging traditional media (distribute press releases, organize press conferences, and issue invitations for local and national media to cover PRO￾Future high-profile events; securing the participation of key influencers such as mayors, entity/canton ministers, and national religious leaders) 2.2.4. Produce content for the NBP website and the monthly NBP newsletter (providing further materials for formal and informal media to use in broadcasts or reports) 2.2.5. Media Metar (Mediacentar to expand its online platform and media fora for critical reflections on media role in reconciliation processes; online platform (www.media.ba) will launch a new section called “Media Metar” that will aim to bring together media critics, analysts, and columnists; each year, publish up to 20 columns/commentaries/interviews/features on the role of media in reconciliation process, conflict and post-conflict societies; and on the aspects of peace journalism; organize in Years 2 and 4 two media fora that will bring together the media community, civil society, and other key stakeholders to discuss and advance the role of media in the post-conflict society of BiH) Output 2.3: Religious, education, and government leaders understand their role in promoting reconciliation through both word and deed. 2.3.1. Municipal leadership roundtables (FTC will hold one roundtable in each target municipality pair the first year [15] and two joint roundtables per year for each paired community in years 2–4 [60]; address such themes as return issues, education, religion, minority issues, and gender issues to develop a shared vision for strengthening communities; plan additional events to address questions or topics of interest to the communities; host “Open Space” meetings, where participants will determine the specific agenda based on a broad, pre-determined topic; participants to include political leaders, religious leaders, community leaders, youth leaders, women’s groups’ leaders, CSO activists, school representatives, and business sector representatives) 2.3.2. Inter-Religious Council (engage diverse religious leaders to promote reconciliation initiatives, participate in the planning and implementation of the roundtables, and get their colleagues to participate) 2.3.3. Association of Parent Student Councils (engage and influence ministers of education and principals to support and participate in reconciliation projects, facilitate dialogues about challenging reconciliation topics related to education, such as two schools under one roof, and developing a unified curriculum; participate in MWGs and roundtables) 2.3.4. Platform for Building Peace (cohesive policy framework to support community reconciliation on a national level; in cooperation with key influencers from the 30 municipalities, CSOs, and religious institutions and in consultation with the NSC, MWGs, and NBP partners, a draft Platform document will be written in the first year, after which country-wide consultations will take place, to result in a proposed Platform to be shared with the NSC to secure state and entity government support; in the third year, the Platform is expected to be adopted by the state government and start a national dialogue on peace building with NBP members advocating across the country at all levels for the Platform to be adopted as the policy document; FTC to organize some of its roundtable discussions around the Platform) IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 85 Exhibit 11: Role of Implementers in PRO-Future (per June 2014 Award Modification) Organization Role Comparative Advantage Catholic Relief Services – Lead Agency Overall project management and leadership, reporting to USAID -Agency commitment to partnership and local capacity development -Existing tools in project design, M&E system design and implementation -Recognized success in peace building and multi-ethnic cooperation in BiH through CPT and other projects - Extensive experience managing USG resources and complying with applicable regulations Compliance with applicable USG regulations, generally accepted accounting principles, and the terms and conditions of the Award Authority to ensure high standards of technical and managerial quality Lead role in implementing NBP-based activities, municipal peace building training, drafting Platform for Building Peace Mozaik Foundation Disbursement of small grants, including Youth Initiative funds, NGO small grants - Expertise in adapting and implementing the globally recognized Youth Bank model in BiH - Experience in managing large-scale grants with USAID and EU - Experience in disbursing funds according to USG regulations - Nationally recognized expertise in peace building and reconciliation Caritas BK BiH/Vive Zene Training for war victims’ associations, organization of speaking-out events - Unique experience working with war victims’ associations since 2004 - Caritas network coverage throughout BiH Forum of Tuzla Citizens Organization and management of leadership roundtables in 30 municipalities - 20 years of leadership in regional and BiH reconciliation initiatives - Expertise in organizing leadership roundtables Identification of and coordination with religious, political, and other leaders in 30 municipalities Helsinki Citizens’ Association Leader of Network for Building Peace - Premier agency in BiH for women’s empowerment, peace building, and civil society programming Provision of annual Srđan Aleksić Award 86 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Exhibit 12: PRO-Future Activities and Small Grants per Location, based on PRO-Future Database Municipality Pair/Municipality of Implementation PRO-Future Activities PRO-Future Small Grants Total Beneficiaries Direct Beneficiaries Indirect Beneficiaries # of Events Direct Beneficiaries Indirect Beneficiaries # of Grants Direct Beneficiaries Indirect Beneficiaries 1 Novi Grad and Bosanska Krupa 740 520 25 1,557 8,825 37 2,297 9,345 2 Prijedor and Sanski Most 2,047 418 41 2,081 11,820 26 4,128 12,238 3 Drvar and Glamoč 295 0 16 125 950 4 420 950 4 Gornji Vakuf Uskoplje and Prozor Rama 894 400 28 671 1,800 20 1,565 2,200 5 Teslić and Tešanj 648 0 26 760 1,483 17 1,408 1,483 6 Doboj and Maglaj 1,624 20 39 2,336 15,831 57 3,960 15,851 7 Gračanica and Petrovo 743 0 20 635 19,132 16 1,378 19,132 8 Modriča and Gradačac 2,227 0 32 2,780 8,765 36 5,007 8,765 9 Ugljevik and Teočak 464 0 21 209 19,550 13 673 19,550 10 Kladanj and Vlasenica 745 350 26 1,166 3,543 16 1,911 3,893 11 Bratunac and Srebrenica 755 400 35 1,373 22,895 30 2,128 23,295 12 Goražde and Novo Goražde 311 200 21 1,164 2,028 16 1,475 2,228 13 Foča FBiH and Foča RS 317 0 17 232 0 6 549 0 14 Mostar and Nevesinje 1,210 1,415 52 1,015 100 19 2,225 1,515 15 Istočno Novo Sarajevo 965 0 32 989 1,690 13 1,954 1,690 16 Sokolac 338 150 22 348 734 11 686 884 TOTAL in 30 targeted municipalities 14,323 3,873 453 17,441 119,146 337 31,764 123,019 Total other municipalities 15,599 18,201 285 4,215 200 46 19,814 18,401 Total campaign 0 941,561 1 0 941,561 Total various municipalities and unspecified locations 6,322 7,760 50 2 6,322 7,760 TOTAL ALL 36,244 971,395 789 21,656 119,346 385 57,900 1,090,741 Source: Authors’ calculations based on PRO-Future Database IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 87 Exhibit 13: PRO-Future Small Grants per Purpose, Based on PRO-Future Database Small Grant Per Type as per PRO￾Future Database on Small Grants Number of Events Number of Direct Beneficiaries Number of Indirect Beneficiaries Amount in KM Local Contribution in KM 1 Community Empowerment 75 1,553 72,828 508,868 23,801 2 Culture 32 5,483 50 39,491 123,953 3 Educational 176 4,729 24,468 701,488 305,782 4 Environmental 15 364 15,320 102,556 36,420 5 Media Campaign 1 44 4,675 3,500 1,200 6 Public Speaking 2 94 0 10,916 4,174 7 Religious 5 1,305 20 5,292 3,935 8 Round Table 6 92 0 19,260 5,897 9 Sports 47 1,142 400 117,052 47,728 10 Local Initiatives 26 3,096 0 12,200 12,400 TOTAL 385 17,902 117,761 1,520,623 565,290 Source: Authors’ calculations based on PRO-Future Database 88 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Exhibit 14: PRO-Future Small Grants in the 30 Target Municipalities, Based on PRO-Future Database Source: Authors’ calculations and mapping based on PRO-Future database IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 89 Exhibit 15: PRO-Future Activities per Purpose, based on PRO-Future Database Type of Event as per PRO-Future Database on Activities Number of Events Number of Direct Beneficiaries Number of Indirect Beneficiaries 1 Advocacy Campaigns (Ambassador’s visit, Blood Is All the Same, One Whole Life, Celebrating Local Responsibility for Peace, Platform for Peace, Local Heroes and Heroines) 10 500 942,321 2 Community Empowerment (Local Heroes and Heroines) 2 51 3 Conferences (e.g., Peace Event, conference for journalists, youth conference, project presentation, presentation of results of “Stigmatization of Sexual Violence Survivors”) 7 3,699 2,650 4 Culture (e.g., exhibitions, performances, visit to parliament, “Let’s Have a Coffee Together,” iftars, art workshops) 62 8,945 2,333 5 Educational (e.g., youth summer camps, training for war victims, art workshops, awards for essays, conflict transformation and peacebuilding for implementers, NDC inter-ethnic education through dialogue, Living Library, panel discussions) 153 4,162 5,197 6 Local Initiatives Fund (e.g., project proposal writing, book clubs, running for health) 19 2,250 0 7 Long-Term Activity (Pro-Future Newsletter and art exhibition, “Medicine for the Soul”) 2 20 8 Media Campaign (e.g., documentary film, journalist award, MoU signing, Local Heroes and Heroines) 10 61 0 9 Meeting (e.g., grant signing, implementers’ internal meeting, meetings of implementers with MWGs, Expert Group meetings, grant selection meetings, peace camp follow-up meeting) 119 950 10,000 10 Presentation of the Project (e.g., grant call campaigns, NBP meetings, grant inauguration) 73 1,787 167 11 Public Forum (e.g., My Contribution to Lasting Peace, March 8 gathering, speaking-out events, panel discussions) 30 1,133 535 12 Public Speaking (e.g., My Contribution to Lasting Peace, My Story, Faith and Forgiveness) 94 3,739 0 13 Religious (Christmas concert) 1 150 14 Remembering Event (We Need to Know, Human Rights Day, Memorial Places, Peace Day) 8 3,955 1,000 15 Round Table (Culture of Dialogue in Peace Building, Education for Peace, Culture of Memory, Faith and Forgiveness, Trust and Development, Role of Religions in Peace Building, Role of Women in Peace Building, Why Are People Leaving BiH, Why Is It Necessary to Work on Peace Building) 69 2,294 16 Sports (Youth Sport Games, Football Competition, Sports Tournament, Volleyball Tournament Workshop for Football Coaches) 6 1,372 642 17 The Day of Open Doors 13 459 550 18 Video/Publication (2016 Calendar, Documentary on War Victims, PRO-Future Newsletter, Social Media) 9 10 6,000 19 Working Group Meetings 102 707 TOTAL 789 36,244 971,395 Source: Authors’ calculations based on PRO-Future Database 90 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Exhibit 16: PRO-Future Direct Beneficiaries in Target and Other Municipalities, Based on PRO-Future Database Source: Authors’ calculations and mapping based on PRO-Future Database IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 91 ANNEX VI: DATA COLLECTION INSTRUMENTS END-LINE HOUSEHOLD SURVEY QUESTIONNAIRE GENERAL SECTION Q1. When you think about reconciliation among the three major ethnic groups in BiH (Bosniaks, Serbs and Croats), what comes to your mind? (RECORD THE EXACT WORDS AS SPOKEN BY THE RESPONDENT) ______________________________________________________________________________________________ Q2. In your opinion, what is required so that the country can advance with a reconciliation process? (RECORD THE EXACT WORDS AS SPOKEN BY THE RESPONDENT) ______________________________________________________________________________________________ Q3. In your opinion, is reconciliation achievable in BiH? (ONE ANSWER ONLY) 1. Yes ► Skip to Q4 2. No Q3A. Why not? Please specify! (RECORD THE EXACT WORDS AS SPOKEN BY THE RESPONDENT) ______________________________________________________________________________________________ Q4. Thinking about the future of the country, which of the following statements would you say are the most important outcomes to aim for? Please rank them according to the degree of importance, where 1 is the most important, 2 is the second most important etc. SHOW CARD Issues Rank 1. The economic development of the country 2. Resolving unemployment issue 3. Developing and applying environmental sustainability policies 4. Strengthening social insurance system (including retirements) 5. Reduction of poverty 6. Ending nepotism and corruption 7. Ensuring quality of education 8. Reform of health care system 9. Ensuring security of citizens 10. Improve coexistence and cooperation among all ethnic groups 99. (Do not read) DK/NA 92 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Q5. Please tell me…. READ OUT EACH STATEMENT. SHOW CARD Items Not at all Not really Yes to some extent Yes very much (Do not read) Not applicable DK/NA 1. To what extent do you feel financially secure in your current job? 1 2 3 4 9 2. To what extent do you feel safe from the interethnic conflicts? 1 2 3 4 9 3. To what extent do you have sufficient access to professional health services? 1 2 3 4 9 4. To what extent do you feel safe from crime (e.g. theft)? 1 2 3 4 9 5. To what extent do you feel safe from violence in your daily life? 1 2 3 4 9 Q6. Which of the following identities best describes you? (ONE ANSWER ONLY) 1. Being a BiH citizen 2. Being a citizen of my Entity 3. Being a citizen of my Canton 4. Belonging to my ethnic group 5. Belonging to my religious group 6. (Do not read!) None of the above is important to me/I value different, individual aspects of my identity ► Skip to Q8 7. (Do not read!) DK/NA ► Skip to Q8 Q7. Based on the selection of identity you made above (see previous question), please indicate how important this identity is for you by answering the following questions: READ OUT (ONE ANSWER FOR EACH STATEMENT) Items Not at all Not really Yes to some extent Yes very much (Do not read) DK/NA 1. Being a ---- is an important part of my self-image. 1 2 3 4 9 2. Overall, I am glad to be a ---- 1 2 3 4 9 3. Being a ---- is an important reflection of who I am 1 2 3 4 9 4. Sometimes I wish I was not a ---- 1 2 3 4 9 TRUST Q8. In general, do you believe that people can be trusted or that it is better to be cautious with people? (ONLY ONE ANSWER!) 1. People can be trusted 1 2. It is better to be cautious with people 2 9. (Do not read!) Hard to say 9 IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 93 Q9. To what degree do you trust people from following ethnic groups? (ASK FOR EACH ETHNIC GROUP! RECORD ONE ANSWER FOR EACH ETHNIC GROUP!) No trust at all Somewhat distrust Neither trust nor distrust Somewhat trust Completely trust (Do not read!) Hard to say 1. Bosniaks 1 2 3 4 5 9 2. Serbs 1 2 3 4 5 9 3. Croats 1 2 3 4 5 9 4. Roma 1 2 3 4 5 9 5. Bosnians and Herzegovinians 1 2 3 4 5 9 6. Jewish 1 2 3 4 5 9 7. Other 1 2 3 4 5 9 Q10. To what degree do you trust people from following social groups? (ASK FOR EACH ETHNIC GROUP! RECORD ONE ANSWER FOR EACH ETHNIC GROUP!) No trust at all Somewhat distrust Neither trust nor distrust Somewhat trust Completely trust (Do not read!) Hard to say 1. Politicians 1 2 3 4 5 6 2. Journalists 1 2 3 4 5 6 3. Religious leaders 1 2 3 4 5 6 4. Civil society leaders 1 2 3 4 5 6 5. Women 1 2 3 4 5 6 6. Youth 1 2 3 4 5 6 7. Education officials, educators 1 2 3 4 5 6 8. Business owners 1 2 3 4 5 6 Q11. What is your opinion on the people from the following ethnic groups? (ASK FOR EACH ETHNIC GROUP! RECORD ONE ANSWER FOR EACH ETHNIC GROUP!) Very negative Somewhat negative Somewhat positive Very positive Don't know/ Refused Bosniaks 1 2 3 4 9 Serbs 1 2 3 4 9 Croats 1 2 3 4 9 Roma 1 2 3 4 9 Bosnians and Herzegovinians 1 2 3 4 9 Jewish 1 2 3 4 9 94 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY INTERACTION AMONG THE KEY INFLUENCERS Q12. In your opinion, to what extent did the interaction between key influencers (political, religious, or civil society leaders, media business owners, education officials) change when it comes to building trust among different ethnic groups in the community in the past three years? (READ OUT THE ANSWERS! ONLY ONE ANSWER!) 1. Very decreased 1 2. Somewhat decreased 2 3. Remained the same 3 4. Somewhat increased 4 5. Very increased 5 9. (Do not read!) Hard to say 9 Q13 In your opinion, to what extent did he interaction among the key influencers (political, religious, or civil society leaders, media business owners, education officials) change when it comes to peace building in the community in the past three years? (READ OUT THE ANSWERS! ONLY ONE ANSWER!) 1. Very decreased 1 2. Somewhat decreased 2 3. Remained the same 3 4. Somewhat increased 4 5. Very increased 5 9. (Do not read!) Hard to say9 Q14. In your opinion, to what extent did the interaction among the key influencers (political, religious, or civil society leaders, media business owners, education officials) change when it comes to reconciliation in the community in the past three years? (ONLY ONE ANSWER!) 1. Very decreased 1 2. Somewhat decreased 2 3. Remained the same 3 4. Somewhat increased 4 5. Very increased 5 9. (Do not read!) Hard to say9 INTERETHNIC EXPOSURE/CONTACT Q15. Which statement best describes your interaction with people from other ethnicities in BiH within your circle of family and friends? (READ OUT THE ANSWER OPTIONS! NOTE DOWN ONE ANSWER ONLY!) 1. Many of my family members or friends are from other ethnicities 1 2. Some of my family members or friends are from other ethnicities 2 3. Few of my family members or friends are from other ethnicities 3 4. None of my family members or friends are from other ethnicities 4 5. (Do not read) Refuses to answer 5 IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 95 Q16. Which statement best describes your interaction with people from other ethnicities in BiH in terms of your work colleagues, people with whom I interact through work or in school/university? (READ OUT THE ANSWER OPTIONS! NOTE DOWN ONE ANSWER ONLY!) 1. Many of my work colleagues/ people with whom I interact through work or in school/university are from different ethnicities 1 2. Some of my work colleagues/ people with whom I interact through work or in school/university are from different ethnicities 2 3. Few of my work colleagues/ people with whom I interact through work or in school/university are from different ethnicities 3 4. None of my work colleagues/ people with whom I interact through work or in school/university are from different ethnicities 4 5. (Do not read) Refuses to answer 5 Q17. If you do have actual communication with members of these groups where does it usually take place? READ OUT EACH GROUP. SHOW CARD Groups Randomly in the streets, neighborhoods Cafeterias/ Bars/Restaurants/ Stores Work/ professional contacts In educational institutions (schools, universities) Home visits Other (specify) (Do not read) DK/NA 1. Bosniaks 1 2 3 4 5 6 9 2. Serbs 1 2 3 4 5 6 9 3. Croats 1 2 3 4 5 6 9 Q18. In the past 12 months, did you have any contacts with members of ethnic groups other than yours? (ONE ANSWER ONLY) 1. Yes 2. No ► Skip to Q21 9. (Do not read!) Hard to say ►Skip to Q21 Q19. On average, how often did you have contacts with members of these groups in the past 12 months? (ONE ANSWER ONLY) 1. Once or twice in the past 12 months 2. Once or twice per month 3. Once or twice per week 4. Every day 9. (Do not read!) Hard to say Q20. Overall, were these contacts mainly positive or negative? Would you say that they were very negative, somewhat negative, somewhat positive or very positive? (ONE ANSWER ONLY) 1. Very negative 2. Somewhat negative 3. Somewhat positive 4. Very positive 9. (Do not read!) Hard to say 96 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY ATTITUDES AND FEELINGS Q21. We would like to know your general feelings toward the following groups. On a scale from 0 to 10, where 0 means “very negative” and 10 means “very positive”, please circle the number that best represents how you feel toward each group. Groups Very Negative Very Positive NA 1. Bosniaks 0 1 2 3 4 5 6 7 8 9 10 99 2. Serbs 0 1 2 3 4 5 6 7 8 9 10 99 3. Croats 0 1 2 3 4 5 6 7 8 9 10 99 Q22. When encoutering/or interacting with members of other communities, I usually feel... ? (ASK FOR EACH EMOTION SEPARATELY! RECORD ONE ANSWER PER ROW!) Not at all 2 3 4 A lot Anxious 1 2 3 4 5 Fearful 1 2 3 4 5 Distrust 1 2 3 4 5 Comfortable 1 2 3 4 5 Q23. How acceptable do you find having members of these ethnic groups as neighbors? (ASK FOR ALL ETHNIC GROUPS SEPARATELY! RECORD ONE ANSWER FOR EACH ETHNIC GROUP!) Completely unacceptable Somewhat unacceptable Neither acceptable nor unacceptable Somewhat acceptable Completely acceptable DK/NA Bosniaks 1 2 3 4 5 9 Serbs 1 2 3 4 5 9 Croats 1 2 3 4 5 9 Roma 1 2 3 4 5 9 Bosnians and Herzegovinians 1 2 3 4 5 9 Jewish 1 2 3 4 5 9 Q24. How acceptable do you find having members of these ethnic groups as working colleagues or co-workers? (ASK FOR ALL ETHNIC GROUPS SEPARATELY! RECORD ONE ANSWER FOR EACH ETHNIC GROUP!) Completely unacceptable Somewhat unacceptable Neither acceptable nor unacceptable Somewhat acceptable Completely acceptable DK/NA Bosniaks 1 2 3 4 5 9 Serbs 1 2 3 4 5 9 Croats 1 2 3 4 5 9 Roma 1 2 3 4 5 9 Bosnians and Herzegovinians 1 2 3 4 5 9 Jewish 1 2 3 4 5 9 IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 97 Q25. To what extent do you agree or disagree with the following statements? (ASK FOR EACH STATEMENT SEPARATELY! RECORD ONE ANSWER FOR EACH STATEMENT!) Strongly disagree Disagree Neither disagree nor agree Agree Strongly agree A. I sometimes think how members of the other groups might have felt during the war. 1 2 3 4 5 B. Usually I am able to understand other groups' point of view, concerning the war. 1 2 3 4 5 C. I try to see things that happened during the war also from the point of other groups. 1 2 3 4 5 D. I try to understand how other groups might feel after everything they have done. 1 2 3 4 5 E. I think it must be difficult for other groups to deal with the consequences of war. 1 2 3 4 5 F. I am ready to accept alternative narratives of the war. 1 2 3 4 5 G. I think that all ethnic groups suffered in the war 1 2 3 4 5 Q26. To what extent do you agree or disagree with the following statements? (ASK FOR EACH STATEMENT SEPARATELY! RECORD ONE ANSWER FOR EACH STATEMENT!) Strongly disagree Disagree Neither disagree nor agree Agree Strongly agree A. I wish I had better relations with other ethnic groups. 1 2 3 4 5 B. I wish I had more friends from other ethnic groups. 1 2 3 4 5 C. I would not mind being in a close relationship with someone who does not belong to my ethnic group. 1 2 3 4 5 D. I would not mind if my first neighbor belonged to other ethnic groups. 1 2 3 4 5 E. I am ready to forgive other ethnic groups for what they did in the war. 1 2 3 4 5 F. I could never forgive the crimes that were committed. 1 2 3 4 5 G. My ethnic group should never forgive other ethnic groups their bad deeds. 1 2 3 4 5 H. I think other ethnic groups can be trusted. 1 2 3 4 5 98 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Q27. Please mark the extent to which you believe the people from other ethnic groups are capable of feeling HAPPY. 1 means "not at all", and 5 means "completely capable". Groups Not at all 2 3 4 Completely capable (Do not read) DK/NA 1. Bosniaks 1 2 3 4 5 9 2. Serbs 1 2 3 4 5 9 3. Croats 1 2 3 4 5 9 Q28. Please mark the extent to which you believe the people from other ethnic groups are capable of feeling FEAR. 1 means "not at all", and 5 means "completely capable". Groups Not at all 2 3 4 Completely capable (Do not read) DK/NA 1. Bosniaks 1 2 3 4 5 9 2. Serbs 1 2 3 4 5 9 3. Croats 1 2 3 4 5 9 Q29. Please mark the extent to which you believe the people from other ethnic groups are capable of feeling LOVE. 1 means "not at all", and 5 means "completely capable". Groups Not at all 2 3 4 Completely capable (Do not read) DK/NA 1. Bosniaks 1 2 3 4 5 9 2. Serbs 1 2 3 4 5 9 3. Croats 1 2 3 4 5 9 Q30. Please mark the extent to which you believe the people from other ethnic groups are capable of feeling SADNESS. 1 means "not at all", and 5 means "completely capable". Groups Not at all 2 3 4 Completely capable (Do not read) DK/NA 1. Bosniaks 1 2 3 4 5 9 2. Serbs 1 2 3 4 5 9 3. Croats 1 2 3 4 5 9 IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 99 Q31. Please mark the extent to which you believe the people from other ethnic groups are capable of feeling GUILT. 1 means "not at all", and 5 means "completely capable". Groups Not at all 2 3 4 Completely capable (Do not read) DK/NA 1. Bosniaks 1 2 3 4 5 9 2. Serbs 1 2 3 4 5 9 3. Croats 1 2 3 4 5 9 Q32. Please mark the extent to which you believe the people from other ethnic groups are capable of feeling COMPASSION. 1 means "not at all", and 5 means "completely capable". Groups Not at all 2 3 4 Completely capable (Do not read) DK/NA 1. Bosniaks 1 2 3 4 5 9 2. Serbs 1 2 3 4 5 9 3. Croats 1 2 3 4 5 9 VISION OF FUTURE/RECONCILIATION ACTIVITIES Q33. What is your vision of BiH of the future? 1. BiH with the current structure of entities and cantons 1 2. BiH without cantons or entities, with the strong local authority of municipalities 2 3. BiH with the current structure of entities, without cantons 3 4. BiH with three entities 4 5. The entities and ethnic groups should each go their separate ways, dividing BiH into three independent States 5 6. Other, please specify: 6 ______________________________________________________________________________________________ 9. (Do not read!) DK/NA 7 Q34. Which of the following would you do in order to change the current conditions in terms of interethnic tensions in your community? READ OUT THE ANSWER OPTIONS! SELECT ALL THAT APPLY! I would do nothing and just stay focused on my own personal and business affairs 1 I would use political and social means of action, but definitely avoid any kind of violence 2 I would use any kind of actions, but definitely avoid violence 3 I would use all means of change available to me, including violence if necessary 4 100 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Q35. Currently, to what extent do you feel ready to advocate for changes in laws, regulations, with representatives of government institutions, parliaments, on local level or higher in order to contribute to peacebuilding and reconciliation? (READ OUT ALL ANSWERS! ONLY ONE ANSWER!) 1. Not at all ready 2. Somewhat not ready 3. Neither ready nor not ready 4. Somewhat ready 5. Completely ready 9. (Do not read!) Hard to say Q36. How much do you agree or disagree with the following statements? ASK FOR EACH STATEMENT SEPARATELY! RECORD ONE ANSWER FOR EACH STATEMENT!) Completely disagree Somewhat disagree Neither agree nor disagree Somewhat agree Completely agree I am capable of speaking out in public confidently. 1 2 3 4 5 I can express my opinion even when others disagree with me. 1 2 3 4 5 I feel that I can contribute significantlz to development of my community. 1 2 3 4 5 I am not interested in advocating for change in my community. 1 2 3 4 5 Q37. How much do you agree with the following statements: Completely disagree Disagree Neither agree nor disagree Agree Completely agree (Do not read) DK/NA The truth about what happened must be told in order for individuals and ethnic groups to move on. 1 2 3 4 5 9 Establishing a shared truth about the past is a prerequisite for reconciliation. 1 2 3 4 5 9 Acknowledging suffering of others is a prerequisite for reconciliation. 1 2 3 4 5 9 Prosecution of war criminals is a necessary precondition of reconciliation. 1 2 3 4 5 9 People should stop talking about past, and focus on economic and social development issues important for everyday life 1 2 3 4 5 9 Q38. In order for life in BiH to be of better quality, it is necessary to work on...? 1. Improving the economy 2. Improving the interethnic relations 3. Improving the political/institutional system 4. Other, specify: ____________________________________ 5. (Do not read) DK/NA IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 101 Q39. Would a serious attempt to build relations of trust and honesty between ethnic groups have any impact in BiH? Yes No On BiH level 1 2 On territory of my city and the surrounding area 1 2 On broader region/Region of the former Yugoslavia 1 2 Q40. In your opinion, for reconciliation how important are the topics related to the past of the country? 1. Very important 2. Important 3. Not important 4. It's an obstruction 9. (Do not read!) DK/NA Q41. How important do you find the following activities in regards to achievement of reconciliation? Not at all Not really Somewhat important Very important (Do not read) DK/NA Activities for youth belonging to different ethnic groups (youth camps, small grants for formal and informal youth groups, youth advocacy and peace building campaigns) 1 2 3 4 9 Discussion events (public speaking of war victims, round tables on the pressing issues in the community, public forums on peace and reconciliation) 1 2 3 4 9 Inter-religious events (day of the open doors in religious institutions, public forums on faith and peace-building) 1 2 3 4 9 Educational activities on peace building and reconciliation (trainings, online schools, seminars, conferences) 1 2 3 4 9 Activities for women (women peace initiatives, events related to women rights, small grants for women associations) 1 2 3 4 9 Small grants for connection of various ethnic and faith groups through joint work on peace and reconciliation promotion 1 2 3 4 9 Peace-building campaigns and media products (articles, movies, video, competitions on peace-building topics) 1 2 3 4 9 Art events motivated by peace building and reconciliation (concerts, plays, exhibitions, etc.) 1 2 3 4 9 Sports events with the purpose of promotion of peace building and reconciliation 1 2 3 4 9 Q42. Have you ever participated in any of the following activities organized with the purpose to promote peace building and reconciliation? Activities for youth belonging to different ethnic groups (youth camps, small grants for formal and informal youth groups, youth advocacy and peace building campaigns) 1 Discussion events (public speaking of war victims, round tables on the pressing issues in the community, public forums on peace and reconciliation) 2 Inter-religious events (day of the open doors in religious institutions, public forums on faith and peace-building) 3 Educational activities on peace building and reconciliation (trainings, online schools, seminars, conferences) 4 Activities for women (women peace initiatives, events related to women rights, small grants for women associations) 5 Small grants for connection of various ethnic and faith groups through joint work on peace and reconciliation promotion 6 Peace-building campaigns and media products (articles, movies, video, competitions on peace-building topics) 7 Art events motivated by peace building and reconciliation (concerts, plays, exhibitions, etc.) 8 Sports events with the purpose of promotion of peace building and reconciliation 9 I did not participate in any of these events ►Skip to Q41 99 102 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY ASK FOR EACH TYPE OF ACTIVITY THE RESPONDENT PARTICIPATED IN: Q42a. Do you know who organized the peace building activity you participated in? 99. (Do not read!) Does not know Q43. In future, would you participate in these activities related to peace building and reconciliation if you had the opportunity? Yes No If no, why not? Activities for youth belonging to different ethnic groups (youth camps, small grants for formal and informal youth groups, youth advocacy and peace building campaigns) 1 2 Discussion events (public speaking of war victims, round tables on the pressing issues in the community, public forums on peace and reconciliation) 1 2 Inter-religious events (day of the open doors in religious institutions, public forums on faith and peace-building) 1 2 Educational activities on peace building and reconciliation (trainings, online schools, seminars, conferences) 1 2 Activities for women (women peace initiatives, events related to women rights, small grants for women associations) 1 2 Small grants for connection of various ethnic and faith groups through joint work on peace and reconciliation promotion 1 2 Peace-building campaigns and media products (articles, movies, video, competitions on peace-building topics) 1 2 Art events motivated by peace building and reconciliation (concerts, plays, exhibitions, etc.) 1 2 Sports events with the purpose of promotion of peace building and reconciliation 1 2 Q44. Have you ever heard of … ? Yes No 1. CRS 1 2 2. PRO-Future 1 2 DEMOGRAPHIC SECTION D1. Sex 1. Male 1 2. Female 2 D2. Age: ________ D3. Ethnicity 1. Bosniak 1 2. Serb 2 3. Croat 3 4. Roma 4 5. Jewish 5 6. BiH 6 7. Other 7 8. Refused 8 IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 103 D4. Apart from special occasions, such as weddings and funerals, how often did you attend religious services in the past year? 1. Every day 1 2. More than once a week 2 3. Once a week 3 4. At least once a month 4 5. Only on special holy days 5 6. Seldom 6 7. Never 7 9. (Do not read) Don't know 9 D5. Education 1. No formal education 1 2. Primary 2 3. Secondary 3 4. College 4 5. University, undergraduate degree 5 6. University, master’s degree 6 7. University, PhD 7 D6. Which of the following describes your working status in the last month, taking into account any work done for pay or profit, regardless of whether it is officially registered? READ OUT THE ANSWER OPTIONS! NOTE DOWN ONE ANSWER ONLY! 1. Full time employment 1 2. Part-time employment 2 3. Intern/trainee 3 4. Volunteer 4 5. Unemployed, with occasional jobs, actively looking for work 5 6. Unemployed, actively looking for work 6 7. Unemployed, not looking for work 7 8. Unpaid everyday house/farm work, looking after children or others, not looking for paid work 8 9. Person with disability whose disability is the main reason for not working 9 10. Pupil/student/in specialization 10 11. Retired 11 12. Military (recruit) 12 13. (Do not read!) Refuses to answer 13 D7. Status of residence 1. Domicile 1 2. Internally displaced 2 3. Refugee 3 4. Returnee 4 5. Other 5 ______________________________________________ D8. During the war I was: Check all that applies 1. Soldier 1 2. Civilian 2 3. Refugee 3 4. Captive in a concentration camp 4 5. None of the above 5 9. (Do not read) NA 9 104 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY D9. Which of the following applies to you, if any? During the war 1992-1995... Check all that applies 1. I lost a close family member 2. Me or a close family member was wounded 3. Me or a close family member was in the concentration camp 4. Me or a close family member was a victim of physical abuse 5. Me or a close family member was a victim of sexual abuse 6. I lived in an area in which significant war activities/damages were conducted 7. None D10. Is your ethnic group a majority or minority in this community, or is this an ethnically balanced area? 1. Majority 1 2. Minority 2 3. Balanced 3 D11. Thinking about current finances of your household and your situation when it comes to provision of food, clothes, shelter etc. for you and your family, which of the following statements describe your income status best? a. I live comfortably with current income b. I am getting by with current income c. I am having difficulties to get by with current income d. It is extremely hard for me to get by with the current income D12. How much in KM did you earn in net income in last month for your formal and informal (registered and unregistered) work? D13. What is the monthly net income of your household? NOTE DOWN ONE ANSWER ONLY! 1. No income in this month 1 2. Up to 100 KM 2 3. 101 to 200 KM 3 4. 201 to 300 KM 4 5. 301 to 400 KM 5 6. 401 to 500 KM 6 7. 501 to 600 KM 7 8. 601 to 700 KM 8 9. 701 to 800 KM 9 10. 801 to 900 KM 10 11. 901 to 1000 KM 11 12. 1001 to 1100 KM 12 13. 1101 to 1200 KM 13 14. 1201 to 1300 KM 14 15. 1301 to 1500 KM 15 16. 1501 to 2000 KM 16 17. 2001 to 3000 KM 17 18. Above 3000 KM 18 19. (Do not read!) Does not know/Refuses to answer 19 D14. MUNICIPALITY: _________________________________ D15. TYPE OF SETTLEMENT 1. Urban 2. Rural IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 105 PRO-FUTURE EVALUATION INTERVIEW GUIDES PRO-FUTURE EVALUATION INTERVIEW INSTRUCTIONS The interview guides are intended to serve as semi-structured guides for conversations with key stakeholders of PRO-Future. Do not read the questions or probes word for word. Instead, adapt the wording to match the phrasing used by the respondent and ask only those questions which have not been already addressed by the interviewees during earlier part of the interview. Take notes on key terms or phrases used by the respondents that may be helpful in coding the interview data. Ask for clarification and definitions as needed. Familiarize yourself with the interview protocol guides in advance of your meeting. Skip questions that are not relevant given the interviewee specificities. Highlight the questions you will prioritize if the respondent’s time is limited. Be respectful of the respondent’s time and keep the interview to the agreed length of time. Follow up by phone or email for more information as needed. In addition: ▪ Take notes during the discussion. To ensure we accurately report what is discussed during the interview, we will record this session as well. ▪ As necessary, tailor all questions to fit the individual stakeholders’ relationship with PRO-Future. ▪ The Evaluation Team will ensure that the information shared through these interviews remain strictly confidential. 106 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY INTERVIEW GUIDE FOR MAYORS AND MEMBERS OF THE MUNICIPAL WORKING GROUPS My name is , and these are my colleagues . We are the team of researchers working for the MEASURE-BiH. First of all, we want to thank you for setting aside the time for this conversation. As you know, USAID/BiH has tasked MEASURE-BiH with conducing an independent evaluation of USAID/BiH PRO-Future intervention. Within the evaluation, we will conduct semi-structured interviews with around 50 PRO-Future stakeholders (including USAID/BiH, implementers and lecturers/facilitators, small grant participants, mayors/members of MWG, different types of event participants, and religious leaders) and will also hold several focus groups. Therefore, your views and opinions on the Activity are profoundly important for this evaluation. Through these KIIs, we are looking to gain the in-depth insights into the Activity implementation, the challenges faced along the way and the influence of specific interventions and to hear about stakeholders’ perceptions, lessons learned and recommendations for any possible future donor/government interventions in rebuilding trust and reconciliation in BiH. The information you provide will be used combined with information provided from other stakeholders. Your comments are confidential and you will not be identified by name in any report. will be taking notes while we talk. With your permission, we would also like to record this session. The reason why we are recording is that it is quite hard to actively participate in a conversation and take notes at the same time. Another reason is that we want to analyze the interviews using objective methods, and avoid any bias related to quality of notes and the capacity of interviewers’ memory. Do we have your permission to begin recording? Please do not hesitate to mention anything that you find important, and I miss to ask about it. Do you have any questions before we start? BACKGROUND ▪ In your opinion, what are the major problems facing BiH today? And what are the major problems facing your municipality? ▪ Among all problems in the municipality, where does the reconciliation process stand as a priority? ▪ How did you first hear about PRO-Future and why did you decide to participate? ▪ When you think about reconciliation among the three major ethnic groups in BiH (Bosniaks, Serbs and Croats), what comes to your mind? IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 107 ▪ In your opinion, what is required so that the country can advance with a reconciliation process? In your opinion, is reconciliation achievable in BiH? Why/why not? ▪ In your opinion, how is talking about the past and establishing the shared truth related to reconciliation? How is interethnic exposure and contact related to reconciliation? What type of interethnic exposure and contact do you believe to be most relevant for reconciliation in BiH? How important is for BiH citizens to accept each other’s narratives of the past for building trust needed for reconciliation? Do you believe that the truth about what happened must be told in order for individuals and ethnic groups to move on? Do you believe that establishing a shared truth about the past is a prerequisite for reconciliation and common future of BiH? Do you believe that acknowledging suffering of others is a prerequisite for reconciliation? Do you believe that prosecution of war criminals is a necessary precondition of reconciliation? Do you believe that people should stop talking about past, and focus on economic and social development issues important for everyday life? ▪ Would you say that things in this municipality and overall in BiH are getting better or worse when it comes to interethnic trust, peace building and reconciliation? Why do you think so? ▪ What do you consider to be PRO-Future’s objectives? Are you familiar with specific PRO-Future interventions? If yes, which interventions are these and what is your opinion of these interventions? ▪ Do you believe design of PRO-Future, including the development hypothesis and the two main objectives, is relevant and well-targeted? ▪ What was your role during the PRO-Future implementation? Who did you collaborate with? What were the outcomes of these activities? How were the MWGs formed? How were the members selected? How active were the MWG members? Please describe the nature of relationship and collaboration between MWG members from your and paired municipalities? Please describe the nature of relationship and collaboration between you/mayors from your and paired municipality? Now, I would like to discuss the PRO-Future effects on the citizens of this community. INFLUENCE ON CITIZENS’ ATTITUDES, TRUST, AND BEHAVIOUR ▪ What kind of activities did PRO-Future conduct with citizens of this municipality? How were these activities accepted in your municipalities? Please describe your experiences and role in regard to small grants? ▪ Have these activities affected citizens’ interethnic empathy and tolerance in your municipality? Why or why not? Can you provide some examples? 108 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY ▪ In your opinion, has PRO-Future affected interethnic trust in this municipality? Why or why not? Can you provide some examples? ▪ In your opinion, did PRO-Future have any effect on citizens’ behaviors in this municipality, e.g. their interethnic contacts and relationships or readiness to advocate for changes that would lead to reconciliation? If yes, what has changed in citizens’ behaviors and why? Which activities produced such changes and has PRO-Future contributed to these changes? ▪ In your experience and opinion, what are the main challenges in implementation of activities targeting changes in citizens’ behaviors in this municipality? How did the PRO-Future handle these challenges? What lessons have been learned based on that? ▪ What is the best approach to activate people who are less receptive when it comes to participation in peace building process? Now, I would like to ask you about PRO-Future activities targeting key influencers. ROLE OF KEY INFUENCERS AND THEIR INFLUENCE ON RECONCILIATION ▪ What kind of interventions did PRO-Future have with key influencers in this municipality? Who were the key influencers reached by the activity? ▪ In your opinion, has PRO-Future had influence on the key influencers’ support to local reconciliation initiatives, openness to dialogue and acting for reconciliation in this municipality? If yes, what has changed in key influencers’ behaviors and why? Which interventions produced such changes in your opinion? Please provide examples. ▪ Do the key influencers in this community and overall in BiH understand their role in promoting reconciliation? How do we know this? What are they doing to promote reconciliation in the country? ▪ How were the PRO-Future activities targeting key influencers implemented? What were the main challenges faced during the implementation of activities targeting key influencers, in your opinion? How did the PRO-Future handle these challenges? What are lessons learnt in your opinion? ▪ Have you noticed any media products on peace building and reconciliation? Articles, videos, movies? What do you think about these materials? How do they contribute to peace building and reconciliation in the country? Now that we have an understanding of the PRO-Future implementation, we have some questions about the recommendations and lessons learned from your perspective. CONCLUSIONS IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 109 ▪ Do you believe that PRO-Future affected policy-making in BiH? What about your municipality? If yes, how? If no, why not? ▪ Overall, are the PRO-Future activities relevant for social, political and cultural context of this municipality? And BiH? ▪ Would you say that PRO-Future methodology, objectives and interventions were mutually connected? Were they adjusted to beneficiaries and target groups? ▪ In your opinion, what are the long-term effects of the PRO-Future on… in this municipality: o Citizens’ interethnic attitudes o Citizens’ interethnic trust o Citizens’ behaviors o Key influencers’ behaviors ▪ What specific interventions were most effective in your opinion? Please provide examples. Which interventions were least effective? Please provide examples. ▪ Were there any unintended/unexpected effects of the PRO-Future in your community? ▪ From your perspective, what are the lessons learned from PRO-Future and recommendations for future interventions in reconciliation and peace building in BiH? Thank you for your participation in this interview. Is there anything else you would like to share with us? 110 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY INTERVIEW GUIDE FOR SMALL GRANT RECIPIENTS My name is , and these are my colleagues . We are the team of researchers working for the MEASURE-BiH. First of all, we want to thank you for setting aside the time for this conversation. As you know, USAID/BiH has tasked MEASURE-BiH with conducing an independent evaluation of USAID/BiH PRO-Future intervention. Within the evaluation, we will conduct semi-structured interviews with around 50 PRO-Future stakeholders (including USAID/BiH, implementers and lecturers/facilitators, small grant participants, mayors/members of MWG, different types of event participants, and religious leaders) and will also hold several focus groups. Therefore, your views and opinions on the Activity are profoundly important for this evaluation. Through these KIIs, we are looking to gain the in-depth insights into the Activity implementation, the challenges faced along the way and the influence of specific interventions and to hear about stakeholders’ perceptions, lessons learned and recommendations for any possible future donor/government interventions in rebuilding trust and reconciliation in BiH. The information you provide will be used combined with information provided from other stakeholders. Your comments are confidential and you will not be identified by name in any report. will be taking notes while we talk. With your permission, we would also like to record this session. The reason why we are recording is that it is quite hard to actively participate in a conversation and take notes at the same time. Another reason is that we want to analyze the interviews using objective methods, and avoid any bias related to quality of notes and the capacity of interviewers’ memory. Do we have your permission to begin recording? Please do not hesitate to mention anything that you find important, and I miss to ask about it. Do you have any questions before we start? BACKGROUND ▪ In your opinion, what are the major problems facing BiH today? And what are the major problems facing your municipality? ▪ Among all problems in BiH, where does the reconciliation process stand as a priority? ▪ How did you first hear about PRO-Future and why did you decide to apply for small grant? ▪ When you think about reconciliation among the three major ethnic groups in BiH (Bosniaks, Serbs and Croats), what comes to your mind? IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 111 ▪ In your opinion, what is required so that the country can advance with a reconciliation process? In your opinion, is reconciliation achievable in BiH? Why/why not? ▪ In your opinion, how is talking about the past and establishing the shared truth related to reconciliation? How is interethnic exposure and contact related to reconciliation? What type of interethnic exposure and contact do you believe to be most relevant for reconciliation in BiH? How important is for BiH citizens to accept each other’s narratives of the past for building trust needed for reconciliation? Do you believe that the truth about what happened must be told in order for individuals and ethnic groups to move on? Do you believe that establishing a shared truth about the past is a prerequisite for reconciliation and common future of BiH? Do you believe that acknowledging suffering of others is a prerequisite for reconciliation? Do you believe that prosecution of war criminals is a necessary precondition of reconciliation? Do you believe that people should stop talking about past, and focus on economic and social development issues important for everyday life? ▪ Would you say that things in BiH are getting better or worse when it comes to interethnic trust, peace building and reconciliation? Why do you think so? ▪ What do you consider to be PRO-Future’s objectives? Are you familiar with specific PRO-Future interventions? If yes, which interventions are these and what is your opinion of these interventions? ▪ Do you believe design of PRO-Future, including the development hypothesis and the two main objectives, is relevant and well-targeted? Now, I would like to discuss the PRO-Future effects on the citizens of this community. INFLUENCE ON CITIZENS’ ATTITUDES, TRUST, AND BEHAVIOUR ▪ What kind of intervention you implemented through small grant(s) received form PRO-Future? Please provide details on both design and implementation. ▪ Please describe your experiences with the process and procedures related to small grants allocation. ▪ Have these activities affected citizens’ interethnic empathy and tolerance in your municipality? Why or why not? Can you provide some examples? Would you say that the activity has caused the participants to change in the following manners: o They are better able to understand other groups’ point of view concerning the war o They are more willing to accept alternative narratives of the war o They more willing to accept people from other ethnic groups as neighbors/colleagues/friends/to be in close relationship with them 112 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY o They are more willing to forgive for what happened o They trust people from other ethnic groups more o They have more contacts and relationships with people from other ethnic groups o They more willing to advocate for changes that would lead to reconciliation ▪ What were the main challenges faced during the implementation of your activity? How did you handle these challenges? What lessons have been learned based on that? ▪ In your opinion, has PRO-Future overall affected interethnic trust in BiH? Why or why not? Can you provide some examples? ▪ In your opinion, did PRO-Future overall have any effect on citizens’ behaviors in BiH, e.g. their interethnic contacts and relationships or readiness to advocate for changes that would lead to reconciliation? If yes, what has changed in citizens’ behaviors and why? If yes, what has changed in citizens’ behaviors and why? If not, why not? ▪ What is the best approach to activate people who are less receptive when it comes to participation in peace building process? Now, I would like to ask you about PRO-Future activities targeting key influencers. ROLE OF KEY INFUENCERS AND THEIR INFLUENCE ON RECONCILIATION ▪ Did you conduct any activities targeting key influencers? If yes, who were the key influencers reached? Were there any higher-level politicians? ▪ How was your activity targeting key influencers implemented? What were the main challenges faced during the implementation of activities targeting key influencers? What kind of obstacles did you face? How did you handle these challenges? What lessons have been learned based on that? ▪ Did your activity have any effect on the key influencers’ support to local reconciliation initiatives, openness to dialogue and acting for reconciliation, in your opinion? If yes, what has changed in key influencers’ behaviors and why? Which interventions produced such changes? How do we know that the change is the effect of this activity? ▪ In your opinion, has PRO-Future overall had influence on the key influencers’ support to local reconciliation initiatives, openness to dialogue and acting for reconciliation in this municipality? If yes, what has changed in key influencers’ behaviors and why? Which interventions produced such changes in your opinion? Please provide examples. ▪ Do the key influencers in BiH understand their role in promoting reconciliation? How do we know this? What are they doing to promote reconciliation in the country? IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 113 ▪ Have you noticed any media products on peace building and reconciliation? Articles, videos, movies? What do you think about these materials? How do they contribute to peace building and reconciliation in the country? Now that we have an understanding of the PRO-Future implementation, we have some questions about the recommendations and lessons learned from your perspective. CONCLUSIONS ▪ Do you believe that PRO-Future affected policy-making in BiH? What about your municipality? If yes, how? If no, why not? ▪ Overall, are the PRO-Future activities relevant for social, political and cultural context of this municipality? And BiH? ▪ Would you say that PRO-Future methodology, objectives and interventions were mutually connected? Were they adjusted to beneficiaries and target groups? ▪ In your opinion, what are the long-term effects of the PRO-Future on… in BiH: o Citizens’ interethnic attitudes o Citizens’ interethnic trust o Citizens’ behaviors o Key influencers’ behaviors ▪ What specific interventions were most effective in your opinion? Please provide examples. Which interventions were least effective? Please provide examples. ▪ Were there any unintended/unexpected effects of the PRO-Future in your community? ▪ From your perspective, what are the lessons learned from PRO-Future and recommendations for future interventions in reconciliation and peace building in BiH? Thank you for your participation in this interview. Is there anything else you would like to share with us? 114 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 115 INTERVIEW GUIDE FOR PARTICIPANTS/BENEFICIARIES OF THE PRO-FUTURE EVENTS (INCLUDING YOUTH, VICTIM ASSOCIATION REPRESENTATIVES, AND PARTICIPANTS OF THE SPEAKING OUT EVENTS) My name is , and these are my colleagues . We are the team of researchers working for the MEASURE-BiH. First of all, we want to thank you for setting aside the time for this conversation. As you know, USAID/BiH has tasked MEASURE-BiH with conducing an independent evaluation of USAID/BiH PRO-Future intervention. Within the evaluation, we will conduct semi-structured interviews with around 50 PRO-Future stakeholders (including USAID/BiH, implementers and lecturers/facilitators, small grant participants, mayors/members of MWG, different types of event participants, and religious leaders) and will also hold several focus groups. Therefore, your views and opinions on the Activity are profoundly important for this evaluation. Through these KIIs, we are looking to gain the in-depth insights into the Activity implementation, the challenges faced along the way and the influence of specific interventions and to hear about stakeholders’ perceptions, lessons learned and recommendations for any possible future donor/government interventions in rebuilding trust and reconciliation in BiH. The information you provide will be used combined with information provided from other stakeholders. Your comments are confidential and you will not be identified by name in any report. will be taking notes while we talk. With your permission, we would also like to record this session. The reason why we are recording is that it is quite hard to actively participate in a conversation and take notes at the same time. Another reason is that we want to analyze the interviews using objective methods, and avoid any bias related to quality of notes and the capacity of interviewers’ memory. Do we have your permission to begin recording? Please do not hesitate to mention anything that you find important, and I miss to ask about it. Do you have any questions before we start? BACKGROUND ▪ In your opinion, what are the major problems facing BiH today? And what are the major problems facing your municipality? ▪ Among all problems in the municipality, where does the reconciliation process stand as a priority? ▪ How did you first hear about PRO-Future? ▪ When you think about reconciliation among the three major ethnic groups in BiH (Bosniaks, Serbs and Croats), what comes to your mind? ▪ In your opinion, what is required so that the country can advance with a reconciliation process? In your opinion, is reconciliation achievable in BiH? Why/why not? 116 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY ▪ In your opinion, how is talking about the past and establishing the shared truth related to reconciliation? How is interethnic exposure and contact related to reconciliation? What type of interethnic exposure and contact do you believe to be most relevant for reconciliation in BiH? How important is for BiH citizens to accept each other’s narratives of the past for building trust needed for reconciliation? Do you believe that the truth about what happened must be told in order for individuals and ethnic groups to move on? Do you believe that establishing a shared truth about the past is a prerequisite for reconciliation and common future of BiH? Do you believe that acknowledging suffering of others is a prerequisite for reconciliation? Do you believe that prosecution of war criminals is a necessary precondition of reconciliation? Do you believe that people should stop talking about past, and focus on economic and social development issues important for everyday life? ▪ Would you say that things in this municipality and overall in BiH are getting better or worse when it comes to interethnic trust, peace building and reconciliation? Why do you think so? ▪ What do you consider to be PRO-Future’s objectives? Are you familiar with specific PRO-Future interventions? If yes, which interventions are these and what is your opinion of these interventions? ▪ Do you believe design of PRO-Future, including the development hypothesis and the two main objectives, is relevant and well-targeted? Now, I would like to discuss the PRO-Future effects on the citizens of this community. INFLUENCE ON CITIZENS’ ATTITUDES, TRUST, AND BEHAVIOR ▪ In which PRO-Future interventions/events did you participate, how did you hear about them, and why did you decide to participate? Please provide details on your participation. ▪ If you were a speaker at a speaking out event, how and why did you decide to participate? Did you participate in any trainings conducted within the PRO-Future? If yes, what kind of training? Would you please describe your experience of the training? What did you learn? Would you please briefly describe your experience as a speaker at speaking out events? How did it affect the audience? How did it affect you? Did you receive any psychological support in the process? If yes, by whom? Did it help you? ▪ Have the events/activities in which you participated affected participants’ interethnic empathy, tolerance, and trust? Why or why not? Can you provide some examples? Did your attitudes toward other ethnicities change due to training/participation in the event? o Would you say you are able to understand other groups’ point of view concerning the war? IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 117 o Would you say you are ready to accept alternative narratives of the war? o Do you think that everyone suffered in the war? o Are you more ready to accept people from other ethnic groups as friends/to be in close relationship with them? o Would you say you are ready to forgive for what happened? o Would you say you trust people from other ethnic groups more? o Do you now have more contacts and relationships with people from other ethnic groups? o Would you advocate for changes that would lead to reconciliation? ▪ In your opinion, did PRO-Future overall have any effect on citizens’ behaviors, e.g. their interethnic contacts and relationships or readiness to advocate for changes that would lead to reconciliation? If yes, what has changed in citizens’ behaviors and why? If not, why not? ▪ In your experience and opinion, what are the main challenges in implementation of activities targeting changes in citizens’ behaviors in this municipality and overall in BiH? How did the PRO￾Future handle these challenges? What lessons have been learned based on that? ▪ What is the best approach to activate people who are less receptive when it comes to participation in peace building process? Now, I would like to ask you about PRO-Future activities targeting key influencers. ROLE OF KEY INFLUENCERS AND THEIR INFLUENCE ON RECONCILIATION ▪ In your opinion, has PRO-Future had influence on the key influencers’ support to local reconciliation initiatives, openness to dialogue and acting for reconciliation in this municipality? If yes, what has changed in key influencers’ behaviors and why? If not, why not? ▪ Do the key influencers in this community and overall in BiH understand their role in promoting reconciliation? How do we know this? What are they doing to promote reconciliation in the country? ▪ Have you noticed any media products on peace building and reconciliation? Articles, videos, movies? What do you think about these materials? How do they contribute to peace building and reconciliation in the country? 118 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY SPECIFIC QUESTIONS For Speaking Out Events Speakers/Participants ▪ Please describe your experience with speaking out events? ▪ How were the speaking out events structured? Was there any follow up in the end? ▪ What kind of training did you undergo to become a speaker in speaking out events? For Youth ▪ How did you decide to become active in peace building activities? ▪ What types of activities are the most appealing to youth? What kind of activities make change in youth perceptions, attitudes, and build trust among youth? ▪ To what extent are the youth in BiH willing to actively participate in building peace in their communities? ▪ What is the best approach to activate young people who are less receptive when it comes to participation in peace building process? Now that we have an understanding of the PRO-Future implementation, we have some questions about the recommendations and lessons learned from your perspective. CONCLUSIONS ▪ Do you believe that PRO-Future affected policy-making in BiH? What about your municipality? If yes, how? If no, why not? ▪ Overall, are the PRO-Future activities relevant for social, political and cultural context of this municipality? And BiH? ▪ Would you say that PRO-Future methodology, objectives and interventions were mutually connected? Were they adjusted to beneficiaries and target groups? ▪ In your opinion, what are the long-term effects of the PRO-Future on… in your municipality: o Citizens’ interethnic attitudes o Citizens’ interethnic trust o Citizens’ behaviors o Key influencers’ behaviors IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 119 ▪ What specific interventions were most effective in your opinion? Please provide examples. Which interventions were least effective? Please provide examples. ▪ Were there any unintended/unexpected effects of the PRO-Future in your community? ▪ From your perspective, what are the lessons learned from PRO-Future and recommendations for future interventions in reconciliation and peace building in BiH? Thank you for your participation in this interview. Is there anything else you would like to share with us? 120 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY FOCUS GROUP GUIDE FOR YOUTH PARTICIPANTS/BENEFICIARIES OF THE PRO-FUTURE EVENTS My name is , and these are my colleagues . We are the team of researchers working for the MEASURE-BiH. First of all, we want to thank you for setting aside the time for this conversation. As you know, USAID/BiH has tasked MEASURE-BiH with conducing an independent evaluation of USAID/BiH PRO-Future intervention. Within the evaluation, we will conduct semi-structured interviews with around 50 PRO-Future stakeholders (including USAID/BiH, implementers and lecturers/facilitators, small grant participants, mayors/members of MWG, different types of event participants, and religious leaders) and will also hold several focus groups. Your views and opinions on the Activity are profoundly important for this evaluation and we encourage you to be as open and honest as you can be so that we can get the clearest picture possibly. Everything you share will be confidential and you will not be identified by name in any report. The information you provide will be combined with information provided from other stakeholders. We have planned for this discussion to last for about an hour. We will only focus on 3 or 4 main areas in order to give ample time for everyone to share and converse together. The reason for this discussion is to gain in-depth insights into the Activity implementation, the challenges faced along the way and the influence of specific interventions and to hear about perceptions, lessons learned and recommendations for any possible future donor/government interventions in rebuilding trust and reconciliation in BiH. As this is a semi- structured discussion, as opposed to a direct question and answer individual interview, we encourage you to use the questions as a springboard to jump into a larger discussion. will be taking notes while we talk. With your permission, we would also like to record this session. The reason why we are recording is that it is quite hard to actively participate in a conversation and take notes at the same time. Another reason is that we want to analyze the interviews and focus group discussions using objective methods, and avoid any bias related to quality of notes and the capacity of interviewers’ memory. Do we have your permission to begin recording? In addition to the topics we ask about, please do not hesitate to mention any further information that you feel is important. Do you have any questions before we start? BACKGROUND ▪ We would like to go around the room for introductions, please share your first name and which PRO-Future interventions/events event you participated in? ▪ When you think about reconciliation among the three major ethnic groups in BiH (Bosniaks, Serbs and Croats), what comes to your mind? ▪ In your opinion, what is required so that the country can advance with the process of reconciliation? In your opinion, is reconciliation achievable in BiH? Why/why not? INTERETHNIC EXPOSURE/CONTACT IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 121 ▪ Would you say that things in your municipality and overall in BiH are getting better or worse when it comes to interethnic trust, peace building and reconciliation? Why do you think so? ▪ How would you describe you experiences of personal contact with members of other ethnic groups over the past 12 months? o Would you say these experiences were negative, somewhat negative, somewhat positive, or positive? INFLUENCE ON CITIZENS’ ATTITUDES, TRUST, AND BEHAVIOUR ▪ Have the events/activities in which you participated affected participants’ interethnic empathy, tolerance, and trust? Why or why not? Can you provide some examples? Did your attitudes toward other ethnicities change due to training/participation in the event? Possible Follow-up Questions: o Would you say you are able to understand other groups’ point of view concerning the war? o Would you say you are ready to accept alternative narratives of the war? o Do you think that everyone suffered in the war? o Are you and the participants of the events you attended more ready to accept people from other ethnic groups as friends/to be in close relationship with them? o Would you say you are ready to forgive for what happened? o Would you say you trust people from other ethnic groups more? o Do you now have more contacts and relationships with people from other ethnic groups? o Would you advocate for changes that would lead to reconciliation? CONCLUSIONS ▪ The overarching goal of PRO-Future is for BiH citizens to advocate for social changes in society by striving for two outcomes: 1) accepting alternative political and historical narratives, and 2) supporting local reconciliation initiatives. If you were in the shoes of the PRO-Future directors and organizers and were yourself in charge of implementation, how would you go about trying to accomplish these goals? o What might you do differently? o What interventions do you think are most and least effective? o What are your recommendations for future interventions in reconciliation and peace building in BiH? What is the best approach to activate people who are less receptive when it comes to participation in peace building process? Thank you for your participation in this focus group. Is there anything else you would like to share with us? 122 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY FOCUS GROUP GUIDE FOR PARTICIPANTS OF THE SPEAKING OUT EVENTS My name is , and these are my colleagues . We are the team of researchers working for the MEASURE-BiH. First of all, we want to thank you for setting aside the time for this conversation. As you know, USAID/BiH has tasked MEASURE-BiH with conducing an independent evaluation of USAID/BiH PRO-Future intervention. Within the evaluation, we will conduct semi-structured interviews with around 50 PRO-Future stakeholders (including USAID/BiH, implementers and lecturers/facilitators, small grant participants, mayors/members of MWG, different types of event participants, and religious leaders) and will also hold several focus groups. Your views and opinions on the Activity are profoundly important for this evaluation and we encourage you to be as open and honest as you can be so that we can get the clearest picture possibly. Everything you share will be confidential and you will not be identified by name in any report. The information you provide will be combined with information provided from other stakeholders. We have planned for this discussion to last for about an hour. We will only focus on 3 or 4 main areas in order to give ample time for everyone to share and converse together. The reason for this discussion is to gain in-depth insights into the Activity implementation, the challenges faced along the way and the influence of specific interventions and to hear about perceptions, lessons learned and recommendations for any possible future donor/government interventions in rebuilding trust and reconciliation in BiH. As this is a semi- structured discussion, as opposed to a direct question and answer individual interview, we encourage you to use the questions as a springboard to jump into a larger discussion. will be taking notes while we talk. With your permission, we would also like to record this session. The reason why we are recording is that it is quite hard to actively participate in a conversation and take notes at the same time. Another reason is that we want to analyze the interviews and focus group discussions using objective methods, and avoid any bias related to quality of notes and the capacity of interviewers’ memory. Do we have your permission to begin recording? In addition to the topics we ask about, please do not hesitate to mention any further information that you feel is important. Do you have any questions before we start? BACKGROUND ▪ In your opinion, what are the major problems facing BiH today? And what are the major problems facing your community? ▪ Among all problems in the community, where does the reconciliation process stand as a priority? ▪ How did you first hear about PRO-Future? ▪ When you think about reconciliation among the three major ethnic groups in BiH (Bosniaks, Serbs and Croats), what comes to your mind? How would you define reconciliation? IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 123 ▪ In your opinion, what is required so that the country can advance with a reconciliation process? In your opinion, is reconciliation achievable in BiH? Why/why not? What are the required conditions and/or processes for facilitating reconciliation? ▪ In your opinion, how is talking about the past and establishing the shared truth related to reconciliation? How is interethnic exposure and contact related to reconciliation? What type of interethnic exposure and contact do you believe to be most relevant for reconciliation in BiH? How important is for BiH citizens to accept each other’s narratives of the past for building trust needed for reconciliation? Do you believe that the truth about what happened must be told in order for individuals and ethnic groups to move on? Do you believe that establishing a shared truth about the past is a prerequisite for reconciliation and common future of BiH? Do you believe that acknowledging suffering of others is a prerequisite for reconciliation? ▪ Would you say that things in your community and overall in BiH are getting better or worse when it comes to interethnic trust, peace building and reconciliation? Why do you think so? ▪ What do you consider to be PRO-Future’s objectives? Are you familiar with specific PRO-Future interventions? If yes, which interventions are these and what is your opinion of these interventions? Now, I would like to discuss the PRO-Future effects on the citizens of this community. INFLUENCE ON CITIZENS’ ATTITUDES, TRUST, AND BEHAVIOUR ▪ In which PRO-Future interventions/events did you participate, how did you hear about them, and why did you decide to participate? Please provide details on your participation. ▪ Given that you were a speaker or a member of the audience at a speaking out event, how and why did you decide to participate? Did you participate in any trainings conducted within the PRO￾Future? If yes, what kind of training? Would you please describe your experience of the training? What did you learn? Would you please briefly describe your experience as a speaker at speaking out events? How did it affect the audience? How did it affect you? Did you receive any psychological support in the process? If yes, by whom? Did it help you? ▪ For Speakers: besides the PRO-Future activities, did you ever had an opportunity to share your experience? Was (and if so how) the PRO-Future event different? After actively participating at the Speaking Out Event, are you still willing to continue with public sharing of your stories and experiences? Do you think that such public sharing of war experiences can contribute to reconciliation? Why and how? ▪ Did you personally undergo any specific personal changes or transformation during or after your participation in PRO-Future activities. If so, please describe. 124 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY ▪ Have the events/activities in which you participated affected participants’ interethnic empathy, tolerance, and trust? Why or why not? Can you provide some examples? Did your attitudes toward other ethnicities change due to training/participation in the event? o Would you say you are able to understand other groups’ point of view concerning the war? o Would you say you are ready to accept alternative narratives of the war? o Do you think that everyone suffered in the war? o Are you and other participants of the PRO-Future event(s) you attended more ready to accept people from other ethnic groups as friends/to be in close relationship with them? o Would you say you are ready to forgive for what happened? o Would you say you trust people from other ethnic groups more? o Do you now have more contacts and relationships with people from other ethnic groups? o Would you advocate for changes that would lead to reconciliation? ▪ In your opinion, did PRO-Future overall have any effect on citizens’ behaviors, e.g. their interethnic contacts and relationships or readiness to advocate for changes that would lead to reconciliation? If yes, what has changed in citizens’ behaviors and why? If not, why not? ▪ In your experience and opinion, what are the main challenges in implementation of activities targeting changes in citizens’ behaviors in this community and overall in BiH? How did the PRO￾Future handle these challenges? What lessons have been learned based on that? What are your recommendations for future interventions in reconciliation and peace building in BiH? What is the best approach to activate people who are less receptive when it comes to participation in peace building process? Now that we have an understanding of the PRO-Future implementation, we have some questions about the recommendations and lessons learned from your perspective. CONCLUSIONS ▪ Do you believe that Speaking Out Events as a PRO-Future Activity has any potential to affect policy-making in BiH? What about your community? If yes, how? If no, why not? ▪ Overall, are other PRO-Future activities relevant for social, political and cultural context of this community? And BiH? ▪ Were there any unintended/unexpected effects of the PRO-Future in your community? ▪ From your perspective, what are the lessons learned from PRO-Future and recommendations for future interventions in reconciliation and peace building in BiH? IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 125 ▪ Do you consider it important to continue with speaking out events in the future and why? ▪ Do you think that speaking out events are more effective for specific audiences or are they suitable for the public in general? ▪ Do you have any suggestions in terms of recruitment of speakers and audiences for the future for this type of activity? ▪ Is there anything you would change or improve when it comes to implementation of the speaking out events? ▪ In your view, what is the current and potential role of war veterans’ associations for reconciliation processes? Thank you for your participation in this interview. Is there anything else you would like to share with us? 126 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY ANNEX VIII: DISCLOSURE OF ANY CONFLICT OF INTEREST Name Naida Carsimamovic Vukotic Title Chief of Party Organization IMPAQ International, LLC Evaluation Position? Team Leader Team member (Methods Lead) Evaluation Award Number (contract or other instrument) Monitoring and Evaluation Support Activity (MEASURE￾BiH), implemented by IMPAQ International, LLC, Contract No. 168-C-14-00003 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Trust, Understanding and Responsibility for the Future (PRO-Future), USAID-BiH, implemented by Catholic Relief Services, Award No. AID-168-A-13-00007 I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date April 11th , 2017. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 127 Name Emina Cosic Puljic Title MEASURE-BiH Senior Research Analyst Organization IMPAQ International, LLC Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) Monitoring and Evaluation Support Activity (MEASURE￾BiH), implemented by IMPAQ International, LLC, Contract No. 168-C-14-00003 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Trust, Understanding and Responsibility for the Future (PRO-Future), USAID-BiH, implemented by Catholic Relief Services, Award No. AID-168-A-13-00007 I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 7. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 8. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 9. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 10. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 11. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 12. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date April 11th , 2017. 128 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Name Salminka Fazlic Title MEASURE-BiH Research Analyst Organization IMPAQ International, LLC Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) Monitoring and Evaluation Support Activity (MEASURE￾BiH), implemented by IMPAQ International, LLC, Contract No. 168-C-14-00003 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Trust, Understanding and Responsibility for the Future (PRO-Future), USAID-BiH, implemented by Catholic Relief Services, Award No. AID-168-A-13-00007 I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 13. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 14. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 15. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 16. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 17. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 18. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date April 11th , 2017. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 129 Name Sabina Cehajic Clancy Title External Expert Organization Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) Monitoring and Evaluation Support Activity (MEASURE￾BiH), implemented by IMPAQ International, LLC, Contract No. 168-C-14-00003 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Trust, Understanding and Responsibility for the Future (PRO-Future), USAID-BiH, implemented by Catholic Relief Services, Award No. AID-168-A-13-00007 I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 19. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 20. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 21. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 22. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 23. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 24. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. Has delivered two short lectures on intergroup reconciliation for CRS in Sapna in 12/2016 and in Vakuf in 11/2016. 130 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY Name Siniša Lakic Title External Expert Organization Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) Monitoring and Evaluation Support Activity (MEASURE￾BiH), implemented by IMPAQ International, LLC, Contract No. 168-C-14-00003 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Trust, Understanding and Responsibility for the Future (PRO-Future), USAID-BiH, implemented by Catholic Relief Services, Award No. AID-168-A-13-00007 I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 25. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 26. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 27. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 28. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 29. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 30. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date April 11th , 2017. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 131 Name Sarah More Title External Expert Organization Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) Monitoring and Evaluation Support Activity (MEASURE￾BiH), implemented by IMPAQ International, LLC, Contract No. 168-C-14-00003 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Trust, Understanding and Responsibility for the Future (PRO-Future), USAID-BiH, implemented by Catholic Relief Services, Award No. AID-168-A-13-00007 I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 31. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 32. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 33. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 34. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 35. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 36. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date April 11th , 2017. 132 USAID.GOV IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY REFERENCES 1. Allport, G. W. (1954). The Nature of Prejudice. Massachusetts: Addison-Wesley Publishing Company. 2. Asmal, K., Asmal, L., & Roberts, R.S. (1997). Reconciliation through truth: a reckoning of apartheid’s criminal governance. Capetown: David Phillips Publishers. 3. Bar-Tal, D. (2007). Socio-psychological foundations of intractable conflicts. American Behavioral Scientist, 50(11), 1430-1453. 4. Baum, F., MacDougall, C., & Smith, D. (2006). Participatory action research. Journal of Epidemiology and Community Health, 60(10), 854–857. 5. Binder, J.R., Desai, R.H., Graves, W.W., Conant, L.L. (2009). Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb Cortex, 19(12), 2767-2796. 6. Brandeis University. (2017). Why do we need creative approaches to peacebuilding. Retrieved from http://www.brandeis.edu/ethics/peacebuildingarts/about_us/whycreative.html 7. Brown, R., Maras, P., Masser, B., Vivian, J., & Hewstone, M. (2001). Life on the ocean wave: Testing some intergroup hypotheses in a naturalistic setting. Group Processes and Intergroup Relations, 4, 81-98. 8. Brown, R., & Hewstone, M. (2005). An integrative theory of intergroup contact. In M. Zanna (Ed.), Advances in Experimental Social Psychology, 37, 255–343. San Diego, CA: Academic Press. 9. Cehajic, S., & Brown, R. (2008). Not in my name: A social psychological study of antecedents and consequences of acknowledgement of in-group atrocities. Genocide Studies and Prevention: An International Journal, 3(2), Article 5. 10. Cehajic-Clancy, S., Brown. R., Castano, E. (2008). Forgive and forget? Antecedents and consequences of intergroup forgiveness in Bosnia and Herzegovina. Political Psychology, 29(3). 11. Cehajic-Clancy, S., Goldenberg, A., Gross, J.J., Halperin, E. (2016). Social-psychological interventions for intergroup reconciliation: An emotion regulation perspective. Psychological Inquiry, 27(2), 73-88. 12. Chirwa, W. C. (1997). Collective memory and the process of reconciliation and reconstruction. Development in Practice, 7(4). 13. Eller, A., Abrams, D. (2003). “Gringos” in Mexico: Longitudinal effects of language school-promoted contact on intergroup bias. Special issue on the contact hypothesis. Group Process and Intergroup Relations, 6, 55-75. 14. Gardner Feldman, L. (1999). The principle and practice of reconciliation in German foreign policy: Relations with France, Israel, Poland and Czech Republic. International Affairs, 75(2), 333-356. 15. Grbich, C. (1999). Qualitative Research in Health; An Introduction. London: Sage. 16. Hakansson, P., & Sjoholm, F. (2007). Who do you trust? Ethnicity and trust in Bosnia and Herzegovina. Europe-Asia Studies, 59(6), 961-976. 17. Hayes, G. (1998). We suffer our memories: Thinking about the past, healing, and reconciliation. American Imago, 55, 29-50. 18. Hayner, P.B. (1999). In pursuit of justice and reconciliation: Contribution of truth telling. In C.J. Aronson (Ed.), Comparative Peace Process in Latin America, 363-383. Stanford: Stanford University Press. 19. Kopstein, J.S. (1997). The politics of national reconciliation: Memory and institutions in German-Czech relations since 1989. Nationalism & Ethnic Politics, 3, 57-78. 20. Kriesberg, L. (1998). Coexistence and the reconciliation of communal conflicts. In Weiner, E. (Ed.), The Handbook of Interethnic Coexistence. New York: The Continuum Publishing Company. 21. Lederach, J.P. (1998). Building Peace: Sustainable Reconciliation in Divided Societies. Washington, DC: United States Institute of Peace Press. 22. Leonard, M., Damjanovic, B., Simic, G., & Aldikact, M.G. (2016). Peace building in Bosnia and Herzegovina: Effects of in-group identification, outgroup trust and intergroup forgiveness on intergroup contact. Peace and Conflict Studies, 23(1), Article 4. 23. Minkler, M., & Wallerstein, N. (Eds.). (2003). Community Based Participatory Research for Health. San Francisco, CA: Jossey-Bass. IMPACT AND PERFORMANCE EVALUATION OF PRO-FUTURE ACTIVITY USAID.GOV | 133 24. Norval, A.J. (1998). Memory, identity and the (im)possibility of reconciliation: The work of the truth and reconciliation commission in South Africa. Constellations, 5, 250-265. 25. Norval, A.J. (1999). Truth and reconciliation: The birth of the present and the reworking of history. Journal of African Studies, 25, 499-519. 26. O’Loughlin, J. (2010). Inter-ethnic friendships in post-war Bosnia and Herzegovina. Ethnicities, 10(1), 26- 54. 27. Pettigrew, T.F. (1998). Intergroup contact theory. Annual Review of Psychology, 49, 65-85. 28. Pettigrew, T.F., & Tropp, L.R. (2006). A meta-analytic test of intergroup contact theory. Journal of Personality and Social Psychology, 90(5), 751-783. 29. Pettigrew, T.F., & Tropp, L.R. (2008). How does intergroup contact reduce prejudice? Meta-analytic tests of three mediators. European Journal of Social Psychology, 38(6), 922-934. 30. Salomon, G. (2004). A narrative-based view of coexistence education. Journal of Social Issues, 60(2), 273- 287. 31. Stephan, W.G., Diaz-Loving, R., Duran, A. (2000). Integrated threat theory and intercultural attitudes: Mexico and the United States. Journal of Cross-Cultural Psychology, 31(2), 240-249. 32. USAID/BiH MEASURE-BiH (2017). Overview of reconciliation conceptual framework in international literature and research on reconciliation in BiH. 33. Whitt, S. (2010). Institutions and ethnic trust: evidence from Bosnia. Europe-Asia Studies, 62(2), 271-292. 34. Zelizer, C. (2005). Artistic responses to the siege of Sarajevo: The cellist and the film festival in Bosnia￾Herzegovina. In van Tongeren, P., Brenk, M., Hellema, M., & Verhoven, J. (Eds.), People Building Peace II, 301-308. Boulder, CO: Lynne Rienner Publishers.