Gori photo.jpg March 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by ME&A. MID-TERM PERFORMANCE EVALUATION FOR PROMOTING INTEGRATION, TOLERANCE, AND AWARENESS (PITA) PROGRAM MID-TERM PERFORMANCE EVALUATION FOR PROMOTING INTEGRATION, TOLERANCE, AND AWARENESS (PITA) PROGRAM Final Report March 2018 Prepared under RFTOP No. SOL-114-17-00007 Submitted to: USAID/Georgia Submitted by: Kathy Selvaggio, Team Leader Ada Huibregste, International Evaluation Expert Konstantine Peradze, Local Expert Natia Gorgadze, Local Expert Contractor: ME&A 4350 East-West Highway, Suite 210 Bethesda, MD 20814 Tel: 301- 652 -4334 www.MEandAHQ.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS EXECUTIVE SUMMARY................................................................................................................................................... I 1.0 EVALUATION BACKGROUND, PURPOSE AND QUESTIONS .......................................................5 1.1 Evaluation Background..........................................................................................................................5 1.2 Evaluation Purpose.................................................................................................................................5 1.3 Evaluation Questions.............................................................................................................................5 2.0 PROJECT BACKGROUND............................................................................................................................6 3.0 EVALUATION METHODS AND LIMITATIONS.....................................................................................8 3.1 Evaluation Methodology .......................................................................................................................8 3.2 Evaluation Limitations............................................................................................................................9 4.0 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS.............................................................10 4.1 EQ 1: How effective has PITA been in building the capacity of Government of Georgia institutions promoting civic integration and tolerance toward minorities?...........................10 4.2 EQ 2: What is PITA’s effectiveness in building a sense of national unity among youth?.....17 4.3 EQ 3: What is PITA’s effectiveness in building public awareness and education?................30 4.4 EQ 4: How well does the Civic Integration and Tolerance Index work to measure civic integration at the national level and for PITA outcomes? .........................................................32 4.5 Project Strategy and Implementation..............................................................................................34 ANNEXES .........................................................................................................................................................................39 Annex A: Statement of Work Mid-Term Evaluation of the PITA ........................................................40 Annex B: List of Interviewees and Focus Group Discussions................................................................45 Annex C: List of Documents Reviewed......................................................................................................48 Annex D: Mini-Survey Questionnaires........................................................................................................51 LIST OF FIGURES Figure 1: Number of Respondents to PITA Survey, by Gender and Ethnicity....................................................9 Figure 2: Number of Youth Center Members by Gender and Year...................................................................18 Figure 3: Number of Youth Center Members, by Ethnicity and Year................................................................18 Figure 4: Annual Relative Participation in Youth Center Activities by Gender and Year..............................18 Figure 5: Years of Youth Participation in Youth Centers, by Age Group .........................................................19 Figure 6: Ethnic Georgians and Ethnic Minorities Are Not Friendly With Each Other.................................21 Figure 7: People of Different Religions Cannot Understand Each Other..........................................................21 Figure 8: Perception of Ease in Communicating with Different Groups...........................................................22 Figure 9: Perceptions that Ethnic Minority Issues Are Addressed by the Government.................................23 Figure 10: Perceptions that Religious Minority Rights Are Addressed by the Government.........................23 Figure 11: Georgian Language Important for Ethnic Minorities, Armenian Respondents..............................24 Figure 12: Main Barriers for Interaction of Youth with Peers in Other Regions.............................................25 Figure 13: Youth Define “Civic Activism” .................................................................................................................27 Figure 14: Number of Instances Discriminatory Language Was Used in Primetime News Programs and Talk Shows........................................................................................................................................................................31 Figure 15: Georgia Civic Integration and Tolerance Index, 2012-2016..............................................................33 ACRONYMS ANI Advancing National Integration Project ATIP Fund State Fund for Protection and Assistance of Victims of Human Trafficking CCIIR Center for Civil Integration and Inter-Ethnic Relations CNM Council of National Minorities CR Council of Religions CSO Civil Society Organization EQ Evaluation Question ET Evaluation Team EU European Union FGD Focus Group Discussion GDI Georgian Democracy Initiative GOG Government of Georgia GPB Georgian Public Broadcasting HEI Higher Education Institution IDP Internally Displaced People IRMS IT, Research & Metadata Solutions KII Key Informant Interview LGBT Lesbian, Gay, Bisexual, and Transgender M&E Monitoring and Evaluation MDF Media Development Foundation MEL Monitoring, Evaluation, and Learning MOES Ministry of Education and Science NATO North Atlantic Treaty Organization NCAP National Concept and Action Plan for Tolerance and Civic Integration NGO Non-Governmental Organization PDO Public Defender’s Office PG Partners-Georgia PITA Promoting Integration, Tolerance, and Awareness Project PMP Performance Monitoring Plan SARA State Agency for Religious Affairs SCEI State Strategy for Civic Equality and Integration SMR State Ministry for Reconciliation and Civil Equality SOW Statement of Work TC Tolerance Center TDI Tolerance and Diversity Institute TSU Tbilisi State University UNAG United Nations Association of Georgia U.S. United States USAID United States Agency for International Development YC Youth Center i EXECUTIVE SUMMARY EVALUATION PURPOSE This is a report on the mid-term performance evaluation of the Promoting Tolerance, Integration, and Awareness (PITA) Project funded by the United States Agency for International Development (USAID) Mission in Georgia. PITA is being implemented by the United Nations Association of Georgia (UNAG) between June 2015 and June 2020. The overall purpose of this performance evaluation is to: examine the progress of PITA’s activities to date; evaluate the prospect for achieving life-of-project results; and identify if any changes need to take place to ensure the latter. The evaluation of PITA was conducted during the period January and February 2018 by a four-person Evaluation Team (ET). The ET consisted of international and local experts and was assembled by ME&A, located in Bethesda, Maryland. The ET was supported by IT, Research & Metadata Solutions (IRMS), a local organization contracted to conduct a survey of youth members and focus group discussions (FGDs) of youth, parents, tutors, and interns. The primary intended audience of the evaluation includes USAID, in particular its Democracy and Governance office, as well as UNAG, the implementing agency. The results may also be shared with project partners and other local stakeholders, such as the State Ministry for Reconciliation and Civic Equality (SMR), the Public Defender’s Office (PDO) and Tolerance Center (TC), Partners-Georgia (PG), other donors working in the relevant sectors, and interested civil society organizations (CSOs). PROJECT BACKGROUND PITA was designed to address ongoing challenges related to the integration of ethnic and religious minorities into the mainstream of Georgian life. Challenges span geographic isolation, linguistic barriers, lack of access to information, and a heavy reliance on Russian media, which spreads disinformation and propaganda. Surveys and polls indicate that ethnic minorities have weaker trust in democratic institutions and stronger alignment with Russian interests. Tensions between the majority and minority populations simmer and periodically erupt through hate speech, discrimination, or violence. Although ethnic tensions seem to have improved in recent years, religious intolerance has heightened, with several high-profile attacks against religious minorities in Georgia. Significant gains in integration have been made in recent years, particularly as a result of Government of Georgia’s (GOG) multi-year plans. The government is implementing its second five-year plan for civic integration of ethnic minorities with PITA’s support. Yet, the scale of these plans is limited. PITA has set out to tackle these issues through a substantial effort to increase interaction between ethnic minorities and the majority population, both at the level of government and at the level of communities. The project’s three objectives are to increase: 1. Interaction between ethnic/religious minorities and GOG; 2. The level of civic engagement and integration among and between youth of diverse ethnic/religious backgrounds; and 3. Public awareness and education about diversity. EVALUATION DESIGN, METHODS, AND LIMITATIONS The ET used a mixed methods approach to respond to the four main evaluation questions (EQs): EQ 1: How effective has PITA been at building the capacity of the GOG, including the SMR, the PDO, and the structures created under PDO including the TC, Council of National Minorities (CNM), and Council of Religions (CR) to fulfill their mandates and achieve PITA’s objectives? EQ 2: How effective has PITA’s approach been in developing a sense of national unity among youth so far? Has this approach been different for girls and boys? ii EQ 3: How effective has PITA been at increasing public awareness and educating the public about diversity? EQ 4: How well does the Civic Integration Index developed by UNAG work in measuring civic integration at the country level, and how does PITA capture its contribution? Should/can it be refined? To determine whether PITA has made progress towards its initial objectives and is on track to deliver the expected results, the ET collected data from a range of stakeholders and beneficiaries. This approach ensured the independence of the evaluation process as well as the accuracy and completeness of the findings, and subsequent conclusions and recommendations. It included: • A thorough review of more than 25 PITA project documents and more than a dozen relevant publications. • Three in-depth interviews with UNAG staff, as well as 45 key informant interviews (KIIs) with partners, beneficiaries, and stakeholders in Tbilisi, Marneuli, Ninotsminda, Gori, and Kutaisi. • KIIs and 12 FGDs with 69 youth, parents, tutors, and ethnic minority interns at five locations mentioned above as well as visits to the Youth Centers (YCs) to observe youth-focused activities. • A telephone mini-survey, pre-tested and conducted by IRMS in Georgian, Armenian, and Azeri, among a random sample of 327 members of the 14 YCs. The survey provided a confidence level of just slightly below 95 percent. The ET encountered limitations inherent to the design of this evaluation and during its fieldwork in Georgia. The most relevant constraints were: 1) biases in data collection methods; 2) limited project monitoring data on outcomes; 3) halo bias; and 4) the short amount of time allotted for a project with diverse components and numerous counterparts and beneficiaries. In addition, concepts such as tolerance and changes in attitudes and behavior toward social groups are difficult to measure with precision and require long-term processes to achieve. Furthermore, the project is designed to have long-term benefits, some of which may have not been fully realized by this mid-term evaluation. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS Findings and Conclusions EQ 1: PITA’s Effectiveness in Building GOG’s Capacity to Fulfill Mandate. SMR can point to important program achievements in recent years, all of which have benefited from PITA’s collaboration and support, including: 1) the decision by the GOG to establish an internship program for ethnic minority university students and graduates, following a successful pilot initiative by SMR and PITA; and 2) two awareness-raising campaigns with ethnic minority communities on issues that are heightened in these communities. In the process, the SMR has built greater ownership for integration of ethnic minorities among some GOG decision-makers. The awareness campaigns have allowed SMR to move beyond national policies and programs to engage local youth activists from PITA’s network and build new partnerships, bringing it closer to the ethnic minorities it serves. As a direct result of the PITA’s financial support and collaboration with its network, the TC, CR, and CNM can point to several strong policy results over the past two years. The most important was a successful campaign by CR and CSO allies preventing Parliament from amending the constitution by imposing limitations on freedom of religion and belief. PITA has also strengthened CNM and CR member organizations in various ways, including facilitated dialogues with government officials. With PITA’s financial and technical support, the TC, CNM, and CR have deepened their credibility and legitimacy with GOG decision-makers and with international human rights bodies as unique voices for ethnic and religious minorities. Nevertheless, they remain highly dependent on the good will of the PDO, which comes under periodic political pressure. PITA and USAID have had to help the TC/CNM/CR navigate challenges to its independence, by intervening with the PDO and helping the councils to establish charters defining institutional roles between the PDO and TC. At present, the iii GOG does not seem to have demonstrated sufficient commitment to upholding ethnic and religious minority rights to ensure that the TC’s current efforts can be sustained if USAID and PITA were to transfer it to the PDO’s direct management and budget control, as originally planned. EQ 2: PITA’s Effectiveness in Promoting National Unity Among Youth. PITA-supported YCs’ membership is growing at a steady rate, exceeding project targets. The majority of youth participate between one and four years, indicating considerable commitment to YC programs. However, more girls are participating than boys and the relative proportion of Georgian youth participating is rising, while the proportion of Armenian youth participating is declining. The gender imbalance in favor of girls among YCs members might find explanation through a range of cultural and economic factors. Boys’ lesser participation is cause for minor concern in a context where disaffected young men often lead movements of intolerance, discrimination, and violence. Similarly, the relative decline in Armenian youth in PITA’s membership, combined with findings about barriers to participation for youth in remote ethnic minority regions due to transportation problems, limited Georgian language skills, and social stigma, are also causes for concern. Respondents to a PITA survey of 327 youth found that the longer a youth was a YC member, the more he or she felt able to communicate with different minority group, and be more open toward people of different ethnicity, religion, or sexual identity. However, the sustainability of this effect comes into question since youth once but no longer active in YCs show a modest decline in openness and tolerance. Furthermore, youth surveyed were in some respects less inclined than Advancing National Integration Project (ANI) respondents to agree with statements indicating strong respect for ethnic and religious rights in Georgia and positive majority-minority relations. This decline might be due to the shift in target age to older youth under the PITA project, who have more maturity, knowledge, critical thinking skills, and deeper sensitivity to minorities due to their work with PITA. Yet, disaggregation of the results indicated minorities were generally more positive than majority respondents about these issues. Survey results also indicate that youth across all ethnic groups do not see ethnicity or religion as major barriers to connection with youth in other regions. In addition, they revealed that knowledge of Georgian language is important for minorities primarily to “feel like a citizen of the country” – an important indication of their sense of national unity. YC activities are having a positive effect on many youth, contributing toward strong intercultural friendships, transforming stereotypes and attitudes, and increasing gender sensitivity. YCs have also established a strong reputation in local communities for providing quality programs for youth and for fostering a sense of civic responsibility. Youth are initiating their own civic improvement projects, and also providing grassroots support for national campaigns. YCs are producing many motivated, engaged future leaders. In addition, a few YCs are able to attract funding from other donors or forge partnerships with businesses, raising the prospect that PITA could decrease or end financial support to these YCs in the future, freeing resources for other uses. EQ 3: PITA’s Effectiveness in Increasing Public Awareness and Education. PITA-supported media watchdog efforts and public awareness of hate speech have been accompanied by a reduction by two-thirds in the number of incidents of hate speech and discrimination during primetime news programs and talk shows over the years 2014-2017. NGOs leading media watchdog efforts have also had complaints to media monitoring bodies upheld. The broad approach to hate speech— encompassing ethnicity, religion, gender, sexual orientation, disability status, and other factors—as well as the efforts to identify and condemn anti-Western statements are sending an important signal to public officials about the limits of intolerance and discrimination in Georgian society. Moreover, in response to persistent advocacy by CR and CNM, Georgian Public Broadcasting (GPB) has recently taken actions to significantly expand news for and about ethnic and religious minorities. Yet more could be done to promote expanded coverage of minorities in the media. iv EQ 4: Effectiveness of the Civic Integration Index in Capturing PITA’s Contribution. Despite its title, the Civic Integration and Tolerance Index has a singular focus on civic integration of and tolerance toward ethnic minorities. It does not fully consider tolerance toward religious minorities, except insofar as it intersects with that of ethnic minorities. Nor does it take into account tolerance and integration toward other minorities, for example Lesbian, Gay, Bisexual, and Transgender (LGBT) people, migrants, or racial groups. The Index falls somewhat short as a tool to measure PITA project outcomes because: 1) it fails to adequately capture religious integration and tolerance; and 2) it measures change at the macro and national levels, while PITA is not quite large enough to drive significant change at the national level. Recommendations EQ 1: PITA’s Effectiveness in Building GOG’s Capacity to Fulfill Mandate. Together, PITA and the SMR should identify “low hanging fruit,” e.g., programs that have a high likelihood of success as well as the possibility of ownership by the GOG. In the process, they should consult with ethnic minority groups to identify issues that are directly relevant to their daily lives. Before transferring the budget and management control of the TC to the PDO, PITA and the TC should study alternative organizational models that would maximize its independence, allow direct channels to government decision-making, and gain sustainable financial support. EQ 2: PITA’s Effectiveness in Promoting National Unity Among Youth. Among various recommendations, PITA should: shift more attention and program resources to YCs in more remote regions, especially Armenian regions, addressing barriers to participation (transportation, lack of Georgian language skills, social stigma); increase the frequency of the exchange program or other approaches that allow youth to build bonds across ethnic, religious, and regional lines; and continue efforts to recruit more boys to YCs. EQ 3: PITA’s Effectiveness in Increasing Public Awareness and Education. PITA should expand efforts to promote more positive media coverage of ethnic and religious minorities, especially among major Georgian TV and radio channels, and local media in minority regions. It should also encourage media to promote the direct voices of ethnic and religious minorities. EQ 4: Effectiveness of Civic Integration Index in Capturing PITA’s Contribution. To make the Civic Integration and Tolerance Index more appropriate to capturing PITA project outcomes, PITA should consider expanding it to separately assess issues concerning religious minorities. It could also consider replicating the national index methodology in minority regions where PITA conducts most of its activities. 5 1.0 EVALUATION BACKGROUND, PURPOSE AND QUESTIONS 1.1 EVALUATION BACKGROUND This is a report on the mid-term performance evaluation of the Promoting Tolerance, Integration, and Awareness (PITA) project funded by the United States Agency for International Development (USAID) Mission in Georgia. PITA is being implemented by the United Nations Association of Georgia (UNAG) between June 2015 and June 2020. The project funding is $4.6 million over five years. The evaluation of PITA was conducted during the period January and February 2018 by a team assembled by ME&A, located in Bethesda, Maryland. The Evaluation Team (ET) comprised four experts: Ms. Kathleen Selvaggio (Team Leader), Dr. Ada Huibregste (Evaluation Expert), Mr. Konstantine Peradze (Local Expert), and Ms. Natia Gorgadze (Local Expert). The ET was supported by IT, Research & Metadata Solutions (IRMS), a local organization contracted to conduct a survey of youth members and focus group discussions (FGDs) of youth, parents, tutors, and interns. The findings, conclusions, and recommendations in this report emanate from the collective efforts of the above-mentioned team. 1.2 EVALUATION PURPOSE According to the Statement of Work (SOW) for this assignment (see Annex A) the overall purpose of this mid-term performance evaluation was to: examine the progress of PITA’s activities to date; evaluate the prospect for achieving life-of-project results; and identify if any changes need to take place to ensure the latter. The intended audience of the evaluation includes USAID, in particular its Democracy and Governance office, as well as UNAG as the implementing agency. The results may also be shared with project partners and other local stakeholders, such as the State Ministry for Reconciliation and Civic Equality (SMR), the Public Defender’s Office (PDO) and Tolerance Center (TC), Partners-Georgia (PG), other donors working in the relevant sectors, and interested civil society organizations (CSOs). In determining PITA’s effectiveness, the ET has kept in mind that the project is designed to have long-term benefits, some of which may have not been fully realized given that this is a mid-term evaluation and that social and attitudinal changes are long-term processes. 1.3 EVALUATION QUESTIONS In line with the Scope of Work (SOW) requirements, the ET was tasked with answering the following questions: 1. How effective has PITA been at building the capacity of the Government of Georgia (GOG), including the SMR, the PDO, and the structures created under PDO including the TC, Council of National Minorities (CNM), and Council of Religions (CR) to fulfill their mandates and achieve PITA’s objectives? a. What are the lessons learned from working with these structures: State Ministers’ SMR, TC, CNM, and CR? What concrete results have these structures achieved so far? b. What is the prospect for sustainability of the TC and the two Councils (CR and CNM) under the PDO? 2. How effective has been PITA’s approach in developing a sense of national unity among youth so far? Has this approach been different for girls and boys? 6 a. How well did the changes to the previous approaches work (e.g., change in the age of the target youth from the school to university age)? 3. How effective has PITA been at increasing public awareness and educating the public about diversity? 4. How well does the Civic Integration Index developed by UNAG work in measuring civic integration at the country level, and how does PITA capture its contribution? Should/can it be refined? For purposes of this evaluation, the terms “effectiveness” and “national unity” are defined as follows: • “Effectiveness” is understood as ability of partner organizations and structures to fulfill their mandates and achieve PITA’s objectives. • “National unity” is understood as the creation of a mutual understanding across ethnic and religious identities within the Georgian territory. The ET’s response to these questions is elaborated in the main body of the report in the following sections and is supported, where appropriate, by tables, figures, and charts. 2.0 PROJECT BACKGROUND The PITA project was designed to address ongoing challenges related to the integration of ethnic and religious minorities into the mainstream of Georgian life. UNAG, which implements the PITA project with USAID’s support, has built the project upon years of previous success of the Advancing National Integration Project (ANI) project to forge closer ties among the Georgian ethnic majority (which comprise 87 percent of the population) and ethnic minorities, including ethnic Azerbaijanis and ethnic Armenians (which together comprise 11 percent of the population), as well as other smaller ethnic groups. Similarly, the project works to promote greater tolerance and understanding among followers of the Georgian Orthodox Church (83 percent of the population), Islam (11 percent) and the Armenian Apostolic Church (3 percent). Ethnic and religious minorities, which intersect to an important degree, tend be located far from the political and economic centers of the country, reinforcing their isolation and marginalization in Georgian life. They often face linguistic barriers to inclusion and tend to have close relationships with people of the same ethnicity in neighboring states. Ethnic minorities tend to be more exposed to Russian propaganda, which is disseminated throughout Georgia, but especially in ethnic minority regions where the non￾Georgian speaking population relies heavily on Russian TV media for information.1 Lack of access to information through Georgian media in these areas leads to insufficient awareness of current events in Georgia which, in turn, contributes to low levels of political participation of minority communities. Tensions between the majority and minority populations simmer, driven in part by deep-seated stereotypes and intolerance. A 2016 report by the European Council against Racism and Intolerance (ECRI) covering the five-year period through mid-June 2015 asserted that “hate speech against ethnic and religious minorities continues to be a widespread problem in Georgia and these groups are still often viewed mainly through a security lens.” Although ethnic tensions seem to have improved in recent years, religious intolerance has heightened, with several high-profile attacks against Muslims and Jehovah’s Witnesses in Georgia. Surveys and polls indicate that ethnic minorities have weaker trust in democratic institutions and stronger alignment with Russian interests. For example, a National Democratic Institute poll on “Public Attitudes in Georgia” conducted in December 2017 reveals some striking differences in attitudes between people 1 See National Democratic Institute Poll June 2016. https://www.ndi.org/NDI-poll-georgian-voters-june-2016 and https://www.ndi.org. 7 in ethnic minority settlements (mostly Armenians and Azerbaijanis settled in the Kvemo Kartli, Samtsakhe￾Javakethe, and Kakheti regions) and citizens in the other regions of the country. Although those living in minority settlements voted at a higher rate in the October 2017 local elections than those in most urban regions of the country, they were two to three times more likely than other citizens to say that they did not have enough information about the parties and the candidates. They also were generally less confident that votes were accurately counted. Furthermore, they were less likely to approve of Georgia joining the European Union (EU) or North Atlantic Treaty Organization (NATO). In fact, 56 percent of minorities indicated that Georgia should join Eurasian Union (Russia, Kazakhstan, and Belarus). Significant gains in integration have been made in recent years, particularly through government multi-year initiatives to promote tolerance and integration of ethnic minorities. The government’s first National Concept and Action Plan for Tolerance and Civic Integration (NCAP) from 2009-2014 was an unprecedented effort among post-Soviet states to address integration of minorities, as well as a unique whole-of-government approach to engaging different ministries in the plan. NCAP has now been superseded by the State Strategy for Civic Equality and Integration (SCEI), 2015-2020, which is taking on new and more focused challenges. At the same time, the scale of these plans tends to be limited. PITA has set out to tackle these issues through a substantial effort to increase interaction between ethnic minorities and the majority population, both at the level of government and at the level of communities. The project has three objectives: 1. Increase interaction between ethnic/religious minorities and the GOG; 2. Increase the level of civic engagement and integration among and between youth of diverse ethnic/religious backgrounds; and 3. Increase public awareness and education about diversity. Toward the first objective, PITA’s main effort centers on supporting the government in the development and implementation of the SCEI and strengthening the TC, a semi-government body tied to the PDO, a national ombudsman’s office charged with monitoring human rights. The TC is an umbrella for two councils that bring together both ethnic minorities and ethnic majorities, which both facilitate direct dialogues between their member organizations and the government, and monitor the government’s actions. Toward the second objective, PITA has identified youth as the primary driver of social change in promoting diversity and inclusion. Through 14 Youth Centers (YCs) established across the country, including in six regions with a high density of ethnic minority population, PITA supports a range of activities to: assist youth individually; raise awareness and foster social interaction between majority and minority youth; and encourage civic activism in a variety of forms. Toward the third objective, PITA is strengthening CSOs to play a critical role as watchdogs on incidents of hate speech and discrimination, particularly among public figures, and call violators to account. PITA also aims to reach a wider audience through various approaches to multiply its reach, such as peer-to-peer awareness raising efforts and promoting media coverage of ethnic and religious minorities. It is not expected that PITA can change these long-term structural issues regarding tolerance and integration in just a few years. However, through steady support built upon a solid track record of past success and relationships with a growing network of partners, PITA is well placed to make a substantial contribution toward the goal. Ultimately, the PITA project is helping to build a social consensus around the values that shape Georgia’s democratic institutions and policies. 8 3.0 EVALUATION METHODS AND LIMITATIONS 3.1 EVALUATION METHODOLOGY The ET relied on mixed qualitative and quantitative methods to respond to the four main evaluation questions (EQs), and determine whether PITA has made progress towards its initial objectives and is on track to deliver the expected results. The ET collected data from a range of stakeholders and beneficiaries to ensure independence of the evaluation process, as well as accuracy and completeness of the findings and subsequent conclusions and recommendations. 3.1.1 Qualitative Research and Analysis The ET undertook a thorough review of more than 25 PITA project documents, including annual and quarterly reports, annual implementation plans, and monitoring and evaluation (M&E) data plans. The team also reviewed more than a dozen publications, some of which were research studies or advocacy documents published with PITA’s support (Annex C). The ET conducted several in-depth interviews with UNAG staff, as well as key informant interviews (KIIs) with partners, beneficiaries, and stakeholders such as the TC under the PDO, SMR, non-governmental organizations (NGOs) receiving Tolerance and Diversity Small Grants from PITA, YC coordinators, external project partners, and others. In all, 45 KIIs were conducted between January 16 and January 27, 2018, and also in follow up interviews in February 2018. In addition to meeting with stakeholders in Tbilisi, the ET traveled to Kutaisi, Marneuli, Gori, and Ninotsminda to: conduct interviews and FGDs with youth, parents, tutors, and interns; visit the YCs; and observe youth-focused activities (for a full list of key informants, see Annex B). The ET retained the services of IRMS, a Georgian firm, to assist in implementing the youth survey and FGDs. To expand upon the information gained through individual interviews, the ET conducted 12 FGDs in five locations with 69 YC members, parents of YC members, tutors to ethnic minority students in the universities, and ethnic minority youth interns. FGDs covered topics including: reasons for joining YCs; level of interaction between ethnic majority and minority groups; contribution of the YCs to youth’s personal development and to the wider community; and the experience of tutors as well as interns. The FGDs were conducted in Georgian, Armenian, and Azeri, depending on the language preference of the participants. Details on the number of FGD participants and locations can be found in Annex B. 3.1.2 Quantitative Research and Analysis Through IRMS, the ET conducted one mini-survey among members of the 14 YCs supported by the project (Annex D). UNAG provided the ET with contact lists for YC members. After examining the mix of ethnicities (Georgian, Armenian, and Azeri) and gender among the youth, the team decided to draw a random sample of 350 respondents proportional to their gender, age, and geographic distribution among the members. After finalization in English, the survey instrument was translated into Georgian, Armenian, and Azeri and pre-tested. Survey data were collected from January 25 to February 5, 2018. The survey team chose to rely upon telephone interviews since some contact information included only telephone numbers and no email addresses, and also to maximize response rates, which tend to be lower online. The data entry system accounted for the logical flow of the questionnaire and applied special rules to prevent incorrect data entry. It should be noted, however, that conducting the survey by telephone had inherent limitations, for example, by posing complicated questions orally rather than in written form. 9 It is important to note that aiming for a target of 350 responses, the survey team had to complete about 900 phone calls, due to non-working phone numbers, (youth tend to change phone numbers often), unanswered calls, and other obstacles. When the survey team was unable to reach a YC member, it randomly chose another YC member, again proportional to the member location, age, and gender. (Note: in the process, the survey team found that the actual ages of YC members differed from those in the PITA database by an average of 1.2 years.) Ultimately, the survey team reached 327 YC members, a confidence level of just slightly below 95 percent relative to the larger population of YC members. Of the 327 survey respondents: • Fifty-four (54) percent were female and 46 percent were male; • Twenty-seven (27) percent were between the ages 14 and 17, 69 percent were between the ages of 18 and 25, and 4 percent were 26 years old or older; and • Eighty-three (83) percent of respondents were currently active in YCs, while 17 percent had previously been active but were no longer active. A breakdown by number, gender, and ethnicity is provided in Figure 1. Figure 1: Number of Respondents to PITA Survey, by Gender and Ethnicity 3.2 EVALUATION LIMITATIONS The ET encountered some limitations inherent to the design of this evaluation and during its fieldwork in Georgia. The most relevant were as follows: Biases in data collection methods. To identify key stakeholders and project beneficiaries, the ET relied to a significant extent on assistance from staff from PITA project and PITA’s partner organizations. Although this led to some risk of selection bias, the ET made efforts to identify additional informants who were not project partners or beneficiaries, and to use random selection methods for FGDs and the survey. Limited project monitoring data on outcomes. Because PITA has mostly been regularly tracking output indicators, it was not possible for the ET to rely on project monitoring data to assess progress towards many outcome indicators identified in the project’s M&E plans. PITA is only consistently reporting on two outcome indicators through its performance monitoring plan (PMP), one of which is the Civic Integration and Tolerance Index, which falls short as a tool for capturing PITA outcomes (see Section 4.4 of this report). Upon inquiry, PITA was able to produce some data toward outcomes, for example, baseline survey of youth attitudes, reports from periodic FGDs with grantees, and references to non-project 104 81 51 40 23 28 0 50 100 150 200 Female Male Number of respondents Gender Azeri Armenian Georgian 10 external research. However, these outcomes are not systematically captured in periodic project reports in order to track progress over time. Even some output data are inconsistently tracked. For these reasons, the ET was heavily reliant on qualitative data and survey data collected for this evaluation to approximate progress toward many project outcomes. Halo bias. There is a known tendency among respondents to underreport socially undesirable answers and alter their responses to approximate what they perceive as the social norm (halo bias). The extent to which respondents were prepared to reveal their true opinions may also have varied for some questions, especially those that called upon respondents to assess the performance of colleagues or those on whom they depend upon for funding or political support (especially in the case of partner NGOs, YC members, and TC staff). In addition, it is possible that growing awareness among PITA-supported youth about the social desirability of tolerant and inclusive attitudes might have colored some of their responses to survey and FGD questions. Time allotted. The Team Leader was in country for three weeks, while the Evaluation Expert did not visit the country at all. The project’s components and activities are diverse, with numerous counterparts in the government and civil society, and beneficiaries in far-flung regions of the country. Finally, it is essential to keep in mind that concepts such as inclusion and tolerance, as well as changes in attitudes and behavior toward social groups, are quite difficult to measure with much precision and inherently require long-term processes to achieve. As a result, this evaluation might not have fully captured the extent of changes brought about through project activities. 4.0 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS 4.1 EQ 1: HOW EFFECTIVE HAS PITA BEEN IN BUILDING THE CAPACITY OF GOVERNMENT OF GEORGIA INSTITUTIONS PROMOTING CIVIC INTEGRATION AND TOLERANCE TOWARD MINORITIES? 4.1.1 Findings State Ministry for Reconciliation and Civic Equality The PITA project supports the SMR in one of its major policy functions: promoting the integration of ethnic minorities through the development and implementation of the SCEI. In 2014, with advice and input from UNAG staff, the SMR elaborated the second strategic plan, covering the years 2015 to 2020. The new plan was well informed by a thorough assessment of strengths and weaknesses of the previous NCAP 2009-2014. The assessment was first suggested by UNAG and USAID and then subsequently developed by the SMR and a group of independent experts, drawing upon feedback from ethnic minority groups. In contrast to the previous strategy, the updated strategy places greater emphasis on engaging the ethnic majority of Georgians in the process of integration, in recognition of evidence that Georgians harbor more skepticism about integration. It also gives special attention to some of the smaller and more vulnerable ethnic minority groups that had been particularly marginalized from Georgian society. Finally, it introduces improved monitoring and reporting mechanisms on the implementation of the plan. With PITA’s support, the SMR coordinates the Inter-Agency Commission responsible for implementation of SCEI, composed of more than 25 representatives of Ministries, national offices, commissions and academies, regional self-government bodies, city governments, and others. By the SMR’s own account, interagency coordination works efficiently and productively. PITA has contributed to effective operations of the Commission through organizing direct consultations with ethnic minority communities, thematic sessions on specific issues, and periodic skills trainings on developing their annual action plans and reports. 11 The Inter-Agency Commission has also established thematic working groups, which engage NGOs and experts to address issue-specific challenges and identify solutions. Currently, various PITA partners participate in these groups. Furthermore, the SMR reports that it works closely with the CNM, which conducts monitoring and periodic reporting to Parliament on the implementation of SCEI. Among many important policy achievements that the SMR has spearheaded at the government level over recent years with PITA’s support are the following: • Internship program. In December 2016, following a very successful pilot initiative led by the SMR and PITA, the government issued an order establishing a new program allowing ethnic minority students in the advanced years of the 1+4 program2 as well as those enrolled in Masters’ degree programs to compete twice a year for internships with state agencies. Although the government had already established a general internship program for university students, this new program was specifically designated for ethnic minority students. It also has proven to be highly valued by students in the 1+4 program. To date, 65 ethnic minority youth have completed three-month internships with 21 state institutions. PITA served on the selection committee for the interns and also provided transportation and accommodation allowances to students performing internships in ethnic minority regions. Up to 10 interns have obtained full-time employment following their internship, but the initiative is still quite young. • Native language instruction. In 2016, the MOES introduced native language instruction in the primary and secondary schools of small ethnic groups (Ossetian, Kurds, Udi, Chechen) that had been overlooked in the past, some of which had also expressed concern that their language was dying out. This policy shift came partly as a result of years of advocacy by the CNM (see below), made possible through PITA’s support. • Public awareness campaigns. Since 2016, the SMR initiated two awareness-raising campaigns in ethnic minority communities, in collaboration with other state institutions. One focused on promoting gender equality, especially in regard to the problems of early marriage and domestic violence, which are particularly acute within some ethnic minority communities. The second focused on Georgia’s integration in the EU, an issue that polls indicate draws much less support among ethnic minority communities than among the majority Georgian population. The SMR forged partnerships with two state organizations for the campaigns: the State Fund for Protection and Assistance of Victims of Human Trafficking (ATIP Fund, a USAID-funded implementing partner for this program); and the Information Center on the EU and NATO. In both cases, the SMR turned to PITA to identify campaign promoters among youth and PITA provided them training to reach out to their communities. For the gender equality campaign, PITA trained approximately 40 youth volunteers from YCs. The youth volunteers then formed Gender Equality Task Forces at YCs comprised of 20 mobile groups of young people. The mobile groups reached out to teachers, students, and parents in more than 60 villages in three ethnic minority regions with information about domestic violence and early marriage. The information was delivered not only in the Georgian language but also in ethnic minority languages. The Gender Equality Task Forces in all YCs continue to function with PITA’s support. According to ATIP, calls to the domestic violence hotline have increased 500 percent compared to the previous year, a result the agency attributed to both the partnership with PITA’s youth network as well as increased efforts by the GOG. 2 Through the 1+4 program, ethnic minorities can enter one of five state universities by taking the entrance exam in their native language and then spend one year mastering the Georgian language before going on to enroll in the four-year bachelor’s program. 12 In many of these initiatives, PITA has provided not only financial support, but also served as a full partner in shaping and implementing the initiative and providing expertise, research, and, most importantly, coordination with its network of regional actors, from YCs to ethnic and religious minority organizations. PITA also ensures that the SCEI and annual action plans are translated into the key minority languages. It is important to note that the SMR concerns itself only with integration of ethnic minorities, leaving matters of religious minorities to the State Agency for Religious Affairs (SARA). As noted below, the SARA, which was created by the government in 2014, has generated controversy, and its commitment to genuine tolerance of religious minorities has been called into question by many PITA stakeholders. Tolerance Center, including the Council of National Minorities and Council of Religions The TC is a consultative body and a tolerance watchdog that spearheads the monitoring of religious and ethnic minorities rights in Georgia through its two entities, the CNM and CR. It is housed under the PDO, a national ombudsman responsible for defending the human rights of all Georgian citizens. The CNM unites over 120 ethnic minority organizations, while the CR brings together 24 religious minority bodies. Both engage the GOG in the defense of minority rights, including through the regular meetings between their members and different government agencies. PITA is the sole donor to the TC, CNM, and CR. Through PITA’s financial and technical support, the CNM and CR conduct awareness raising, monitoring, advocacy, and reporting on minority issues, including contributions to the PDO’s annual report to Parliament. As a direct result of the PITA’s financial support and coordination with PITA’s broader youth and NGO network, the TC, CR, and CNM can point to a number of strong policy and program results over the past two years, demonstrating progress toward PITA’s program outcome of advancing the TC’s advocacy capacity for minority rights: • In 2017, the CR achieved a major policy victory, successfully preventing an effort by the Parliament to amend the constitution by imposing limitations on freedom of religion and belief. Through lobbying and a strong public statement signed by 17 CR members, the CR challenged the draft constitutional amendments to restrict freedom of religion on the grounds of national security, prevention of crime, and implementation of justice, forcing Parliament to reconsider these provisions. • Similarly, in 2016, the CR, together with other CSO allies in the PITA-supported No to Phobia platform, managed to prevent the government from adopting legislation on “religious insults,” successfully arguing that such laws could be used against those supporting minority rights and freedom of religion. The draft legislation mirrored a law adopted by Russia several years earlier. • In 2017, with PITA’s support, the CR released a publication detailing a series of policy and program recommendations to ensure freedom of religion, followed by a series of meetings with state institutions responsible for acting on the recommendations. Successful results from its advocacy are already apparent in recent actions taken by several state institutions. • As a result of their recommendations to Parliament, the CNM and CR also engaged in extensive dialogue with the government Georgian Public Broadcast (GPB) service, eventually persuading them to broadcast the religious and ethnic celebrations of minority groups, and to plan for the launch of broadcasting in Armenian and Azeri languages. • As a result of CR consultations with the Ministry of Education and Science (MOES), the Ministry has committed to reviewing school textbooks, identifying intolerant content and deleting it from textbooks, and applying a diversity lens to the approval of new textbooks. Besides these important national policy successes, the CNM and CR member organizations testify that PITA has contributed considerably to strengthening their organizations through a range of mechanisms 13 including: high-quality trainings on such issues as management, communication, and fundraising; small grants to some CNM/CR member organizations for special initiatives; and facilitation of dialogue with both local government and local communities. One CNM member organization that carried out an awareness campaign about domestic violence among Muslim women in the Azerbaijani communities through a small grant from PITA reported that it was able to forge an ongoing relationship with the Union of Georgian Muslims as a direct result of the grant activities. According to PITA’s project data, up to 85 representatives of CNM and CR member organizations have formally interacted with local and central governments under the PITA project. “I don’t have an opportunity to meet with the Minister of Education but can use the CNM mechanism and advocate to address problems of our local community,” remarked one CNM member. And a CR member related, “PITA is supporting our meetings with the Ministries and government, and the direct relationship with decision-makers. It really changes the atmosphere. We are not criticizing them, we are sharing, telling them how we view the situation, and sometimes we get a better understanding of why they do what they do.” At the same time, the TC, CNM, and CR face ongoing structural and political challenges. In 2015, at the beginning of the PITA project, the TC faced a serious crisis when a previous Public Defender sought to restructure the TC in a way that would have stripped it of its essential roles and independence. It took five months of negotiations with the active intervention of both PITA’s staff and USAID to reach an agreement with the Public Defender to respect the independence of the TC. Subsequently, PITA helped both the CNM and CR to establish charters, which set out roles and procedures for each council and protected their independence in setting agendas, making recommendations, and raising criticism. In 2014, the GOG created the SARA under the offices of the Prime Minister, purportedly to promote religious tolerance, and to monitor and investigate issues related to the exercise of religious freedom. However, unlike the SMR, the SARA does not coordinate closely with the CR, its counterpart under the PDO, but instead seems to frequently challenge the CR in its policy positions and representation before government bodies. While the Georgian Orthodox Church has long received substantial government funding—and benefits from privileged legal status—the SARA began distributing government funds to a handful of minority religious organizations. Many key informants expressed the belief that the SARA is heavily dominated by the Georgian Orthodox Church, and that it was deliberately established to counteract the CR’s defense of minority religions, even to “buy” the loyalty of some minority religious organizations, especially Muslim organizations. In fact, the SARA has taken positions contradicting the demands of minority religions, such as opposing the construction of a mosque in the city of Batumi. Through their active intervention as well as through their excellent reputation among policymakers, PITA and USAID have played an important role in bolstering the voice and legitimacy of the TC in the face of political pressure from the SARA and elsewhere. In addition, international human rights bodies that monitor ethnic and religious minorities rights in Georgia regularly rely upon the work of the TC, CNM, and CR, and the statements of these international actors are often invoked to defend the TC’s credibility.3 Together, this external pressure has worked to maintain the TC’s independence, and to hold the GOG accountable for protecting the rights of minority religions in Georgia. PITA’s original plan has been to transfer budgetary and management control of the TC and its two entities to the PDO by the end of the project period. However, nearly all stakeholders associated with the TC expressed strong apprehension about this prospect. They pointed to the high risk to the TC’s ability to represent the interests and priorities of minority communities without political interference if it comes under the direct control and budget of the PDO. In fact, the TC and CR pointed to a specific example where its independence from the PDO was critical: the PDO did not support the CR’s position opposing the draft constitutional amendments that limit religious freedom, although it also did not actively oppose 3 These include: the United States (U.S.) State Department through its annual Human Rights and Religious Freedom reports; the European Commission on Racism and Intolerance (ECRI); the Council of Europe Advisory Committee on the Framework Convention for the Protection of National Minorities; and Human Rights Watch. 14 the position. The TC’s independence gave it the possibility to launch an advocacy campaign, which eventually succeeded in removing the amendments. There is no equivalent for the TC in the current PDO organizational structure. For example, the PDO’s Equality Department takes up individual cases of human rights violations on the basis of sexual orientation or gender identity, but it does not directly represent CSOs advocating for lesbian, gay, bisexual, and transgender (LGBT) rights, nor does it regularly monitor the rights of LGBT persons or work to promote dialogue between government institutions and LGBT organizations. As a result, it is difficult to determine how these critical functions would continue if the TC is folded into the PDO. National- and Local-Level Civil Society Organizations PITA is supporting various national and local CSOs in efforts to advocate with government institutions and political parties to promote tolerance and integration of minorities. Many CSOs are making significant in-roads by influencing national and local decision-makers, complementing the efforts of the SMR and the TC/CNM/CR. For example, the Georgian Democracy Initiative (GDI) with PITA’s support is close to securing a joint memorandum signed by a wide range of political parties on “Reducing the Use of Hate Speech in Political Discourse.” Such an agreement would complement other PITA-supported efforts around media monitoring (see Section 4.3 below) reducing Islamophobic, homophobic, and other discriminatory statements among political figures. Similarly, PITA also supported dialogues between minority religious organizations and national political parties as well as local political candidates running in recent elections, organized by the Tolerance and Diversity Institute (TDI), a national CSO working to defend religious liberty and tolerance. According to TDI, many political party representatives admitted ignorance about the issues and requested continued dialogues. The Media Development Foundation (MDF), which is leading efforts to monitor hate speech among public figures and the media (detailed below), echoed the important role that PITA and USAID lend to their efforts to influence the government: “The moral backing that USAID provides for us and other leading NGOs is critical. The government cares about us because we have the support of the international community,” MDF’s spokesperson remarked. PITA strengthens CSOs’ capacities to actively engage in the promotion of nationwide diversity through 17 small grants for CR/CNM and issue-based CSOs and annual grants and capacity building of Youth Component Partner NGOs. As a result, 14 CSOs in 12 municipalities and surrounding rural areas now have improved institutional capacities, increased quality of services, and targeted constituency-building practices among local youth and other beneficiary communities. These organizations also grow into a unique resource for Tbilisi-based organizations—including government agencies—which implement different outreach or other activities in the regions of Georgia. 4.1.2 Conclusions SMR. PITA’s support and collaboration has helped the SMR sharpen its vision as well as its strategy in the new SCEI, permitting improved priority-setting and focused approaches. The SMR has demonstrated strong political commitment to advancing tolerance toward and integration of ethnic minorities, and it has developed strong working relationships with other government Ministries as well as with PITA staff. In the absence of substantial budget resources of its own for implementation of SCEI, this close collaboration has worked to build greater ownership among some GOG decision-makers, as demonstrated in the internship program for ethnic minority university students and graduates. At the same time, implementation of the SCEI is limited by meager resources and limited political will among some government stakeholders. For example, the action plan identified the internship program for ethnic minorities as the one action toward the goal of greater integration of ethnic minorities into the political life of the country—an action far from adequate to accomplish that goal. 15 With PITA’s direct support and involvement, the SMR’s focus on awareness campaigns marks an important new direction in several respects. It has allowed the SMR to move beyond broad policies and programs to identify specific issues that are heightened in ethnic minority communities. By drawing upon PITA’s regional network of youth, it has also permitted the SMR to move beyond the national level to engage local activists. The gender equality campaign, focused on the issue of domestic violence and child marriage, seems to have struck a particular chord among youth involved in the PITA project. At least two lessons emerge from the experience of the SMR under the PITA project: • Although broad legal and policy initiatives are essential, ethnic minorities experience the government’s commitment to integration mainly through concrete projects that tangibly improve their quality of life—and in the process they deepen their stake in Georgian economic, political, and social life. PITA is well placed to help the SMR identify such projects through direct consultation with minority communities. • It is often easier to raise issues with ethnic minority communities not through the government but through local citizens, who have the language and trust of the community and can raise sensitive issues about values and culture. Therefore, the SMR’s collaboration with youth through PITA has been instrumental in both raising awareness and conducting dialogues with minority communities. TC/CNM/CR. Key informants report that, over time, the TC/CNM/CR have deepened their credibility and legitimacy with government decision-makers as unique voices for ethnic and religious minorities. Additional evidence for this conclusion lies in the: growing number of ethnic minority organizations that want to join the CNM; central role the CNM and CR play in elaboration of the annual action plans for the state strategy on civic integration; growing interest in the CR and CNM monitoring reports and in their recommendations presented in the PDO’s annual report to the Parliament; and strong endorsement of international human rights organizations. Despite their rising reputation, the TC/CNM/CR remain highly dependent on the good will of the PDO, which itself comes under periodic political pressure. PITA and USAID representatives have played a critical role as arbiters between the TC and the PDO, and their influence has been instrumental in guaranteeing the continuation of the TC’s work. The TC and their members clearly hope to rely on their support if similar crises arise in the future. At present, according to key informants, the GOG does not seem to have demonstrated sufficient commitment to upholding ethnic and religious minority rights to ensure that the TC’s current efforts can be sustained if USAID and PITA were to withdraw their support. PITA’s plans to transfer the TC/CR/CNM to the PDO’s direct management and budget control do indeed pose risks to the voice and independence that the TC has exercised responsibly for more than a decade. Although the CNM and CR have recently established charters with PITA’s assistance that define their roles and responsibilities independent of the PDO, the charters are not legally binding documents, and there is nothing to prevent a Public Defender from replacing them. Furthermore, it is unclear whether the CNM and CR would be able to continue their current roles as representative bodies for non-state minority organizations under the PDO’s control, or whether they would be restricted to the more limited function investigating individual cases of discrimination through the PDO’s Equality Department. 4.1.3 Recommendations Building on its success with initiatives such as the internship program, PITA, in collaboration with the SMR, should continue to pilot initiatives that both meet the immediate needs and interests of ethnic minority groups but also gain the support of government institutions, especially those that have been less active in the implementation of SCEI. To the extent possible, PITA should help the SMR identify “low hanging fruit”—that is, programs that have a high likelihood of success as well as the possibility of ownership by 16 the GOG. The SMR’s current plans to work with the Ministry of Finance to organize small business trainings for ethnic minority women is an example of a targeted program that could positively work to the benefit of both ethnic minorities and the GOG. PITA should also continue to collaborate with the SMR to expand current efforts to launch national awareness-raising campaigns, which are particularly important in light of the Russian-backed disinformation and propaganda that tend to be more prominent in ethnic minority regions. Building upon the particular success of the gender equality campaigns, PITA could assist the SMR in consulting with ethnic minority groups to identify issues that are directly relevant to the daily lives of ethnic minorities and for which greater public awareness is needed. PITA should rethink the plan to transfer the TC to the budget and management control of the PDO at the end of the project period. Instead, it should study alternative organizational models that would maximize its independence, allow direct channels to government decision-making, and gain sustainable financial support. Any model should conform to the international standards for national human rights organizations set out in the Paris Principles.4 Alternative options for the future of the TC/CNM/CR institutional arrangement carry different costs and benefits. Below are several options that the PITA project might consider: • Maintain the status quo, by continuing to support the TC staff and activities through USAID or other donor support, and actively use this position to negotiate for maximum independence of the TC and to allow it to continue its monitoring, reporting, and advocacy functions. Although this option removes the prospect of financial sustainability for the TC in the medium term, it would safeguard the TC from direct political interference, and would ensure that its activities to represent minority communities and engage and influence government continue unabated. • Partially transfer the TC to the PDO, by negotiating with the PDO to cover the costs of activities such as monitoring missions and regional dialogues with CNM and CR members, while donors continue supporting the salaries and expenses of TC staff. This partial solution at least would help safeguard the roles of staff as independent spokespersons who represent the interests of minority communities directly with the government, although the scope of the TC’s primary monitoring and dialogue activities might still be at risk under the direct control of the PDO. • Transition the TC outside of government to civil society. This option would certainly ensure independence from direct political pressure, although the TC would likely continue to depend on donor support, foregoing financial sustainability. However, the TC would sacrifice its direct channels to engaging government decision-makers, such as contributing findings and recommendations in the PDO’s annual report to Parliament and participating in periodic direct dialogues with government officials. Indeed, the CR would cede its primary role of defending freedom of religion within the government to the SARA, which has not shown itself to be responsive or sensitive to the interests of minority religions. Meanwhile, if there is a strong preference for the TC to remain within the GOG, then the TC, in close coordination with PITA’s staff, USAID, and other international partners, might pursue dialogue with supporters within the Georgian Parliament about establishing a legislative mandate for the TC, building upon current institutional charters for the CNM and CR. A legislative mandate would give the TC greater permanence and enshrine its independence into law. 4 A paper discussing various organizational models for national human rights organizations and the Paris Principles can be found at http://www.ohchr.org/Documents/Publications/PTS-4Rev1-NHRI_en.pdf 17 4.2 EQ 2: WHAT IS PITA’S EFFECTIVENESS IN BUILDING A SENSE OF NATIONAL UNITY AMONG YOUTH? 4.2.2 Findings Youth Center Membership and Participation According to PITA’s database, membership across the 14 YCs has more than doubled since the start of the PITA project, from 1,488 in 2015 to 3,277 today. Of these, 2,092 members fall within the primary target group of youth 18-25 years old. With the PITA project almost at the halfway mark, the total membership is more than 80 percent of the life-of-project target of 4,000 youth overall; and membership surpasses the mid-point target of 2,000 for youth 18-25 years old. However, it is important to remember that, by design, these figures represent a cumulative number over time, with some previous YC members who are no longer active included in the overall count. Several YC coordinators interviewed reported that, although in the past they had conducted outreach to build the membership of their centers, now the membership was building itself through the positive image of the centers in the communities and youth members’ outreach to friends and peers. However, YCs in smaller cities and remote regions far from universities, such as Ninotsminda, report that considerable out-migration of youth aged 18 years and older to urban centers, in pursuit of higher education or employment, limits the participation of youth in the 18-25 age range, particularly male youth. It is easier to attract older youth to the centers located in larger urban areas where universities are within close reach, such as Tbilisi, Marneuli, Kutaisi, Batumi, Gori, Akhaltsikhe, and Telavi. In addition, although the Centers have grown in popularity among youth in remote regions, given the general scarcity of other organized activities, youth often face difficulty in accessing transportation that would allow them to participate more frequently. Through FGDs, youth and their parents in remote regions surrounding Marneuli and Ninotsminda also pointed to some social stigma in their communities when girls returned home late from the YCs or were allowed to travel alone to larger cities such as Tbilisi to take part in workshops, sometimes discouraging girls from more active participation. Moreover, youth in densely settled ethnic minority regions often lack the Georgian language skills that allow them to participate more actively in PITA activities that bring together youth from across the country, such as the thematic schools and exchange programs. This reinforces their isolation even within the PITA program. In general, however, the shift in PITA’s target youth to the 18-25 age group, away from the 14-18 age group under ANI, does not appear to have carried significant consequences for the program. In fact, most YC coordinators interviewed do not see the new target age range as a shift but rather as an expansion to include older youth. The YCs established under ANI continue working with high school students and do not exclude interested youth who are under age of 25. The inclusion of older youth has enabled some who were active in the YCs under ANI to continue their involvement under PITA, and to even further strengthen their skills, leadership, and activism. Some serve as facilitators, tutors, and role models for younger peers in the YCs, while others have gone on to use their skills as recipients of PITA’s small grants, becoming agents of social change within their own communities and peer groups. Disaggregation of YC membership data by gender as well as by ethnicity reveals some interesting patterns. First, overall, girls outnumber boys by a 3:2 ratio, and the ratio has slightly increased over the project period. Not all YCs demonstrate a gender imbalance in favor of girls; some report higher numbers of boys. However, some YC coordinators reported that they had made specific efforts to recruit boys in response to the uneven ratio. Second, the Georgian majority members represent more than half of all members, with their share of overall membership growing from 51 percent in the first year to 55 percent in the third year of the PITA project. Although the share of Azeri members has stayed steady at 19 percent of the growing membership 18 ranks, the share of Armenian members has declined from 30 percent to 25 percent over the project period. Figure 2: Number of Youth Center Members by Gender and Year Figure 3: Number of Youth Center Members, by Ethnicity and Year PITA program data also confirm what almost all YC coordinators told the ET in interviews: girls tend to be more motivated and engaged than boys in YC activities and therefore tend to be selected more often for opportunities open to youth who demonstrate leadership. A breakdown of the share of girls and boys participating in three YC activities—training on diversity and tolerance, microgrants, and internships— shows that girls consistently represent a higher proportion (see Figure 4). Figure 4: Annual Relative Participation in Youth Center Activities by Gender and Year 864 1757 2002 624 1151 1275 0 500 1000 1500 2000 2500 Number of members Year 1 Year 2 Year 3 Year Female Male 754 1562 1808 445 773 806 278 545 616 11 28 47 0 500 1000 1500 2000 Year 1 Year 2 Year 3 Number of members Ethnicity Georgian Armenian Azeri Other 51 49 61 39 61 39 63 37 68 32 56 44 63 37 56 44 0 20 40 60 80 Female Male Female Male Female Male "Diversity, tolerane and innovation" participation Micro grant recipients Internship beneficiaries Percentage YC activities Year 1 Year 2 Year 3 19 However, YC coordinators also said that boys who became active members often became very committed to the program, taking up the activities and campaigns with enthusiasm. The survey of 327 youth demonstrates that there is considerable circulation of new youth members into the YCs. About a quarter of members indicated that they had been involved in YCs less than one year, 35 percent from 1 to 2 years, 32 percent from 2 to 4 years, and 5 percent over 4 years. As expected, older youth in general tended to have longer tenure in the YCs than younger youth (see Figure 5). However, the ET was unable to obtain data on the frequency of individual youth participation in many of the key YC activities to examine whether there is considerable turnover among the most active leaders. Figure 5: Years of Youth Participation in Youth Centers, by Age Group 1+4 Program and PITA Support The 1+4 program, launched under the Civic Integration strategy in 2009, has been a centerpiece of efforts to integrate ethnic minorities into mainstream Georgian society. The program was designed to address the problem that many ethnic minority students lacked the Georgian language skills to pass the university entrance exam and to successfully complete higher education in Georgia. Through the 1+4 program, ethnic minorities can enter one of five state universities by taking the entrance exam in their native language (Azerbaijani, Armenian, Abkhazian, or Ossetian), and then spend one year mastering the Georgian language before going on to enroll in the four-year bachelor’s program. These ethnic minority students were also allocated a 12 percent quota among all students admitted to the five state universities. Since the affirmative action program was launched in 2009, ethnic minority enrollment in Georgian universities has grown ten-fold, from 72 in 2008-2009 to 729 in the current academic year. Nevertheless, the 1+4 program faces challenges. It still falls considerably short of its potential, as the quotas remain unfilled. Research by the Center for Civil Integration and Inter-Ethnic Relations (CCIIR) published in 2016 and supported by PITA found that the quotas reserved for ethnic minority students were only 19 percent filled. This result is likely due to lower academic performance among ethnic minority students even in their own language as well as insufficient scholarship funds used to support ethnic minority students under the quota system. Furthermore, the same research found that only 15 percent of students who began the program in 2010 completed it within the allotted five years. The 1+4 program was slated to expire next year, following a 10-year run. However, the relative success of the 1+4 program and high interest among ethnic minority students suggest that MOES will likely decide to continue the flagship program into the future. The PITA project does not directly finance the 1+4 program. Yet, partly in response to the problems identified through CCIIR research, it has provided crucial support to ethnic minority students enrolled in the program over the past three years. 9 15 11 24 6 26 5 0 5 10 15 20 25 30 14-17 years old 18-25 years old Percentage Age up to 1 year 1 to 2 years 2 to 4 years 4+ years 20 In 2015, PITA launched an initiative to assist 1+4 students, by replicating a successful Georgian-language tutor program piloted at Ilia University and establishing YCs at both Ilia University and Tbilisi State University (TSU). In addition to tutoring services, the YCs provide a range of educational and social opportunities, from assistance in finding housing in Tbilisi to cultural excursions to organized volunteer initiatives. Various key informants and FGDs with students themselves indicate that this support was very enriching for ethnic minority students, facilitating not only their academic success but also their socialization into university life. FGDs with tutors also showed that experience was also very enriching for them, and they suggested greater recruitment of tutors for the program and more training for tutors. Just-completed research on the 1+4 program carried out by the Center for Training and Consultancy with PITA’s support found that 39 percent of ethnic minority students enrolled in the 1+4 program at TSU and Ilia University were engaged with the PITA-supported YC activities. Moreover, a survey of these youth cited a range of positive benefits from the program: • Ninety-four percent of students reported that involvement with the YCs increased their number of friends from different ethnic and religious backgrounds. • More than 80 percent of respondents said it improved their knowledge of the Georgian language, their communication skills, and also increased their motivation to learn. • More than 70 percent of respondents said that it helped them become more active citizens, made them more aware of the issues confronting the country, and also made it easier to confront social problems and daily routines. Youth Attitudes and Opinions: Survey Findings Survey respondents were asked to rate on a scale of 1-5 their agreement or disagreement with a series of statements about respect for the rights of ethnic and religious minorities, and relations between minorities and majorities. These same questions were also asked of ANI survey respondents four years earlier. The results show that respondents to the PITA survey are in some cases less inclined than ANI respondents to agree with statements reflecting respect for ethnic and religious rights in Georgia— especially Georgian respondents. PITA’s survey respondents also stated more frequently that ethnic and religious differences are the source of lack of understanding among different groups. Results were statistically significant (e.g., 95 percent certainty that the results exist in among all PITA and ANI youth); controlling for other characteristics, the most persistent result was project effect (PITA-ANI). Nevertheless, some results diverge when disaggregated by majority and minority status (see Figures 6-10 below). Reflections from Tutors “I knew how non-Georgian speakers struggled so I decided to help.” - Azeri male tutor, Ilia University “A tutor is not a teacher. A tutor is a friend who helps in integration, not only at university but outside as well…Our Muslim friends went to the church, we also [went] to mosque.” -- Georgian female tutor, Ilia University “I’m from Azerbaijan, before I didn’t have any communication with Armenians. I perceived them as enemies. When I got here, I became really good friends with Armenians.” - Azeri female tutor, TSU “It gave me the feel of responsibility – I was something for someone.” - Georgian female tutor, Ilia University “I can honestly say that with this program, I became more tolerant. I never thought about having Armenian and Azerbaijani friends, and now I am friends with them. For my bachelor’s degree I wrote about ethnic minorities, and they all helped me very much.” - Georgian female tutor, Ilia University 21 For example, between the ANI and PITA project periods, the share of ethnic Georgians disagreeing or totally disagreeing with the statement that “ethnic Georgians and ethnic minorities are not friendly” declined by 5 percent (although the share agreeing with the statement also declined by 2 percent). However, the share of minorities disagreeing or totally disagreeing with the statement jumped by 15 percent, while those agreeing or totally agreeing with it dropped by 20 percent (Figure 6). Figure 6: Ethnic Georgians and Ethnic Minorities Are Not Friendly With Each Other In addition, the share of ethnic Georgians and ethnic minorities disagreeing or totally disagreeing with the statement that “people of different religions cannot understand each other” declined by 26 and 28 percent, respectively. The share of Georgians agreeing or totally agreeing with the statement climbed by almost 12 percent, although the corresponding share of ethnic minorities declined by 9 percent (Figure 7). Overall, this shows deterioration in perceptions of the relations between people of majority-minority religions. Figure 7: People of Different Religions Cannot Understand Each Other PITA’s youth survey respondents were also asked about their ability to communicate with people of different sexual identity, different religious background, and different ethnic background. The results show that, while a majority of respondents feel able to communicate with people of different ethnicity or religion, the share drops noticeably for people of different sexual identity. [Note: although the term “sexual identity” was used in English (ს ე ქ ს უა ლური ი დე ნ ტობ ა in Georgian), Georgian and Armenian speakers confirmed that the translation in those languages would have been interpreted only as LGBT, and not also as people of the opposite sex.] 38 11 25 30 8 32 16 28 30 30 32 33 16 20 14 5 8 7 9 2 0 20 40 60 80 100 120 Majority Minority Majority Minority ANI PITA Percentage Majority minority group 5 - totally agree 4 3 2 1- totally disagree 72 32 32 12 6 30 19 22 15 14 25 49 6 5 10 2 2 3 7 0 0 18 6 11 0 10 20 30 40 50 60 70 80 Majority Minority Majority Minority ANI PITA Percentage Majority minority category 1 - totally disagree 2 3 4 5 - totally agree Don’t know 22 Figure 8: Perception of Ease in Communicating with Different Groups Further analysis of survey data shows that youth’s perception of their ability to communicate with different groups increases with their status as active members of YCs (versus “inactive,” or those who were previously active but are no longer active) as well as the duration of their membership. For example: • For every additional year in YC membership, youth are more than 1.2 times more likely to be comfortable communicating with people with different religion. Compared to youth inactive in PITA’s YCs, active youth are 1.8 times more likely to be comfortable to communicate with people with different religion. • For every two additional years in youth membership, youth become over 1.3 times more likely to be comfortable communicating with people of different ethnic background. And, compared to youth inactive in PITA’s YCs, active youth are 1.7 times more likely to be comfortable communicating with people from different ethnic background. In addition, as respondents move from a lower age category (14-17 years old) to a higher one (18-25 years old), they are 1.6 times more likely to be comfortable communicating with people from different ethnic background. • Interestingly, however, there are more striking differences in responses for people of different sexual identity. Compared to girls, boys are 57 percent less likely to be comfortable communicating with people of different sexual identity. Compared to youth inactive in PITA’s YCs, active youth are two times more likely to be comfortable communicating with people of different sexual identity. Analysis of survey data also found that Muslims and followers of the Armenian Church are 40 percent less likely to be comfortable communicating with people of different sexual identity than Orthodox Christians. When asked whether minority issues were addressed by the government, results showed a decline in ratings among both majority and minority respondents between the ANI and PITA project periods (Figures 9 and 10). Interestingly, however, results showed a sharp decline in ratings among majority respondents between the ANI and PITA project periods, but a very small decline in ratings among minority respondents. 33 11 26 12 17 58 16 20 2 5 57 17 18 3 5 0 10 20 30 40 50 60 70 5 -I can communicate 4 3 2 1 - I can't communicate Percentage Minority groups People with different sexual identity People with different religious background People with different ethnic background 23 Figure 9: Perceptions that Ethnic Minority Issues Are Addressed by the Government Figure 10: Perceptions that Religious Minority Rights Are Addressed by the Government Finally, in both ANI and PITA projects, there was a general consensus across all ethnic groups that every citizen must have knowledge of the state Georgian language. When asked the reasons for their answer, respondents cited “to feel like a real citizen of the country” as the most important reason in the PITA survey, in contrast to the ANI survey where respondents cited “communication with ethnic Georgians” and “participation in society” as the most important reasons. Overall, a higher share of PITA than ANI respondents said that it is useful for education, employment, public services, or trade. However, most of these gains for education, employment, and trade seem to come from the ethnic Georgian respondents, while the gain in public services came largely from both ethnic Azeri and Georgian respondents. There is a notable difference between Armenian and Azeri youth in their statements about the importance of knowing the Georgian language. And from the ANI to the PITA projects, there are large declines in the share of Armenian youth who find value in knowing Georgian for the purposes of education or 5 0 15 4 0 0 0 5 37 39 40 44 32 27 16 28 18 14 11 11 1 11 8 7 0 5 10 15 20 25 30 35 40 45 50 Majority Minority Majority Minority ANI PITA Percentage Majority/minority status, by ANI and PITA project 1 - totally disagree 2 3 4 5 - totally agree Don’t know 9 11 12 5 5 14 17 9 17 30 31 42 20 20 22 21 49 11 12 11 0 14 0 1 0 10 20 30 40 50 60 Majority Minority Majority Minority ANI PITA Percentage Majority/minority status, by PITA and ANI project 1 - totally disagree 2 3 4 5 - totally agree Don’t know 24 communicating with ethnic Georgians. Among Armenian youth, gains in knowing Georgian to access information, public services, or trade are insignificant. Yet, in contrast to Armenian respondents to the ANI survey, Armenian respondents to the PITA survey cited “to feel like a citizen of the country” as the top reason (Figure 11). Figure 11: Georgian Language Important for Ethnic Minorities, Armenian Respondents YC Benefits for Youth Through FGDs as well as interviews, many youth expressed strong appreciation for the personal advancement opportunities afforded by the YCs, ranging from academic support to information and guidance about scholarships or academic exchange programs to career development opportunities. Indeed, these individual advancement opportunities, along with social interaction with peers, often seemed to be the major factors attracting youth to the centers—as well as securing the support of their parents—more than a specific interest in diversity or inclusion across ethnic and religious lines. However, in the process, students eventually come to discover and embrace the values of diversity, multiculturalism, and tolerance almost as a byproduct of the personal advancement. Interviews as well as survey results indicate that the YCs are indeed succeeding in building bridges between youth of diverse ethnic and religious backgrounds. The integration appears to be working in 0 10 20 30 40 50 60 70 80 90 For communication with ethnic Georgian… Education To feel like a real citizen of the country Participate in society Employment To access information To receive public service Trade Health care Percentage Armenian ANI Armenian PITA Integration and Personal Transformation Through YCs “One young girl made xenophobic statements, however she managed to challenge herself and made good friends with minorities from the Azerbaijani and Armenian communities. Now she is a prominent defender of minority rights.” UNAG Coordinator, Kutaisi “We reached a point where parents want their kids to attend the schools that offer Georgian language. Eighty percent of youth in the Georgian￾language schools in this region are Azerbaijani.” Marneuli YC Coordinator “The younger generation [of ethnic minorities] sees their life here in Georgia, as opposed to their parents, who still feel excluded.” Ilia University YC Coordinator “Until recently youth had requested exchanges with youth in other Georgian majority towns. But lately, they have been demanding more exchanges with minority towns. They have established good connections and have mutual interest in one another.” UNAG Coordinator, Kutaisi “One young man who is a member of the YC didn’t understand why he would want to learn about gender equality. Now he calls himself a feminist, and he is very openly discussing the challenges and the violence in his family.” Kutaisi YC Coordinator 25 both directions, with ethnic minorities feeling a greater sense of belonging to Georgian society, and ethnic Georgian youth embracing their relationships with ethnic minorities. YC coordinators and youth members often pointed to the opportunities to engage in exchange programs among youth in different regions and from different ethnic, cultural, and religious backgrounds as the most valuable PITA activity and, almost without exception, they called for an increase in the frequency of exchange programs, to multiple times a year. Many lamented that the exchange programs and summer camps, which served a similar purpose, had become less frequent under PITA than under ANI. At the same time, YCs in more remote, isolated regions that are dominated by a single ethnic group, such as Ninotsminda, report less progress toward integration due to fewer opportunities for youth, distance from urban settlements, the expense of transportation, and lack of Georgian language skills. This result also corresponds with survey findings. For example, across eight locations where youth were surveyed, cell phone Internet access ranges from 92 to 100 percent, except for Ninotsminda, where 38 percent do not have cell phone Internet access. Disaggregation of the same responses by ethnicity showed that 18 percent of Armenian respondents say that they have no cell phone Internet access. In addition, Figure 12 below shows youth’s perception of the main barriers for interaction with peers in other regions. According to survey data, nearly 80 percent of Armenian youth perceive language as the most significant barrier, and more than half of Azeri youth point to transportation as a key barrier. Significantly, however, ethnic and religious differences are perceived as the lowest barrier for interaction with peers. Figure 12: Main Barriers for Interaction of Youth with Peers in Other Regions Some YC coordinators also prioritize the inclusion of youth who are marginalized in other ways—for example, youth with disabilities, LGBT youth, or youth in conflict with the law. In Kutaisi, for example, a social worker with the National Probation Agency reported that since 2016 the agency has been making participation in Youth Centers a condition of probation for many youth who have had brushes with the law, with their consent. “We know that youth that we refer to [YCs] will find themselves in safe and accepting environment, with kids who serve as tutors and role models. We also had a case of LGBT youth who underwent a successful integration process with help of the Youth Center,” she said. The YC Coordinator for Kutaisi related anecdotes of remarkable rehabilitation for some youth referred by the Probation Agency. 28 35 53 18 78 45 30 10 26 56 9 29 14 1 18 14 0 13 0 20 40 60 80 100 Georgian Armenian Azeri Percentage Roads/transportation Language Youth attitudes toward their peers in other regions Youth do not find it important Ethnic origin Religious belonging 26 The ET was not able to measure the extent of the multiplier effects by YC youth on their peer groups and larger communities. However, according to YC coordinators, many youth are quite active on social media, sharing their experiences in the centers with a wider circle of friends and acquaintances through posts, tags, and discussions. YC members’ influence on their peers is credited with growing the YC membership, and many YC activities involve peer-to-peer education and training. Youth also testified about the impact on their parents. “The youth have an impact on their parents as well. They make their parents change their minds and support them,” remarked one YC member in Kutaisi. Parents themselves confirmed this statement through FGDs in Marneuli, Ninotsminda, and Gori. In addition to the multiple improvements they saw in their children’s self-confidence, motivation, communication, and social skills, parents also applauded the way that youth were overcoming the limitations of their isolated communities. Several noted that they overcame initial skepticism to become strong boosters of the YC. Youth Activism and Leadership Survey respondents were asked to define civic activism in an open-ended question. Notably, nearly a quarter of respondents answered, “I don’t know,” although this is a significant decline from the 50 percent share that responded the same way in the ANI survey four years ago. Yet 14 percent of PITA survey respondents also gave an unclear response to the question, an increase from 8 percent of ANI respondents. For the remainder, answers that mentioned being active in political and social life, involvement in issues and activities, and resolving issues figured larger in the responses, as seen in the word cloud, next page (Figure 13). Many YC coordinators report that the centers have established a strong reputation in local communities for providing quality programs for youth and for fostering a sense of civic responsibility, an assertion echoed by several external stakeholders. In many cases, the youth themselves initiate projects to solve local community problems—for example, installing street lights in dark areas, establishing safety measures such as crosswalks for pedestrians or ramps for people with disabilities, placing trash bins or recycling containers around the city to discourage littering and waste, and charitable drives for disadvantaged groups. Such initiatives are more significant in a societal context where volunteerism is still uncommon. As a result, local governments are increasingly seeking out partnerships with YCs on civic improvement projects and local businesses are asking for their referrals for employment for youth. Youth are not only launching their own initiatives, but they are also providing grassroots support for local as well as national initiatives spearheaded by state institutions as well as national NGOs. As mentioned above, as the MDF, the ATIP Fund, and the Center for EU/NATO Integration have trained youth to monitor local media and to conduct local awareness campaigns on issues such as domestic violence, early marriage, and the promotion of Western values of democracy and freedom. In this way, youth are helping to build a broader constituency for ideas and values that uphold human rights and democracy. Reflections from Parents Parents of youth in Marneuli YC: “[Youth] have found their place in society. YC activities helped overcome barriers; it’s not simple to become integrated into the city from the small settlements. Now those youth are integrated.” “The youth helped overcome the old mentality [in our community]. People see active youth and begin encouraging their children to get an education. They don’t let their children get married at an early age. The community became more informed from the youth.” Parent of youth in Ninotsminda YC: “In the past, we were shy, and we didn’t discuss such topics [as early marriage]. My daughter is 18 and not only does she know everything, but she feels comfortable to discuss it with us, ask questions.” Parent of youth in Gori YC: “[My daughter] started to view things differently. My attitudes were also changed. We are a different generation. This generation is more educated, more open.” 27 Figure 13: Youth Define “Civic Activism” However, in a few cases, youth ran into roadblocks. YC coordinators and PITA-supported youth NGOs in Marneuli, Ninotsminda, and Gori related problems gaining the cooperation of local high school authorities in order for youth to conduct education and awareness campaigns with students. They pointed to various reasons, from the reluctance of school directors to take responsibility for managing activities outside of the school curriculum to school directors’ disagreement with some of the campaign messages. Such restrictions limit the opportunities for youth to reach peers who for various reasons do not participate actively in YC activities. By contrast, youth in Kutaisi related the story of a high school director who was initially suspicious of YC activities. However, after attending a YC event, he began contacting the Center to ask them to present more educational events for his high school students. Youth are also having a direct impact on local and national decision-making. For example, YC members at TSU conducted a survey of internally displaced people (IDP) in Gori, analyzed the results, and presented their findings and recommendations directly to the State Minister of IDP who responded that he had been unaware of many of the problems. He eventually addressed 8 of the 12 problems raised by the students. In Kutaisi, both the YC Coordinator and a youth member sit on a city council established by the mayor and are directly involved in decision-making. The ET heard numerous stories of youth alumni from the YCs going on to become strong leaders in a variety of sectors. For example, in Marneuli, an alumnus from the YC program is now a chief of staff for a local government office. He returns to the YC periodically to consult with youth on how the local government can help them. Another alumnus from the Gori YC has gone on to work with the Peace Foundation, a policy think tank that focuses on the peaceful resolution of conflict. In this role, she has organized a public discussion on the most prevalent phobias in Georgian society. She is also a member of South Caucasus Youth Parliament. Another active member of the Kutaisi YC inspired many youth around him when he was selected to participate in the Open World Leadership Program, an exchange program organized by the United States (U.S.) Congress that brings together current and future leaders from the U.S. and from countries of the former Soviet Union. 28 Internship Program As mentioned above, the internship program piloted by the SMR in collaboration with the PITA project has proven so popular and successful that in 2017 the Georgian government adopted it as an ongoing program—a major policy achievement for the PITA project. So far, 65 ethnic minority youth have completed three-month internships with 21 state institutions, of which 37 were girls, 28 were boys, 33 were Azerbaijani, and 32 were Armenian. Approximately 10 obtained full-time employment following the internship, while many of the remainder reportedly continued on with graduate studies. One MOES official who supervised several interns through the program praised the special value an Armenian intern brought his department: “She was gradually given more responsible assignments and also made several presentations about higher education institutions in Georgia, Armenia, Azerbaijan, and Estonia. She was able to draw upon information in the Armenian language, which was not accessible to Georgians,” he remarked. In FGDs, some interns themselves spoke about the striking effect that the program had on their self￾concept. “It was a different feeling when you go to work and do your job like one of Georgians,” remarked one former intern who is an ethnic minority. Some reflected on the practical experience gained and the professional relationships built. “I was like one of the employees, I was involved in everything,” the former intern said. Others, by contrast, did not have such positive experience. “It was all theory, no practice. It could be better,” remarked one intern, a sentiment echoed by several others. They seemed to agree that the experience depended largely on the direct supervisor, and his/her efforts in planning for and guiding the intern. Some expressed disappointment about the lack of internship opportunities at state institutions addressing issues closer to their field of study. So far, the internship program has placed university students or graduates only in some state institutions and has not extended to the private sector or CSOs. Diversified Funding A handful of YCs in larger urban areas have demonstrated some capacity for attracting diversified funding. For example, the YC in Marneuli benefits from very generous support from Socar, the state-owned oil and gas company of Azerbaijan, which has extensive ties with Georgia’s Azeri community. The YC in Kutaisi has received financial support from the EU and has also forged a partnership with the local McDonald’s, which has provided food for several of the center’s events. Youth in many of the YCs have received training on developing project proposals and are putting these skills to good use. Notably, the two YCs at TSU and Ilia University receive financial support from several other donor organizations (the EU, Open Society Foundation, and an international NGO were cited) as well as direct support from the universities. Furthermore, YC coordinators in several regions report that parents are very willing to provide in-kind contributions for the youth exchange program by providing accommodation, food, and transportation for visiting youth. 4.2.2 Conclusions By all accounts, the YC activities are having a strongly positive effect on many youth, contributing toward strong intercultural friendships, transforming stereotypes and attitudes, and increasing gender sensitivity. The exchange programs appear to be one of the most effective PITA activities contributing to this outcome—and one of the most highly demanded by youth. Furthermore, the activities appear to be producing many motivated, engaged future leaders, by not only providing them with skills but also linking them with opportunities to shape their own lives and the lives of their communities. Indeed, many have already made their mark on their own communities and even on national discourse and debates through their activism and advocacy. The numbers of youth participating in YCs seem to be growing at a steady rate, with almost 70 percent of youth actively participating between one and four years—an indication that they find their experience meaningful. The gender imbalance favoring girls in PITA’s overall membership can likely be explained by several factors. First, in some cultural contexts, especially in ethnic minority communities, it is less socially 29 acceptable for adolescent girls than for boys to engage in social activities outside the home, and the YCs may offer one of the limited opportunities available to girls in a safe, supportive environment. Second, with the shift in PITA’s target age to youth ages 18-25, male youth who are not bound for the university might feel more pressure than female youth of the same age to generate income and have less time for non-formal and volunteer activities. Third, girls’ greater performance under PITA is consistent with a general pattern of girls’ greater educational attainment rates and higher scores on standardized exams, according to national data supplied to the ET by PITA staff. Thus, girls’ greater leadership and activism in YCs can be explained and, to an important extent, lauded since they signal growing opportunities for girls to exercise voice and leadership in a society that is still male-dominated in many sectors. At the same time, the lesser involvement of boys and young men gives reason for mild concern. Throughout the world, disaffected young men often lead movements of intolerance, discrimination, and even violence against minority groups, with some susceptible to extremist ideologies. Indeed, a review of the PITA-supported No to Phobia campaign’s monitoring efforts reveals that the large majority of those promulgating hate speech are male. This suggests the importance of cultivating greater male involvement in PITA-supported youth program. Survey results show active participation in YCs and longer duration of involvement in YCs make youth more likely to be able to communicate with different minority groups, and be more open toward people of different ethnicity, religion, or sexual identity. On the other hand, once youth remove themselves from the YC settings in which openness and tolerance are promoted, their ability to communicate with different groups deteriorates somewhat. In addition, the deterioration in youth’s assessment of the situation of minority rights and the relations between minorities since the ANI project period was a surprising finding of the survey. There are several possible explanations for this trend: • Events in the recent period have heightened tensions, among religious groups in particular, leading to less trust between majority and minorities. • The change in age group with PITA might also have contributed to change in attitudes and perceptions, because older youth are more attentive to current events and also more capable of critical thinking than younger teenagers. • PITA participants are more aware and knowledgeable about minority rights (as result of their participation in PITA and even ANI) than ANI participants were—and therefore are more critical of the current state of minority rights in Georgia. However, when survey results are disaggregated between ethnic majority and minorities, minorities often tend to have a more positive view than the majority. Furthermore, the barriers of transportation, language, and social stigma some youth face limiting participation in many of the more remote settlements need to be addressed. This is especially the case given that many of remote settlements are in ethnic minority regions, where integration of minorities into the life of Georgia is most needed. Similarly, Armenian youth are a declining share of YC membership. They also show less integration, less access to Internet by cell phone, and less belief in the value in knowing Georgian for the purposes of education or communicating with ethnic Georgians. These are worrisome trends for PITA. On the other hand, the fact that youth do not find religion or ethnicity to be a major barrier to connection with youth in other regions and that they consider mastery of Georgian language essential to “feel like a citizen of the country” shows a sense of national unity among majority and minority youth. Regarding financial sustainability, a number of YCs have attracted funding from other donors or from local partnerships with businesses. Such diversification of funding sources is encouraging and raises the 30 possibility that PITA could reduce or even terminate its financial support to some YCs after nurturing their development for several years, freeing financial resources for other purposes. 4.2.3 Recommendations The ET suggests that PITA’s project management consider the following recommendations for shifts in directions in the remaining project time period: 1. Explore launching YCs and tutoring programs at the three other state universities hosting the 1+4 program: Georgian Technical University, Tbilisi State Medical University, and Samtskhe-Javakheti State University. Students, faculty, and tutors at TSU and Ilia University could share their best practices and lessons learned, including their success in attracting funding from other donors. Furthermore, at all universities hosting the program, more should be done to spread the word about tutoring and the YC program among both ethnic minorities and Georgian students to recruit more participants. 2. Explore the expansion of the successful internship program for ethnic minority youth into more state institutions as well as NGOs and the private sector, possibly starting with businesses that have already demonstrated support for YCs or that PITA has engaged in diversity training. 3. Engage USAID and appropriate officials in the SMR as well as the MOES, requesting that the latter issue guidance to school districts allowing greater access to high school students by YCs and youth-affiliated NGOs. 4. Shift more attention and program resources to YCs in more remote regions, especially Armenian ethnic minority regions, addressing barriers to participation such as transportation to YCs, lack of Georgian language skills necessary for cross-regional program activities, and the social stigma that some youth (especially girls) face in participation. Also, PITA should specifically explore launching a YC in the Pankisi Valley, a region that several key informants identified as vulnerable to extremist ideologies. 5. Increase the frequency of the exchange program or pursue other approaches that allow youth from diverse regions of the country to build bonds across ethnic, religious, and regional lines, such as summer and winter camps. Allow some youth to participate in such programs not only individually but also in small groups, which might help youth in some regions confront the stigma in their communities around traveling alone outside of their villages. 6. Continue efforts to recruit more boys and young men to YC activities, perhaps enlisting the help of more committed male YC members and those who have served as role models. 7. Model those YCs that have attracted alternative sources of funding from the private sector and other donor institutions, in order to encourage other YCs to seek diversified financial support and to become more financially sustainable. PITA might look to reduce or even eventually eliminate its support for some YCs. 4.3 EQ 3: WHAT IS PITA’S EFFECTIVENESS IN BUILDING PUBLIC AWARENESS AND EDUCATION? 4.3.1 Findings Please note: Findings and conclusions related to public awareness activities involving youth at the regional level were addressed in the previous two sections of this report. PITA has supported the MDF, a national NGO, in its daily monitoring of news channels, print media, and online media sites for hate speech and discrimination based on ethnicity, religion, sexual orientation, gender, or political grounds, as well as anti-Western sentiments (the latter was a late addition to the media monitoring efforts, in response to a rise in Russian-influenced propaganda against Western values). The 31 MDF is also aided by the No to Phobia campaign, a coalition of 13 NGOs launched in 2014, which similarly monitors media and public statements by officials and mounts pressure campaigns. Many of the identified incidents as a result of this monitoring were documented in a publication, Hate Speech by Public Authorities/State Officials, published with PITA’s support in 2016, and are also listed on the No to Phobia website. Interestingly, the majority of the hate speech incidents documented by MDF and the No to Phobia campaign concern homophobic statements. National NGOs have also turned to PITA’s youth network to train them to monitor local media for hate speech. Media monitoring efforts and public awareness of hate speech have led to some dramatic results. Data compiled by MDF demonstrate a reduction by two-thirds in the number of incidents of hate speech and discrimination during primetime news programs and talk shows over the years 2014-2017 (see Figure 14). In addition, a pressure campaign aimed at the Deputy Minister of Diaspora Issues following his repeated homophobic and xenophobic remarks led to his resignation in 2016. Figure 14: Number of Instances Discriminatory Language Was Used in Primetime News Programs and Talk Shows In addition to media monitoring, since 2015, PITA-supported NGOs active in the No to Phobia campaign lodged a series of formal complaints about intolerant statements with self-regulation bodies of media organizations, four of which have been upheld. The campaign’s pressure on the GOG to review its advertising contracts with media outlets that engage in hate speech, a demand echoed by an international human rights organization, might also have contributed to greater media self-regulation. Supporting these statistics, several key informants interviewed noted that the trend in hate speech among media figures as well as politicians has markedly decreasing. They attribute this change directly to No to Phobia’s efforts because there was little objection to hate speech in the past. “It was unimaginable for any politician to talk about LGBT rights in 2008. Now even intolerant politicians are staying away from derogatory comments on LGBT people, at least in public statements,” remarked one informant. Nevertheless, informants commented that the trend for hate speech directed at immigrants and Muslims has slightly risen, along with ongoing discriminatory statements against Armenians. Besides supporting efforts to challenge hate speech, PITA is also committed to promoting positive media coverage of ethnic and religious minorities. In response to persistent advocacy by the CR and CNM, GPB 307 246 98 0 100 200 300 400 2014-2015 2015-2016 2017-present Number of instances Years “This past year, the Public Broadcasting Service sent journalists to work with Churches that celebrate Christmas on December 25. Different churches spoke about their liturgy, their celebrations. The attitude was very positive. It was a high-quality broadcast and a real breakthrough. And I have confidence they [Public Broadcasting] will continue. This is what we dreamed of, and all of the sudden it happened.” Member of the Council of Religions 32 has recently taken a series of actions to significantly expand news for and about ethnic and religious minorities. It adopted a multi-year strategy on integration and inclusion, and created an advisory group composed of ethnic and religious minorities. Its website now presents news information in seven languages and it is preparing to launch television and radio broadcasts in the Azeri and Armenian minority languages. It has increased the number of minority representatives among its staff by more than 50 percent, from 17 in 2016 to 26 in 2017. This past December, for the first time, it broadcasted the celebrations of minority Christian churches that celebrate Christmas on December 25, rather than January 7, the Orthodox Church Christmas. Yet, as promising as these developments are, other informants observed that GPB has very low viewership and ratings, especially with minority viewers. 4.3.2 Conclusions PITA-supported watchdog efforts are clearly producing impressive results, not only with the media, but also with policymakers and other public spokespersons. The broad approach to hate speech, covering discriminatory remarks on the basis of ethnicity and religion as well as on the basis of gender, sexual orientation, disability status, and other factors, is also significant in signaling the comprehensive values of human rights and inclusion. Similarly, the decision to expand watchdog efforts to include anti-Western sentiments in a context of widespread disinformation and propaganda is laudable. Such efforts exert significant checks and balances on media, government, and other institutions—an essential ingredient in a well-functioning democracy. The recent actions by GPB signal an important breakthrough in positive coverage of ethnic and religious minorities, which have had very limited visibility in Georgian media. At the same time, much more could be done to promote expanded and improved coverage of minorities in the media. 4.3.3 Recommendations For the remainder of the project period, it is recommended that PITA: • Continue to support the monitoring efforts to hold the media, as well as policymakers and other public figures, accountable for remarks reflecting discrimination or prejudice, maintaining the pressure that has so effectively discouraged hate speech. • Expand efforts to promote more positive coverage of ethnic and religious minorities. Given the low viewership for GPB, PITA might explore outreach to major Georgian media channels and local media in minority regions, especially TV, radio, and social media. In addition, PITA should encourage media to promote the direct voices of ethnic and religious minorities as subjects, not just objects, of media reporting. 4.4 EQ 4: HOW WELL DOES THE CIVIC INTEGRATION AND TOLERANCE INDEX WORK TO MEASURE CIVIC INTEGRATION AT THE NATIONAL LEVEL AND FOR PITA OUTCOMES? 4.4.1 Findings The Civic Integration and Tolerance Index is a tool to measure national progress in the tolerance toward and integration of minority communities in Georgia. First developed by UNAG in 2006 with support from Freedom House Europe, the tool was subsequently revamped in 2012. It determines the level of civic integration across seven dimensions: rule of law; political integration and civic participation; governance; education and the state language; culture; social and regional integration; and media and access to information. The national index has changed quite modestly from year to year. In 2016, it scored 3.9 (on a scale of 1 = greatest integration/tolerance and 7 = least integration/tolerance), a slight improvement from a score of 4.1 four years earlier (Figure 15). A look at the trend in the sub-measures indicates greater improvements over this time period in the areas of governance, education and state language, and culture and identity 33 (0.2 improvement on the 7-point scale), although the area of media and access to information actually declined in 2016 after improving slightly in previous years. Figure 15: Georgia Civic Integration and Tolerance Index, 2012-2016 The index methodology is modeled upon that of USAID’s CSO Sustainability Index, employing a thorough desk review of relevant literature as well as gathering the opinions of about 15 experts in Georgia. Experts receive careful guidance about how to apply scoring methods. UNAG reports that the measurements are regularly refined for greater validity. For example, the 2016 index review team collected more information about local developments and experiences in the regions to verify natonal findings and observations with local experts, and to make the report more informative. The ET did not attempt to assess the strengths and weaknesses of the index’s methodology. Rather, it assumed that the methods meet acceptable standards of validity, reliability, and objectivity. Instead, the ET examined questions related to the scope of the index, its use, and its relevance to the PITA project. Despite its title, the Civic Integration and Tolerance Index has a singular focus on tolerance toward ethnic minorities, although review experts did take into account cases of discrimination and hate speech against religious minorities in their discussions to determine scores for the 2015 and 2016 indices. However, tolerance toward religious minorities is considered only insofar as it intersects with that of ethnic minorities, not as a separate measurement—even recognizing that ethnicity and religion overlap in Georgia to an important degree. The index also does not take into account tolerance and integration toward other minorities, for example, LGBT people, migrants, or racial groups. The Civic Integration and Tolerance Index is published as a 25-page report and is only available in Georgian language. It is not searchable on the Internet. The index has been used primarily by UNAG as an internal document, although a limited circle of NGOs, state institutions, and academics who focus on ethnic minority issues also use it as a reference tool. For example, the SMR staff confirmed that they refer to the index periodically. Notably, PITA’s annual M&E plan cites the index as a key data source for measuring progress toward its objectives related to: 1) increasing ethnic/religious interaction with the government; and 2) increasing public awareness and education around diversity. In the course of information gathering for this report, the ET discovered an international index that offers an alternative to UNAG’s Civic Integration and Tolerance Index. The Social Progress Index is a composite index compiling data across 12 different dimensions of social advancement, including one on Tolerance and Inclusion. This sub-index draws upon existing global data to assess tolerance across five different dimensions: tolerance for migrants, tolerance for homosexuals, discrimination and violence against minorities, religious tolerance, and community safety. Perhaps most striking, Georgia’s score for Tolerance and Inclusion is the lowest by far among all 12 scores of social progress for the country. 4.1 4.0 4.0 3.9 3.9 3.8 3.9 4.0 4.1 2012 2013 2014 2015 2016 Tolerance index Year 34 4.4.2 Conclusions The Civic Integration and Tolerance Index serves an important purpose for UNAG and its immediate circle of partners engaged in improving the situation of the country’s ethnic minorities. However, the index has a number of limitations. Its title implies a broad scope of measurement of tolerance and integration, but in fact it is confined to measuring the integration of ethnic minorities. Its publication in a lengthy report format only in Georgian language and its lack of availability on the Internet limit its use by a wider audience. For the purpose of the PITA project, the index falls short as a tool to measure project outcomes for two main reasons. First, even recognizing the close alignment between ethnicity and religious affiliation in Georgia, the index fails to adequately capture religious integration and tolerance, a central objective of the PITA project. Second, it measures change at the macro and national levels. PITA, while broad in geographic reach as well as in its activities at the national as well as regional levels, is not quite large enough to drive significant change at the national level. The discrepancy between the index’s very modest climb over recent years, and the positive PITA project outcomes detailed in this report and elsewhere underline this mismatch. At the same time, the ET acknowledges the difficulty in measuring outcomes such as social norms, attitudes, and behaviors relevant to tolerance and inclusion, and understands that the Civic Integration and Tolerance Index, if revised to include religious tolerance, can serve as one among other tools to reflect broad PITA outcomes. However, it should be supplemented by other measures as well. 4.4.3 Recommendations For purposes of the PITA project, the ET recommends that PITA consider expanding the Civic Integration and Tolerance Index to separately assess issues concerning religious minorities. This addition would likely require the development of new measurements related to religious education, freedom of religious expression, religious discrimination, and other relevant dimensions. In addition, the index could be much more relevant as a measure of outcomes for the PITA project if either or both of the following steps were taken: 1) replicating the national index methodology in the minority regions where PITA conducts most of its activities, such as Kvemo Kartli, Samtskhe-Javaketi, and Kakheti; or 2) measuring some of the index’s more relevant sub-indices (possibly political integration and civic participation, education and the state language, culture and identity, social and regional integration) across Georgia, comparing regions where PITA is most active with those where it is not active. However, both approaches would require a greater investment of resources. Separate from the PITA project, UNAG could also consider widening the scope of the Civic Integration and Tolerance Index to capture even broader dimensions of tolerance and inclusion for minorities such as LGBT persons, migrants, and racial minorities. Alternatively, the index could rely upon existing measurements, such as the Tolerance and Inclusion sub-index of the Social Progress Index mentioned above. In addition, the Civic Integration and Tolerance Index has the potential for wider use if it were summarized in a user-friendly format, perhaps capturing key findings in charts and graphic illustrations. Moreover, it would attract other audiences such as the diplomatic corps, NGOs/CSOs, politicians, and academics at both the national and international level if it were translated into other key languages such as English and Russian. 4.5 PROJECT STRATEGY AND IMPLEMENTATION 4.5.1 Findings PITA has recently articulated strategic priorities in the following areas: consideration of regional context; multiplier effect; synergies; partnerships; and M&E. Although the ET did not directly attempt to assess the 35 emerging project strategy and implementation as part of its work, some observations surfaced in the course of the evaluation. Regional context. PITA pays attention to regional context in various ways. Six of PITA’s 14 YCs are located in predominantly ethnic minority regions. Through finance, links to its network, and facilitation, PITA has supported periodic consultations between state officials with minority communities in their regions. PITA has taken responsibility for translation of the SCEI and actions plans as well as some PITA project materials into minority languages (Azeri and Armenian) and ensured that trainings provided by two youth task forces in Gender Equality and EU integration were delivered not only in Georgian but in minority languages as well. Nevertheless, as noted above, YC members and their families in the more remote ethnic minority regions face barriers to more active participation in YC project activities, including transportation, knowledge of the state language, and social stigma surrounding the participation of girls in activities at night or far from home, leading to some divisions based on ethnicity even within the PITA project. Multiplier effect. As mentioned in Section 4.2 above, PITA’s work with youth appears to be having a multiplier effect among youth’s peers, parents, and local communities. The ET found that parents as well as external community stakeholders who participated in interviews or FGDs for the evaluation related their positive perceptions of the program and pointed to some of the direct effects on personal relationships and on communities outside the YCs. In addition, many YC training activities are designed to be replicated by youth not only among other YC members, but also among peers outside the YCs, and with teachers, parents, and community leaders. Furthermore, PITA works to strengthen the TC’s monitoring and reporting on ethnic and religious minority rights, and the reports are regularly used by international human rights organizations, multiplying their impact both outside and inside Georgia. However, it is difficult to assess the extent of the multiplier effect in the absence of mechanisms to track the degree of outreach. Synergy. PITA plays a critical role in facilitating connections among beneficiary state institutions, NGOs, and its broader network of YCs members and CNM/CR members. There are growing linkages among both the stakeholders and the activities across the three main project components. This appears to be leading to multiple synergies and cross-fertilization, for instance between national NGOs monitoring media and YC members trained to monitor media at the regional level, between the SMR and other state institutions and youth on public awareness campaigns, and between members of the CNM and CR with state institutions on a variety of issues. Although few youth in the regions seemed to be directly aware of the roles of the SCEI or the TC, the YC coordinators at the universities in Tbilisi are fostering periodic dialogues between their youth members and government officials, and the internship programs is providing a direct experience with state institutions for a limited number of youth. Partnership. In many ways, PITA does not direct the project as much as it coordinates and accompanies different project partners, as they shape their own initiatives and programs. Virtually without exception, partners were very positive in their assessments of their cooperation with PITA and found little to recommend in the way of improvements. They pointed to PITA’s clear guidance on program and financial reporting, its responsiveness, and the quality of trainings that it provides. They had high praise for PITA’s flexibility and the collaborative nature of the working relationship as well as for PITA’s well-established reputation, which gives them power to facilitate key connections. Similarly, on the strength of its reputation, PITA has forged partnerships with several external organizations, which are willing to provide resources for activities without charge. 36 Reflections from PITA Project Partners “Current support from PITA is not the major source of funding for us, but there are many other pluses from our cooperation with PITA. PITA staff persons have such expertise, understanding, sensitivity and values around these issues, they help us shape successful initiatives. When we have their involvement, it is easier to bring in other key stakeholders. PITA has opened doors for us.” National NGO receiving small grant “PITA is very flexible, they respond to our requests to address issues that come up, that weren’t planned under the project, like the anti-Western propaganda.” National NGO supported by PITA “PITA is supporting the project in the right way. They work with us through the process, addressing challenges, not only because we need something from them, but also because they might need something from us. There is a good exchange of information and very smooth working relations with PITA.” PITA Direct Partner “It is an exceptional situation, when you trust the donor [PITA], have their support, not just financial support. The main thing that is important is mutual trust.” Member of the Council of Religions “PITA is very supportive. It is not [a] very bureaucratic relationship…We don’t have the problem of holding most our conversations about satisfying the donor, but instead on implementation of the project, which is very good.” PITA Direct Partner Monitoring, Evaluation, and Learning. Monitoring, evaluation, and learning (MEL) is an area of weakness for PITA. There are inherent shortcomings in the project’s M&E framework. For example, there are neither indicator definitions nor is there an M&E plan detailing how the indicators will be operationalized through data collection approaches and timelines. Some indicators lack clarity. Although PITA’s quarterly and annual project reports are exhaustive in relating details of project activities, they do not always consistently report on output benchmarks set out in the M&E plans. While many benchmarks that are tracked show that the project met or exceeded its targets—e.g., numbers of trainings delivered, advocacy events organized, microgrants distributed—progress towards other benchmarks is not reported. Perhaps most important, PITA’s M&E plans set out a total of 27 outcome indicators, which are supposed to be tracked annually, but very few are consistently tracked or reported through annual progress reports. There are no baselines or targets for most outcome indicators. Upon inquiry, the PITA project was able to provide some data toward several outcomes, for example, baseline survey of youth attitudes, reports from periodic focus group discussions with grantees, and references to non-project external research. However, in general, it is very difficult to track progress toward outcomes on the basis of project data and PITA project reports. In addition, PITA’s quarterly reports have only just begun reflecting more deeply on larger strategies, challenges encountered and lessons learned (see Year 3, Q2 report submitted January 9, 2018). Although the PITA project staff conducts annual sessions with partners on project experience and challenges, it is unclear to the ET how this reflection is used for purposes of learning and course correction. PITA did share with the ET data reported to USAID/Georgia on two country-level PMP outcomes on an annual basis: 1) increased inclusion of target population; and 2) (a sub-outcome for the first outcome) increased integration of Georgia’s ethnic minorities. However, the Civic Integration and Tolerance Index is used as the indicator for measuring progress toward the first outcome even though, as the previous section argues, it does not adequately capture PITA’s outcomes. The second outcome, measured through “changes in beneficiary attitudes, perceptions, and behavior with regard to integration and tolerance,” shows very little change in actual measurements for the two years for which data are presented (FY 2015 and FY 2016). 37 Many of these same problems were identified during the ANI project period, and it appears that little has been done to remedy the situation. 4.5.2 Conclusions PITA recently identified strategic directions, and has begun accounting for the implementation of the strategy in its quarterly reports. Although most directions do not appear to be new, the fact that PITA is documenting and accounting for them is a welcome development. PITA’s two strategies of greatest strength are: 1) creating strong synergies between its different program components, and between its direct project partners and a growing circle of external partners; and 2) collaborative partnerships, which are highly trusted and valued among all direct and external partners that the ET interviewed. However, PITA could do more to identify ways to increase attention to the regional context as well as to measure and expand its multiplier effect. M&E is one area where there is still considerable room for improvement. It is impossible to see the forest for the dense trees in PITA’s project reports, much less any directions that the forest might be growing or shrinking. Furthermore, the monitoring system is not well designed or constructed, leading to considerable difficulty in accessing data to determine progress toward project results, especially beyond the activity level. The ET was not made aware of any regular process by the project for reflecting on lessons learned and adapting strategies and approaches accordingly. Although the PITA project experienced a significant delay in the work of the TC at project start-up in 2015 due to the PDO’s request for modifications to the project agreement, the ET heard of no other significant management or project implementation problems worth noting. 4.5.3 Recommendations The PITA project should organize a regular process for reflection and learning, perhaps engaging key project counterparts, and make sure to document the results of such sessions. The sessions should not only reflect on PITA’s internal approaches and experience, but also on the external environment, which might enable or hinder PITA’s project goals. Such sessions should be used to revisit its strategy and management approaches to make sure that they are still appropriate to current challenges, needs, and lessons learned. In addition, it could consider the following recommendations for its current strategy. • Monitoring, Evaluation, and Learning. Overall, PITA could benefit by shifting some attention away from activity details to progress toward larger project outcomes. Even at this stage, it could re-conceptualize the MEL Plan in a more synthesized manner, beginning by defining key indicators more clearly, and establishing systems for regular monitoring and reporting. This will also require reducing the number of outcome indicators that are expected to those that can be estimated relatively accurately and are feasible within PITA’s human and financial resources. PITA can use one integrated process or mechanism to collect all outcome indicators annually or biennially in order to maximize its resources. A simplified and reduced MEL plan can allow for streamlining of a MEL system into PITA’s work. Additionally, quality assurance procedures as part of the MEL system can minimize errors and allow for accurate performance data. • Regional context. As already recommended, in order to be more responsive to the regional context of ethnic minorities, PITA should devote more project energy and resources to overcoming the barriers faced by youth in remote villages of ethnic minority regions. This might involve different approaches to YCs in these regions, for example, establishing smaller YC “hubs” in village clusters far from the main YC, investment in long-term low- or no-cost Georgian language programs for YC ethnic minority members, and targeted outreach to parents who resist their children’s participation in YC activities. • Multiplier effect. PITA might try to put in place some mechanisms to be able to measure its effects on those beyond its direct beneficiaries. In addition, PITA might also increase its multiplier 38 effect if it supported a more proactive approach to spreading ideas, images, and voices of ethnic and religious minorities through broadcast media, rather than focus primarily on reacting to negative media. 39 ANNEXES 40 ANNEX A: STATEMENT OF WORK MID-TERM EVALUATION OF THE PITA 41 SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK I. SUMMARY INFORMATION Non-personal services for a mid-term program evaluation of the Promoting Integration, Tolerance, and Awareness (PITA) activity. The contractor will be required to answer all evaluation questions listed under III below. The contractor must provide the following deliverables within the terms defined by the contract: • Detailed evaluation design to be submitted with the proposal. • In-brief and out-brief with preliminary findings with the USAID management and staff upon arrival to and departure from Georgia. • Final evaluation report in accordance with the USAID Reporting Guidelines, including quantitative data collected. The timeframe to be covered by the evaluation is from the start of the project in June 2015 through the initiation of this evaluation. Project Numbers: Cooperative Agreement No. AID-114-A-15-00005 Project Dates: 06/04/15-06/03/2020 Project Funding: $4,600,000 Implementing organization: United Nations Association of Georgia(UNAG) Agreement Officer's Representative (AOR): Tamara Sirbiladze II. PURPOSE OF THE EVALUATION AND ITS INTENDED USE The purpose of this mid-term performance evaluation is to examine the progress of PITA's activities to date, evaluate the prospect for achieving life-of-project results, and identify if any changes need to take place to ensure the latter. The results of the evaluation will be used by USAID/Georgia for improving ongoing interventions in the area of national integration and tolerance and make relevant shifts and changes at the mid-point of implementation, if needed. The audience of the evaluation will be USAID/Georgia and in particular its Democracy, Governance, and Social Development (DGSD) office. The results of the study will be shared with local stakeholders, The State Ministry of Reconciliation and Civic Equality, Public Defender's (Ombudsman's) Office (PDO), other partner NGOs, the United Nations, and European Union. Finally, the evaluation results will also be used for reporting purposes to Washington-based stakeholders. III. EVALUATION DESIGN AND METHODOLOGY The program evaluation design will be submitted by the contractor in response to the Request for Task Order Proposals (RFTOP) and reviewed by USAID. The finalized evaluation design must be submitted to the Task Order Contracting Officer Representative (TOCOR) three workdays prior to the team's arrival in-country. The evaluation design must outline in detail what methods the contractor will use to get answers for each evaluation question. The evaluation design must include a detailed evaluation matrix (including the key questions, methods, and data sources used to address each question and the data analysis plan for each question), draft questionnaires and other data collection instruments or their main features, known limitations to the evaluation design, a work plan, and a dissemination plan. Methodology section in the proposal should address strengths and weakness of the proposed methodology, and how the latter will be mitigated. This information together with the Mission's comments will be discussed in detail during the in-brief meeting with USAID. Unless exempted from doing so by the Contracting Officer Representative (COR), the design will be shared with country-level stakeholders as well as with the implementing partner for comment before being 42 finalized. The work plan must include the anticipated schedule and logistical arrangements and delineate the roles and responsibilities of members of the ET. The evaluation matrix below is illustrative and USAID expects that the contractor will suggest the best methods that would generate most reliable and evidence-based answers to the key evaluation questions. Illustrative evaluation matrix: Research Question Data Source Methodology 1 How effective has PITA been at building the capacity of the GOG, including the State Ministry for Reconciliation and Civic Equality (SMR), The Public Defender’s Office (PDO), and the structures created under PDO: TC, CNM, and CR to fulfil their mandates and achieve PITA’s objectives? What are the lessons learned from working with these structures (SMR, PDO, TC, CNM, and CR)? What concrete results have these structures achieved so far? What is the prospect of sustainability for the Tolerance Center and the two Councils (CNM and CR) under PDO? Project documentation (narrative reports, M&E plan, results framework, etc.) Project staff Public Defender Tolerance Center staff and documentation CNM and CR members State Ministry of Reconciliation and Civic Equality Centers’ documentation (on their funding) Review of project progress reports, government and civil monitoring reports, diversity.ge and tolerance.ge entries, and other documents Interviews with Public Defender, TC, members of CNM and CR, and State Ministry of Reconciliation and Civic Equality Interviews with ethnic and religious minority representatives Interviews with human rights NGOs Documentation review 2 How effective so far has been PITA’s approach in developing a sense of national unity among youth? Has this approach been different for girls and boys? PITA’s approach to develop a sense of national unity among youth of different ethnic origin was built on USAID’s previous work on ethnic minority integration. How well did the changes to the previous approaches work (e.g., change in the age of the target youth from the school to university age)? How effective has PITA been at increasing public awareness and educating the public about diversity? Project documentation, quarterly reports, M&E plan, results framework Project staff Partner NGOs’ staff School staff involved in project activities Project youth participants and their parents Document reviews Interviews with project and partner NGOs’ staff members Focus groups with youth participants Focus groups with parents Interviews or focus groups with school staff Mini survey of students 4 How well Civic Integration Index developed by UNAG works in measuring civic integration at the country level, and how PITA captures its contribution? Should/can it be refined? Project documents and staff Documentation review Interviews with project staff 43 IV. EVALUATION SCHEDULE The following levels of effort are illustrative and should serve only as an example of the staff which may be mobilized under this Task Order. These levels may not reflect the actual level of effort contracted, and the contractor will be expected to submit its own estimate of the level of effort needed to fulfill the objectives. A six-day work week will be authorized in Georgia with no premium pay. Schedule of Evaluation Timing (Anticipated Months or Duration) Proposed Activities January 8-9, 2018 Initial telecom with USAID Georgia; Preparation work and finalization of the evaluation design; Document review January 10-11, 2018 Travel and preparations for data collection January 12, 2018 In-briefing January 13-27, 2018 Data collection (in Georgia) January 30, 2018 Out-briefing February 5-15, 2018 Data analysis and report writing February 16, 2018 Draft report February 20-March 2, 2018 USAID review of Draft Report March 5-14, 2018 Incorporate USAID comments and prepare Final Report March 15, 2018 Deadline for Final Report Level of Involvement (LOE) in days for a team of three Task LOE for Expat Team Lead LOE for Expat Evaluation Specialist LOE for Local Expert Total LOE in Days Document review, desk review, work planning, meetings in Washington (evaluation design) + Preparation for travel and data collection 5 5 5 15 In-brief, evaluation design (including meetings with USAID) 1 1 1 3 Preparations for data collection (scheduling) 1 3 5 9 Data collection days 15 0 15 30 44 Task LOE for Expat Team Lead LOE for Expat Evaluation Specialist LOE for Local Expert Total LOE in Days Out-briefing 1 1 1 3 Data analysis 5 5 3 13 Draft Final Report 5 5 3 13 Final Report 4 2 1 7 Totals 37 22 34 93 Estimated LOE in days by position for a team of three Position Preparation Travel To/From Country In-Country Data Collection Report Writing/Finalization Total LOE in Days Expat Team Leader 7 4 16 14 41 Expat Evaluation Specialist 9 0 1 12 22 Local Specialist 11 0 15 8 34 Totals 27 4 32 34 44 V. PROJECT DOCUMENTS FOR REVIEW AND LOGISTICS The PITA will put the contractor in contact with its sub-partners and may provide help with organizing a small number of meetings, such as meeting with the GOG and other USG agencies, if needed. Relevant reports and other project documentation will be provided by the Mission to the contractor prior to travel to Georgia. These documents are: • Project Description as is stated in the award: • Implementing partners Quarterly Reports: • Initial list of in-country contacts; • Project results framework; • PMP indicator tables; • M&E plans submitted and approved by USAID; • Approved Annual Implementation Plans; • Annual Reports; • Other deliverables (expert reports, publications) produced by partner; • USAID’s previous ethnic minority integration activity midterm evaluation report (2013); and • Other documents as needed. While in Georgia, the contractor will conduct most of the meetings in Tbilisi. Some meetings will require traveling to regions outside Tbilisi inhabited by ethnic and religious minorities such as Samtskhe-Javakheti, Kvemo-Kartli, also Imereti (Kutaisi) and Adjara (Batumi). 45 ANNEX B: LIST OF INTERVIEWEES AND FOCUS GROUP DISCUSSIONS 46 List of Key Informants Organization Individuals Interviewed United Nations Association of Georgia (UNAG) Lasha Jugheli, Current Chief of Party Ramaz Aptsiauri, Former Chief of Party Rusudan Chanturia, Deputy Chief of Party Otto Kantaria, Chief Development Officer Ilia Natsvaladze, UNAG Coordinator in Kutaisi State Ministry for Reconciliation and Civic Equality Ketevan Tsikhelashvili, State Minister Tina Gogheliani, Head of IR Department Eliso Lomidze, Head of Civic Integration Unit Public Defender’s Office (PDO) Ketevan Shubashvili, Head of the Equality Department Ana Iluridze, Head of Gender Department Tolerance Center Beka Mindiashvili, Head of Tolerance Center Council of Religions (CR) Mariam Gavtadze, CR Expert Council of National Minorities (CNM) Koba Chopliani, Coordinator, CNM Izabela Osipova, CNM Expert Ministry of Education and Science Nugzar Chitaia, Head of Higher Education Department National Probation Agency Tinatin Chelishvili, Social Worker Partners Georgia Sofiko Shubladze, Director Tea Bulia, Training Coordinator Evangelical Baptist Church Rusudan Gotsiridze, Bishop Tolerance and Diversity Institute Eka Chitanava, Director Georgia Democracy Initiative Georgi Mshvenieradze, Director Media Development Foundation Tamar Kintsurashvili, Executive Director Center for Civil Integration and Inter-Ethnic Relations Shalva Tabatadze, Chairman Center for Training and Consultancy Irina Khantadze, Director Center for Study of Ethnicity and Multiculturalism Giorgi Sordia, Director EU-NATO Information Center Nino Bolkvadze, Director State Fund for Protection and Assistance of Victims of Human Trafficking Nino Davitashvili, Manager, Prevention of Domestic Violence Tbilisi State University Youth Center (YC) Kakha Gabunia, Professor and Head of Center Mariam Kartvelishvili, YC Coordinator Ilia University Youth Center Zurab Guntsadze, ALPE Director Ivlita Lobzhanidze, Ilia University YC Coordinator Marneuli Youth Center Rena Nurmamedova, YC Coordinator Self-Development University Rashan Ziadaliev, Director Union of Young Azerbaijanis in Georgia Leila Mamedova, Director Paros (Ninotsminda YC Host NGO) Narine Ginosyan, Director Irina Karslyan, Ninotsminda YC Coordinator Kutaisi Youth Center Eka Arveladze, YC Coordinator Madona Basiladze, Head of Imereti KEDEC (Kutaisi YC Host NGO) Lika Kiladze, Executive Director Public Defender’s Office Madona Basiladze, PDO in Imereti, Racha-Letchkhumi, and Kvemo Svaneti Gori Youth Center Pikria Gelashvili, YC Coordinator Gori Municipality Mayor’s office Dima Gogiashvili, Department of Youth Affairs Gori Photographers Club Otar Gvritishvili Kelktseula Village Marina Tabatadze, School Director Equality Movement Levan Berianidze, Executive Director 47 List of Focus Group Discussions Youth Center Location Focus Group Participants Georgian Azeri Armenian Ilia University Youth Center (Tbilisi) 12 tutors 5 females 1 male 1 female 4 females 1 male Tbilisi State University 4 tutors 3 females 1 female Marneuli Youth Center 8 youth 5 parents 4 females 4 males Parents: 3 females 2 males Kutaisi Youth Center 4 youth 2 females 2 males Ninotsminda Youth Center 11 youth 7 parents 5 females 6 males Parents: 5 females 2 males Gori Youth Center 4 youth 1 parent 3 females 1 male Parent 1 female Interns 8 youth 1 female 1 male 5 females 1 male Tbilisi TSU Youth Center 2 youth 1 female Tbilisi ILIAUNI Youth Center 3 youth 3 males 48 ANNEX C: LIST OF DOCUMENTS REVIEWED 49 Pita Project Documents Cooperative Agreement Between UNAG and USAID, dated June 4, 2015 (Annex A) Annual Implementation Plan for Promoting Integration, Tolerance, and Awareness Program in Georgia Annual Implementation Plan, Program Year One, June 4, 2015 – June 3, 2016 Annual Implementation Plan, Program Year Two, July 1, 2016 – June 30, 2017 Annual Implementation Plan, Program Year Three, July 1, 2017 – June 30, 2018 FY1, Quarter 1 Performance Report, Reporting Period June 4 – September 30, 2015 FY1, Quarter 2 Performance Report, Reporting Period October 1-December 31, 2015 FY1, Quarter 3 Performance Report, Reporting Period January 1-March 31, 2016 Annual Report FY 2015-2016, dated July 31, 2016, reporting period June 4, 2015-June 30, 2016 FY2, Quarterly Progress Report One, Reporting Period July 1, 2016- September 30, 2016 FY2, Quarterly Progress Report Two, Reporting Period October 1, 2016- December 31, 2016 FY2 Quarterly Progress Report Three, Reporting Period January 1, 2017- March 31, 2017 Annual Report FY 2016-2015, dated July 31, 2017, reporting period July 1, 2016 - June 30, 2017 FY3, Quarterly Progress Report One, Reporting Period July 1, 2017- September 30, 2017 FY3, Quarterly Progress Report Two, Reporting Period October 1, 2017- December 31, 2017 Monitoring and Evaluation Plan for Year One Monitoring and Evaluation Plan for Year Two (Starting July 1, 2016 – ending June 30, 2017) Monitoring and Evaluation Plan for Year Three (Starting July 1, 2017 – ending June 30, 2018) PITA Logframe PITA Workplan for Year One FY 2016 PPR Data Collection Template FY 2017 PPR Data Collection Template Mid-Term Performance Evaluation of the Advancing National Integration (ANI) project in Georgia (January 2014) In Georgian language: PITA Baseline Survey Report (2016) Tolerance Index Report (2016) Grant Recipient Feedback Report (2017) IP & Youth Beneficiary Feedback Survey (2017) Charter of Council of National Minorities Charter of Council of Religions 50 Other Documents Center for Civil Integration and Inter-Ethnic Relations. “The Effectiveness of the One-Year Georgian Language Program for Ethnic Minorities at HEIs.” 2016. Council of Europe. Advisory Committee on The Framework Convention for the Protection of National Minorities: Second Opinion on Georgia adopted on 17 June 2015. Strasbourg, 11 January 2016. Council of Europe. Third Report submitted by Georgia pursuant to Article 25, paragraph 2 of the Framework Convention for the Protection of National Minorities. Received on 12 July 2017. Council of Religions. Recommendations by Council of Religions under the Auspices of the Public Defender of Georgia. 2017. European Commission Against Racism and Intolerance. ECRI Report on Georgia, Fifth Monitoring Cycle. Council of Europe. Adopted 8 December 2015, Published 1March 2016. Georgian Democracy Initiative. Hate Speech by Public Authorities/State Officials and the Issues of Disciplinary Punishment 2014-2016. 2016. Government of Georgia, Ministry of Reconciliation and Civic Equality. “National Concept for Tolerance and Civil Integration and Action Plan 2009-2014.” Kharatiani, Keti and Khatuna Nachkebia. Assessment of Georgian Language Training Educational Program. Center for Training Consultancy 2018. Media Development Foundation. Anti-Western Propaganda: Media Monitoring Report. 2016. National Democratic Institute, “Public Attitudes in Georgia: Results of December 2017 Survey,” carried out for NDI by CRRC Georgia. January 2018. United Nations Association of Georgia. The Civic Integration and Tolerance Index 2016. U.S. Department of State, Bureau of Democracy, Labor and Human Rights. “Country Reports on Human Rights Practices for 2016: Georgia.” U.S. Department of State, Bureau of Democracy, Labor and Human Rights. “International Religious Freedom Report for 2016: Georgia.” 51 ANNEX D: MINI-SURVEY QUESTIONNAIRES 52 PITA SURVEY QUESTIONNAIRE Introduction to survey: The following is read to each respondent before beginning the interview. The respondent must agree to continue the interview after the following is read to him or her. “IT, Research and Metadata Solutions is conducting a survey. This survey is to learn about perceptions, attitudes, and behaviors of youth in Georgia regarding civic engagement, tolerance, and ethnic minorities. This survey is being done among youth who are members of Youth Centers established by UNAG. It is sponsored by the US Agency for International Development. You’re taking the survey will help us to identify issues related to national integration and civic engagement by youth in their community. It will also help to improve services and programs provided by the Youth Centers to their members.” “Three hundred and fifty youth have been randomly selected to take the survey. The interview will take about 15 minutes to complete. Any information you provide will be maintained in a secure manner. No one but the interviewer will know how you answered the questions. We will not use your name when we report results of the survey.” “Taking the Youth Integration Survey is your choice. If you feel uneasy with any of the questions, you can refuse to answer. You may also skip questions you do not want to answer. You can stop the interview at any time. If you have any questions about this survey, you may call Giorgi Giorgadze, survey project manager.” DO YOU AGREE TO CONTINUE WITH THE INTERVIEW? DEMOGRAPHIC INFORMATION about a respondent Q1. Gender Female 1 Male 2 Q2. Indicate your age. ___________________________ Q3. To which ethnic group do you belong? Georgian 1 Armenian 2 Azeri 3 Russian 4 Greek 5 Abkhazian 6 Osethian 7 Respondent ID Interviewer code: Settlement: ____________________ 53 Kurd 8 Ukrainian 9 Jewish 10 Other (indicate) 11 Q4. To which religious group do you belong? Orthodox Christian 1 Catholic 2 Muslim 3 Protestant 4 Jewish 5 Follower of the Armenian Church 6 I do not follow any of the religions 7 Other 8 Refused -2 Q5. Please indicate your highest level of education. Primary school (1 to 9 grades) 1 Secondary / High school student (10 to 12 grades) 2 Vocational educational 3 High education (BA/MA) 4 Other (please specify) 5 Refused -2 Q6. What language do you mostly speak at home? Georgian 1 Russian 2 Armenian 3 Azeri 4 Other 5 Refused -2 Q7. Do you speak Georgian? Georgian is my native language 5 54 Yes, completely (I can freely speak, read, and write) 4 I can speak, communicate 3 I can understand all but can’t speak 2 I can hardly understand, I can’t speak 1 I do not speak it at all 0 Refused -2 YOUTH INTEGRATION QUESTIONS Q8. For how long have you been a member of the YC [Interviewer Notes: put “0” in the corresponding field if respondent names only years or month. Put “-1” if respondent can’t answer, or “-2” refuses to answer the questions] Month: ____________ Year: ________________ Q9. Do you think that youth in your city are or are not able to influence different aspects of public life? Please indicate a scale between 1 to 5 (1 = are not able influence; 5 = are able to influence) for the following statements Statement Ethnic Minorities Religious Minorities I am able to make my own life choices about interaction with people of different I am able to influence my own family members in their views about I am able to influence my own peers in their views about Students of my school/university are able to influence the school/university life on concerns related to diversity and inclusion of Together with other members of society, I am able to positively influence local issues related to I am able positively influence the public opinion on attitudes and beliefs about Q10. How would you define civic activism? [Interviewer Notes: Put “-1” if respondent can’t answer, or “-2” refuses to answer the questions] __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ ____________ Q11. On a scale of 1-5, where 1 is “no responsibility” and 5 is “full responsibility” please indicate to what extent you believe you have a responsibility to participate in activities that benefit the public. 55 No responsibility 1 Little responsibility 2 Some responsibility 3 Considerable responsibility 4 Full responsibility 5 Don’t know -1 No response -2 Q12. How well are youth in your region connected with other regions? No contacts 1 Weak contacts 2 Some contacts 3 Strong contacts 4 Don’t know -1 No response -2 Q13. What are the key factors that create obstacles to strong connection between you and your peers from other regions? [check all that apply] Roads/transportation 2 Language barrier 3 Youth attitudes towards their peers in other regions 4 Youth don’t find it important 5 Ethnic origin 6 Religious belonging 7 Don’t know -1 No response -2 Q14. Communication with different groups [1 – I can’t/won’t communicate with such people; 5 – I am completely comfortable communicating with such people] Rank Group of people with different sexual identity Group of people with different religion Group of people with different ethnic background Q15. Have you ever felt awkward while communicating with other due to the following factors? [check all that apply] 56 Yes No Because of my accent when I speak Georgian 1 0 Because of ethnic background 1 0 Because of religious background 1 0 Because of language you are speaking 1 0 Other (specify) 1 0 Q16. On a scale of 1 to 5 where 1 = totally disagree and 5 = totally agree please tell us if you totally disagree or totally agree with the following statements regarding ethnic minorities in Georgia. [Interviewer Notes: Put “-1” if respondent can’t answer, or “-2” refuses to answer the questions] Statement Scale 1. Ethnic minority issues are addressed by the government. 2. People of different religions have different values. 3. Religious minority rights are addressed by the government. 4. Ethnic Georgian and ethnic minority values are very different. 5. Ethnic minorities have bigger political power in ethnic minority regions than they should have. 6. Ethnic Georgians do not have enough respect for ethnic minorities. 7. Ethnic minorities do not have enough respect for ethnic Georgians. 8. People of different religions in Georgia do have enough respect for each other. 9. Ethnic Georgians and ethnic minorities cannot understand each other. 10. It is hard to imagine a friendship between people of different religions. 11. People of different religions cannot understand each other. 12. Young people tend to have more respect for ethnic minorities than older people do. 13. Young people tend to have more respect for people of different religions than older people do. Q17. On a scale of 1 to 5 where 1 = totally disagree and 5 = totally agree please tell us if you totally disagree or totally agree with the following statements about relationships between ethnic Georgians and ethnic minorities. [Interviewer Notes: Put “-1” if respondent can’t answer, or “-2” refuses to answer the questions] Statement Scale 1. Youth are poorly aware of the traditions and lifestyles of people of different ethnic origins. 2. Ethnic Georgian and ethnic minorities are not friendly with each other. 3. Exchange of mockery happens frequently. 4. Physical offenses happen frequently. 57 5. Verbal insults happen frequently. Q18. In which language do you receive most of your information? Check all that apply. Georgian 1 Russian 2 Armenian 3 Azerbaijani 4 Ossethian 5 English 6 Other (please specify) 7 Q19. Can you access the Internet from home? Yes No Cell phone Internet 1 0 Fixed Internet (MAGTI Fiber optics or SILK NET) 1 0 Q20. Do you think that every citizen must have knowledge of the state language? Yes 1 No 0 Don’t know -1 Q22 No response -2 Q21. Why is it important for ethnic minorities to know the Georgian language? (check all that apply) Yes No 1. For communication with ethnic Georgian population 1 0 2. Education 1 0 3. To feel like a real citizen of the country 1 0 4. Participate in society 1 0 5. Employment 1 0 6. To access information 1 0 7. For public service 1 0 8. Trade 1 0 9. Health care 10. Don’t know/Can’t answer -1 58 11. No response -2 12. Legal skip -3 Q22. What type of gender issues or challenges exist in your community? [Interviewer note: Don’t read the answer options] [Check all that apply] Yes No Don’t Know 1. Girls/young women have less opportunity than boys/young men to education in Georgia. 1 0 -1 2. Girls/young women have less opportunity to good employment than boys/young men in Georgia. 1 0 -1 3. Girls/young women are unable to stop harmful practices such as early marriage or domestic violence. 1 0 -1 4. Laws and policies are insufficient to protect girls and women from harm. 1 0 -1 5. Boys/young men are generally unsupportive of equal status for girls/young women in Georgia. 1 0 -1 Q23. Comments __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ _______________________________________________