DISCLAIMER This publication was produced for review by the United States Agency for International Development. It was 
prepared by NORC at the University of Chicago. The authors’ views expressed in this publication do not necessarily reflect the 
views of the United States Agency for International Development or the United States Government.
IMPACT EVALUATION OF THE 
OBIRODH – ROAD TO TOLERANCE 
YOUTH LEADERSHIP TRAINING 
PROGRAM IN BANGLADESHI 
UNIVERSITIES 
FINAL REPORT
Prepared under Contract No. GS-10F-0033M / Order No. AID-OAA-M-13-00013, Tasking N058
DRG LEARNING, EVALUATION, AND 
RESEARCH ACTIVITY
IMPACT EVALUATION OF THE OBIRODH – 
ROAD TO TOLERANCE YOUTH 
LEADERSHIP TRAINING PROGRAM IN 
BANGLADESHI UNIVERSITIES
FINAL REPORT
July 2021 
Prepared under Contract No. GS-10F-0033M / Order No. AID-0AA-M-13-00013, Tasking N058
Submitted to:
Mousumi Sarkar, COR
Submitted by:
Peter Vining, NORC Principal Investigator (New York University)
Cyrus Samii, NORC Principal Investigator (New York University)
Michael Gilligan, NORC Principal Investigator (New York University)
Contractor:
NORC at the University of Chicago
Attention: Matthew Parry, Program Manager
3450 East-West Highway, Suite 800
Bethesda, MD 20814
Telephone: (301) 634-5444
E-mail: Parry-Matthew@norc.org
DISCLAIMER 
The authors’ views expressed in this publication do not necessarily reflect the views of the United 
States Agency for International Development or the United States Government.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | i
TABLE OF CONTENTS
ACRONYMS III
EXECUTIVE SUMMARY 1
1. INTRODUCTION 4
2. VIOLENT EXTREMISM IN CONTEMPORARY BANGLADESH 5
3. BACKGROUND ON PREVENTING AND COUNTERING VIOLENT EXTREMISM (P/CVE) 
PROGRAMS 11
4. THEORY OF CHANGE 14
FACTORS INHIBITING THE IMPACT OF BYSTANDER INTERVENTION TRAINING 18
ADAPTING THE BYSTANDER INTERVENTION MODEL TO REDUCE VIOLENT EXTREMISM 
ANTECEDENT BEHAVIORS 20
5. IMPACT EVALUATION METHODOLOGY 21
INTERVENTION DESIGN: ADAPTING BIT TO BANGLADESHI UNIVERSITIES 22
SITE SELECTION, PARTICIPANT RECRUITMENT, AND RANDOMIZATION PROCEDURE 24
SAMPLE SIZE, DATA COLLECTION STRATEGY, AND TIMELINE 26
OUTCOMES OF INTEREST AND MEASUREMENT STRATEGY 28
NORMS 29
SKILLS 32
CONFIDENCE AND SELF-ASSESSED COMPENTENCE TO ACT 33
WILLINGNESS TO INTERVENE 34
PEER LEADERS VS TRADITIONAL AUTHORITY FIGURES 35
BASELINE CHARACTERISTICS COVARIATE BALANCE 35
6. DATA ANALYSIS 38
7. RESULTS 41
NORMS 41
SKILLS 44
CONFIDENCE AND SELF-ASSESSED COMPETENCE TO ACT 45
WILLINGNESS TO INTERVENE 46
PEER LEADERS VS TRADITIONAL AUTHORITY FIGURES 48
PLACEBO OUTCOMES: DO NORMS AND CONFIDENCE TO ACT CHANGE WITH RESPECT 
TO UNRELATED TOPICS 51
LIMITATIONS 52
SUMMARY 53
8. DISCUSSION AND CONCLUSIONS 54
APPENDIX A. 56
APPENDIX B. 63
TABLES
Table 1: Participating Public and Private Universities in Dhaka and Rajshahi 24
Table 2: Data Collection Timeline 28
Table 3: Pre-Treatment Covariate Balance 37
Table 4: Effects of Programming and Tolerance BIT Program on Norms of Tolerance 56
Table 5: Effects of Programming and Tolerance BIT Program on Religious Tolerance (Web￾Based Stimulus Activity) 56
Table 6: Effects of Programming and Tolerance BIT Program on Rejection of Violence 57
Table 7: Effects of Programming and Tolerance BIT Program on Internalized Skills 57
Table 8: Effects of Programming and Tolerance BIT Program on Confidence to Act 58
Table 9: Effects of Programming and Tolerance BIT Program on Willingness to Intervene 58
Table 10: Participant Identification with Peer Leaders, compared to Traditional Authority Figures, 
among Those Enrolled in Tolerance BIT Program 59
Table 11: Effect of Peer Leaders, compared to Traditional Authority Figures, on Participant 
Evaluation of Social Environment among Those Enrolled in Tolerance BIT Program 59
Table 12: Effect of Peer Leaders, compared to Traditional Authority Figures, on Participant 
Identification with Program Goals among Those Enrolled in Tolerance BIT Program 59
Table 13: Impact of Peer Leaders, compared to Traditional Authority Figures, on Outcomes of 
Interest among Those Enrolled in Tolerance BIT Program 60
Table 14: Summary Statistics for Outcomes of Interest 60
Table 15: Standardized Baseline Covariate Balance by Attrition 62
Table 16: Gross Effect of Programming and Effect of Tolerance BIT Curriculum on Placebo 
Norms and Confidence Outcomes. 62
FIGURES
Figure 1: Pathways from BIT Programming to Willingness to Intervene 17
Figure 2: Effects of Programming and Tolerance BIT Curriculum on Norms of Tolerance 42
Figure 3: Effects of Programming and Tolerance BIT Curriculum on Norms of Religious Group 
Status Measures 42
Figure 4: Effects of Programming and Tolerance BIT Curriculum on Rejection of Violence and 
VEOs 43
Figure 5: Effects of Tolerance BIT Curriculum on Common Knowledge Assessment Scores 44
Figure 6: Effects of Programming and Tolerance BIT Curriculum on Internalized BIT Skills 45
Figure 7: Effects of Programming and Tolerance BIT Curriculum on Confidence Act 46
Figure 8: Effects of Programming and Tolerance BIT Curriculum on Willingness to Intervene 47
Figure 9: Participant Identification with Peer Leaders, compared to Traditional Authority 
Figures, among Those Enrolled in Tolerance BIT Program 48
Figure 10: Participant Identification with Social Environment of Programming, comparing Peer 
Leaders to Traditional Authorities among Those Enrolled in Tolerance BIT Program 49
Figure 11: Participant Identification with Programming Goals, Comparing Peer Leaders to 
Traditional Authorities, among Those Enrolled in Tolerance BIT Program 50
Figure 12: Impact of Peer Leaders, compared to Traditional Authority Figures, on Outcomes of 
Interest among Those Enrolled in Tolerance BIT Program 51
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | ii
ACRONYMS
BIT Bystander Intervention Training
CVE Countering Violent Extremism
DID Difference in Differences
DIM Difference in Means
GTD Global Terrorism Database
IE Impact Evaluation
IPA Innovations for Poverty Action
ITT Intent-to-Treat
LGAP Local Government Accountability and Performance
LGBT Lesbian Gay Bisexual Transgender
NPC No-program Control
P/CVE Preventing and Countering Violent Extremism
RCT Randomized Controlled Trial
SMS Short Message Service (text messaging using standard communication protocols)
UNICEF United Nations Children’s Fund
USAID United States Agency for International Development
USD U.S. Dollar
VEO Violent Extremist Organizations
WGI Worldwide Governance Indicators
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | iii
EXECUTIVE SUMMARY
This report presents the results of a rigorous randomized impact evaluation of a youth leadership 
training program aimed at preventing violent extremism in Bangladesh. The program was part of the 
broader USAID-sponsored Obirodh: Road to Tolerance project in Bangladesh, which aimed to support 
Bangladeshi civil society actors in their work to promote tolerance and mitigate the spread of 
extremism. The USAID Center for Democracy, Human Rights, and Governance (DRG) worked with the 
USAID Bangladesh mission to commission an evaluation as part of the DRG Learning, Evaluation, and 
Research (DRG-LER) Activity. The key finding from the evaluation was that the program significantly 
improved participants’ norms of tolerance and their expected willingness and capacity to intervene safely 
and effectively when they witness intolerant and extremist behavior. Thus, the program represents a 
promising approach to addressing violent extremism that could be applied in a variety of contexts.
PROGRAM DESCRIPTION
Preventing and countering violent extremism (P/CVE) suffers from a needle-in-a-haystack problem: 
becoming a violent extremist is a very low probability event. Therefore, targeting programs on at-risk 
persons wastes resources on de-radicalization programs for people who never would have been 
radicalized in the first place. In addition, strategies attempting to deter or use surveillance on entire “at￾risk” groups in an aggressive and coarse manner may do more to alienate members of that group, 
undermining the potential for enlisting the support of those who may actually have no interest in violent 
extremism. Moreover, the context in which a P/CVE program is being implemented may greatly 
constrain the menu of feasible strategies. Approaches taken from within industrialized liberal 
democracies that rely on the police, justice system, or youth service institutions may not be appropriate 
in contexts where civil liberties are constrained or police forces are partisan or abusive. The program 
examined by this evaluation sought to avoid these pitfalls by focusing instead on making the social 
climates in which violent extremists might recruit less hospitable to violent extremism. It did so by 
training youth to recognize expressions of intolerant and extremist views and to intervene safely and 
effectively to counter such expression. Specifically, rather than attempting to identify and neutralize 
individual extremists, this program attempts to enable those who do not espouse intolerant or 
extremist views to contribute to a general climate in which expressions of intolerance or violence 
would be countered. By training individuals to publicize and reinforce norms against extremist views and 
behaviors, the programming is meant to both decrease the likelihood of extremist expression among 
those holding attitudes that might lead them to express it, and empower others who hold attitudes 
opposed to extremist behaviors to further reinforce norms against it. The program thus offers a new 
approach to P/CVE not relying on surveillance or aggressive police action, and an approach to which 
broader segments of the population falsely assumed to be “at-risk” may be more receptive and 
responsive.
The youth leadership training program focused on university students because the majority of those 
involved in recent violent attacks in Bangladesh have been university graduates with educated, middle￾class origins. We selected six universities as sites for the study, three in Dhaka and three in Rajshahi. 
The program drew inspiration from “bystander intervention training” (BIT) programs implemented in 
other contexts to address anti-social behaviors. The program trained university students to identify and 
intervene in situations in which their peers expressed extreme intolerance or aggression toward 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 1
vulnerable groups or minorities, with a particular focus on religion and gender. The impact evaluation 
used a randomized controlled trial design, testing the pro-tolerance BIT curriculum against a placebo 
curriculum and against a no-program control group. This allows the evaluation to estimate both the 
gross effect of the program, which involves both novel social interactions and generic prompts to think 
critically along with the content of the curriculum, and the effect of the BIT curriculum itself. The 
evaluation also examined whether it would be more effective to have peer leaders (those similar to the 
trainees) rather than traditional authority figures (professors, in this case) deliver the training. The study 
used surveys and a web-based stimulus activity to measure attitudes, the internalization/retention of 
knowledge and skills, and behavioral intentions. 
FINDINGS
The program led to improved norms of tolerance among participants, although much of this 
improvement appears attributable to the nature of the social interactions created by the program and 
the generic prompts to get subjects to think critically rather than the BIT curriculum itself. It is plausible, 
for example, reduced sexism observed among participants is attributable to the mixed-gender nature of 
the programming, rather than the BIT curriculum content. We can distinguish these sources of the 
effects when we compare how BIT program participants fared relative to those who received no 
program at all and relative to those who received a placebo curriculum (that focused on health and 
sanitation). The BIT program successfully improved participants’ skills and competence to intervene in a 
potentially effective way when presented with scenarios of aggression, with similar caveats suggesting 
these effects are partially driven by social interactions within the program. The BIT program also 
improved participants’ eagerness and confidence to act when they witnessed intolerance, and these 
effects appear mainly to be derived from the content of the BIT curriculum itself. Finally, we find the BIT 
program increased participants’ willingness to intervene (the primary outcome of interest, as measured 
by behavioral intentions in response to vignettes) in a variety of perpetrator-victim scenarios. 
Importantly, the curriculum was able to increase tolerant attitudes not only towards groups directly 
addressed by the curriculum (women, non-Muslims), but also in an area our implementing partner 
deemed too controversial to include, namely LGBT rights. Students appear to have extended lessons 
learned on tolerance to this latter group even though the curriculum did not directly address it. 
Moreover, the program increased willingness to intervene against violent extremism, even though the 
curriculum was focused mostly on positive reinforcement of tolerance and anti-violence norms. 
With regard to peer versus traditional authority facilitators, the evidence suggests on the whole, the 
traditional authority facilitators were more effective. These findings suggest a “social climate” strategy, 
i.e., a strategy designed to empower and encourage those already holding tolerant attitudes to safely 
protect and promote norms of tolerance within their social milieus, may be potentially effective for 
P/CVE purposes, especially in contexts with heightened political sensitivity and frequent rights violations. 
While encouraging, these findings do not shed light on the longer-term impact of such programming, or 
whether changes within the social microclimates of participants have endured following, for example, the 
disruptions of the Covid-19 pandemic. In addition, while this program was designed as a “megadose” 
(about 28 hours) of BIT when compared to similar initiatives, it is unclear to what extent scaling the 
programming (either through larger class sizes and/or shorter duration) would potentially erode efficacy. 
Finally, although the research design sought to minimize demand effects and social desirability bias in 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 2
survey responses and activities, it is nevertheless possible some of the attitudinal measures we use 
reflect these issues. 
IMPLICATIONS
The implications of our results are that indirect, “social climate” strategies are a compelling approach to 
address violent extremism. While further study should be conducted on the durability of these 
interventions before strong policy conclusions are drawn, the BIT curriculum implemented in 
Bangladesh was successful and should be considered for other contexts. To that end, this evaluation’s 
findings suggest some lessons learned for future P/CVE endeavors. In future programming, relying on 
peer facilitators appears to be somewhat risky because peer facilitators appeared to be less adept 
instructors. Thus, we recommend relying on professional instructors for future training programs like 
the one studied here. Second, this evaluation suggests that a BIT approach focusing on empowering 
those holding tolerant attitudes to safely protect and promote tolerant social norms can cultivate 
tolerant attitudes toward groups too politically or socially sensitive and controversial to address 
directly. 
The program in this study was designed to make the environment less hospitable to violent extremists 
by training members of civil society to recognize intolerant and extremist views and take appropriate 
action. The program was successful in meeting these goals, producing significant improvements in 
participants’ norms of tolerance and their willingness and ability to intervene safely and effectively when 
they witness intolerance and extremism. These improvements were reflected in survey instruments 
used to measure attitudinal changes, skills/knowledge retention and application using aptitude measures, 
and behavioral intentions in response to vignettes. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 3
1. INTRODUCTION 
As part of the DRG Learning, Evaluation, and Research (DRG-LER) Activity, USAID requested that 
NORC design and implement an impact evaluation (IE) to study a youth leadership training program 
aimed at preventing violent extremism in Bangladesh. The program was part of the broader USAID￾sponsored Obirodh: Road to Tolerance project in Bangladesh, which aimed to support Bangladeshi civil 
society actors in their work to promote tolerance and mitigate the spread of extremism. The youth 
leadership training program focused on university students, given that the majority of those involved in 
recent violent attacks in Bangladesh have been university graduates with educated, middle class origins. 
Countering violent extremism is fraught with the reality that such extremism is, by its nature, rare, and 
strategies that attempt to deter or use surveillance on presumed “at-risk” communities in an aggressive 
and coarse manner may do more to alienate those who have no interest in violent extremism than 
identify or temper those who are genuinely at-risk. Such programs, like the New York Police 
Department’s “Muslim Surveillance and Mapping Program” were found to be both illegal in profiling faith 
communities and also stifling of potential cooperation from community members. We discuss examples 
of such approaches and their negative consequences in an overview of P/CVE strategies found in section 
3 of this report.
Working with university students in Bangladesh, the youth leadership training program sought to avoid 
such pitfalls by using positive reinforcement of tolerance and anti-violence norms to cultivate a social 
“climate” against extremism. The program drew inspiration from “bystander intervention training” (BIT) 
programs implemented in other contexts, such as universities, secondary schools, and organized sports, 
to address anti-social behaviors. It sought to train university students to identify and intervene, in ways 
safe for themselves and others, in situations in which their peers expressed extreme intolerance or 
aggression toward vulnerable groups or minorities, with a particular focus on religion and gender. The IE 
used a randomized controlled trial (RCT) design, testing the pro-tolerance BIT curriculum against a 
placebo curriculum and against a no-program control group. The placebo was designed as a parallel 
leadership development program meant to build critical thinking skills with a focus on civic issues 
unrelated to tolerance. Using a placebo allowed the evaluation to estimate both the gross effect of the 
program, which involves both novel social interactions in a programmatic context and generic prompts 
to think critically, and the valued added by the content of the curriculum per se. The evaluation also 
studied whether it would be more effective to have peer leaders (those similar to the trainees) leading 
sections of the programming as “facilitators,” rather than traditional authority figures (i.e., professors in 
this case).
This report presents the results of the IE. We find that the BIT program led to improved norms of 
tolerance among participants, although much of this improvement appears attributable to the nature of 
the social interactions and generic prompts to think critically that were created by the program. We 
also find that the BIT program successfully improved participants’ skills and judgement (competence) 
when presented with scenarios of aggression, with similar caveats suggesting that these effects are 
partially driven by the general social interactions within the program. Furthermore, we find that the BIT 
program improved eagerness and confidence to act among participants, which we attribute mainly to the 
focused curriculum of the BIT program. Finally, we find that the BIT program led to willingness to 
intervene (the primary outcome of interest) when participants were presented with a variety of 
perpetrator-victim scenarios. Presumably, willingness to intervene is driven by the effect of confidence 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 4
and self-assessed competence to act among BIT participants. The results are promising for indirect, 
social climate strategies to counter violent extremism.
With respect to the aforementioned outcomes (norms, competence, confidence, and willingness to 
intervene), we found suggestive evidence that peer leaders may be less effective than traditional 
authority figures (namely, professors) at leading a program of this nature, although these differences 
were not large. Differences between each type of facilitator appear more pronounced regarding 
participant perceptions of the facilitator leading their groups, which may be driving small differences in 
the other outcomes of interest. Not surprisingly, traditional authority figures (i.e., professors) were 
significantly more likely to be seen as experienced, while peer leaders were seen as youthful. 
Interestingly, the traditional authority figures were perceived as more approachable than the peer 
leaders. These results speak to the importance of experience and professional skill in administering 
training programs.
In section 2 of this report (below), we begin with a discussion of the context, describing the nature of 
violent extremism in contemporary Bangladesh. Violent extremism in Bangladesh regularly involves 
educated young adults, which motivates our focus on university students. In section 3, we review 
current strategies for countering violent extremism, explaining why we think the bystander intervention 
training approach is most suitable for university students in the present context. In section 4, we present 
the theory of change for the youth leadership training program. Section 5 describes the research design 
and data collection plan for the IE. Section 6 then present our statistical analysis of the results of the 
evaluation. A concluding section 7 summarizes what we have learned, derives policy implications, and 
offers suggestions for further research.
2. VIOLENT EXTREMISM IN CONTEMPORARY BANGLADESH
Violent extremism in Bangladesh has increased noticeably in recent years, threatening development goals 
and political stability in the world's eighth most populous country. The Institute for Economics and 
Peace’s Global Terrorism Index records an increase on its 10 point “Impact of Terrorism" scale from 4.1 
to 5.9 between 2012 and 2015 in Bangladesh.1 Estimates of the number and severity of extremist attacks 
in recent years vary, depending on the data source and criteria used in defining such attacks. One 
conservative estimate claims that the 2013-2015 period was marred by at least 47 major extremist 
attacks, killing 54 and injuring hundreds more, in contrast to just 92 attacks during the entire decade 
prior2. The number of known violent extremist organizations (VEOs) also varies widely, as groups often 
merge, splinter, rebrand, or operate in conjunction with non-violent political wings; estimates have 
ranged from 12 to 70 organizations.3
An independent analysis of data provided by the Global Terrorism Database (GTD)4, which records 
deliberate acts of violence for political, religious, economic or social purposes, suggests that five major 
VEOs have carried out 114 attacks, killing at least 127 people and injuring another 348 people within 
1 Riaz, A. (2016a). Who are the Bangladeshi ‘Islamist Militants? Perspectives on Terrorism, 10 (1).
2 Rahman, M. A. (2016). The Forms and Ecologies of Islamist militancy and terrorism in Bangladesh. Journal for Deradicalization, 
(7), 68-106.
3 Ahsan, Z. (2005). Trained in foreign lands: The spread inland. The Daily Star, 21 August 2005, and Rahman, M. A. & Kashem, 
M. B. (2011). Understanding religious militancy and terrorism in Bangladesh. Dhaka: ICA Bangladesh.
4 National Consortium for the Study of Terrorism and Responses to Terrorism (Start). (2017). Global Terrorism Database 
[Data File]. Retrieved from https://www.start.umd.sdu/gtd.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 5
Bangladesh from 2013-2016. These include VEOs with an international presence (Islamic State in 
Bangladesh and Al Qaeda in the Indian Subcontinent, known within Bangladesh as Ansar-al-Islam), as well 
as Bangladeshi organizations (Ansarullah Bangla Team, Jamaat-E-Islami Bangladesh, and Jama'atul 
Mujahideen Bangladesh). These organizations are usually responsible for the most high-profile and mass￾casualty events, such as the July, 2016 Holey Cafe attack and hostage incident, which killed 28 people 
and wounded another 30. This attack was claimed by Islamic State, and likely also involved Jama'atul 
Mujahideen Bangladesh. 
Many VEO-sponsored attacks in Bangladesh have targeted individuals rather than institutions, usually for 
sectarian purposes, and as part of an ongoing campaign to intimidate and terrorize different minority 
groups within the country. Targets have included ethnic, religious and sexual minorities, as well as 
foreigners, academics, journalists, and those deemed to have committed acts of blasphemy or apostasy 
against extremist interpretations of Sunni Islam. GTD data indicate that of the 59 lethal VEO-related 
attacks recorded from 2013-2016, at least 35 were murders of targeted individuals; victims have ranged 
from Hindu, Shia and Christian religious leaders5 to university professors,6 atheist and secular bloggers,7
lesbian, gay, bisexual, and transgender (LGBT) rights activists,8 foreign aid workers,9 and others. The 
April, 2016 murder of LGBT activist and US Embassy employee Xulhaz Mannan along with colleague 
Mahbub Rabbi Tonoy by Ansar-al-Islam is a prototypical example of these targeted killings, which are 
often preceded by death threats and carried out by groups of men wielding knives, machetes, or other 
crude weapons. 
In addition to these high-profile attacks carried out by or on behalf of known extremist organizations, 
Bangladesh has also experienced a troubling wave of sectarian violence carried out by individuals or 
unorganized small groups against the aforementioned minority groups. The Global Terrorism Database 
contains information on many attacks whose perpetrators remain unidentified, but clearly fall within this 
narrative. Oftentimes, these attacks include difficult-to-prosecute property crimes (such as arson) and 
are carried out using machetes, knives, rocks, petrol bombs and crude incendiary devices. Examples 
from the 2013-2016 period include attacks against Sufi Muslims10, Hindu temples11 and homes12, as well 
as religious minority leaders.13
Profiles of the individual militants themselves contradict a common narrative that they tend to be 
uneducated and unskilled. Many arrestees from recent plots and attacks have been university graduates 
with educated, middle class origins, suggesting that economic deprivation offers limited explanatory 
power for why individuals join VEOs.14 An analysis of the socio-demographic profiles of Bangladeshi 
5 "IS 'beheads' Hindu priest in Bangladesh," BBC, February 21, 2016; "Top Shia Preacher Killed in IS Claimed Attack in 
Bangladesh," Outlook India, March 15, 2016; "Bangladesh: Another Hindu priest murdered," The Daily Star Online, July 1, 2016;
"B'desh Christian priest attacked in his house by armed men," Deccan Herald, October 6, 2015. 
6 "Professor murder: Militants claim responsibility on Facebook," Dhaka Tribune, November 16, 2014.
7 "Ananta Bijoy Das Hacked To Death In Bangladesh In Third Such Killing Of Atheist Bloggers," International Business Times, 
May 12, 2015; "Second blogger hacked to death this year in Bangladesh," Reuters, March 30, 2015.
8 "Islamist Militants Suspected in Killing of Gay Rights Activist in Bangladesh," The New York Times, April 26, 2016. 
9 "Australia: Bangladesh Opposition officials among seven charged over Italian aid worker murder," ABC Online, June 28, 2016.
10 "3 Sufi Muslims attacked in B’desh," New Delhi Pioneer, July 30, 2016.
11 "Dhaka: Fresh Attacks on Hindu Temples Create 'Widespread' Panic Among Minorities," The Daily Star Online, March 9, 
2013.
12 "Bangladesh: 10 Hindu houses torched in Dinajpur," Dhaka Tribune Online, December 4, 2016.
13 "Leader of religious minority forum attacked in Bangladesh," Deutsche Presse-Agentur, November 24, 2015.
14 Rahman, M. A. (2016). The Forms and Ecologies of Islamist militancy and terrorism in Bangladesh. Journal for Deradicalization, 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 6
militants arrested from January 2014 through June 2015 is consistent with this claim; the majority of 
them are young, well-educated men from middle class backgrounds. Out of 112 alleged militants 
arrested during this time period, 52 were men between the ages of 18 and 30. Out of 65 for whom 
occupations could be identified, 53 held professional or middle-class jobs/career tracks. Nine were 
university students, nine were engineers, five were teachers, and the rest were a variety of other 
middle-class professions including business owners, travel agents, IT experts, and office clerks. In 
contrast, only 13 of the 65 held jobs in working class or manual labor positions. Regarding education, 
nearly all of the 53 individuals in middle class career tracks hold or are pursuing college degrees, and 
several hold advanced degrees. Only 13 were madrassa students or teachers.15
Evidence from student and school administrator focus groups can provide some insight into possible 
explanations for the aforementioned patterns in VEO perpetrators and targets. Focus groups of 
university and post-secondary level madrassa students held by Breakthrough Media found that 
intolerance was fostered by the Bangladeshi secondary education system, due to divisions that the four 
different types of secondary schools (English, Bangla, Quomi madrassa and Aliya madrassa) create and 
exacerbate along lines of gender, wealth, religion, and religiosity, without building a sense of empathy for 
different backgrounds. In addition, all four school types were found to promote a religiously/culturally 
narrow definition of Bangladeshi national identity that is closely tied to Sunni Islam. These findings from 
student focus groups were corroborated by a survey of school administrators in which 72% responded 
that the “diverse streams of education [i.e., the four different school types] contribute to radicalization 
in Bangladesh.”16 Apparently, despite universities being more integrated than the secondary schools, the 
economic and cultural divisions originating in the primary and secondary education systems persist 
within the university system while the financial and social vulnerabilities of university students – who are 
typically facing academic pressures, and the pressures of early adulthood and identity formation in the 
new and unfamiliar settings of a university campus, potentially with few preexisting social connections￾also make them more susceptible to outside influences and pressures.17
Country-wide survey evidence from Bangladesh also suggests a concerning political and social 
environment. Although three-quarters of respondents in a 2012 country-wide survey of Bangladeshi 
Sunni Muslims felt that suicide bombings and other attacks against civilian targets were rarely or never 
justified, roughly a quarter responded that they were sometimes or often justified; these levels of 
support for suicide attacks were also found to be substantially higher than those within the broader 
region (including Pakistan, Malaysia and Indonesia).18
Further analysis of the survey reveals a population that appears to be split between strict and moderate 
cultural interpretations of Sunni Islam with respect to a variety of issues. While a majority of 
respondents stated that women should have the right to divorce their husbands and choose whether to 
wear the hijab, sizable minorities (about 30%) responded that they should not; moreover, a sizable 
minority (35%) responded that honor killings of women by family members are often or sometimes 
justified. Regarding attitudes towards minority groups, Sunni Muslims in Bangladesh tend to be 
(7), 68-106.
15 Riaz, A. (2016). Who are the Bangladeshi ‘Islamist Militants? Perspectives on Terrorism, 10(1).
16 Breakthrough Media. (2017). “Research Paper for Credible Voices Bangladesh.” Report prepared for USAID. 
17 Ibid. 
18 Fair, C. Christine, Ali Hamza, and Rebecca Heller. “Who Supports Islamist Militancy in Bangladesh: What the Data Say?” 
Available at SSRN 2804275 (2016). 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 7
conservative. About 70% responded that homosexual behavior was “morally wrong,” 68% responded 
that they would be “not too” or “not at all” comfortable if a son of theirs someday married a Christian, 
and about 16% did not consider Shia to be Muslims. Regarding attitudes towards the West, only about 
17% responded that Western media have hurt morality within Bangladesh.19 Regarding attitudes towards 
their fellow Muslims, majorities or near-majorities favored the stoning of people who commit adultery 
(52%), punishments like whipping or cutting off of hands for the crimes of theft or robbery (50%), and 
the death penalty for people who leave the Muslim religion (44%).20
A more recent set of surveys conducted between 2012-2014 commissioned by PACOM and carried out 
by Nielson-Bangladesh roughly corroborates many of the findings in the 2012 Pew study regarding 
perceptions on violence and foreign countries; surveys in 2012, 2013, and 2014, all found that roughly 
half of respondents agreed that the use of violence was acceptable “against immoral people,” and “to 
maintain the culture and traditions of society,” while between 80-90% agreed that violence was 
acceptable “in defense of one’s religion.” Regarding perceptions of Western countries, less than half of 
respondents reported positive perceptions of the United States’ influence on Bangladeshis’ way of life, 
while less than a quarter reported positive perceptions of the United Kingdom.21
Finally, recent work has also suggested that many of the aforementioned patterns both in the individuals 
recruited/targeted by VEOs, as well as the prevailing social/political context, are in part exacerbated by 
the proliferation of social media use (especially Facebook) in an environment that is still characterized by 
significant levels of illiteracy and a lack of critical thinking education.22 Both a SecDev study on behalf of 
UNDP Bangladesh and the Breakthrough Media focus groups reported that Facebook (which about 80% 
of the population use) has contributed both to increasing “cultural tribalism” via information echo 
chambers, while also offering a tool for VEO messaging towards vulnerable populations, despite findings 
that very few Bangladeshis actually seek out extremist pages and content. Interestingly, Madrassa student 
focus groups reported that they viewed news shared on Facebook as more trustworthy than “the 
media,” due to their ability to self-select their content.23 Extremist groups make use of this cultural 
tribalism in developing graphical/video content designed to provoke emotional reactions and reinforce a 
sense of victimhood among Sunni Muslims at the hands of purportedly hostile minority groups and 
foreign powers, often by connecting them to injustices and conflict narratives abroad.24
The aforementioned context and profiles of those carrying out recent extremist attacks suggest that 
economic “countering violent extremism” (CVE) interventions (i.e., jobs programs, cash transfers, etc.), 
which have been attempted in numerous other contexts with frequently unconvincing results, are likely 
ill-suited as effective strategies for the Bangladeshi setting.25 Given that most recent perpetrators have 
19 The Pew Research Center. (2012). The World’s Muslims [Data File]. Retrieved from http://www.pewforum.org/datasets/2012/. 
20 Ibid. 
21 “Vulnerability to Extremist Influence in Bangladesh.” (June, 2012). Report commissioned for the United States Pacific 
Command (PACOM). 
22 The SecDev Group. (2016). “Violent Extremist Narratives and Social Media in Bangladesh.” Report commissioned by the 
UNDP Bangladesh. 
23 “Vulnerability to Extremist Influence in Bangladesh” p. 7. 
24 “Violent Extremist Narratives and Social Media in Bangladesh.” (p. 7); “Bangladesh Country Needs Assessment: Drivers of 
Radicalization and Recruitment, and Community Resilience to Violent Extremism.” (March, 2016). Assessment conducted for 
the Global Community Engagement and Resilience Fund by the Royal United Services Institute. 
25 For a more thorough discussion of economic CVE interventions and their merits, see Gilligan, M., Samii C., and Vining, P. 
(2017). Evidence Review Paper: Countering Violent Extremism. Paper prepared for USAID Impact Evaluation Clinic, March 27-
31, 2017. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 8
not only been gainfully employed, but are often educated members of the middle class, it is far more 
likely that CVE efforts need to incorporate a psychosocial understanding of radicalization and prevention 
efforts. Specifically, such efforts should be premised on the understanding that individuals who are 
successfully targeted for recruitment by VEOs often have the same prosocial dispositions as those who 
become involved in other social movements and organizations; by becoming involved in VEOs, they are 
attempting to fulfill their psychosocial needs and desires in ways that traditional kinship, peer and 
community ties are apparently unable to. While involvement in VEOs carries potentially high costs to 
those individuals, the payoffs of fulfilling a “quest for significance” can outweigh those costs, especially if 
the broader political/social context reduces barriers to entry and the risks of punishment.26
It is well known that individuals are more willing to sacrifice their own welfare in order to pursue 
feelings of efficacy in service of a cause, oftentimes in response to events that lead to feelings of 
“significance loss." 27 Such significance loss events may include sources of trauma that cause feelings of 
loss, shame, humiliation, deprivation, disenfranchisement, vulnerability, reduced self-worth, 
isolation/loneliness, conflicted identity, and/or disillusionment with one’s current social role.28 Such 
feelings lead to cognitive openings within the individual, i.e., an openness to reorientation of their beliefs. 
In order for an individual’s quest for renewed significance to manifest as participation in violent 
extremism, however, these psychological motivations require (1) “moral warrants" that justify the use of 
violence, as well as (2) social processes of introduction and involvement to exploit the identified 
cognitive openings. VEOs thus step in to fulfill these necessary conditions by providing moral/ideological 
warrants for violence in socially empathetic settings, in addition to identifying, grooming, and at times 
even creating vulnerable individuals.29 Crucially, social affirmation is understood to be a big part of how 
participation in violent extremism fulfills the need for significance. The type of climate strategy studied in 
the current IE attempts to create a counterweight to social affirmation through extremism.
Introduction to VEOs often occurs via kinship and friendship networks, although some perpetrators 
were found to have “self-radicalized.” Introduction is usually gradual, starting with passive participation 
in informational events or low-risk activism activities. Assuming that the VEO is interested in more 
involvement from a potential recruit, further indoctrination occurs via radicalized elites, usually in the 
context of a new and supportive community. This often occurs concurrently with isolation from and 
rejection of moderating influences and old social ties. As a part of this socialization process, sacrificial 
behavior may be required as a signal of reliability for future group contributions.30
A wide variety of programs and initiatives have attempted to disengage and deradicalize individuals from 
VEOs. Almost exclusively, these programs have targeted those who have either already committed, or 
were at imminent risk of committing an act of violence. The choice of this target population is largely 
due to the difficulties of detecting and targeting interventions at those in earlier stages of radicalization. 
Most deradicalization programs have attempted to leverage some combination of implicit interventions 
26 Ibid. 
27 Dugas, M., Bélanger, J. J., Moyano, M., Schumpe, B. M., Kruglanski, A. W., Gelfand, M. J., … & Nociti, N. (2016). The quest for 
significance motivates self-sacrifice. Motivation Science, 2(1), 15. 
28 Bushman, B. J. (Ed.). (2016). Aggression and Violence: A Social Psychological Perspective. Psychology Press; Agnew, R. (2010). A 
general strain theory of terrorism. Theoretical Criminology, 14(2), 131-153.
29 Kruglanski, A. W., Gelfand, M. J., Bélanger, J. J., Sheveland, A., Hetiarachchi, M., & Gunaratna, R. (2014). The psychology of 
radicalization and deradicalization: How significance quest impacts violent extremism. Political Psychology, 35(S1), 69-93.
30 Berman, E. (2011). Radical, religious, and violent: The new economics of terrorism. MIT press; Wiktorowicz, Q. (2005). 
Radical Islam rising: Muslim extremism in the West. Rowman & Littlefield Publishers.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 9
(i.e., those addressing the underlying reasons for joining) and explicit interventions (i.e., those countering 
the radicalized worldview).31 The efficacy of these programs is difficult to evaluate, due to: (1) a lack of 
RCTs used to evaluate their effects, (2) strong demand effects leading participants to report being 
“cured” (which often leads to leniency, reduced sentencing, and/or other incentives), and (3) because 
post-program monitoring makes participants less-appealing for risk-averse VEOs to approach for 
recidivism. 
One example of a series of interventions designed to address radicalization at earlier stages was the 
German HAYAT program, which seemed to offer a fruitful model for preventing highly radicalized (but 
not yet violent) individuals from emigrating to fight on behalf of jihadist groups in Syria/Iraq during the 
latter 2010’s. The program featured an innovative, community-based approach that sought to leverage 
friend and family networks in order to deter a radicalized individual from departing.32 Still, the success of 
an individually-focused model for those at earlier stages of radicalization presumes a social and political 
context in which such attitudes and behaviors noticeably depart from prevailing social norms, are 
detectable by the individual’s social networks, and are therefore likely to put the individual at risk of 
ostracization and/or harm should they become further involved in VEO activity. Moreover, the success 
of such a model presumes that safe “off-ramps” are available for the individual’s close network to pursue 
in order to intervene without risking legal consequences or ostracization for the radicalized individual or 
themselves. In contexts where individual rights are regularly violated, these conditions may not be in 
place. 
Overall, while the psychosocial perspective and potential interventions derived from it offers a nuanced 
post-hoc explanation for why individuals are targeted by, join, and carry out acts of violence on behalf of 
VEO’s, it offers frustratingly few prescriptions for effective and targeted approaches to prevent or 
counter violent extremism at the individual level, and at stages of involvement that precede imminent 
plots and acts. Most of the aforementioned findings are based on studies of individuals who have either 
been arrested/imprisoned following VEO-related offenses,33 or who have voluntarily disengaged/de￾radicalized and agreed to share their accounts with researchers. Other findings are based on voluntary 
accounts given by radicalized, but explicitly non-violent individuals in contexts where such organization 
and activism is legal as protected speech.34
Finally, because the population of potentially at-risk individuals is enormous while the population of 
those who actually radicalize, join, and go on to commit acts of violence is so small, it is likely more 
useful to target early-stage preventing and countering violent extremism (P/CVE) approaches at the 
community level, rather than at the individual level. Specifically, rather than attempting to identify and 
treat “at-risk individuals,” strategies that attempt to address social norms regarding behaviors and 
31 Kruglanski et al. (2014); Refer to Gilligan, Samii and Vining (2017) for examples. 
32 Koehler, D. (2013). Family Counselling as Prevention and Intervention Tool Against ‘Foreign Fighters’. The German ‘Hayat’ 
Program. Journal Exit-Deutschland. Zeitschrift für Deradikalisierung und demokratische Kultur, 3, 182-204.
33 See, for example, Kruglanski, A. W., Gelfand, M. J., Bélanger, J. J., Sheveland, A., Hetiarachchi, M., & Gunaratna, R. (2014). The 
psychology of radicalization and deradicalization: How significance quest impacts violent extremism. Political Psychology, 35(S1), 
69-93; Bélanger, J. J., Caouette, J., Sharvit, K., & Dugas, M. (2014). The psychology of martyrdom: making the ultimate sacrifice 
in the name of a cause. Journal of Personality and Social Psychology, 107(3), 494.
34See, for examples, Wiktorowicz, Q. (2005). Radical Islam rising: Muslim extremism in the West. Rowman & Littlefield Publishers; 
Kenney, M., Horgan, J., Horne, C., Vining, P., Carley, K. M., Bigrigg, M. W., ... & Braddock, K. (2013). Organisational adaptation 
in an activist network: Social networks, leadership, and change in al-Muhajiroun. Applied Ergonomics, 44(5), 739-747.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 10
attitudes towards groups targeted by VEOs are likely to offer an approach to P/CVE to which those “at￾risk” and their close-networks may be more receptive and responsive. 
3. BACKGROUND ON PREVENTING AND COUNTERING 
VIOLENT EXTREMISM (P/CVE) PROGRAMS
A variety of P/CVE programs have been implemented by governments and communities throughout the 
post-9/11 period to date.35 These programs can be loosely defined as those designed to prevent or 
mitigate violent extremism by using a range of soft policy interventions36 that target individuals, groups, 
communities and/or societies. While many of these initiatives have been successfully implemented from 
a programmatic standpoint,37 very few of them have been implemented in such a way that enables 
policymakers to determine whether they actually work- i.e., whether they achieve their goal of 
preventing and/or countering radicalization and involvement in VEOs. Among the few P/CVE programs 
that have included an evaluation component, virtually none have been designed in such a way that 
enables causal inference from program effects. A 2017 systematic review of 73 CVE studies, for 
example, found that only 14 were “effect evaluations” (i.e., those addressing causal effects of the 
program), of which only 5 were quantitative IEs, and only 2 of which were quasi-experimental.38 To our 
knowledge, no P/CVE programs have yet been implemented as RCTs, which are the gold standard for 
evaluating program effects.
The challenges to rigorous evaluation of P/CVE program effects are numerous, but fairly well-known. 
First among these challenges is correctly reaching “at-risk” individuals or groups, without generating 
unacceptable levels of false positives and/or exacerbating the problem by alienating or stigmatizing them. 
Attempts to correctly profile “at-risk” individuals have largely proven fruitless, and have sometimes 
backfired in the form of racial or religious profiling. The United Kingdom’s “Prevent” program, for 
example, was largely blamed for resulting in the stigmatization of UK Muslims who expressed 
conservative viewpoints; the mere inclusion of such individuals in programming designated as 
“preventing violent extremism” made individuals feel targeted as “potential terrorists.”39 Likewise, “at￾risk community” strategies risk similar blowback, especially when targeted at faith communities. Such 
programming, like the New York Police Department’s “Muslim Surveillance and Mapping Program,” can 
result in discriminatory profiling that ultimately alienates members of the communities they are 
ostensibly intended to engage cooperatively and who are best-positioned to detect and potentially 
intervene in actual cases of violent extremism.40 
35 For a helpful and comprehensive review of CVE program studies prior to 2017, see: Gielen, Amy-Jane. "Countering Violent 
Extremism: A Realist Review for Assessing What Works, for Whom, in What Circumstances, and How?" Terrorism and Political 
Violence (2017): 1-19.
36 IE those short of legal or security measures such as arrests, freezing of finances, police surveillance, etc. 
37 Usually measured in terms of beneficiaries reached, program retention/completion rates, and other metrics that indicate 
whether the program operates as designed and reaches the intended participants. Some programs do include participant 
feedback (either qualitative in the form of focus groups, or quantitative, usually in the form of evaluation surveys), but this 
feedback usually revolves around similar measures which Gielen (2017) describes as “process evaluation.” 
38 Ibid (p. 6)
39 Thomas, Paul. "Failed and friendless: the UK's ‘Preventing Violent Extremism’ programme." The British Journal of Politics and 
International Relations 12.3 (2010): 442-458.
40 Schanzer, David H., et al. The challenge and promise of using community policing strategies to prevent violent extremism: A 
call for community partnerships with law enforcement to enhance public safety. Durham: Triangle Center on Terrorism and 
Homeland Security, 2016.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 11
A second major challenge to studying P/CVE program impact is with respect to program design and 
implementation itself: at what level of aggregation should programming be targeted? And which stage of 
radicalization or VEO involvement should be addressed by the program? These issues are examined 
further in the next section of this report, with the case of Bangladesh in mind. A third major 
challenge is in measurement of program effects: even if a P/CVE program is correctly implemented as 
designed and reaches its target audience, how would we know whether participants receiving 
“treatment” are actually being affected by it in ways consistent with the program’s policy goals? 
Outcomes of interest (which include concepts such as radicalization and the risk of participating in 
violent extremism/violent extremist groups, or violent extremism antecedent attitudes/behaviors41) are 
rare and difficult to detect in the form of observable behaviors. Attitudinal measures (i.e. surveys) must 
contend with social desirability bias, which can present challenges when using them for measuring 
program effects.42
In summary, the paucity of convincing evidence for the efficacy of P/CVE programs is not due to 
incompetence or carelessness on the part of researchers and practitioners, nor is it due to poorly￾designed programming. Rather, it is due to these aforementioned methodological challenges and the 
“needle-in-the-haystack” nature of the substantive problem itself. Despite these challenges, some notably 
well-designed studies have been helpful in clarifying areas where P/CVE programming might be most 
promising. Work by Aldrich (2012; 2014) studying USAID radio programs in Mali, Chad and Nigeria, for 
example, has provided quasi-experimental evidence that CVE messaging can alter norms of civic 
participation and behavior among individuals in ways suggesting a greater willingness to work with the 
West in combatting terrorism.43 
An evaluation by Williams et al. (2016) has provided evidence that community-focused secondary 
prevention programs may be effective when targeted towards peer “gatekeepers.”44
Given the difficulties in implementing and assessing efficacy of programs that attempt to identify at-risk 
individuals and/or intervene in cases further down the radicalization pathway, a more promising 
approach for P/CVE programming is likely in the area of prevention. Prevention can be achieved through 
counter-communication, resilience-building, and/or the provision of support to key networks and actors 
who are likely most efficacious in prevention efforts. 45 A useful taxonomy of P/CVE prevention 
41 Attitudes and behaviors that are often associated with those who later go on to participate in violent extremist groups 
and/or acts of political violence. These attitudes may be context-dependent in their specific manifestation, but often have 
common themes which include (but are not limited to) out-group disgust (i.e. racism, homophobia, anti-Semitism, etc.), 
misogyny, authoritarian personalities and views, a need for closure, or black-and-white worldviews. Associated behaviors might 
include, for example, hate speech, violence against members of the out-group (or the advocacy thereof), intolerance for 
differing beliefs/worldviews, or risk-taking behavior that is indicative of fatalism. It is important to note that violent extremism 
antecedent attitudes/behaviors are not ubiquitous among those involved in violent extremism, nor are those who exhibit such 
attitudes/behaviors certain to radicalize and pursue violence. For a further discussion, see: Gambetta, Diego, and Steffen 
Hertog. Engineers of Jihad: The curious connection between violent extremism and education. Princeton University Press, 2017. 
42 This problem is discussed at length by Horgan & Braddock (2010) in evaluating “de-radicalization” programs, which are often 
implemented in such a way that incentivizes participants to exhibit reformed attitudes and behaviors, often to avoid punishment 
or secure better prison conditions, early release, etc.; see” Horgan, John, and Kurt Braddock. "Rehabilitating the terrorists? 
Challenges in assessing the effectiveness of de-radicalization programs." Terrorism and Political Violence 22.2 (2010): 267-291.
43 Aldrich, Daniel P. "Radio as the voice of god: Peace and tolerance radio programming’s impact on norms." Perspectives on 
Terrorism 6.6 (2012): 34-60; Aldrich, Daniel P. "First steps towards hearts and minds? USAID's countering violent extremism 
policies in Africa." Terrorism and Political Violence 26.3 (2014): 523-546
44 Williams, Michael J., John G. Horgan, and William P. Evans. "The critical role of friends in networks for countering violent 
extremism: toward a theory of vicarious help-seeking." Behavioral Sciences of Terrorism and Political Aggression 8.1 (2016): 45-65.
45 Harris-Hogan, Shandon, Kate Barrelle, and Andrew Zammit. "What is countering violent extremism? Exploring CVE policy 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 12
programs is inspired by public health models, and separates such efforts into those deemed primary 
prevention (aimed at decreasing “new cases,” or incidence of radicalized individuals), secondary 
prevention (aimed at decreasing the prevalence of radicalized beliefs within a network and dissuading 
radicalized individuals from going on to potentially engage in violent extremism), and tertiary prevention 
(aimed at convincing those already engaged in violent extremism to disengage and de-radicalize).46
Notably, this model of radicalization remains agnostic to the potential actions taken by radicalized 
persons (which are not necessarily violent). Moreover, not all of those who commit acts of violence on 
behalf of VEOs have been radicalized; however, for non-conflict zones (and certainly the case of 
Bangladesh), radicalization is generally prerequisite to involvement in (and violent acts on behalf of) 
VEOs. 
Regarding this taxonomy of prevention-focused P/CVE programs, tertiary efforts pose the greatest 
difficulty for targeting and for evaluation, due to the methodological reasons discussed above. While 
“soft” programming that targets individuals already involved in violent extremism likely has an important 
place in policymakers’ repertoire of P/CVE tools, such efforts in practice are usually targeted to those 
who have already had contact with state security efforts (i.e., prisoners), and thus fail to reach those 
currently engaged in violent extremist activities for the obvious reason that such individuals are doing so 
clandestinely or are not otherwise reachable.
Primary and secondary prevention efforts form the bulk of prevention-focused P/CVE programming 
attempted thus far. The sparse evidence available from past evaluation paints a mixed picture regarding 
the efficacy of primary prevention P/CVE efforts, i.e., those designed to “inoculate” a population against 
extremist ideology through programs that address root causes of radicalization. These primary 
prevention programs are usually attempted through direct counter-messaging efforts, or indirectly by 
addressing psycho-sociological causes of radicalization, by imbuing critical thinking skills, or by raising 
awareness among families and support services. For example, several studies have more or less 
concurred that interventions aiming to address inter-group prejudice or extremism can be effective in 
changing norms of behavior, though not necessarily in changing the underlying attitudes and beliefs of 
individuals.47 Such findings suggest that prevention-focused P/CVE programs designed to persuade those 
already holding violent extremist antecedent attitudes, or who may sympathize with violent extremist 
groups, are likely to be ineffective, particularly if the program is short term and/or of low duration. 
Unlike efforts to change attitudes and beliefs, relatively well-documented findings are available to suggest 
that primary P/CVE interventions can change behavioral norms in areas substantively related to violent 
extremism. This evidence comes from evaluations of programs designed to combat racism, gender 
violence, and school bullying (among others).48 The hypothesized mechanism behind such programming 
and practice in Australia." Behavioral Sciences of Terrorism and Political Aggression 8.1 (2016): 6-24
46 Ibid.
47 Examples are varied, and include studies of interventions focusing directly on attitudes towards violent extremism (Aldrich 
2012; 2014), as well as those meant to address antecedent behaviors such as out-group prejudice, and violence against women 
(see: Scacco, Alexandra, and Shana S. Warren. "Can social contact reduce prejudice and discrimination? Evidence from a field 
experiment in Nigeria." American Political Science Review (2018): 1-24; Coker, Ann L., et al. "Evaluation of Green Dot: An active 
bystander intervention to reduce sexual violence on college campuses." Violence against Women 17.6 (2011): 777-796.) Doubts 
about the efficacy of persuasion attempts via short and medium-term programming are apparent in substantive foci beyond CVE 
and antecedent behaviors as well; regarding attitudes on dating violence, for example, Storer et al. (2016)’s review of 15 
programs found mixed evidence that bystander intervention programming reduced rape myth acceptance among men despite 
being effective at changing behavioral norms.
48 See, for example, Coker, Ann L., et al. "Evaluation of Green Dot: An active bystander intervention to reduce sexual violence 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 13
is that, by publicizing and reinforcing a norm against some socially undesirable behavior, it both decreases the 
likelihood of its expression among those holding attitudes that might lead them to express it, and empowers 
others who hold attitudes opposed to the behavior to further reinforce the norm against it. Applying such 
findings to primary prevention-focused P/CVE programs suggests that the most promising approaches 
might focus on changing the social milieu of acceptable behaviors regarding violent extremism 
antecedent behaviors, rather than attempting to directly change individual-level attitudes. 
An obvious impediment to primary prevention-focused P/CVE efforts is one of program scale, given that 
the potential social reservoir for “at-risk” individuals covers a broad swath of the population, even when 
accounting for some regularities among the eventual perpetrators of violent extremist acts.49 Moreover, 
while such efforts may be feasible in contexts with available resources to devote to wide-spread 
implementation of such programming, the very correlates that put a society at high risk of violent 
extremism- challenges such as unequal development and income inequality, political polarization, social 
cleavages, distrust in the criminal justice system, and strained social support services- also leave it 
unequipped to implement such measures at scale. Take the public health model of prevention, for 
example: deploying such efforts in a resource-limited context would be like implementing a hand￾washing campaign in a society that lacks clean water infrastructure and soap. 
Given the targeting and methodological impediments to tertiary prevention CVE programs, and the 
practical impediments to primary prevention, secondary prevention CVE programs are likely to offer the 
most cost-effective and potentially fruitful approach to addressing violent extremism in a context such as 
Bangladesh. Such efforts attempt to prevent further radicalization of individuals holding violent 
extremism antecedent attitudes, while empowering individuals within the networks of those at-risk to 
act upon witnessing violent extremism antecedent behaviors and therefore reduce their expression and 
diffusion. Specifically, an application of the BIT model for secondary prevention might offer a promising 
and practical method for CVE in a context such as Bangladesh. The next section of this report details 
the BIT model and its adaptation for secondary P/CVE purposes in Bangladesh. 
4. THEORY OF CHANGE
This evaluation considers a capacity-building secondary P/CVE strategy for creating social environments 
hostile to violent extremism, focusing on promoting the social norm of “tolerance” through bystander 
intervention training (BIT). Furthermore, the evaluation considers a BIT strategy focusing on “peer 
gatekeepers,” both as the individuals delivering the training, and those receiving it. As discussed, 
programs that attempt to target at-risk youth are usually of limited success because they require finding 
the proverbial needle in the haystack. A more effective strategy, then, may be to change the permissive 
social milieus in which hateful speech, harassment, and other antecedent behaviors of violent extremism 
are left unchallenged, leaving an open path for escalation toward actual violence. The principal 
investigators therefore proposed testing programs to change the social environments that may allow 
on college campuses." Violence against Women 17.6 (2011): 777-796.)
49 It is well-known, for example, that most individuals who commit acts of terrorism are young males, with some evidence 
suggesting that educated and middle-class males may be especially vulnerable, and even more so if they experience “significance 
loss events.” However, even with slightly better specific demographic and psychological predictors- the observation in 
Bangladesh, for example, that perpetrators tend to come from the educated middle class- the target population for primary 
prevention CVE programming would be too vast to realistically and efficiently reach. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 14
violent extremism to grow, delivered by those whose identities or social positions among the target 
audience would potentially facilitate said changes.
We ask: 
1. What is the effect of a tolerance-focused bystander intervention training P/CVE program on 
effective intervention to promote and protect norms and behaviors against P/CVE antecedent 
attitudes and behaviors among those receiving it?
2. Are trained peer leaders more effective at delivering said bystander intervention training program 
than traditional authority figures for the context?
We theorize that the tolerance BIT program will affect change through two pathways. First the program 
will change norms within the subject pool to reject hate and be more tolerant of various outgroups, 
such as Hindus, women, and Westerners. Second, the program will increase subjects’ capacities to take 
pro-tolerance actions and make the subjects more effective as pro-tolerance interveners by training 
them in effective social intervention techniques. 
The first pathway in the theory of social change is through improving norms of tolerance. The use of the 
term norm is unfortunately quite varied, referring sometimes to how the members of a group do behave 
and other times to how such members should behave. The former type of norm, how members of a 
group do behave, is called a descriptive norm and the latter type of norm, describing how members of 
the group should behave, is called a prescriptive norm.50 In theory individuals comply with norms because 
failure to do so risks some form of social censure (for example embarrassment, reprimand and 
exclusion). Psychologists frequently argue that individuals are often imperfectly informed about actual 
norms due to limited opportunities for observation and cognitive shortcomings.51 Thus, psychologists 
are often more interested in perceived norms rather than actual norms. We follow that convention in 
this report: when we use the term norms we are actually speaking of perceived norms. Most of our 
analysis focuses on prescriptive norms, although we do not ignore descriptive norms entirely.
Norms are to be distinguished from what psychologists call attitudes but what economists might call 
preferences. Attitudes/preferences are an individual taste for an activity. A person who engages in 
tolerant behavior because they like pleasant social interactions or were born with an elevated sense of 
empathy are indulging a preference for tolerance. Programs typically attempt to bring about social 
change by changing perceived norms rather than attitudes/preferences because the latter are thought to 
be harder to change than the former.52 Compliance with perceived norms is undoubtedly aided by a 
preference for the normative behavior but it is not necessary. Compliance with perceived norms 
operates through a desire for social acceptance. For example, people may recycle even when they find it 
50 Prentice, Deborah A. 2007. Norms, Prescriptive and Descriptive. Encyclopedia of Social Psychology, Roy F. Baumeister, and 
Kathleen D. Vohs, Sage Publications, 1st edition.
51 Tankard Margaret E. and Elizabeth Levy Paluck. 2016. Norm Perception as a Vehicle for Social Change. Social Issues and Policy 
Review. Vol. 10:1 181—211; Perkins, H. W. (2002). Social norms and the prevention of alcohol misuse in collegiate contexts 
Journal of Studies on Alcohol, Supplement, 14, 164–172 ; Perkins, H.W.,Meilman, P.W., Leichliter, J. S., Cashin, J. R., & Presley, C. 
A. 1992). Misperceptions of the norms for the frequency of alcohol and other drug use on college campuses. Journal of American 
College Health, 47(6), 253–25.
52 Tankard Margaret E. and Elizabeth Levy Paluck. 2016. Norm Perception as a Vehicle for Social Change. Social Issues and Policy 
Review. Vol. 10:1 181—211
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 15
inconvenient for fear of social sanction if they do not. Thus, changing perceived social norms can affect 
broad social change even when underlying attitudes or preferences remain fixed.53
Tankard and Paluck (2016) list three sources of information that can change perceived norms: the 
behavior of salient individuals, descriptive information about the community and institutional signals. 
They give the example of a hypothetical program to increase recycling in a community: “…a norm 
change intervention may, for example, expose people to a popular peer who recycles, provide people 
with information that most of their peers recycle, or advertise new recycling guidelines from an 
important and trusted community institution.” (p. 183).54
Recent research has indicated that changing individuals’ perceived norms can successfully produce 
changes in individuals’ behavior and broader social change through these three channels. For example, 
Paluck and Shepherd (2012)55 and Paluck, Shepherd, and Aronow (2016)56 report on an intervention 
that reduced high-school bullying by changing perceived norms through the first channel mentioned 
above: popular peers. Perkins and Craig (2006) employed the second channel, information about average 
peer behavior, to reduce alcohol abuse in college.57 Tankard and Paluck (2017) show how a Supreme 
Court decision changed the norm on same-sex marriage via the third channel, i.e., a trusted community 
institution, even when underlying attitudes on same sex marriage had not necessarily changed.58
The BIT program studied in this report aimed to change perceived norms through the first channel, i.e., 
through statements/behavior of salient individuals (the trained facilitators). We tested the ability of two 
sorts of salient individuals to convey the message of norms of tolerance: peer leaders and college 
instructors. 
We propose that effective intervention (i.e., the promotion and protection of desirable norms- in this 
case norms of tolerance and norms against violence) requires individuals to have internalized the 
prescriptive norms of tolerance, while also having both the competence and the confidence to publicly 
defend those norms in the face of hateful speech or actions. An individual’s confidence and competence 
to promote and defend prescriptive norms, in turn, is predicated on (1) the individual’s internalization of 
those norms, and (2) the internalization of capacities (i.e., skills) necessary to safely promote and defend 
the norms.59 We expect both a tolerance-focused bystander intervention curriculum, coupled with a 
53 Ibid
54 Ibid
55 Paluck, E. L., & Shepherd, H. (2012). The salience of social referents: A field experiment on collective norms and harassment
behavior in a school social network. Journal of Personality and Social Psychology, 103, pp. 899–915.
56 Paluck, E. L., Shepherd, H., & Aronow, P. 2016. Changing climates of conflict: A social network driven experiment in 56 
schools. Proceedings of the National Academy of Sciences
57 Perkins, H. W., & Craig, D. W. (2006). A successful social norms campaign to reduce alcohol misuse among college student￾athletes. Journal of Studies on Alcohol, 67(6), 880–889.
58 Tankard M.E., Paluck E.L. (2017). The Effect of a Supreme Court Decision Regarding Gay Marriage on Social Norms and 
Personal Attitudes. Psychological Science. 28(9):1334-1344. 
59 Note that the internalization of desirable (i.e. tolerant) norms is considered prerequisite to skill-building too. This reflects the 
fairly straightforward supposition that a person will not intervene to promote/protect a social norm if they themselves have not 
internalized it. Indeed, we expect that the majority of participants in a program of this nature will either (1) already have
tolerant norms, or (2) will be open to more tolerant views. We do not expect individuals with intolerant viewpoints to remain 
in a program of this nature by choice. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 16
trained facilitator implementing programmatic elements of the curriculum, to promote the 
internalization of norms and skills necessary for effective intervention. 
Our second proposed pathway is to build individual citizen’s capacity to safely confront and discredit 
hateful extremist views so that those who hold extremist views know that society will not tolerate 
them. We expect that this outcome (which we dub “willingness to intervene”) requires both the 
internalization of social norms (discussed above; in this case of tolerance) as prerequisite to an 
individual’s choice to intervene on behalf of those norms, and the internalization of capacity-building 
skills required to intervene. We expect that the internalization of norms and skills needed for 
willingness to intervene is driven both by participant exposure to a program designed to impart and 
teach those respective norms/skills, as well as the participant’s “buy-in,” or the extent to which they 
identify with the program goals, with those of their co-participants, and with the facilitator leading the 
programming. Finally, we expect that the use of program-taught skills for willingness to intervene 
requires a participant’s confidence and competence to actually act, should they find themselves in 
circumstances where willingness to intervene is possible. We visualize the aforementioned set of 
pathways from programming to willingness to intervene in Figure 1.
Figure 1: Pathways from BIT Programming to Willingness to Intervene
Programs based on the first proposed pathway—cultivating social norms of tolerance—have many 
models. One approach is the Teaching Tolerance curriculum of the Southern Poverty Law Center 
(http://www.tolerance.org). Teaching Tolerance is a literacy-based program that can be folded into 
elementary and high schools’ existing curricula. Their curriculum is targeted toward an American 
audience to address racism, religious intolerance, and anti-immigrant sentiment. However, we believe 
we have adapted it appropriately to the Bangladeshi social context in order to spread norms of 
tolerance, non-violence, and acceptance of secularism and other differences.
A second approach to changing norms is the Roots curriculum developed by Princeton Psychologist 
Elizabeth Levy Paluck and her colleagues.60 The curriculum was designed to reduce bullying in American 
60 See http://www.betsylevypaluck.com/roots-curriculum/.
Internalize 
Norms
Tolerance BIT 
Curriculum
Facilitator 
Identity
Internalize 
Skills
Confidence/ 
Competence 
to Act
Effective 
Intervention 
on Behalf of 
Victim
Identification 
with Facilitator 
and Program 
Goals
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 17
high schools, but the social theory behind it is widely applicable to many undesirable social behaviors, 
including extremism. For this program, the social networks of schools were mapped. Important “nodes'' 
in this social network—people with many friends, or people who are widely admired by their peers—
are enlisted and taught how to spread the desired norm. The theory behind the program is that these 
important nodes in the social networks are norms creators and spreaders, and their important social 
status can be leveraged to spread the desired norms and reduce antisocial behavior. A challenge to using 
this model for P/CVE purposes is that it requires the mapping of an established social network. Our 
target audience—first and second-year university students—included a large number of participants who 
had not yet established their own social networks in a new setting, therefore rendering a process of 
identifying central nodes, similar to that of the Roots program, difficult.61 Moreover, the university 
context of Bangladesh presents additional complications, given that student leaders are sometimes 
affiliated with student political groups known for violence and hazing rituals. Because we were interested 
in comparing the efficacy of peer leaders to that of traditional authority figures for the context, we 
deferred to university officials working with our implementation partner (Rupantar) to identify upper￾level university students with prior leadership experience (which is not tied to partisan organizations) 
for this purpose. 
Programming based on the second pathway—improving capacity and skills to safely intervene and 
counter hateful and violent extremist speech—can be modeled on a variety of “bystander training” 
programs developed for American campuses and high schools. In a randomized study in 26 Kentucky 
high schools over five years, one such program, The Green Dot (https://www.livethegreendot.com), was 
shown to reduce multiple forms of interpersonal violence.62 The program includes schoolwide training 
on bystander intervention by Green Dot educators and, like the Roots programs described above, it 
leverages student popular opinion leaders to change norms. StepUP (http://stepupprogram.org/),
developed at the University of Arizona, is probably the most widespread and commonly used program 
on college campuses in the United States. (The Active Zone bystander intervention program at NYU, 
where we work, is based on it). We are unaware of any randomized studies of the program, but it has 
been labeled “best practice” by the NCAA (National Collegiate Athletic Association), and has won the 
NASPA (Student Affairs Administrators in Higher Education) Gold Award. The program teaches a 
simple five-step decision making process for bystander intervention that should be easily translatable 
into the Bangladeshi context.63
FACTORS INHIBITING THE IMPACT OF BYSTANDER INTERVENTION TRAINING
Conditions in the study sites in Bangladesh are such that the BIT program will not emphasize identifying 
and connecting at-risk individuals with police or other P/CVE service providers. Even in the US, studies 
have found that while friends of at-risk individuals are usually best positioned for noticing early signs of 
violent extremism,64 they are typically reluctant to connect the at-risk person to police or P/CVE service 
providers.65 A series of studies on this issue found that the reasons cited for this reluctance mainly 
61 We found in baseline surveys, for example, that many students had difficulty naming five friends within their university who 
would potentially be interested in future iterations of the programming. 
62 Coker AL et al. 2017. RCT Testing Bystander Effectiveness Training to Reduce Violence. American Journal of Preventive 
Medicine. 52:566-78 (gated link to the study).
63 During December 2019 site visits with university officials, we found that program trainers and small groups of trainees 
believed that the “5 D’s” of bystander intervention training were easily understood and remembered by program trainees. 
64 Williams et al., (2016a)
65 Williams et al., (2016a)
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 18
centered on a fear of damaging their relationship with the person, fear of getting the person into 
trouble, and fear of getting into trouble themselves, should they attempt to connect the person with a 
CVE provider.66 Therefore, any programming that focuses on training peer gatekeepers should account 
for this reluctance in its design.67 We would assume that such reluctance would be relevant in our 
Bangladesh sites as well.
The bulk of research on secondary P/CVE programs has taken place in North America and Western 
Europe, where P/CVE service providers have been able to establish themselves as trusted intermediaries 
between communities and law enforcement. Such programs are often implemented as a bundle of 
interventions, which may include community education, volunteering and multicultural programming, in 
addition to cultivating resilience against extremism and building ties between communities and law 
enforcement.68 The success of a P/CVE service provider model is predicated on an assumption that 
neither the referring gatekeeper, nor the at-risk referent, will face punishment or stigma (either by their 
own communities or the criminal justice system/security apparatus), should an intervention take place. 
Unfortunately, this assumption is unlikely to hold in a context such as Bangladesh for a variety of 
reasons, including a closing civic space, a zero-tolerance approach by security forces towards those 
expressing sympathy with extremist groups, and widespread stigma around issues of mental health. Even 
if trusted secondary prevention CVE providers were to be available in this context, well-positioned 
gatekeepers such as friends would likely be even more reluctant to connect at-risk individuals with the 
providers for these reasons. When accounting for the lack of such providers, reluctant gatekeepers may 
also feel powerless to render help to an at-risk friend, given that doing so may result in extreme 
consequences for themselves and the friend. 
The aforementioned trade-offs between a gatekeeper’s social proximity to an at-risk person and their 
reluctance to intervene due to fear of repercussions suggests that adopting a capacity-building program 
to a context such as that of Bangladesh will require re-thinking what constitutes realistic and appropriate 
forms of intervention/assistance, and how to train individuals to safely implement a decision to assist 
that accounts for risks to themselves and the person they’re attempting to help. In contexts of extreme 
reluctance among potential gatekeepers, a best alternative approach may be to empower bystanders to 
engage directly and privately with those they notice are at-risk of violent extremism, while providing 
them with an awareness of resources that can help them offer instrumental, informational, affiliative, or 
emotional support—hence the potential value of a bystander intervention training program, which is 
designed to train participants how they can respond as individuals to “social emergencies” without the 
immediate availability of authorities/service providers.69 BIT can also equip individuals to safely signal to 
their peers that they themselves can serve as a confidential resource for those concerned about others, 
either by helping them to accurately identify warning signs, or to steer them towards an intervention 
plan. 
66 Ibid; Williams et al. (2016a) note that variation in reluctance likely exists among individuals by personality type and 
attachment style. 
67 The BIT program we designed for Bangladeshi universities does not rely on P/CVE “service providers.” In addition to the 
peer reluctance issues, our scoping missions found that very few potential P/CVE service providers exist for the context. 
68A prominent example being the World Organization for Resource Development and Education (WORDE) “portfolio 
approach” to countering violent extremism, which consisted of several interlocking programs; see evaluation by Williams et al. 
(2016b). 
69 Ibid.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 19
Finally, the challenges of social and political contexts like those of Bangladesh require a focus on helping 
individuals at earlier stages of the radicalization process than those in Western contexts, due to the 
risks of stigma and/or harm to both a bystander who intervenes, and to the individual they are trying to 
help. It is well-established, for example that those who sympathize with or go on to support and join 
violent extremist groups frequently (though not always) develop and exhibit attitudes and behaviors that 
dehumanize/express disgust towards targeted social groups, and ascribe blame and sanctioning to 
members of that outgroup for perceived transgressions. Other common antecedent attitudes/behaviors 
include sentiments of in-group victimhood, collective narcissism, and/or in-group nationalism. Violent 
extremist groups both promote the development of such attitudes in their messaging, and recruit from 
those exhibiting them. Thus, in a context where P/CVE interventions focusing directly on those who 
have radicalized and are at high risk of violent extremism are unlikely to be fruitful. A better alternative 
may be to focus on combating norms of identity-based intolerance and promoting norms of out-group 
tolerance. 
ADAPTING THE BYSTANDER INTERVENTION MODEL TO REDUCE VIOLENT EXTREMISM 
ANTECEDENT BEHAVIORS
The classic model of bystander intervention (focusing on medical emergencies), described by Latane and 
Darley (1968), includes five steps from the occurrence of an emergency to a person’s intervention in 
order to render assistance. These steps include (1) noticing the event, (2) interpreting it as an 
emergency, (3) assuming responsibility to act, (4) knowing an appropriate form of assistance, and (5) 
implementing a decision to assist. Each of these steps pose challenges which may lead to a failure to act. 
The well-known “bystander effect,” for example, is caused by a failure of individuals to assume personal 
responsibility to act, which increases in likelihood with the presence of more bystanders (i.e., the 
diffusion of responsibility). The goal of BIT programs is to train individuals to recognize these sources of 
intervention failure, take responsibility for, and then safely and effectively respond to problematic 
situations in lieu of official help channels. In order to achieve this goal, the training usually focuses on 
addressing impediments to acting by empowering individuals with a toolkit of potential actions they 
could take under various circumstances.70
A key advantage of adopting a BIT model for P/CVE purposes is that it focuses on intervening in ways 
that are safe. BIT explicitly addresses and prioritizes safety for trainees and others, by teaching trainees 
how to evaluate the urgency and risks posed by different situations in which they might find themselves. 
Although BIT is designed to teach trainees to overcome the “bystander effect,” it does not encourage 
trainees to engage in “heroic” behaviors which would put themselves (or potentially those they are 
trying to help) at risk of injury that can result from escalating a situation.71 The training usually 
discourages the use of direct confrontation unless the bystander has safety in numbers, and instead 
70 A canonical example of such a program for medical emergencies is training provided by the American Red Cross, which 
teaches bystanders how to recognize a medical emergency, assess the safety of the situation/area, direct others in the area to 
provide support by alerting emergency services, clearing a location, etc., and then rendering first aid until help arrives. For 
versions adopted to preventing sexual assault/harassment, this protocol might take the form of recognizing aggressive behavior 
at a party, assessing whether it has the potential to escalate, recruiting assistance from others nearby if available, and then 
intervening usually by distracting the perpetrator and then checking in with the victim once the incident ends. 
71 The issue of safety and potential escalation was also a key area of focus during September, 2019 facilitator training sessions. 
For the context of Bangladesh, facilitators noted that caveats applied to the use of some of the “5 D’s” strategies. For example, 
the “documenting” strategy (in which a bystander documents evidence of a perpetrator’s actions for potential later use in legal 
proceedings) might be inappropriate to use under circumstances when perpetrators have connections or influence over local 
security authorities.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 20
promotes de-escalatory strategies such as the creation of a distraction in order to interrupt a 
perpetrator’s behavior before it becomes dangerous.
An additional advantage of adopting a BIT model for secondary prevention P/CVE in a context such as 
Bangladesh is that it does not presume or require the availability of P/CVE service providers, which 
generally have been the crux of community-focused secondary P/CVE programming implemented in 
Western democracies. Given shrinking civic space, crackdowns by the state security apparatus, and 
enduring stigma towards the discussion and treatment of mental health issues, a secondary P/CVE model 
premised on gatekeeper referrals to P/CVE service providers in the Bangladeshi context would likely be 
ineffective at best, and potentially dangerous to both referents and referees at worst. As discussed, prior 
findings (in the US) suggest that peer gatekeepers would hesitate to refer an at-risk friend to support 
services due to a fear of harming their relationship with the friend;72 such concerns would likely be 
exacerbated within the Bangladeshi context. Thus, a BIT model could provide individuals with a toolkit 
of approaches they can use upon witnessing incidents of violent extremism antecedent behavior that do 
not necessarily require involving third parties, or escalation beyond the immediate incident at hand. The 
goal of such a program would be to empower individuals to recognize and safely respond to violent 
extremism antecedent behaviors, focusing in this case on the antecedent behaviors of intolerance 
towards out-groups.73 We detail in the following sections how we designed an intervention that adopted 
a tolerance-focused BIT program to the Bangladeshi context and implemented it as an RCT. 
5. IMPACT EVALUATION METHODOLOGY
The main purpose of this IE is to estimate and compare the effects of the aforementioned BIT P/CVE 
strategy for (1) reducing the support for and expression of violent extremist antecedent behaviors, and 
(2) promoting the safe exhibition of pro-social behavioral norms in order to create social climates 
hostile to violent extremism in the form of bystander intervention. We pursued this goal through three 
steps, detailed in this section of the report. First, we partnered with a Bangladeshi NGO (Rupantar) to 
adopt a tolerance-focused BIT program for Bangladeshi university students and train university￾nominated peer and authoritative influencers to deliver it. As a feature of the experimental design, we 
also assisted Rupantar in developing a placebo curriculum of a similar duration/pacing as the BIT 
treatment. Furthermore, during April 2019 design meetings with Rupantar, we trained members of the 
implementation team to implement RCTs.74
Second, after working with Rupantar to develop a Bangladeshi BIT curriculum and train facilitators to 
correctly implement it, we conducted a randomized, placebo-controlled trial featuring three treatment 
statuses over two treatment waves into which individual participants were recruited and randomized, as 
follows:
72 Williams et al. (2016a).
73 “Out-groups,” in this case, are defined as those which have experienced recent violence and targeted hate speech in 
Bangladesh. These have included (but are not limited to): Hindus, Buddhists, Christians, Shia Muslims, Atheists, LGBTQ 
persons, Westerners/foreigners, and those holding secular and/or socially liberal political attitudes.
74 These training sessions were extremely helpful to ensuring that the experimental protocol was followed. Namely, Rupantar 
team members followed the randomization protocol closely and there were fewer than 5 participants (out of more than 1400) 
who were enrolled within a different treatment group than that to which they were randomly assigned. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 21
• Treatment 1: Delivery of a secondary P/CVE program in the form of a tolerance-focused BIT 
youth leadership program via top-down authoritative influencers, 
• Treatment 2: Delivery of a secondary P/CVE program in the form of a tolerance-focused BIT 
youth leadership program via horizontal peer influencers, 
• Placebo: Delivery of a youth leadership program focused on challenges of development within 
Bangladesh, led by authoritative influencers as in Treatment 1 above. 
• No-Program Control (Wave 2 only): Given that applicants to the program exceeded available 
slots (and that many interested students could not participate due to schedule conflicts), we added a 
no-program control to the study during wave 2 incentivized with a small gift card to a popular online 
merchant. Unfortunately, recruitment for the no program control was disrupted due to country￾wide closures of the university system in late March, 2020 because of the Covid-19 pandemic. We 
discuss implications in subsequent sections. 
Third, in order to measure intervention efficacy, participants across all treatment statuses completed 
baseline and endline surveys that included measures related to norms of tolerance, radicalization 
antecedent attitudes, and views on violent extremism. In addition, the surveys included several vignettes 
designed to elicit whether and to what extent participants would report their likelihood of reacting to a 
situation presented to them using one or more bystander intervention tactics. The surveys also included 
perceptions and evaluations of the participant’s facilitator, which are used in standard course 
evaluations. Moreover, information was collected on participants’ demographic and socio-economic 
characteristics, as well as a battery of psychosocial and community characteristics at baseline. The 
following subsections detail the intervention design, site selection, participant recruitment, and 
randomization procedure. We discuss our measurement strategy and data collection in subsequent 
sections. 
INTERVENTION DESIGN: ADAPTING BIT TO BANGLADESHI UNIVERSITIES 
What should bystander intervention training to promote tolerance and deter hateful, intolerant 
behaviors and speech look like in the Bangladeshi context? There were no analogous initiatives within 
Bangladesh at the time of program development; the consideration of P/CVE issues and violent 
extremist antecedent behaviors have largely been considered criminal justice matters until recently.75 In 
order to design the programming, we drew on content from existing publicly available BIT programs 
that have been implemented within U.S. universities and other youth settings, and worked closely with 
Rupantar to adapt the curriculum for university students in Bangladesh. Rupantar is a well-established 
NGO working within Bangladesh, and was chosen as implementation partner for the programming due 
to their extensive prior experience working within the university system, as well as their ability to scale 
for reaching sufficient enrollments for conducting the program as an RCT. 
75 This became a clear theme to us from November, 2017 and June, 2018 interviews with multiple representatives from NGOs 
and universities engaged in programming designed to build confidence and promote mental health among young people. 
Although a variety of NGOs are engaged in programming meant to, for example, built leadership skills among Bangladeshi 
youth, most are focused on building career-oriented skillsets. While mental health programming is becoming more common in 
Bangladesh, there are still social taboos associated with discussing mental health issues and how they might be associated with 
vulnerability to radicalization. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 22
The final version of the tolerance BIT program76 consisted of 14, two-hour sessions, for a total of 28 
hours of programming. Our evaluation of evidence for the efficacy of BIT programs targeting different 
anti-social behaviors in universities suggested that the frequency and duration of such programs matter 
for retention and application of the training. The vast majority of programs targeting, for example, sexual 
harassment and dating violence, only seem to show long-term impact among participants when the 
programming is implemented across multiple sessions.77
In most cases, participants met twice a week for the sessions, although exact schedules varied by 
university and facilitator. The programming was designed to accommodate early drop-outs and 
replacements, as well as the measurement strategy we devised for evaluating efficacy. The bulk of the 
substantive programming was delivered during sessions 3-11. Early university students (in their first or 
second-year) received the programming in small/medium-group format, with classes capped at 20 and 
usually ranging from 10-15 after accounting for attrition.78
The structure of the curriculum itself focused on developing (1) domain knowledge development 
through lecture and class discussion about the issues of tolerance/intolerance within Bangladesh, 
followed by (2) bystander intervention training. Because the program was designed as an RCT with a 
placebo, the curriculum was marketed as a “youth leadership training program” and also integrated 
sessions meant to develop generic leadership skills (such as public speaking and group presentations). A 
short summary of the topics of focus for each of the 14 sessions is as follows:
• Sessions 1-2: Leadership training (i.e., public speaking skills, group presentations, debate)
• Sessions 3-6: Domain knowledge training (intolerance within Bangladesh)
• Sessions 7-11: Bystander intervention training 
• Sessions 12-14: Application of BIT module via activities (role-playing, implementation 
intentions, skits).
Finally, a separate placebo curriculum was developed with the help of Rupantar. The placebo had 
participants meeting for the same frequency/duration in small groups with facilitators. However, the 
placebo curriculum focused on the challenges associated with rapid development in Bangladesh, including 
transportation issues (i.e., urban traffic), environmental issues (i.e., pollution), and public health issues 
(i.e., smoking and fast food). We were careful during the design of the placebo to avoid creating content 
that would promote focused discussion on intolerance and/or issues related to P/CVE. Moreover, 
Rupantar was well aware of the importance of this as a key feature in the experimental design, following 
our training sessions with them. Joint curriculum development and training sessions with Rupantar 
occurred during and after our first meetings with them in April, 2019. 
76 The detailed curriculum is available in the appendices of this report, and separately in the document entitled. 
77 We were aware of a multitude of other P/CVE programs implemented under the Obirodh initiative as shorter duration 
programs- usually in the form of single, day-long workshops. We learned early during scoping missions that various self￾improvement workshops are extremely common, and that they are often competing for limited student attention/time. 
78 Most university classes in Bangladesh are extremely large. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 23
SITE SELECTION, PARTICIPANT RECRUITMENT, AND RANDOMIZATION PROCEDURE
Parallel to curriculum development and training sessions with Rupantar, we solicited interest from 
Bangladeshi universities that were (1) interested in hosting the programming within the targeted cities of 
Dhaka and Rajshahi,79 (2) large enough for sufficient subject enrollment to take place for the needs of 
the RCT, and (3) somewhat representative of the diversity of Bangladeshi higher education institutions 
and students enrolled within them. A total of 9 schools (5 in Dhaka, 4 in Rajshahi) were formally 
approached following earlier scoping missions, or at the suggestion of Rupantar. Memorandums of 
understanding (MOUs) were eventually signed with 6 schools (Table 1). 
Table 1: Participating Public and Private Universities in Dhaka and Rajshahi
Location Universities
Dhaka Daffodil International University (Private)
Jahangirnagar University (Public)
University of Dhaka (Public)
Rajshahi Rajshahi College (Public)
University of Rajshahi (Public)
Varendra University (Private)
Besides offering large resident student populations from which to recruit participants, the public 
universities above host students from across the entire country. While we found that the bulk of 
participants were from the Dhaka or Rajshahi divisions, there was meaningful representation (at least 
100 participants) from those whose home villages fell within 7 of the 8 administrative divisions of 
Bangladesh. Moreover, we were keen to include some of the private universities within the study in 
order to attract interest from students from differing socioeconomic backgrounds.80 The six schools 
above signed MOUs with Rupantar in order to begin implementing the programming beginning 
September, 2019. 
Trainers (called “facilitators”) were recruited during August 2019 by their respective universities, 
subject to constraints on age and career progression that were requested and later verified as a feature 
of the research design comparing peer leaders to authoritative figures.81 The facilitators were provided 
79 Following consultations with USAID, Rajshahi replaced Chittagong as the second targeted city due to Rupantar’s prior access 
to universities within it, and the fact that Rajshahi has experienced problems with violent extremist organizations. 
80 Generally, private universities reflect students from higher socioeconomic backgrounds with local origins. The public 
universities, which are heavily subsidized by the government, reflect a much broader (albeit high-achieving) cross-section of the 
population due to the national exam system, which provides a merit-based admissions criterion. This was borne out in our data 
collection, which revealed participants from a majority of the 64 administrative districts (and all 8 divisions) of Bangladesh. 
Missing from the study is representation from those within the post-secondary parochial (madrassa) system, though the 
participants study exhibited wide variation in, for example, religiosity and religious conservatism, among other factors we might 
expect to distinguish a madrassa from universities. 
81 With input from USAID, we defined “peer leaders” as upper-level university students or recent graduates from the university 
under the age of 25 who hold/held positions of leadership within the school. As we discussed, we avoided using a peer￾nomination process in order to choose these individuals due to the influence of student political groups. “Authoritative leaders” 
were defined as university professors/lecturers currently employed by the university. Aside from asking participants how they 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 24
with both pedagogical training for moderating small-group student-led discussions,82 as well as 
substantive training for the programming during a 4-day training session. The training sessions and a 
short pilot study of the curriculum content were organized and led jointly by Rupantar and by a member 
of the IE team during early September 2019 in Dhaka. Facilitator input was repeatedly solicited during 
the training and pilot sessions in order to ensure (1) that the curriculum elements (and BIT especially) 
were being properly translated for the target audience, and (2) that the facilitators themselves had 
agency over shaping a program that they would later be expected to lead.83
Participants were recruited by Rupantar team members using flyers with QR codes leading to a short 
online interest form. Assistance was also provided by a small group of students (one from each 
university) hired to work as technical assistants for the programming. The programming was officially 
marketed as a youth social leadership training seminar for first and second year students. A generic 
program summary was made available detailing some of the soft skills training which was embedded into 
both treatment and placebo curricula. As an incentive to participate, those who attended at least ten of 
the fourteen sessions the program were offered a certificate of completion84, provided by 
representatives at NORC at the University of Chicago and New York University. Rupantar also 
frequently provided snacks and/or lunch in some cases, while the universities furnished meeting spaces 
and other necessary logistics. A voluntary program of this nature was expected to draw those interested 
in “leadership training” who had the schedule availability to do so. It is thus expected that the 
participants generally reflect those who are prosocial and outgoing among the student body; although, 
mitigating this to some extent is the fact that leadership qualities are highly valued soft skills within the 
context, which likely explains the widespread interest in the program.85 One could imagine a version of 
the programming (beyond the capacity of the study) that was mandatory, perhaps implemented as a 
required “first year seminar” in which all students were expected to attend. Although we cannot directly 
compare the profiles and personalities of participants to those who did not participate, we provide 
some comparisons between baseline characteristics of those who finished the program, and those who 
did not finish the program (detailed in the baseline characteristics section, below). 
We received interest from over 3,400 applicants (about 2,600 of whom were deemed eligible first and 
second-year students within the age range) across the six schools over the two waves of programming. 
Applicants were then asked to complete a secondary enrollment form, including requisite consent for 
research forms, scheduling questions for logistics, and a link to part 1 of the baseline survey,86 which 
perceived their facilitators (which indicated meaningful differences in perceptions towards the two categories), we also 
collected objective demographic/background data on each facilitator (age, gender, highest degree, prior teaching experience, 
and current employment). We were able to verify, for example, that the median age of “peer leaders” was 24, versus 36.5 for 
the professors.
82 A separate “teaching guide” was created for the purposes of training in order to assist facilitators in understanding the 
differences between lecturing, group discussion, and group activities. 
83 Training sessions with the authoritative figures (i.e., university professors) featured much more back-and-forth discussion 
about the appropriateness of certain topics and activities. For example, some time was spent discussing whether to integrate 
LGBT issues into the curriculum- the facilitators warned that because LGBT issues remain relatively taboo within Bangladeshi 
society, formally integrating LGBT topics into the curriculum would pose an attrition risk. Instead, it was decided that 
facilitators would moderate and allow open discussion of these topics as students raised them during sessions, without formally 
introducing them in presentation materials or activities. 
84 Facilitators were given some latitude to make case by case basis decisions about attendance requirements. Facilitators were 
also tasks with keeping attendance records, which were later provided to the PIs both for use in the IE and for awarding 
completion certificates. 
85 Field notes from PI scoping missions and input from both Rupantar and participating universities.
86 Administered using Qualtrics, an online survey platform. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 25
contained most of the questions regarding demographics, psychosocial characteristics, and other 
background questions. Completion of the secondary enrollment form enabled placement into the 
randomized lottery for admission into the programming. 
In order to facilitate replacing no-shows and early drop-outs (which we expected to be common for a 
new program that is competing for the attention of busy university students), applicants were 
randomized into one of three ordered admission lists. Rupantar was instructed to offer admission to 
participants into the programming in strict order of appearance on this list. Admitted participants who 
were ultimately unable to attend the sessions due to scheduling or exam conflicts, who failed to respond 
to attempts at contact post-application, or who dropped out after up to two sessions, were quickly 
replaced with participants further down the list.87 Rupantar was given leeway to replace drop-outs 
within the sessions of each facilitator up until the third session, at which point the treatment and 
placebo curricula diverged. This strategy was designed to ensure that each facilitator’s sessions were 
kept as full as possible, without violating the randomization procedure, and while fulfilling our 
commitment to using a fair lottery for participation. 
Finally, during wave 2, we opted to add a no-program control (NPC) to the study due to a large number 
of interested participants who did not participate in the programming because of scheduling conflicts, 
exams, or other logistical reasons. Participation in the NPC was incentivized by offering a small gift card 
(500 BDT) to a popular Bangladeshi online merchant Daraz.com for completing baseline/endline surveys. 
Recruitment for the NPC was disrupted by the country-wide shutdown of universities due to the Covid￾19 pandemic, resulting in only 204 participants completing endline activities. 
SAMPLE SIZE, DATA COLLECTION STRATEGY, AND TIMELINE
We calculated minimum sample sizes of about 310 subjects per treatment condition (assuming attrition) 
for minimum detectable effects (MDE) of 0.2 standard deviations in outcome measures. We ultimately 
proposed sample sizes of 400 per treatment condition in order to account for potential attrition during 
programming and any missing/incomplete data. Through the application process discussed in the prior 
section, 2,383 eligible applicants (across all universities and both waves of the study) were randomized 
into admission lists. Eventually, 1,662 were assigned to facilitators, meaning that they did not drop out 
due to scheduling or exam conflicts, or failed to respond to Rupantar’s attempts at coordinating 
schedules. A total of 1,468 attended at least 1 session of the programming, and 1,120 attended at least 
ten sessions of the programming. Importantly, 437 and 439 of those assigned to peer and authoritative 
treatment statuses completed at least seven sessions of the programming (meaning they were exposed 
to the BIT module). 
We collected all data electronically88 using Qualtrics and Inquisit in order to enable participants to 
respond to questions in private. All surveys and other measurement materials were presented in 
87 Attrition rates varied across universities and waves, but averaged around 20%. Most participants who dropped out did so 
before session 1 of the programming, and cited schedule or exam conflicts as the reason. For a new program competing for 
time/attention of university students, this was roughly what we’d expect. 
88 Our meetings with Rupantar suggested that device/internet access would not be a problem, given that the vast majority of 
students at the universities participating in the programming had access to some form of smartphone and internet. We verified 
this to be the case in baseline surveys; about 96% of participants reported having access to either a smartphone and/or laptop. 
The remaining participants likely completed surveys in university computer labs. Electronic/remote data collection proved 
fortuitous given that Covid-19 related university shutdowns and revised IRB protocols would have rendered a paper and/or live 
interviewer approach impossible for wave 2 participants. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 26
bilingual (English and Bangla) format.89 Participants had the ability to save and return to partially 
completed surveys during the survey periods, due to survey length. We were able to collect complete 
baseline/endline data from 418 and 422 of those assigned to authoritative/peer tolerance BIT 
treatments, respectively, which met our calculated sample size requirements for comparison. Moreover, 
we collected data from 406 placebo participants and 204 no-program control participants. 
The context and subject matter of questions being asked required the data collection and measurement 
strategies to consider and mitigate demand effects and social desirability bias. The latter issue (social 
desirability bias) was taken especially seriously, given that participant safety issues also had to be 
considered. For example, the political and social context within Bangladesh rendered direct questions 
about specific violent extremist organizations or political groups undesirable for inclusion in surveys, 
both due to safety concerns for participants and because answers to such questions would likely suffer 
from social desirability bias. Although the use of a placebo should account for demand effects90 while 
mitigating social desirability bias to some extent,91 it is possible that participants enrolled in the 
tolerance BIT program perceived “tolerant” answers to questions as socially desirable. We discuss how 
the measurement strategy accounted for some of these issues in a subsequent subsection on outcomes 
(below). Features of data collection protocols to mitigate these issues are described as follows. 
In order to mitigate social desirability bias as much as possible within the data collection protocols, all 
data collection activities that included sensitive questions and those used as outcomes92 were 
administered separately from programming. Participants were asked to complete these activities in 
private, within a generous time frame. Most participants used their personal devices or laptops to 
complete them, although some likely completed them on lab computers or a friend’s device.93 The PI’s 
made it very clear throughout programming and measurement activities (both to facilitators and to the 
participants themselves) that responses they provide in all surveys would be stored in US-based servers, 
only accessible by the NYU-based principle investigators. We clarified this in plain (English and Bangla) 
language by asking facilitators to tell their students that “no person in Bangladesh will ever see the 
answers you provide.”94 (This was also printed within all data collection emails/SMS messages used). 
During training sessions with the facilitators, the PI’s included a segment on data privacy and 
confidentiality, where it was also emphasized that the purpose of participant survey responses was not 
to evaluate their individual job performance. Finally, NORC at the University of Chicago hired and 
trained several “technical coordinators” at each of the host universities, who were available to assist 
participants with technology/access issues regarding data collection activities.95 Thus all data collection 
activities were directly and exclusively overseen by the PIs, with any ground-level technical assistance 
89 Rupantar translated all measurement materials, while a Bangla-speaking research scientist at NORC at the University of 
Chicago assisted with verifying translation accuracies. English and Bangla were juxtaposed within the same survey/measurement
items in order to enable participants (the vast majority of whom had some level of bilingual proficiency) to read in their 
dominant language, while gleaning additional clarification/nuance from the second language if needed. 
90 The ability of participants to infer the purpose of measurement activities and thus provide “correct answers” should not 
differ between treatment and placebo. 
91 All participants were aware, for example, that they were participating in a civic-focused USAID-funded leadership program. 
92 Part 1 of the baseline, which included demographic and psychosocial baseline characteristics, was completed at the end of 
session 1 of programming; this survey was framed as a “background and personality survey.” 
93 Via anecdotal reports from NORC-hired “technical coordinators.” 
94 Students were also informed that their personally identifiable information would be removed from any final data sets, 
whereas any publicly accessible data will contain only de-identified data. 
95 Technical coordinators were provided with laptops which could be lent to participants for completing survey activities. The 
coordinators were instructed to ensure that the participants could do so privately. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 27
provided by those without a contractual relationship with Rupantar or the trained facilitators. Finally, 
participants were also reminded during measurement activities (both in the text and at the beginning of 
programming) that their survey/activity responses were not a factor in determining eligibility for the 
certificate of completion awarded for participation.96
Table 2 summarizes the timing of data collection for each treatment status and wave of the study.
Table 2: Data Collection Timeline
Treatment Placebo
Peer
Tolerance
Prof/
Tolerance Placebo
Peer/
Tolerance
Prof/
Tolerance
Pure 
Control
Wave 1 1 1 2 2 2 2 
Intake data October 2019
(N=240)
October 2019
(N=243)
October 2019
(N=241)
January/ 
February 2020
(N=266)
January/ 
February 2020
(N=278)
January/ 
February 2020
(N=277)
January/ 
February 2020
(N=347)
Baseline October 2019
(N=196)
October 2019
(N=202)
October 2019
(N=199)
February 2020
(N=241)
February 2020
(N=249)
February 2020
(N=234)
February 2020
(N=347)
Endline 1 December 
2019/ January 
2020
(N = 185)
December 
2019 /January 
2020
(N= 190)
December 
2019/ January 
2020
(N= 190)
March, 2020
(N=221)
March 2020
(N=227)
March 2020
(N=229)
March 2020
(N=204)
Endline 
Behavioral 
April 2020
(N= 57)
April 2020
(N= 48)
April 2020
(N= 51)
April 2020
(N=173)
April 2020
(N=178)
April 2020
(N=163)
April 2020
(N= 98)
Endline 2 May 2020
(N=149)
May 2020
(N=172)
May 2020
(N=161)
Notes Jahangirnagar 
University had 
a three week 
delay.
Jahangirnagar 
University had 
a three week 
delay.
Jahangirnagar 
University had 
a three week 
delay
Endline 2 used 
for those 
affected by 
Covid-19 
delay
Endline 2 used 
for those 
affected by 
Covid-19 
delay
Endline 2 used 
for those 
affected by 
Covid-19 
delay
OUTCOMES OF INTEREST AND MEASUREMENT STRATEGY
We measure participant norms, skills, confidence and competence to act, and willingness to intervene in 
defense of a victim using a combination of survey instruments and a web-based stimulus activity. 
Summary statistics for all outcomes are presented in table 14 (appendices). Each component of the 
measurement strategy (following the pathways articulated in section 4) is discussed in the following 
subsections below: 
96 Participants were told that attendance of at least 12 of the 14 sessions (with some variation depending on school 
circumstances, and some exceptions made on a case by case basis by the facilitator) was the only requirement for receiving the 
certificate. In practice, the vast majority of participants who attended at least 10 sessions were awarded the certificate. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 28
NORMS
We consider two categories of social norms that the tolerance BIT programming was meant to 
promote: (1) norms of tolerance and (2) the rejection of violence. These, in turn, were measured as 
follows:
NORMS OF TOLERANCE AND INTOLERANCE
We constructed indices for a battery of outcomes measuring changed norms of tolerance and 
intolerance. The theme of promoting and protecting norms of tolerance was a key component of the 
program. The broad nature of the concept of tolerance implied that we should operationalize it by 
measuring changed attitudes regarding a variety of salient social cleavages in Bangladesh. We also 
focused on religious tolerance in depth by using an Inquisit-based stimulus activity designed to sensitize 
respondents to contemporary religious conflict before measuring attitudes. We divide tolerance and 
intolerance measures into two groupings, by modality of measurement.97 First, we create aggregated 
indices measuring the following, both at baseline and endline:
• Religious Outgroup Disgust: An index of survey items asking the respondent the extent to which 
they agree that different religious traditions (omitting their own) are a “corrupting influence within 
Bangladesh.” The intention of the program was to promote norms against such disgust.
• Hostile and Benevolent Sexism: Adapted from the Glick and Fisk (1996) Ambivalent Sexism 
Inventory Scale.98 We distinguish hostile sexism (which is a measure of the extent to which the 
respondent perceives women in society as threatening) from benevolent sexism (which is a measure 
of the extent to which the respondent believes that women should be protected and cherished 
within society). The intention of the program was to promote norms against such sexism.99
• Anti-LGBT Disgust: Although the programming did not include formal content on LGBT issues, we 
included a measure of attitudes towards LGBT persons in baseline/endline surveys as a single 
question adopted from Harek’s (1984) 20 item attitude towards LGBT persons. The intention of the 
program was to promote norms against such disgust.
• Anti-Western Attitudes: An index of questions asking the participant the extent to which they 
believe that secularism and the West are corrupting influences in Bangladesh. The intention of the 
program was to promote norms against such attitudes.
• Distrust of Foreigners: An index of questions asking the participant the extent to which they are 
suspicious of foreigners, and the extent to which they believe that foreign customs are changing 
97 We divide the analysis by measurement modality because April 2020 Covid-related shutdowns impacted data collection for 
the Inquisit experiments. The experiments were originally planned with an on-campus computer lab option with assistance from 
a trained student “tech coordinator.” However, due to Covid-19 shutdowns, they were instead administered entirely remotely 
using the Inqusit app. While this proved successful for about half of the sample, many participants could not successfully run the 
app on their devices, given that the software required Android versions 6.0 or above at the time. For the statistical analysis, we 
have thus separated the Inquisit-based measures from Qualtrics survey measures due to smaller sample sizes. 
98 Glick, Peter, and Susan T. Fiske. "The ambivalent sexism inventory: Differentiating hostile and benevolent sexism." Journal of 
Personality and Social Psychology 70.3 (1996): 491.
99 The curriculum did not explicitly distinguish the technicalities that differentiate hostile sexism from benevolent sexism, 
though we believe that examples of both were discussed in sessions. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 29
their own customs too much. The intention of the program was to promote norms against such 
distrust.
We further explored the concept of religious tolerance/intolerance by using an endline-only web-based 
activity that presented respondents with a vignette designed to sensitize them to contemporary religious 
conflict. Respondents were asked to read an excerpt from an April 2020 BBC article documenting anti￾Muslim riots occurring in neighboring India.100 Following the stimulus, respondent attitudes were 
measured (using a Likert agreement scale) for the following concepts, which we collectively dub 
“religious group status:” 
• Religious Nationalism: A statement that “Bangladesh is a Muslim country above all else” which 
measures the extent to which the respondent ties his/her national identity to their religious identity. 
The intention of the program was to dampen attachment to such nationalism.
• Religious Segregation Preferences: An index of two items stating that Hindus (Muslims) who live 
in non-Hindu (non-Muslim) countries should move to Hindu (Muslim) countries, respectively. 
• Religious Collective Narcissism: A statement that in Muslim countries, Muslims should have more 
political rights than other groups. 
• Religious Perceived Group Victimhood: A statement that Muslims around the world are under 
threat.
Together, these measures capture the extent to which participants perceive the world around them in 
terms of potential group status loss, characterized by religious identity. We expect that evidence of 
program efficacy would be reflected in relatively lower levels of these measures in those receiving 
tolerance-focused BIT treatment when compared to those in placebo and no-program control groups, 
following exposure to being primed with contemporaneous information about anti-Muslim violence in a 
neighboring country. 
REJECTION OF VIOLENCE NORM
The second major social norm targeted by the curriculum focused on use of violence.101 We evaluated 
participant attitudes and norms about violence as follows at endline:
• Violence Disapproval: An index of questions drawn from Moskalenko & McCauley’s (2009) 
“Radicalism Intentions Scale”.102 This scale asks the respondent to name a cause that a close friend 
of theirs cares deeply about, followed by questions measuring the extent to which they would 
approve of the close friend engaging in a range of activities on behalf of that cause. For our 
100 At the time, anti-Muslim riots in India had a great amount of salience within Bangladeshi media and society, as they were 
occurring concurrently with changes being made to Indian citizenship laws that were widely perceived as discriminatory 
towards Indian Muslims. We chose this specific article because we wanted to measure facets of religious separatism/supremacy 
among participants (the vast majority of whom identified as Sunni) when exposed to a current events story they’d realistically 
encounter over social media. 
101 Via endline surveys prior to web-based Inquisit activities. 
102 Moskalenko, Sophia, and Clark McCauley. "Measuring political mobilization: The distinction between activism and radicalism." 
Terrorism and Political Violence 21.2 (2009): 239-260.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 30
purposes, we used two questions defined by Williams, Horgan, and Evans (2016) as “amped political 
extremism,” which focus on the use of violence on behalf of the cause.103
• Violence Worriedness: This index uses the same “radicalism intentions” scale as that of “violence 
disapproval,” but instead asks the respondent how worried they would be about a friend engaging in 
the aforementioned violent activities. 
• Rejection of VEO Goals: A question measuring the extent to which the respondent believed that 
groups committing violent acts sometimes had legitimate goals. We designed this battery of 
questions in order to avoid asking the respondent directly about their attitudes towards specific 
VEOs, due to safety concerns. Given that the programming was designed to promote and reinforce 
norms against violence (without focusing on any particular violent extremist 
organization/movement), we expect that evidence of anti-violence norm internalization would be 
reflected in rejection of the notions that VEO goals can be legitimate.104 
• VEO Awareness: An index of questions we created to measure the respondent’s views regarding 
the perceived popularity and reach of violent extremist organizations within Bangladesh and among 
those like themselves. We designed this battery of questions in order to avoid asking the 
respondent directly about their attitudes towards specific VEOs, due to safety concerns. We expect 
that evidence of an anti-violence norm internalization would be reflected in an increased recognition 
that VEOs are present and actively recruiting within Bangladesh, and that they frequently target 
those similar to themselves.105 
• Openness to VEO Demobilization: Finally, we probed the extent to which participants 
distinguished the use of violence for pursuing extremist goals from the non-violent pursuit of 
extremist goals. An index of two questions was constructed measuring the extent to which the 
respondent agreed that VEOs should be permitted to compete in the political process if they 
renounce the use of violence, and whether it’s okay to agree with the goals of VEOs while 
disagreeing with their methods of pursuing the goals. We expect that evidence of anti-violence norm 
internalization would be reflected in acceptance of the notion that a VEO which renounced the use 
of violence should be permitted to participate in the political process. 
103 Williams, Michael J., John G. Horgan, and William P. Evans. "Evaluation of a multi-faceted, U.S. community-based, Muslim-led 
CVE program." (2016). 
104 Design meetings with Rupantar and consultations with USAID emphasized that asking respondents directly about whether 
they supported the goals of VEOs would be unsafe, even with the privacy and confidentiality steps we took. Moreover, doing so
would likely yield untruthful responses due to social desirability bias. Instead we framed these items using indirect statements 
such as “the political goals of violent extremist organizations like ISIS are popular within Bangladesh,” or “It is okay to agree 
with the political goals of violent extremist organizations without supporting their methods.” Framing the items in this way 
carries the downside of less precision in the concept, an issue we discuss further in the results. 
105 Design meetings with Rupantar and consultations with USAID emphasized that asking respondents directly about whether 
they supported the goals of VEOs would be unsafe, even with the privacy and confidentiality steps we took. Moreover, doing so
would likely yield untruthful responses due to social desirability bias. Instead we framed these items using indirect statements 
such as “the political goals of violent extremist organizations like ISIS are popular within Bangladesh,” or “It is okay to agree 
with the political goals of violent extremist organizations without supporting their methods.” Framing the items in this way 
carries the downside of less precision in the concept, an issue we discuss further in the results. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 31
SKILLS
We measure skills using two sets of outcomes, which gauge (1) participant retention of key concepts 
from the tolerance-focused BIT curriculum in the form of a knowledge assessment, and (2) the mastery 
of skills specifically required for effective bystander intervention.
First, measured participant retention of key concepts from the curriculum by making use of a 10-
question common knowledge assessment (CKA), which was developed by Management Systems 
International (MSI). 106 The CKA consisted of 10 questions which focused on the drivers of peace, 
harmony, stability (and instability) in Bangladeshi society, as well as the potential drivers of violent 
extremism within Bangladesh. Participants could select as many answers as they felt were applicable. 
From the CKA, we developed a “rubric” which awarded higher scores for selected answers that were 
consistent with the programming,107 while penalizing answers that were not consistent with the spirit of 
the programming.108 Specifically, CKA scores were constructed measuring:
• Tolerance CKA: Generated from 6 CKA questions for which tolerance (and related concepts) 
were relevant potential answers to questions about peace, harmony and lack thereof in Bangladesh. 
• Violent Extremism Drivers CKA: Generated from 2 CKA questions which ask the participant about 
the drivers of violent extremism. Higher scores were those that linked intolerance (and related 
concepts) to violent extremism. 
Second, effective bystander intervention requires mastery of skills to go from noticing a social 
emergency to actually acting in order to de-escalate it or otherwise help in a safe manner. In order to 
measure the BIT skills in participants, we draw on Burn’s (2009)109 operationalization of the Latane and 
Darley (1970)110 five barrier situational model of bystander intervention. This battery of questions 
measures skill-based impediments to intervention, starting with their likelihood of noticing a problematic 
situation in the first place. We adopted this battery of 14 questions to the context of Bangladesh by 
focusing each hypothetical statement on situations of religious, and sex-based harassment and 
discrimination. These 14 questions were then aggregated into outcome indices as follows:
• Failure to Notice: Two items positing the likelihood that the respondent would be unlikely to 
notice perpetrator behaviors at a gathering. 
• Failure to Identify an Emergency: Three items positing the likelihood that the respondent would 
be unlikely to correctly identify a situation that has a high potential for escalating to violence, despite 
having noticed such a situation. 
106 MSI was awarded the prime contract for managing the implementation of the Obirodh Project, which included this program, 
among others. 
107 For example, one question asks “what causes societal conflict?” Participants would score higher on the CKA if they selected 
the answer “misunderstanding of different cultures and practices.”
108 For example, one question asks “what makes a society cohesive?” Participants would score lower on the CKA if they 
selected the answer “a society with the same culture, religion and clothes.”
109 Burn, Shawn Meghan. "A situational model of sexual assault prevention through bystander intervention." Sex Roles 60.11 
(2009): 779-792.
110 Latané, Bibb, Bibb Latané, and John M. Darley. The unresponsive bystander: Why doesn't he help? Appleton-Century-Crofts, 
1970.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 32
• Failure to Assume Responsibility: Five items positing the likelihood that the respondent would be 
unlikely to assume that they have personal responsibility to act in a given situation, even if it is a 
clear emergency.
• Failure due to Skill Deficit: Two items positing the likelihood that the respondent would be 
unsure of what to say or do even if they believed they had responsibility to intervene.
• Failure due to Audience Inhibition: Two items positing the likelihood that the respondent would 
be afraid to act even if they knew how to, due to fear of judgement from others or that they would 
not receive help from others. 
CONFIDENCE AND SELF-ASSESSED COMPENTENCE TO ACT 
Beyond the internalization of norms and skills, effective bystander intervention requires a sense of self￾efficacy, i.e., the self-evaluated sense of confidence and competence to act. Thus in addition to 
internalized norms and skills associated with the programming, we measure participant evaluations of 
their own sense of agency by constructing three indices of questions adapted from Banyard et al.’s 
(2014)111 “readiness to help” scale for measuring bystander behaviors related to sexual assault on 
university campuses. This set of items asks participants to evaluate their willingness, readiness, and ability 
to act in order to address a variety of hypotheticals associated with sexual assault. Items include, for 
example: “I think I can do something about sexual assault,” and “There is not much need for me to think 
about sexual assault.” We adapted 9 of these items for each of three topic areas relevant to the 
Bangladeshi context: 
• Religious Intolerance: Items describing confidence/competence to act under scenarios of religious 
intolerance being expressed on campus. 
• Violence on Campus: items describing confidence/competence to act under scenarios of violence 
on campus. We chose to leave the nature of hypothetical campus violence vague because much of 
campus violence in Bangladesh is political violence carried out by student political groups.
• Extremism: items describing confidence/competence to act under scenarios of violence on campus. 
“Extremism” and “violent extremism” are viewed interchangeably in Bangladesh and have the 
same Bangla translation, unless explicitly differentiated as “non-violent extremism.”
In addition to these self-evaluations of confidence/competence, we also included a battery of items 
evaluating the respondents’ perceptions that their friends would act under a variety of context-relevant 
hypotheticals. These 11 items were drawn from Banyard et al.’s (2014) Perceptions of Peer Helping 
Index.112 A participant’s perceptions about their friends/social circle’s ability and willingness to help in a 
social emergency reflects their own sense of confidence to affect change in such a situation. The 
participant’s perceptions about their friend’s willingness to help is a direct indicator of perceived social 
norms regarding helping/intervention behavior.113
111 Banyard, V. L., et al. "How do we know if it works? Defining measurable outcomes in bystander-focused violence 
prevention." Psychology of Violence 4.1 (2014): 101-15.
112 Ibid.
113 This is distinct from social norms of tolerance. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 33
WILLINGNESS TO INTERVENE
Finally, we evaluated effective bystander intervention capacity among participants by presenting them 
with four vignettes, each describing different perpetrator/victim interactions focused on the following 
topics:
• Religious Discrimination: In this vignette, the participant is asked to put themselves in the place of 
a person witnessing a perpetrator who uses an anti-Hindu slur. 
• Sexual Harassment (“Eve-teasing”): In this vignette, the participant is asked to put themselves in 
the place of a person witnessing a perpetrator who “eve-teases” a woman walking by. 
• Anti-LGBT Bullying: In this vignette, the participant is asked to put themselves in the place of a 
person witnessing a perpetrator who bullies a student who is rumored to be homosexual.
• Radicalization: In this vignette, the participant is asked to put themselves in the place of someone 
who witnesses a friend become angry at a news story, discuss having met a new group of friends 
online who plan to take action for a cause, and mentions that they are “willing to fight, and even die, 
to make a difference .”114
For each of these four vignettes, participants were presented with an expanded list of potential 
responses and asked their likelihood of choosing that response for the scenario. Some of the responses 
offered included bystander intervention training strategies (which the participants were taught as part of 
the training), whereas others were considered “do nothing” responses (i.e., “I would wait and see what 
happens”), and others are considered “join perpetrator” responses (i.e., “I would encourage him (the 
radicalized person) if it were for the right reasons.” From these menus of responses, we constructed 12 
outcome variables (three for each of the four topic areas), as follows:
• BIT Strategies Used: An index of respondent responses to their likelihood of using one or more 
BIT strategies in response to the vignette. So, for example, respondents who indicated they were 
highly likely to “attempt to create a distraction” or “delay your response and check in later with the 
victim” ranked higher on this index. 
• No Inaction: A reversed index of responses to the likelihood of a passive response/inaction in 
response to the scenario. Respondents indicating they were highly unlikely to “do nothing” or 
“watch and see what happens” scored higher on this index.
• Not Joining Perpetrator: Finally, respondents who indicated that they were highly unlikely to join 
the perpetrator in the given vignette scored higher on this metric (which was also reversed).
114 We verified that this scenario would be understood as a case of likely radicalization through a small pilot study using 
Mechanical Turk (in which respondents overwhelmingly described it as such a scenario in free responses). Rupantar also 
confirmed that it was almost certainly likely to be understood this way. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 34
PEER LEADERS VS TRADITIONAL AUTHORITY FIGURES
IDENTIFICATION WITH FACILITATOR
In addition to comparing the efficacy of peer leaders and traditional authority figures on the previously 
discussed outcomes, participants were asked to respond to a battery of items rating their facilitator on a 
variety of personal traits, in order to verify the extent to which the “peer leader” and “traditional 
authority” facilitators were perceived as we would expect them to be. We aggregated these trait-related 
items into the following:
• Relatability: An index of four items rating the extent to which the facilitator is relatable, including 
statements such as “my facilitator has walked a similar path as me” and “my facilitator is just like 
me.” 
• Experience: An index of six items rating the extent to which the facilitator is experienced, including 
statements such as “my facilitator is knowledgeable,” and “my facilitator has seniority.”
• Youthfulness: An index of two items rating the extent to which the facilitator is youthful, including 
the statements “my facilitator is youthful” and “my facilitator is enthusiastic.”
• Approachability: An index of four items rating the extent to which the facilitator is approachable, 
including statements such as “I would feel comfortable telling my facilitator about a personal 
problem,” and “I would feel comfortable asking my facilitator to explain something I don’t 
understand.” 
IDENTIFICATION WITH SOCIAL ENVIRONMENT OF PROGRAM
A second set of facilitator-driven outcomes we evaluate are participant perceptions of the social 
environment of the programming. Specifically, the extent to which the participant feels socially 
connected to their classmates (which is moderated by the facilitator) is likely to affect their identification 
with program goals. We measure participant social experiences using two variables: the extent to which 
they feel that they made new friends from the programming, and the extent to which they would 
recommend the programming to their friends. Table 11 depicts results from the omnibus “mean effects” 
test for each social outcome, followed by item-level effects.
BASELINE CHARACTERISTICS COVARIATE BALANCE 
Table 3 displays covariate balance across the four treatment contrasts. In addition to basic 
demographics, we compare participants on a variety of baseline psychosocial variables and attendance 
rates (attrition). We aggregate conceptually similar, correlated measures into indices.115 Overall, the 
contrast between tolerance BIT program recipients and placebo (either placebo or tolerance BIT) was 
well-balanced on demographic and psychosocial baseline variables (Table 3 Col 8). An omnibus test of 
differences in baseline demographic and psychosocial covariates suggests no systematic differences. 
115 For example, we combine Likert-scale measures of depression, emotional stability, economic optimism, and recent loss into 
an index we label “mental health.” We use a similar approach for aggregating anger, hostility, verbal aggression, and hostile 
attribution bias into an index we label “anger.”
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 35
When comparing peer and traditional authority groups within the BIT tolerance programming (Table 3, 
Col 9), we found that participants in traditional authority-led groups were more likely to report prior 
familiarity with their facilitator than those enrolled in the peer leaders group. This was not surprising, 
given that the participating students may have had classes before with the professors serving as 
facilitators (or know of them through their departments). We also found that those enrolled in 
authoritative facilitator-led groups were more likely to report good relationships with their family 
members, less likely to report having had negative prior interactions with those from different religious 
and/or ideological backgrounds, and more likely to report prior exposure to classes/programming that 
develops critical thinking skills. We include family relations quality, prior negative interactions with out￾groups, and prior critical thinking skills class/program exposure in specifications contrasting peer and 
authoritative influencer groups.116
Those who opted to participate in a wave 2 no-program control were more likely to be older, and male 
than those enrolled in the pooled tolerance BIT programming sample (Table 3 Col 6). Other 
demographics and psychosocial measures were not found to be unbalanced. No-program control 
participants were incentivized with a small gift card, and they included a combination of those from the 
randomized programming list who did not participate in the programming and some who were recruited 
later via flyers. We believe that no-program control participants may have been unbalanced on age and 
gender demographics due to covid-19 related shutdowns, which necessitated participants’ connections 
from their home villages via their personal devices and/or laptops. This problem was mitigated to some 
extent by the provision of data packages by NORC at the University of Chicago for remote 
connectivity. However, we suspect that older male students are more likely to have personal devices 
and/or laptops (or access to them) in the first place when no longer on their university campuses. We 
control for age and gender in specifications that include the no-program control participants.
Regarding attrition, attendance levels (total classes attended), dropout rates (those who started but did 
not finish the program), and “no-show” rates (those who completed a baseline but never started the 
program and were replaced) were not found to be correlated with treatment assignment. We also 
conducted baseline covariate balance tests comparing finishers (defined as those who completed at least 
10 of the 14 sessions), drop-outs (those who completed between 1-9 sessions), and no-shows (those 
who were admitted and completed intake data, but never showed up for the first session and were 
replaced). Of the various demographic and psychosocial background traits we measured, we only found 
differences in reported social media use; those who never showed up and those who dropped out 
reported greater baseline social media use than those who finished the program. 
Greater levels of social media use among drop-outs was unexpected; if anything, we would have 
expected social media use to be a proxy for digital connectivity (which would be associated with having 
finished, given that many wave 2 students finished programming online). It is possible that those 
reporting higher levels of social media use were also more likely to find other (competing) activities, 
given that Facebook is commonly used to organize and advertise extracurriculars. 
Baseline covariate balance by attrition status is depicted in Table 15 (appendices). 
116 No-program control participants were not assigned questions related to the programming/facilitators, given that they did 
not participate in programming. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 36
Table 3: Pre-Treatment Covariate Balance117
Baseline 
Covariate
NPC 
(204) 
Mean 
[SE]
Placebo 
(406) 
Mean 
[SE]
Auth 
(418) 
Mean 
[SE]
Peer (422) 
Mean 
[SE]
Diff
(t-test) 
(NPC vs 
BIT All)
Diff 
t-test) 
(NPC + 
Placebo vs 
BIT All
Diff
(t-test) 
(Placebo 
vs BIT 
All)
Diff
(t-test) 
(BIT Auth 
vs BIT 
Peer)
(Omnibus Test) -0.30 [0.06] <0.01
[0.02]
< 0.01 
[0.02]
<0.01 
[0.02] <0.01*** 0.77 0.81 -0.84
Age 0.24 
[0.09]
0 
[0.05]
-0.06 
[0.05]
-0.06 
[0.05] 0.29*** 0.12*** -0.01 0.01
Male 0.71 
[0.03]
0.57 
[0.03]
0.56 
[0.03]
0.61 
[0.02] 0.13*** 0.04 0.03 -0.05
Sunni 0.78 
[0.03]
0.79 
[0.02]
0.78 
[0.02]
0.75 
[0.02] 0.02 0.03 0.05 0.03
Econ. Welfare -0.12 
[0.08]
0.06 
[0.05]
0.03 
[0.05]
0.01 
[0.05] -0.13 -0.01 0.05 0.01
Anger 0.05 
[0.08]
0.01 
[0.05]
-0.05 
[0.05]
-0.02 
[0.05] 0.1 0.07 0.05 -0.05
Grit -0.13 
[0.07]
0.08 
[0.05]
0.03 
[0.05]
-0.03 
[0.05] -0.15 -0.02 -0.05 0.04
Deliberativeness -0.08 
[0.08]
-0.01 
[0.05]
0.09 
[0.05]
0.05 
[0.05] -0.14 -0.08 -0.04 0.08
Mental Health 0.04 
[0.07]
0.02 
[0.05]
0.08 
[0.05]
0.02 
[0.05] 0.04 -0.01 -0.06 0.06
Pro-sociality 0.04 
[0.07]
-0.02 
[0.05]
-0.07 
[0.05]
-0.03 
[0.05] 0.1 -0.01 -0.01 0.02
Social Media Use -0.07 
[0.07]
-0.02 
[0.05]
-0.07 
[0.05]
0 
[0.05] -0.04 -0.02 -0.03 -0.07
Family Relations -0.09 
[0.08]
0.01 
[0.05]
0.13 
[0.04]
-0.03 
[0.05] -0.12 -0.06 0.01 0.17***
Family Overseas 3.18 
[0.16]
3.46 
[0.11]
3.41 
[0.11]
3.49 
[0.11] -0.27 -0.09 0.01 -0.07
Religiosity 
(Reversed)
0.08 
[0.07]
-0.01 
[0.05]
-0.02 
[0.05]
-0.01 
[0.05] 0.12 0.05 -0.08 0.01
Diversity of 
Friends
-0.06 
[0.07]
-0.07 
[0.05]
0.03
[0.05]
0 
[0.05] -0.06 -0.07* -0.08 0.06
117 Robust standard errors in brackets. The value displayed for t-tests are the differences in the means across the groups. All 
estimation regressions include school and wave fixed effects. ***, **, and * indicate significance at the 1, 5, and 10 percent 
critical level.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 37
Baseline 
Covariate
NPC 
(204) 
Mean 
[SE]
Placebo 
(406) 
Mean 
[SE]
Auth 
(418) 
Mean 
[SE]
Peer (422) 
Mean 
[SE]
Diff
(t-test) 
(NPC vs 
BIT All)
Diff 
t-test) 
(NPC + 
Placebo vs 
BIT All
Diff
(t-test) 
(Placebo 
vs BIT 
All)
Diff
(t-test) 
(BIT Auth 
vs BIT 
Peer)
Negative Out￾Group 
Interactions 
-3.96 
[0.17]
-4.14 
[0.12]
-4.25 
[0.12]
-3.93 
[0.12] 0.12* -0.01 0.03 -0.36**
Critical Thinking 
Exposure
0 
[0.07]
-0.02 
[0.05]
0.02 
[0.05]
-0.12 
[0.05] -0.09* -0.01 -0.09 0.14**
Prior 
Programming 
Exposure
NA -0.08 
[0.05]
0.01 
[0.05]
0.05 
[0.05] NA NA 0.12* -0.07
Prior Facilitator 
Familiarity NA 0.09 
[0.06]
0.08 
[0.06]
-0.14 
[0.04] NA NA 0.04 0.22***
Finished 
Program NA 0.75 
[0.02]
0.78 
[0.02]
0.80 
[0.02] NA NA -0.03 -0.02
Sessions 
Attended NA 8.84 
[0.22]
9.04 
[0.21]
9.00
[0.22] NA NA -0.18 0.04
No-Show NA 0.18 
[0.02]
0.16 
[0.02]
0.19 
[0.02] NA NA 0.17 -0.02
6. DATA ANALYSIS
Our data analysis aims to addressing the following research questions:
RQ1. 
Does the P/CVE Bystander Intervention Training (BIT) Curriculum increase participants’ 
motivation and capacity to challenge individuals taking actions that reflect violent extremism? In 
particular, does it cause participants to:
(1) Internalize norms against extremism,
(2) Possess skills for recognizing and knowing how to safely challenge behaviors and 
expressions of attitudes linked to violent extremism,
(3) Possess confidence in their ability to challenge such behaviors and attitudes, and 
(4) Put all of these things together effectively to challenge behaviors and expressions of 
attitudes linked to violent extremism?
RQ2. Is the program more effectively delivered by trained peer leaders or by traditional 
authority figures?
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 38
Following our theory of change, we estimate effects on items measuring the following:
• Internalization of norms, including the following:
− Intolerance (we expect these to be reduced)
− Rejecting violence (we expect these to be increased)
• Skill- and knowledge-based barriers to challenging violent extremism antecedents (we expect these 
to be decreased)
• Confidence and self-assessed competence to challenge violent extremism antecedents (we expect 
these to be increased)
• Willingness to Intervene in scenarios modeling violent extremism antecedents (we expect these to 
be increased)
• Identification with facilitator (we expect this to vary dependent on peer versus traditional authority 
facilitator).
We discuss the specific measures in the “outcomes of interest” subsections below that describe our 
results. 
For RQ1 regarding the effect of the program on norms, skills, confidence, and action against violent 
extremism, we report estimates from regression models specified as follows: 
Endline Outcome = a + b*BIT Curriculum + c*Baseline Outcome + University FE
+ Wave FE + Unbalanced Covariates + e (1) 
where a is an intercept term that measures mean outcomes in the absence of the BIT intervention, b
measures the effect of the BIT intervention, c measures how the baseline measures of the outcomes 
correlate with endline measures, the University FE and Wave FE terms refer to indicator variables that 
control for variation at the level of universities and data collection waves, and the Unbalanced Covariates
are control variables determined on the basis of analysis of pre-treatment covariate data, and e is an 
error term that captures other idiosyncratic variation from individual to individual. We estimate the 
regression using weighted least squares, weighting by the inverse probability of treatment assignment. 
We use heteroscedasticity robust standard errors, given that the treatment assignment was at the level 
of individuals.118
We report two types of effects for RQ1. The first effect is based on the contrast between tolerance BIT 
program participants versus those in the no-program control condition. We do this by estimating 
equation (1) on a sample that includes program participants and those who were not engaged in any 
program at all, and excludes those who received the placebo curriculum. This estimates the gross 
impact of the program, including the effect of the BIT curriculum per se, as well as any effects 
attributable to getting people together in a novel social situation to think critically. The second effect 
118 As a technique of accounting for class cohort spillovers, we also ran specifications that clustered standard errors on 
facilitator class cohorts. Results were found to be negligibly different from those reported in the results section below. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 39
isolates the effect of receiving the BIT curriculum as compared to receiving the placebo curriculum. This 
effectively controls for the aforementioned social interaction effects, which would be operative in both 
the BIT and placebo curricula (therefore enabling comparison of differences between the tolerance BIT 
and placebo). To do this, we estimate specification (1) on a sample that includes those in the BIT 
curriculum and those in the placebo curriculum, excluding those who were not engaged in any program 
at all. This estimates the added value of the curriculum per se, above and beyond the effect of the novel 
social interactions and critical thinking exercises that were also included in the placebo curriculum.119
For RQ2, which measures the effect of having a peer versus traditional authority facilitator, we use the 
following specification: 
Endline Outcome = a + b*Peer Facilitator + c*Baseline Outcome + University FE
+ Wave FE + Unbalanced Covariates + e (2) 
where, in this case b is the effect of having a Peer Facilitator instead of a Traditional Authority facilitator, 
and all of the other terms have the same meaning as in equation (1). Estimation again uses weighted least 
squares with heteroscedasticity robust standard errors. This effect characterizes whether peer 
facilitators generate any different effects with the tolerance curriculum as compared to traditional 
authorities.
We use a hierarchical testing and statistical significance strategy to first evaluate overall effects on sets of 
items, and then to evaluate item-level effects that are adjusted to account for the fact that we are doing 
many tests (i.e., to address the “multiple comparisons problem”). As discussed above, we have sets of 
items measuring our concepts. For example, for norms of intolerance, we have six items: a Religious 
Outgroup Disgust index, Hostile Sexism index, Benevolent Sexism index, Anti-LGBT Disgust index, 
Anti-Western Attitudes index, and Distrust of Foreigners index. To test overall effects, we use an 
omnibus “mean effect” test constructed as the average of z-score standardized effects for each item.120
We implement this using the mseffect command in Stata. Then, within each group of items, we report 
item-level effects and with multiple comparisons false discovery rate (FDR) p-value adjustment using the 
mhtreg command in Stata.121 Methodologists refer to such adjusted item-level p values as “q values.” The 
item-level results allow us to assess which items may be driving the overall omnibus effect, and the q
values allow us to test for statistical significance in a way that accounts for the multiple testing problem. 
In our discussion, we take effects with p or q values less than 0.05 are considered to be statistically 
significant, while when we have p or q values between 0.05 and 0.10, we refer to the finding as 
“suggestive.”
119 As a technicality, we estimate this curriculum effect using two approaches: one includes the no-program control group so 
only to get more precise estimates on the control variables, and the other excludes them. Both specifications target the same
effect, and yield essentially the same results.
120 This follows the approach of Kling, J.R., Liebman, J.B. and Katz, L.F. (2007), Experimental Analysis of Neighborhood Effects. 
Econometrica, 75: 83-119. For an explanation of hierarchical testing strategies for program evaluation, see also Casey, K., 
Glennerster, R., & Miguel, E. (2012). Reshaping institutions: Evidence on aid impacts using a preanalysis plan. The Quarterly 
Journal of Economics, 127(4), 1755-1812.
121 This follows the approach presented in Michael L. Anderson (2008) Multiple Inference and Gender Differences in the Effects 
of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects, Journal of the American 
Statistical Association, 103:1481-1495.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 40
7. RESULTS
In this section, we report results for the effects of the tolerance BIT programming, followed by the 
effects of peer leaders (compared to traditional authority figures) in delivering the programming. For 
both components of the field experiment, we present results according to the causal chain specified in 
figure 1 (above). Results are depicted as coefficient plots for all outcomes in this section, whereas tables 
reporting coefficients, standard errors, p-values and FDR (q-values) are presented in the appendices, and 
referenced where appropriate. As discussed in the research design, outcomes are generally constructed 
as aggregated additive indices of Likert-scale survey and post-activity measures, which were then 
standardized using pooled group mean and standard deviation. Therefore, all effects that we report 
below are in pooled standard deviation units. Summary statistics for each outcome category and scaling 
are also presented in the appendices. We begin by reporting the program’s estimated effects on social 
norms of tolerance and rejection of violence
NORMS
Per the research design, we consider two categories of social norms that the tolerance BIT 
programming was meant to promote: (1) norms of tolerance and (2) the rejection of violence. I 
NORMS OF TOLERANCE AND INTOLERANCE
Figures 2-4 depict coefficient plots for the omnibus “mean effects” test for each category of outcomes 
(two focusing on norms of tolerance and one on rejection of violence), followed by item-level effects for 
each outcome index. All specifications include unbalanced covariates and wave/school fixed effects. 
Tables 4-6 (appendices) report coefficients, standard errors, p-values and q-values.
In terms of gross effects of programming, we find that intolerance was reduced by about an eighth of a 
standard deviation (ca. 0.12 sd; Table 4, column 1); this represents a modest although statistically 
significant effect. Changes were most evident in benevolent sexism, with changes also apparent in anti￾LGBT, anti-Western and anti-foreigner sentiment. For benevolent sexism, it is possible that the nature 
of the coed interactions themselves (which were in place under both the BIT and placebo curricula) had 
an effect on its own, independent of the content of the curriculum. Changes in anti-LGBT sentiment 
were interesting, given the aforementioned discussion about the decision not to include curriculum 
content focusing on LGBT issues in any program elements. 
About half or a little more of the summary index effect appears attributable to the tolerance BIT 
curriculum per se—0.07 standard deviation (Table 4, column 4). The extent to which the tolerance BIT 
curriculum mattered varies by item. The curriculum seemed to be associated with decreases in anti￾religious outgroup sentiments and anti-Western attitudes, with suggestive evidence pointing to declines 
in hostile sexism. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 41
Figure 2: Effects of Programming and Tolerance BIT Curriculum on 
Norms of Tolerance
Figure 3: Effects of Programming and Tolerance BIT Curriculum on 
Norms of Religious Group Status Measures
For the web-based stimulus activity on religious status, we find an omnibus effect of similar magnitude as 
the previous results (Table 5, column 1) and consistent with our expectations. Because of the smaller 
sample size, the differences are not statistically significant for the items. However, the curriculum was 
associated with a decline in these metrics via the omnibus, likely reflecting greater power from the 
additional placebo participants in the analysis. 122
122 Note that the smaller sample size is due to the fact that only about half of the sample was able to successfully complete the
activities, following Covid-19 shutdowns.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 42
Figure 4: Effects of Programming and Tolerance BIT Curriculum on 
Rejection of Violence and VEOs
NORMS AGAINST VIOLENCE
Regarding the rejection of violence and violent extremist organizations, we find that the summary effects 
(Table 6, columns 1 and 4) are small when examining gross effects of programming. Curriculum effects 
seem to be more pronounced; the tolerance curriculum was associated with about one sixth of an 
increase in rejection of violence, driven specifically by increased disapproval and worry about a 
potentially violent friend. 
Findings for gross effects of programming appear driven by the fact that we have positive effects for the 
items pertaining to concern for/disapproval of friends’ involvement in violence, but then an unexpected 
negative effect for rejection of VEO goals among those in all programming; As noted, this item is 
measuring the extent to which the goals of VEOs can be legitimate despite the fact that VEOs use 
violence to pursue them. The findings here make this distinction important: goals despite violence, 
versus violence as a tactic per se. It is therefore plausible (though this is speculative) that prompting 
subjects to thinking critically (whether in focusing on gender and religious tolerance in the BIT 
curriculum or on more neutral subject matter in the placebo) would actually make people more likely to 
distinguish between goals, which can be legitimate on their own, and means of pursuing those goals, in 
this case, through violence. This finding is also important with regard to the concern about social 
desirability bias. If subjects were merely parroting what they took to be the socially desirable response, 
rather than responding in a way that reflect genuine reflection and judgment, we certainly would not 
expect subjects to evaluate VEO goals in this way. This effect suggests that subjects were prompted 
to apply a more critical mindset, rather than mindlessly parroting norms of tolerance. Further support 
for this interpretation comes from the findings with respect to “openness to VEO demobilization”, 
which specifically examine the participant’s willingness to accept those connected to VEOs if they 
renounce violence. The programming increased participants’ rejection of the use of violence among their 
friends/peer groups, but it also prompted subjects to distinguish goals from means and to be less 
reflexive in rejecting the possibility that those connected to VEOs are irredeemable because of part 
connection to violence. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 43
Overall, we observe improved tolerance among participants. We can attribute about half of the effect to 
the curriculum and the rest to the novel social interactions and general prompts to think critically. 
Program effects on norms related to violence and VEOs suggest improved norms against violence, 
though with some nuance regarding subjects’ increased willingness to distinguish goals from actions, and 
to be accepting of past VEO-affiliates if they renounce violence. Fortunately, our evaluation does not 
stop at evaluating norms and attitudes. What matters ultimately is how subjects act on these new norms 
and attitudes. We show encouraging results on that score below, after presenting results on skills and 
confidence. 
SKILLS
Figures 5 and 6 (below) depict results from the omnibus “mean effects” test for all skills outcomes 
followed by item-level effects for each outcome index. All specifications include unbalanced covariates 
and wave/school fixed effects. Tables 7 and 8 (appendices) report coefficients, standard errors, p-values 
and q-values.
Figure 5: Effects of Tolerance BIT Curriculum on Common Knowledge 
Assessment Scores123
Compared to placebo, those enrolled in the tolerance BIT programming scored about one eighth of a 
standard deviation higher on CKA items measuring retention of tolerance-related programming 
concepts, and about one fifth of a standard deviation higher on CKA items measuring retention of the 
drivers of violent extremism. While it is perhaps unsurprising that those enrolled in a tolerance-focused 
program would score higher on tolerance-related knowledge assessments than placebo, it is 
nevertheless an important indicator that the participants absorbed and retained the curriculum content.
123 Gross effects of programming cannot be distinguished from curriculum effects because the CKA was not used for no￾program control participants. Figure 5 reports those enrolled in tolerance BIT programming versus placebo only. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 44
Figure 6: Effects of Programming and Tolerance BIT Curriculum on Internalized 
BIT Skills
Relative to no program control, we observe strong effects across the board on exercise of skills and 
judgment in scenarios of aggression (Table 7, column 1). The curriculum per se (Table 7, column 4) 
accounts for about a third of the effect. The tolerance BIT curriculum seemed to be associated with 
significant improvements over placebo with respect to training participants to assume personal 
responsibility when encountering such scenarios. There is suggestive evidence that the curriculum also 
successfully improved participants’ ability to notice and identify an emergency situation, however with a 
high q value (0.42). Regarding the impact of the placebo (and the relatively weaker value added of the 
tolerance BIT program), it is possible that having participants reflect on their civic responsibility, and 
creating group interactions on that basis, can be efficacious in imparting some of the skills required for 
effective bystander intervention. Indeed, skills such as noticing aggression, or recognizing and 
overcoming audience inhibition, are potentially both improved by any type of programming that asks 
participants to engage in prosocial solutions to societal problems. The concentrated attention to BIT 
and norms per se therefore may not improve these particular skills much further, although it does not 
hurt either.
CONFIDENCE AND SELF-ASSESSED COMPETENCE TO ACT
Figure 7 (below) depicts results from the omnibus “mean effects” test for all confidence/competence 
outcomes followed by item-level effects for each outcome index. All specifications include unbalanced 
covariates and wave/school fixed effects. Table 9 (appendices) report coefficients, standard errors, p￾values and q-values.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 45
Figure 7: Effects of Programming and Tolerance BIT Curriculum on 
Confidence Act
Regarding confidence to act, we found that the tolerance BIT curriculum (figure 7, plot 2) appears to 
matter a lot. Program participants (Figure 7, plot 1) appeared to exhibit more confidence to act across 
the board when presented with scenarios of religious intolerance, scenarios of violence, and scenarios of 
extremism. More than half of these effects are accounted for by the tolerance BIT curriculum (appendix 
Table 9, column 4), which presumably is attributable to having participants role play and practice the 
skills they learned as a part of the training. Overall, these effects regarding confidence/competence to 
act highlight the potential importance of more focused training in prompting participants to progress 
from judging a situation (which seems easy enough to do) to seeing themselves as having personal agency 
and efficacy in making a difference in the outcome of that situation. It is at this stage in the causal chain 
that the BIT curriculum per se starts to matter a lot.
WILLINGNESS TO INTERVENE
Figure 8 (below) depicts results from the omnibus “mean effects” test for all willingness to intervene 
outcomes, followed by item-level effects for each outcome index. This is ultimately what the 
intervention was intended to inspire. All specifications include unbalanced covariates and wave/school 
fixed effects. Table 10 (appendices) report coefficients, standard errors, p-values and q-values. The 
results are promising.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 46
Figure 8: Effects of Programming and Tolerance BIT Curriculum on Willingness 
to Intervene
We see that from the gross effects (Figure 8 graph 1) and then from the test of the tolerance BIT 
curriculum against the placebo (Figure 8, graph 2) that the BIT curriculum matters substantially: the 
effects on prior parts of the causal chain carry through to ultimate willingness to intervene. Notably, we 
observe effects on two of the three religious discrimination items, two of the three anti-LGBT items, 
and one of the three eve-teasing and radicalization items, respectively. Participants were more likely to 
select one or more BIT strategies in order to handle scenarios of religious discrimination, sexual 
harassment (eve teasing), LGBT harassment, and probable extremist radicalization. The tolerance BIT 
programming (Figure 8, graph 2 and appendix Table 10, column 4) accounted for about half of these 
effects for three of the four topic areas (the exception being anti-LGBT bullying). Measures of whether 
the participant might “do nothing” or “join the perpetrator” in the aforementioned scenarios yielded 
less-clear evidence of an effect, although the finding that respondents were less likely to join a 
perpetrator in anti-LGBT bullying potentially reflects participant reconciliation between prevailing 
societal norms regarding homosexuality and those targeted by the programming. For some participants 
holding conservative social views on the subject, the choice to not join a perpetrator in such activities 
potentially suggests a reevaluation of how they choose to express those views. 
The findings for anti-LGBT bullying might also be suggestive that item floor effects account for a lack of 
movement in the other items. (A substantial majority of respondents disagreed moderately or strongly 
with the proposition that they would join a perpetrator in these situations, whereas responses were 
less-clearly distributed for the LGBT question). For the other items, this is perhaps not surprising, given 
that participants who joined and then completed a program of this nature were already likely to exhibit 
prosocial characteristics. 
Insofar as willingness to intervene is ultimately what matters, these effects are promising. The expected 
effects on norms and skills appear to translate into behavioral propensities that create a climate that 
promotes tolerance and challenges hateful speech and actions that form the basis of violent extremism. 
These effects on behavioral tendencies stand despite the unexpected effect on subjects’ willingness to 
distinguish, and accept, the goals of VEOs despite the means that they use to pursue them. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 47
PEER LEADERS VS TRADITIONAL AUTHORITY FIGURES
We now turn to the results regarding the effectiveness of peer leaders as compared to traditional 
authority figures, which in our case were university professors. We first consider whether participants 
evaluated or identified with peer leaders differently than traditional authority figures. We then examine 
the implications for whether peer or traditional leaders serve the goals of the program better.
IDENTIFICATION WITH FACILITATOR
Figure 9 (below) depicts effects for each outcome index. For these specifications, we do not present an 
omnibus “mean effects” test, because each of the traits we evaluate are conceptually distinct, and in 
some cases are in contrast (i.e., youthfulness versus experience). All specifications include unbalanced 
covariates and wave/school fixed effects. Table 11 (appendices) reports coefficients, standard errors, p￾values and q-values.
Figure 9: Participant Identification with Peer Leaders, compared to 
Traditional Authority Figures, among Those Enrolled in Tolerance BIT Program
Regarding identification with their facilitator, Table 11 shows that respondents had mixed assessments. 
Traditional authority figures were significantly more likely to be seen as experienced- this was not 
surprising, given that traditional authorities were older and had higher degrees/job ranks than peer 
leaders. Also, not surprisingly, we find evidence that peer leaders were more likely to be seen as 
youthful, though with the caveat of a fairly high q value (0.16). However, perhaps unexpectedly, 
traditional authority figures were perceived by participants to be more approachable than were peer 
leaders. This could potentially be due to clearer institutional relationships between participants and 
traditional authority figures (i.e., that of students and professors), whereas such relationships between 
participants and their older peers are potentially more nebulous. Likewise, peer leaders themselves 
might have felt less confident leading training sessions given their relative lack of teaching experience, 
which may have been perceived by participants as an impediment to approachability. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 48
IDENTIFICATION WITH SOCIAL ENVIRONMENT OF PROGRAM
Figure 10 (below) depicts results from the omnibus “mean effects” test for each social outcome, 
followed by item-level effects. Table 12 (appendices) reports coefficients, standard errors, p-values and 
q-values. 
Figure 10: Participant Identification with Social Environment of Programming, 
comparing Peer Leaders to Traditional Authorities among Those Enrolled in 
Tolerance BIT Program
Figure 10 shows that we do not find clear differences between peer leaders and traditional authority 
figures with respect to perceptions of the social environment created by the program. Although 
participants with peer leader facilitators were slightly less likely to report making new friends in the 
program, and slightly less likely to recommend the program to their other friends, differences were 
small and not statistically significant. 
We next examine the outcomes of participant identification with programming goals using two items: 
the extent to which participants learned about tolerance, and the extent to which participants learned 
about bystander intervention training. Figure 11 depicts results from the omnibus “mean effects” test for 
each programmatic outcome, followed by item-level effects. Table 13 (appendices) report coefficients, 
standard errors, p-values and q-values.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 49
Figure 11: Participant Identification with Programming Goals, Comparing Peer 
Leaders to Traditional Authorities, among Those Enrolled in Tolerance BIT 
Program
From Table 13 coefficients (column 1), we see that participant self-assessments of the extent to which 
they learned key elements of the tolerance BIT curriculum offer some evidence that traditional authority 
figures may have been more effective than peer leaders. Namely, those enrolled in peer-led sessions 
were less likely to agree with items positing that they learned about BIT strategies. 
EFFECT OF PEER LEADERS ON INTERVENTION-RELATED OUTCOMES OF INTEREST
Finally, we compare the efficacy of peer leaders with traditional authority figures with respect to the 
other outcomes of interest that we evaluate and present in figures two through eight (above): program 
norms (tolerance and rejection of violence), program skills, confidence/competence to act, and effective 
use of BIT. For brevity, Figure 12 (below) depicts results from the omnibus “mean effects” test for all 
intervention pathway outcomes. For clarity, we print the direction that each relationship between peer 
leaders and outcomes would be expected to show if peer leaders were more effective than traditional 
authorities. 
Regarding the outcomes of interest for the broader program, Figure 12 (below) shows that we do not 
find strong differences between using peer leaders to lead programming when compared to traditional 
authority figures for the context for most outcomes. To the extent that we do, students of traditional 
facilitators seem to do better with respect to anti-violence norms, and with a few component items of 
other outcome categories. For brevity, we note these as follows, per each family of outcomes with p￾values and q-values reported in footnotes:
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 50
Figure 12: Impact of Peer Leaders, compared to Traditional Authority Figures, on 
Outcomes of Interest among Those Enrolled in Tolerance BIT Program
• Norms: Traditional authority-led participants had anti-foreigner sentiment that was about one sixth of 
a standard deviation lower,124 and participant worry about a potentially violent friend was about one-sixth 
of a standard deviation higher,125 than peer-led participants.
• Skills: Traditional authority-led participants were about one sixth of a standard deviation less likely 
to face impediments to intervention rooted in skill deficits than peer-lead participants.126
• Effective Intervention: Traditional authority-led participants were about one fifth of a standard 
deviation less likely to respond passively to an “eve-teasing” scenario.127
It is possible that these scattered findings in favor of traditional authority figures are consistent with 
narratives that those who have prior experience teaching may be better suited for such positions; 
however, overall differences between the two groups were not sufficiently clear to conclude that one 
provided clearly superior training for those enrolled in the tolerance BIT program.
PLACEBO OUTCOMES: DO NORMS AND CONFIDENCE TO ACT CHANGE WITH RESPECT 
TO UNRELATED TOPICS
Finally, we make use of respondent answers to several placebo questions in order to interrogate the 
extent to which participants may have simply attempted to provide survey responses reflecting positive 
outcomes for the youth social leadership programming that they participated in. Specifically, we 
examined whether tolerance BIT program participants exhibited (1) changed norms and (2) changed 
levels of confidence to act on issues related to the environment and public health. If tolerance BIT 
participants were simply attempting to answer our survey questions in ways that reflected “good 
124 p = 0.01; q = 0.07
125 P = 0.03; q = 0.13
126 P = 0.03; q = 0.14
127 p < 0.01; q = 0.16
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 51
leadership,” or if they were attempting to provide us with “good survey answers,” then we would 
expect to also observe changed norms and increased confidence to intervene with respect to questions 
focusing on “civic virtues” that are completely unrelated to out-group tolerance and violence. 
Regarding unrelated norms, an index was created from three questions measuring participant agreement 
on items related to (1) punishments for littering, (2) banning smoking from indoor places, and (3) 
whether “fast food” was making Bangladeshis sick. Regarding confidence to act, we adapted Banyard et 
al (2014)’s “readiness to help” index to a battery of questions related to environmental action, which 
appeared in addition to those focusing on tolerance and violence.128 Using the same analysis framework 
articulated in section 6 (above), we test whether gross effects of programming and tolerance BIT 
curriculum affected change in these metrics among participants. Results are presented in Table 16 
(appendices). 
In short, we do not find evidence of changes in norms regarding these unrelated civic topics, nor do we 
find evidence of changed confidence to act on environmental issues among those in the tolerance BIT 
programming. We interpret this as further evidence that participants critically engaged with questions.
LIMITATIONS
We find consistent evidence that the tolerance BIT programming was efficacious in its primary goal: 
training participants how to safely act in defense of social norms of tolerance and anti-violence in a 
context where institutional constraints limit the menu of potential programming options. We believe 
that this goal has implications for P/CVE, given that the behaviors and attitudes that are being targeted 
by such a “social climate” strategy are antecedents to radicalization. Furthermore, implementing the 
programming as an RCT with a placebo enabled us to conclude that changes in program participants 
were attributable to the programming, while also enabling us to measure the extent to which those 
changes can be attributed to program content versus the social, effects of participating in a multi-week 
mixed-group experience focused on civic issues. Despite these aforementioned findings, several study 
limitations should guide future research on this important topic. 
First, although the programming clearly resulted in changes within participants for the outcomes we 
measured, measurement activities generally took place shortly after the end of programming.129 Thus 
the long-term efficacy of programming is not currently known. Generally, there is scant evidence 
examining the long-term impact of bystander intervention training programs focused on similar anti￾social behaviors.130 As discussed, most BIT programs are designed as short-duration training sessions 
(often in the form of a single workshop, or an online training). This program was deliberately designed 
as a large dose (28 contact hours) with the goal of long term impact. 
Second, we have taken great efforts to design and implement our measurement strategy in such a way 
that mitigates the possibility that respondent answers were driven by social desirability bias and/or 
demand effects. Regarding the former (which we have discussed at varying points throughout this 
report), we have good reason to believe that participants felt comfortable and secure answering all 
128 Banyard, V. L., et al. "How do we know if it works? Defining measurable outcomes in bystander-focused violence 
prevention." Psychology of Violence 4.1 (2014): 101-15.
129 Most participants finished endline activities within about two weeks following programming end.
130 Nelson, Jacqueline K., Kevin M. Dunn, and Yin Paradies. "Bystander anti‐racism: A review of the literature." Analyses of Social 
Issues and Public Policy 11.1 (2011): 263-284
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 52
questions privately and with knowledge that their responses would not be observed by any person 
within their country. Importantly, the measurement activities were carried out separately from program 
facilitators and Rupantar, and respondents had the opportunity to save their progress and return for a 
comfortable amount of time. Moreover, our findings with respect to perceptions of the legitimacy of 
VEO goals (in spite of violence) suggest to us that participants critically engaged with the questions, and 
did not feel social pressures to parrot “correct answers;” indeed, many of the participants appeared 
willing to agree with the premise that VEO goals can potentially be legitimate. 
Regarding the latter, we deliberately designed the surveys and web-based stimulus in attempts to avoid 
demand effects- the surveys, for example, were presented to respondents as research on different 
personality traits and styles of leadership. The surveys themselves contained many batteries of questions 
measuring personality, social characteristics, worldviews, and other traits which we deemed relevant to 
the study, but which did not focus on the topics of out-group tolerance or bystander intervention 
training. Moreover, participants were assured that their receipt of the completion certificate was not 
tied to any of their responses to surveys. Finally, we have attempted to include behavioral measures in 
the form of web-based activities that simulate (1) participant exposure and reaction to a news story 
about religious persecution and (2) a follow-up activity designed to directly test intervention behaviors 
in the form of a ball toss game. Unfortunately, our implementation of this second behavioral activity 
yielded only half of our intended sample size, because of covid-19 related shutdowns that rendered 
many participant devices incompatible with the software. We presented findings from the first activity 
(despite this problem) because we observed an effect of programming, despite the underpowered 
sample. 
Third and finally, ultimately whether these participants actually intervene in real-world situations where 
they witness intolerant and/or violent behaviors cannot easily be assessed. During scoping missions, we 
asked potential participant universities whether, for example, discipline reports were available which 
would include incidents of intolerant behaviors. Although the universities do discipline students for a 
variety of offenses, our general sense was that they did not collect information on the outcomes that 
this research focuses on.131 In addition to this measurement issue (i.e., records of intolerant behavior), 
programming would need to be scaled beyond these 6 implementation sites in order to assess broader 
societal impacts. 
SUMMARY
In summary, we find consistent evidence that the tolerance BIT programming improved participant’s 
willingness to intervene (particularly with the use of BIT strategies when presented with a variety of 
perpetrator/victim scenarios). Moreover, we find that this effect was attributable to the BIT program’s
improving of participant competence/confidence to act. While the tolerance BIT programming also 
appeared to improve participant norms and skills which we believe are prerequisite to willingness to 
intervene, the value-added of the tolerance BIT programming per se was not substantially greater than 
that of the placebo programming, which suggests that these prerequisite norms and skills can potentially 
be developed by any program that provides an inclusive social setting in which participants focus on civic 
engagement. We discuss implications of this finding further in the concluding remarks of the report. 
131 Moreover, soliciting the provision of private student records as a part of the research design would have been complicated 
and ethnically perilous, with the potential to deter interest in programming.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 53
Regarding the type of facilitator used for the tolerance BIT programming, we do not find substantial 
differences between peer leaders and traditional authority figures (namely, professors). To the extent 
that we do find differences within scattered measures, the results suggest that traditional authority 
figures may be better at leading programming of this nature, and that this advantage could rooted in 
participant perceptions of experience and approachability, as well as (potentially) participant perceptions 
that they learned better with the traditional authorities leading the classes. 
8. DISCUSSION AND CONCLUSIONS
This report documents the design and evaluation of a secondary preventing/countering violent 
extremism (P/CVE) strategy implemented in Bangladeshi universities from September 2019 through May 
2020. The strategy tested was a tolerance-focused bystander intervention training (BIT) program, which 
we chose due to its extensive prior use as a method of combating other hateful and antisocial behaviors 
among young people, and due to its potential viability in the challenging political and social context of 
Bangladesh. To our knowledge, this is the first program of its kind to be implemented in a non-Western 
context as a randomized controlled trial. The preponderance of findings presented above suggest 
programming of this nature can be impactful in increasing the likelihood participants will implement 
bystander intervention strategies to promote a climate of tolerance and to challenge hateful speech and 
actions that are the basis of violent extremism. The impact is due to the program’s efficacy in improving 
norms, skills, perceptions of confidence, and competence to act effectively, where the curriculum itself is 
especially important for confidence and competence to act. 
Interestingly, many of the intermediary outcomes we deemed prerequisite to willingness to intervene 
were impacted positively by both the tolerance BIT programming, and the placebo, when compared to 
participants enrolled in a no-program control group. Although some caveats apply to these findings, 
given that the no-program control was not subject to the same randomization protocols as the three 
treatment statuses of the field experiment, we nevertheless believe this contrast is informative. The 
social interactions and general prompts to think critically, and the civic focus, may have resulted in 
“virtuous spillovers,” leading subjects to defend norms of tolerance and against violent extremism even 
when they were not explicitly instructed on these issues. 
The implications of our results are that indirect, “social climate” strategies are a compelling approach to 
address violent extremism, particularly in contexts of high political sensitivity and where individual rights 
are vulnerable. As we noted in the introduction, countering violent extremism is fraught by the fact such 
extremism is, by its nature, rare. This “needle in the haystack” problem means any attempt to target at￾risk individuals directly may be inefficient, given the extreme difficulty of identifying them. Moreover, 
strategies using group-based surveillance or otherwise target entire groups in an aggressive and coarse 
manner may do more to alienate members of that group, undermining the potential for enlisting the 
support of group members who oppose violence extremism. Given these challenges and our findings, 
this report makes the following recommendations for future programming:
• For contexts where institutional resources, political and/or security constraints limit available 
secondary P/CVE options, targeting “peer gatekeepers” with capacity-building programming offers a 
promising avenue for promoting social climates hostile to violent extremism and its antecedents. In 
such contexts, those that internalize (or have already internalized) a desired social norm offers a 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 54
potentially valuable resource to P/CVE practitioners if they can be trained to safely and effectively 
promote and protect those norms within their social climates. 
• Within sensitive contexts, programming meant to address a negative social/behavioral norm can 
potentially be achieved by targeting closely related social/behavioral norms. For secondary P/CVE 
programming, this implies programming does not necessarily need to explicitly target violent 
extremism or specific violent extremist organizations in program content to have the desired effects 
of reducing behaviors and attitudes thought to be antecedent to the problem. 
• P/CVE programming targeting “peer gatekeepers” should be designed in such a way that includes 
domain knowledge development as a precursor to capacity-building, with requisite time allocated for 
each component.
• Scaled P/CVE programming targeting “peer gatekeepers” can likely be delivered effectively by fellow 
trained peer gatekeepers without substantial loss of quality, though with the obvious caveat that 
experience likely improves quality and outcomes.
What we have demonstrated is that it is possible to empower youth and create a social “climate” 
against extremism through positive reinforcement of tolerance and anti-violence norms. Such a climate 
should make it less likely at-risk youth travel in an unimpeded manner down the road of violent 
extremism. Our results are indicative of the promise of such a strategy. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 55
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 56
APPENDIX A.
Table 4: Effects of Programming and Tolerance BIT Program on Norms of 
Tolerance132
Outcome Expectation
Gross Effect of Programming
(N=1,025)
Effect of Tolerance BIT 
Curriculum (N= 1,424)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Intolerance (ME) - -0.12 [0.04] 0.01** -0.07 [0.03] 0.02**
Relig. Outgroup Disgust - -0.06 [0.07] 0.43 1.00 -0.11 [0.06] 0.04** 0.22 
Hostile Sexism - -0.12 [0.08] 0.12 0.75 -0.1 [0.05] 0.07* 0.41 
Benevolent Sexism - -0.18 [0.08] 0.02** 0.12 -0.03 [0.06] 0.57 1.00 
Anti-LGBT Disgust - -0.18 [0.08] 0.02** 0.09 -0.05 [0.05] 0.27 1.00 
Anti-Western Attitudes - -0.17 [0.08] 0.03** 0.21 -0.11 [0.05] 0.04** 0.22 
Distrust of Foreigners - -0.25 [0.08] <0.01*** 0.01 -0.06 [0.06] 0.26 1.00 
Table 5: Effects of Programming and Tolerance BIT Program on Religious 
Tolerance (Web-Based Stimulus Activity)
Outcome Expectation
Gross Effect of Programming 
(N=537)
Effect of Tolerance BIT 
Curriculum (N= 766)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects - -0.12 [0.08] 0.12 -0.11 [0.05] 0.04**
Religious Nationalism - -0.22 [0.12] 0.06* 0.26 -0.04 [0.08] 0.59 1.00 
Collective Narcissism - -0.14 [0.12] 0.22 0.89 -0.14 [0.08] 0.08* 0.33 
Segregation Preference - -0.12 [0.12] 0.32 1.00 -0.13 [0.08] 0.08* 0.33 
Group Victimhood - -0.04 [0.13] 0.77 1.00 -0.12 [0.08] 0.11 0.46 
132 ***, **, and * indicate significance at the 1, 5, and 10 percent critical level for all tables. 
Table 6: Effects of Programming and Tolerance BIT Program on Rejection of 
Violence
 
Gross Effect of Programming
(N=1044)
Effect of Tolerance BIT 
Curriculum 
(N= 1450)
Outcome Expectation
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects + -0.3 [0.06] <0.01*** -0.09 [0.04] 0.03**
Friend Disapprove + -0.39 [0.09] <0.01*** 0.00 -0.1 [0.06] 0.09* 0.44
Friend Worry + -0.31 [0.09] <0.01*** 0.00 -0.07 [0.06] 0.23 1.00
VEO Rejection + -0.47 [0.09] <0.01*** 0.00 -0.15 [0.06] 0.01** 0.06
VEO Awareness + -0.18 [0.09] 0.05* 0.25 -0.1 [0.06] 0.08* 0.42
Open to VEO 
Demobilization + -0.25 [0.09] 0.01** 0.03 -0.04 [0.06] 0.50 1.00
Table 7: Effects of Programming and Tolerance BIT Program on 
Internalized Skills
Gross Effect of Programming 
(N=1044)
Effect of Tolerance BIT Curriculum 
(N= 1450)
Outcome Expectation
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects - 0.36 [0.07] <0.01*** 0.22 [0.05] <0.01***
No Noticing - 0.5 [0.08] <0.01*** 0.00 0.28 [0.06] <0.01*** 0.00
No ID Emergency - 0.45 [0.1] <0.01*** 0.00 0.22 [0.07] <0.01*** 0.01
No Responsibility - 0.26 [0.07] <0.01*** 0.00 0.2 [0.05] <0.01*** 0.00
Skill Deficit - 0.21 [0.09] 0.02** 0.08 0.14 [0.06] 0.02** 0.09
Audience Inhibition - 0.36 [0.07] <0.01*** 0.22 [0.05] <0.01***
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 57
Table 8: Effects of Programming and Tolerance BIT Program on Confidence 
to Act
Outcome Expectation
Gross Effect of Programming 
(N=1,002)
Effect of Tolerance BIT 
Curriculum (N= 1,387)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects + 0.39 [0.07] <0.01*** 0.2 [0.05] <0.01***
Act Religious Intolerance + 0.37 [0.07] <0.01*** 0.00 0.22 [0.05] <0.01*** 0.00
Act vs Violence + 0.47 [0.09] <0.01*** 0.00 0.2 [0.06] <0.01*** 0.01
Act vs Extremism + 0.29 [0.06] <0.01*** 0.00 0.16 [0.04] <0.01*** 0.00
Friends Act + 0.22 [0.09] 0.01** 0.05 0.09 [0.06] 0.14 0.57
Table 9: Effects of Programming and Tolerance BIT Program on Willingness 
to Intervene
Outcome Expectation
Gross Effect of Programming 
(N=1044)
Effect of Tolerance BIT 
Curriculum (N= 1450)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects + 0.22 [0.05] <0.01*** 0.12 [0.03] <0.01***
BIT in Religious Discrim. + 0.52 [0.09] <0.01*** 0.00 0.26 [0.06] <0.01*** 0.00
BIT in Eve Teasing + 0.6 [0.09] <0.01*** 0.00 0.23 [0.06] <0.01*** 0.00
BIT in LGBT + 0.51 [0.09] <0.01*** 0.00 0.18 [0.06] <0.01*** 0.02
BIT in Radicalization + 0.37 [0.09] <0.01*** 0.00 0.17 [0.06] <0.01*** 0.04
No Passivity Religious Discrim. + 0.05 [0.09] 0.55 1.00 0.08 [0.06] 0.19 1.00
No Passivity Eve Teasing + 0.06 [0.09] 0.49 1.00 0.03 [0.06] 0.66 1.00
No Passivity LGBT + 0.21 [0.09] 0.01** 0.16 0.07 [0.06] 0.24 1.00
No Passivity VEO + 0.16 [0.09] 0.07* 0.86 0.03 [0.06] 0.61 1.00
No Perpetrating Religious Discrim. + 0.07 [0.08] 0.42 1.00 0.13 [0.06] 0.04** 0.45
No Perpetrating Eve Teasing + 0.03 [0.08] 0.72 1.00 0.08 [0.06] 0.18 1.00
No Perpetrating LGBT + 0.21 [0.08] 0.01** 0.15 0.16 [0.06] 0.01** 0.11
No Perpetrating VEO 
No Perp VEO + 0 [0.09] 0.96 1.00 0.02 [0.06] 0.78 1.00
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 58
Table 10: Participant Identification with Peer Leaders, compared to Traditional 
Authority Figures, among Those Enrolled in Tolerance BIT Program
Outcome
Peer Leaders vs Traditional Authority Figures (N=778)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
Relatability 0.06 [0.06] 0.30 1.00
Experience -0.27 [0.06] <0.01*** 0.00
Youthfulness 0.13 [0.06] 0.04** 0.17
Approachability -0.12 [0.05] 0.01** 0.06
Table 11: Effect of Peer Leaders, compared to Traditional Authority Figures, on 
Participant Evaluation of Social Environment among Those Enrolled in Tolerance 
BIT Program
Outcome
Peer Leaders vs Traditional Authority Figures (N=778)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
Mean Effects -0.07 [0.06] 0.30
Made New Friends -0.03 [0.07] 0.67 1.00
Recommends to Friends -0.09 [0.07] 0.21 0.43
Table 12: Effect of Peer Leaders, compared to Traditional Authority Figures, on 
Participant Identification with Program Goals among Those Enrolled in Tolerance 
BIT Program
Outcome
Peer Leaders vs Traditional Authority Figures (N=770)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
Mean Effects -0.04 [0.06] 0.49
Learned About Tolerance -0.03 [0.05] 0.51 1.00
Learned About BIT -0.1 [0.07] 0.13 0.27
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 59
Table 13: Impact of Peer Leaders, compared to Traditional Authority Figures, on 
Outcomes of Interest among Those Enrolled in Tolerance BIT Program
Outcome (Mean Effects)
Peer Leaders vs Traditional Authority Figures (N=778)
(1)
Coeff [SE]
(2)
p-value
(3)
N 
Intolerance 0.01 [0.04] 0.92 778
Religious Supremacy -0.06 [0.07] 0.40 416
Rejection of Violence -0.05 [0.03] 0.05* 778
Skill Impediments 0.06 [0.05] 0.27 778
Confidence/Competence -0.04 [0.05] 0.40 778
Effective BIT Use -0.03 [0.04] 0.38 778
Table 14: Summary Statistics for Outcomes of Interest133
Outcome Count Mean St. Dev Min Max
Religious Outgroup Disgust 1,451 20.22 6.46 6 42
Hostile Sexism 1,451 15.73 4.84 4 28
Benevolent Sexism 1,451 20.59 4.41 4 28
Anti-LGBT Disgust 1,451 4.18 1.92 1 7 
Anti-Western Attitudes 1,451 7.31 2.76 2 14
Distrust of Foreigners 1,451 8.37 2.25 2 14
Violence: Disapprove Of Friend Involvement 1,451 10.61 2.70 2 14
Violence: Worry About Friend Involvement 1,451 8.92 3.26 2 14
VEO Rejection 1,451 22.63 5.82 5 35
Religious Nationalism 768 3.09 1.23 1 5 
Religious Collective Narcissism 767 2.67 1.12 1 5 
Religious Segregation Preference 768 3.67 1.76 2 10
Religious Perceived Group Victimhood 768 3.84 0.98 1 5 
Failure to Notice 1,451 7.86 2.66 2 14
Failure Via Audience Inhibition 1,451 7.45 2.63 2 14
133 Unstandardized values depicted for transparency of scaling; standardized values of additive indices were used for the analysis. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 60
Outcome Count Mean St. Dev Min Max
Failure to Take Responsibility 1,451 16.74 5.64 5 35
Failure to ID Emergency 1,451 12.21 3.19 3 21
Failure Via Skill Deficit 1,451 8.31 2.74 2 14
Would Act in Religious Intolerance 1,451 34.98 7.83 9 55
Would Act vs Violence 1,451 39.76 8.17 9 59
Would Act vs Extremism 1,451 37.63 8.17 9 58
Friends would Act 1,451 58.02 11.97 11 77
BIT responses chosen Religious Discrim. Scenario 1,451 29.66 6.14 6 42
BIT responses chosen Eve Teasing scenario 1,451 5.66 1.65 1 7 
BIT responses chosen LGBT scenario 1,451 5.79 1.59 1 7 
BIT responses chosen VEO scenario 1,451 4.42 1.86 1 7 
Would not do nothing Eve Teasing scenario 1,451 29.18 6.73 6 42
Would not do nothing LGBT scenario 1,451 29.44 6.59 6 42
Would not do nothing Religious Discrim. Scenario 1,451 27.77 5.49 6 42
Would not do nothing VEO scenario 1,451 9.49 2.68 2 14
Would not join perp Eve Teasing scenario 1,451 9.20 2.67 2 14
Would not join perp LGBT scenario 1,451 9.57 2.71 2 14
Would not join perp Religious Discrim. Scenario 1,451 9.47 2.75 2 14
Would not join perp VEO scenario 1,451 5.86 1.59 1 7 
Facilitator Relatability 1,247 19.84 3.99 4 28
Facilitator Experience 1,247 33.59 5.44 7 42
Facilitator Youthfulness 1,247 11.15 2.14 2 14
Facilitator Approachability 1,451 21.75 5.57 4 28
Made New Friends 1,247 12.06 1.84 2 14
Recommends Program to Friends 1,216 6.13 0.94 1 7 
Learned About Tolerance 1,218 5.86 1.09 1 7 
Learned About BIT 1,451 12.12 1.84 2 14
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 61
Table 15: Standardized Baseline Covariate Balance by Attrition
Variable 
(Standardized)
No-shows (192) 
Mean [SE]
Drop-Outs (316) 
Mean [SE]
Finishers 
(1154) 
Mean [SE]
Diff (t-test) 
No-Shows vs 
Finishers
Diff (t-test) 
Drop-outs vs 
Finishers
Age -0.01 [0.07] -0.17 [0.05] -0.01 [0.03] 0.00 -0.17
Male 0.59 [0.04] 0.64 [0.03] 0.58 [0.02] 0.01 0.05
Sunni 0.78 [0.04] 0.78 [0.03] 0.78 [0.01] 0.00 0.00
Econ. Welfare 0.01 [0.07] 0.09 [0.06] 0.02 [0.03] -0.02 0.07
Anger -0.09 [0.07] 0 [0.05] -0.01 [0.03] -0.08 0.01
Grit 0.07 [0.07] -0.02 [0.06] 0.04 [0.03] 0.02 -0.07
Deliberativeness 0.02 [0.08] -0.02 [0.06] 0.03 [0.03] -0.01 -0.05
Mental Health 0.15 [0.07] -0.03 [0.05] 0.03 [0.03] 0.12 -0.06
Pro-sociality 0.01 [0.08] -0.1 [0.06] -0.01 [0.03] 0.02 -0.09
Social Media Use 0.13 [0.08] 0.16 [0.06] -0.06 [0.03] 0.19** 0.22***
Family Relations 0.02 [0.08] -0.07 [0.06] 0.04 [0.03] -0.02 -0.11
Family Overseas 3.92 [0.19] 3.51 [0.13] 3.45 [0.07] 0.47 0.06
Religiosity (Reversed) 0.06 [0.07] -0.01 [0.06] -0.02 [0.03] 0.07 0.01
Diversity of Friends 0.08 [0.08] 0.02 [0.06] -0.01 [0.03] 0.09 0.03
Negative Out-Group 
Interactions -4.25 [0.18] -4.05 [0.14] -4.16 [0.07] -0.09 0.11
Critical Thinking 
Exposure 0.11 [0.08] 0.02 [0.06] -0.02 [0.03] 0.13 0.04
Prior Programming 
Exposure -0.04 [0.15] -0.05 [0.06] 0.02 [0.03] -0.06 -0.07
Table 16: Gross Effect of Programming and Effect of Tolerance BIT Curriculum 
on Placebo Norms and Confidence Outcomes.
Placebo Outcome
Gross Effect of Programming 
(N=1025)
Effect of Tolerance BIT 
Curriculum (N= 1,424)
(1)
Coeff [SE]
(2)
p-value
(3)
Coeff [SE]
(4)
p-value
Norms: Littering, Smoking and Junk Food (Reversed) -0.10 [0.31] 0.75 0.27 [0.23] 0.24
Confidence to Act: Environmental Action 0.50 [4.59] 0.91 0.33 [0.47] 0.48
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058 USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 62
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013 / DRG-LER I TASKING N058
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 63
APPENDIX B.
DISCLAIMER This publication was produced for review by the United States Agency for International Development. It was 
prepared by NORC at the University of Chicago. The authors’ views expressed in this publication do not necessarily reflect the 
views of the United States Agency for International Development or the United States Government.
IMPACT EVALUATION OF THE OBIRODH 
– ROAD TO TOLERANCEYOUTH 
LEADERSHIP TRAINING PROGRAM IN 
BANGLADESHI UNIVERSITIES
LONG TERM FOLLOW-UP
AUGUST 2022
Prepared under Contract No. GS-10F-0033M / Order No. AID-OAA-M-13-00013, Tasking N058
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 65
DRG LEARNING, EVALUATION, AND 
RESEARCH ACTIVITY
IMPACT EVALUATION OF THE OBIRODH –
ROAD TO TOLERANCE YOUTH LEADERSHIP
TRAINING PROGRAM IN BANGLADESHI
UNIVERSITIES
LONG TERM FOLLOW-UP 
AUGUST 2022
Prepared under Contract No. GS-10F-0033M / Order No. AID-0AA-M-13-00013, Tasking N058
Submitted to:
USAID DRG Center
Submitted by:
Peter Vining, NORC Principal Investigator (New York University)
Cyrus Samii, NORC Principal Investigator (New York University)
Michael Gilligan, NORC Principal Investigator (New York University)
Contractor:
NORC at the University of Chicago
Attention: Matthew Parry, Program Manager
3450 East-West Highway, Suite 800
Bethesda, MD 20814
Telephone: (301) 634-5444
E-mail: Parry-Matthew@norc.org
DISCLAIMER 
The authors’ views expressed in this publication do not necessarily reflect the views of the United States 
Agency for International Development or the United States Government
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 66
TABLE OF CONTENTS
1. EXECUTIVE SUMMARY 68
2. LONG TERM FOLLOW-UP INTRODUCTION 68
EVALUATING LONG-TERM IMPACT THROUGH RETROSPECTIVE AND 
DEMONSTRATED USE OF TRAINING 69
VIOLENCE REJECTION NORMS – ADDED MODULES ON THE JUSTIFIABILITY OF 
VIOLENCE 70
WILLINGNESS TO INTERVENE – ADDED MODULES ON RETROSPECTIVE NOTICING 
OF AND INTERVENTION IN REAL LIFE AND SOCIAL MEDIA 71
3. LONG TERM FOLLOW-UP METHODOLOGY 73
SAMPLE SIZE, DATA COLLECTION STRATEGY, TIMELINE, AND ANALYSIS 73
BASELINE CHARACTERISTICS COVARIATE BALANCE BY LTF ATTRITION 74
DATA ANALYSIS 76
4. LONG TERM FOLLOW-UP RESULTS 77
NORMS 77
NORMS OF TOLERANCE AND INTOLERANCE: LONG TERM EFFECTS OF 
PROGRAMMING 77
NORMS REGARDING THE USE AND JUSTIFIABILITY OF VIOLENCE: LONG TERM 
EFFECTS OF PROGRAMMING 79
SKILLS: LONG TERM EFFECTS OF PROGRAMMING 81
CONFIDENCE AND SELF-ASSESSED COMPETENCE TO ACT 82
WILLINGNESS TO INTERVENE 83
SUMMARY 85
5. LONG TERM FOLLOW-UP DISCUSSION AND CONCLUSIONS 85
APPENDICES 87
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 67
TABLES
Table 1: Long Term Follow-up Data Collection Timeline 74
Table 2: Standardized Baseline Covariate Balance by Attrition: Long Term Follow-up vs Endline 75
Table 3: Long Term Effects of Programming and Tolerance BIT Program on Norms of Tolerance 87
Table 4: Long Term Effects of Programming and Tolerance BIT Program on Rejection of 
Violence 88
Table 5: Long Term Effects of Programming and Tolerance BIT Program on Justifiability of 
Violence Beliefs (All Participants) 88
Table 6: Long Term Effects of Programming and Tolerance BIT Program on Dominant In-Group 
Justifiability of Violence Beliefs (Jihadist Violence Included; Muslim Participants Only) 89
Table 7: Long Term Effects of Programming and Tolerance BIT Program on Internalized Skills 89
Table 8: Long Term Effects of Programming and Tolerance BIT Program on Confidence to Act 90
Table 9: Long Term Effects of Programming and Tolerance BIT Program on Willingness to 
Intervene 91
Table 10: Long Term Effects of Programming and Tolerance BIT Program on Retrospective 
Evaluations of BIT Use in Real Life and on Social Media 92
Table 11: Summary Statistics for Long Term Follow-Up Additional Outcomes of Interest 93
FIGURES
Figure 1: Long Term Effects of Programming and Tolerance BIT Curriculum on Norms of 
Tolerance 78
Figure 2: Long Term Effects of Programming and Tolerance BIT Curriculum on Rejection of 
Violence and VEOs 79
Figure 3: Long Term Effects of Programming and Tolerance BIT Curriculum on Beliefs Regarding 
the Justifiability of Violence (All Respondents) 80
Figure 4: Long Term Effects of Programming and Tolerance BIT Curriculum on Beliefs Regarding 
the Justifiability of Violence (Justifiability of Jihadist Violence Module Included; Muslim 
Respondents Only) 81
Figure 5: Long Term Effects of Programming and Tolerance BIT Curriculum on Internalized BIT 
Skills 82
Figure 6: Long Term Effects of Programming and Tolerance BIT Curriculum on Confidence Act 83
Figure 7: Long Term Effects of Programming and Tolerance BIT Curriculum on Willingness to 
Intervene 84
Figure 8: Long Term Effects of Programming and Tolerance BIT Curriculum on Retrospective 
Bystander Intervention Behaviors 85
Figure 9: Pathways from BIT Programming to Willingness to Intervene 87
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 68
1. EXECUTIVE SUMMARY
This addendum details findings from long-term follow-up research with participants in the USAID￾sponsored Obirodh: Road to Tolerance bystander intervention training (BIT) randomized controlled trials, 
held in six Bangladeshi Universities during late 2019 and early 2020. The endline evaluation found that 
the programming significantly improved participants’ norms of tolerance and their expected willingness 
and capacity to intervene when witnessing intolerant and extremist behaviors. Following these positive 
impacts on program participants, the USAID DRG Center commissioned this long-term follow-up 
research to understand which (if any) of these effects persisted roughly 20 months after the final wave of 
programming concluded. The key findings from this long-term follow-up research are as follows:
• BIT programming had enduring effects on participants’ expected willingness to intervene when 
witnessing intolerant and extremist behaviors.
• Improvements in norms of tolerance also sustained among program participants (when 
compared to those in a no-program control group). However, these sustained improvements 
were also present among participants in a placebo program (that focused on transportation, 
environmental, and public health challenges). 
In short, while the tolerance BIT program was successful in having long-term efficacy in training 
participants to have confidence and thus willingness to intervene, it appears that long-term 
improvements in tolerance norms can also be sustained indirectly through programming that facilitates 
improved critical thinking skills and dialogue among mixed groups of young people. 
2. LONG TERM FOLLOW-UP INTRODUCTION
A key finding from the original impact evaluation is that the program significantly improved participants’ 
norms of tolerance, and their expected confidence, willingness, and capacity to intervene safely and 
effectively when they witness intolerant and extremist behavior. While encouraging, these findings were 
based primarily on endline data collected approximately one to two weeks after the end of 
programming (during late 2019 for wave 1 participants, and early 2020 for wave 2 participants). The 
findings thus do not shed light on the longer-term impact of such programming, or whether changes 
within the social micro-climates of participants have endured following, for example, the disruptions of 
the Covid-19 pandemic, or the growth and maturation of the young participants themselves. 
This addendum details the results of a long-term follow-up of those enrolled in the treatment 
programming, as well as those enrolled in placebo and no-program control. The long-term follow-up 
was conducted from January-March, 2022, and successfully surveyed 1,013 individuals who participated 
in programming or no-program control. In summary, we find an enduring effect of the tolerance 
bystander intervention training (BIT) curriculum on participants’ confidence to act when presented with 
scenarios of religious intolerance, scenarios of violence, and scenarios of extremism. Moreover, we find 
some additional suggestive evidence that this increased confidence to act may be resulting in a greater 
willingness to intervene. While we find numerous other enduring effects of programming on norms of 
tolerance and norms against violence, they cannot be attributed specifically to the tolerance BIT 
curriculum. Finally, while we investigated long-term differences between those enrolled in peer and 
authoritative facilitator groups within the tolerance BIT programming, we did not find any substantively 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 69
meaningful differences. For brevity, we focus this analysis on comparing the tolerance BIT program to 
placebo and no-program control. 
In addition to surveying program participants who were willing to participate in the long-term follow-up 
roughly two years after the programming ended, an attrition analysis was conducted to investigate 
whether 2022 long-term follow-up participants systematically differ from 2019-2020 program 
participants who submitted an endline on outcomes of interest, but who did not participate in the long￾term follow-up. We find minimal differences between these groups in baseline covariates. Moreover, we 
did not find an association between treatment assignment and participation in the long-term follow-up. 
The following sections detail the design and data collection for the long-term follow-up, as well as 
results and discussion. 
EVALUATING LONG-TERM IMPACT THROUGH RETROSPECTIVE AND DEMONSTRATED 
USE OF TRAINING 
A survey framework nearly identical to that used in endline data collection was developed for 
conducting the long-term follow-up. Regarding outcomes of interest, we replicate question modules 
used for the main results presented in the final report. These include the measures of tolerance and 
rejection of violence norms, skills, confidence/competence to act, and willingness to intervene. In 
addition to replicating these primary outcomes of interest, we developed and employed several 
additional question modules designed to further probe participant norms regarding the justifiability of 
violence. Moreover, we developed and employed several measures designed to elicit retrospective 
estimations of whether and how many times the respondent noticed and/or acted (either in real life 
and/or over social media) in response to situations characterized by religious intolerance, sexism, 
extremism, and the use of violence. 
Finally, we replicated the web-based stimulus activity (designed to test whether programming had an 
inoculative effect against in-group collective narcissism and justifiability of violence in defense of one’s 
religious in-group) using a contemporaneous international news excerpt detailing sectarian violence in 
either India or Bangladesh. Variants of the activity (for Muslims, Hindus, Christians, and Buddhists) were 
created using the same article structure, but adapted using contemporaneous acts of sectarian violence 
targeting religious minorities within the region. (So, for example, anti-Hindu violence within Bangladesh 
itself was presented to Hindu participants; in contrast, anti-Muslim violence in India was presented to 
Muslim participants). In adapting the article, the researchers took care to avoid deception of any kind; all 
versions detailed real and recent acts of sectarian violence within Bangladesh and/or India that featured 
similar anecdotes (such as a viral video of a person being attacked). The stimulus activity was followed 
by a reading comprehension test meant to serve as an attention check, and then followed by survey 
modules measuring religious collective narcissism and justifiability of violence. 
To accommodate these aforementioned survey additions and other changes, nearly all programmatic 
and facilitator-related questions were removed from the long-term follow-up survey, given that we 
observed few differences between authoritative and peer-led groups at endline. In their place, we ask 
respondents (who participated in programming) a brief set of questions measuring retrospective 
program salience; we did not observe differences between treatment and placebo on these metrics, and 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 70
do not present them in the analysis.1 The following section details each of the new measures 
incorporated into the survey framework. 
VIOLENCE REJECTION NORMS – ADDED MODULES ON THE JUSTIFIABILITY OF VIOLENCE
In addition to tolerance norms, a major focus of the study was whether programming had an impact on 
participant norms regarding the use of violence. At time of programming, it was decided that only 
indirect questions about the acceptability/justifiability of violence were to be used in endline surveys, out 
of an abundance of caution against the possibility that program participants (some of whom were sharing 
devices to complete surveys) might inadvertently reveal and/or be observed providing responses to 
sensitive violence-related questions. Researchers thus intentionally avoided including any named violent 
extremist organizations (VEOs) within programming materials or measurement activities during program 
implementation, due to the risk that attention placed on the programming by authorities (as well as the 
small group dynamics of the programming) would potentially put participant safety in question should 
such questions be asked of them. 
For the long-term follow-up, we incorporated three additional question modules that more directly 
measure participant perceptions regarding circumstances under which violence might be justified, though 
we continue to avoid the use of any questions that prompt the respondent to evaluate a single specific 
named VEO.2 These additional violence-related questions were asked only during long-term follow-up, 
after it was determined that the risk of a participant's responses being observed by others was very low, 
and fully under control of the participant himself/herself, given that the survey was conducted remotely 
without using enumerators. The additional modules are as follows: 
• Support for Violence in Defense of Just Causes: An index of four items measuring the 
participant's belief that violence is sometimes necessary or justified if in defense of a just/worthy 
cause, or in defense of one's values or beliefs. These questions were derived from a 2021 study 
of violent extremist attitudes in young adults by Nivette (2021). 3 These questions were phrased 
without reference to a particular cause, faith tradition, or other identity group and thus asked of 
all participants. 
• Support for Jihadist Violence (Muslim Participants Only): An index of seven items asked 
only of participants who identified as Muslim, measuring agreement with statements focused on 
whether violence carried out by an assortment of international jihadist groups can be justified. 
1 Specifically, we asked those in tolerance BIT programming and placebo whether (1) they had remained friends with at least 
one person from their leadership training sessions, and (2) whether they remained in contact with their facilitator to construct a 
retrospective programmatic salience index. We also asked program participants how many times they had used their training 
over the past 18 months (without specifying which type of training). We found no difference between treatment and placebo 
on these measures, suggesting that both those assigned to the tolerance BIT treatment and those assigned to the placebo 
curriculum felt roughly the same about the program’s enduring salience. 
2 One question within the “support for jihadist violence” module, asks participants to indicate level of agreement with the 
statement “[t]he fighting of groups like the Taliban, Al Qaida, and ISIS is justified.” We view this question as the most sensitive 
violence-related question asked within the long-term follow-up survey framework. In contrast (and for demonstrative 
purposes), violence-related questions in endline were phrased in ways such as “It's okay to agree with the goals of violent 
extremist groups like ISIS but disagree with their methods of pursuing those goals” and “[t]he goals of violent extremist groups 
like ISIS are popular in Bangladesh.”
3 Nivette, Amy, et al. "Understanding changes in violent extremist attitudes during the transition to early adulthood." Journal of 
Quantitative Criminology (2021): 1-30. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 71
These questions were derived from a recent study of sacred values and vulnerability to violent 
extremism by Gomez et al (2017).4
• Support for Violence on Behalf of Religious In-Group: An index of seven questions 
evaluating the participant’s agreement with statements related to the justification and 
defensibility of using force in defense of one’s religious tradition, adapted from Swann et al 
(2009).5 These questions were added to the web-based stimulus activity post-activity survey, and 
adapted such that the in-group being referenced matched the religious tradition indicated earlier 
in the survey by the participant. All participants were thus presented with this question module. 
For the subsequent long-term follow-up, these question modules were combined with the index 
measuring collective narcissism following the stimulus activity, as well as the violence worriedness index 
(which measured worriedness following a friend’s hypothetical engagement with violent activities), both 
of which were directly replicated from endline activities. 
Altogether, these measures capture the extent to which participants reject violence as a legitimate 
means of pursuing goals. It is expected that evidence of program efficacy would be reflected in relatively 
lower levels of these measures in those receiving tolerance-focused BIT treatment when compared to 
those in placebo and no-program control groups. 
WILLINGNESS TO INTERVENE – ADDED MODULES ON RETROSPECTIVE NOTICING OF AND 
INTERVENTION IN REAL LIFE AND SOCIAL MEDIA 
For the long-term follow-up, indicators of effective intervention on behalf of a victim were replicated 
using the same series of vignettes that were employed during endline data collection activities. 
Moreover, with the benefit of more than 18 months during which the participant might have potentially 
drawn on this training in real-world (or social media) situations where intervention may have been 
helpful, we sought to directly ask respondents to provide retrospective assessments of any BIT 
behaviors during the 18 months prior to the long-term follow-up (LTF) survey. Retrospective BIT 
behaviors were measured using counts of BIT since the end of programming. The retrospective use 
measures were developed by the researchers, and worded to avoid referencing the programming, or 
BIT itself (to accommodate those in the no-program control).6 Participants were asked the following:
• Noticing (Any): Participants were first asked a screener question measuring if they had ever 
noticed any incidents of religious intolerance, gender intolerance, extremist beliefs, and/or 
violent incidents in real life or on social media during the past 18 months. 
• Noticing in Real Life: Participants who responded in the affirmative to having noticed any of 
the aforementioned incidents were asked to estimate the number of times they noticed them in 
real life during the past 18 months. 
4 Gómez, Ángel, et al. "The devoted actor’s will to fight and the spiritual dimension of human conflict." Nature Human Behaviour
1.9 (2017): 673-679. 
5 Swann Jr, William B., et al. "Identity fusion: the interplay of personal and social identities in extreme group behavior." Journal of 
Personality and Social Psychology 96.5 (2009): 995. 6 A separate and smaller module of questions was asked of all program participants to assess whether their training had been 
used; in this module, type of training was not specified (in order to accommodate both BIT treatment and placebo). This 
module was used to measure enduring programming salience.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 72
• Noticing on Social Media: Participants who responded in the affirmative to having noticed 
the aforementioned incidents were asked to estimate the number of times they noticed them on 
social media during the past 18 months. 
• Acting (Any): Participants were asked a screener question measuring if they had ever acted in 
response to any incidents of religious intolerance, gender intolerance, extremist beliefs, and/or 
violent incidents in real life or on social media during the past 18 months. 
• Acting in Real Life: Participants who responded in the affirmative to having acted in response 
to the aforementioned incidents were asked to estimate the number of times they had done so 
in real life, for each type of incident during the past 18 months. 
• Acting on Social Media: Participants who responded in the affirmative to having acted in 
response to the aforementioned incidents were asked to estimate the number of times they had 
done so on social media, for each type of incident during the past 18 months. 
From these aforementioned question modules, four indices were constructed measuring retrospective 
bystander intervention (noticing and acting in real life and on social media) for scenarios characterized 
by (1) religious intolerance, (2) sexism, (3) extremism, and (4) violence. These four indices are then used 
in the subsequent analysis. Finally, while we were unable to directly measure interventions occurring in 
participants’ day-to-day lives, we added a behavioral measure into this module of questions to give 
participants an opportunity to show an example of a time when they noticed or acted in response to 
these types of scenarios over social media, in the form of a screenshot from their personal social 
media.7
• Willingness to Upload a Screenshot of Intervention on Social Media Finally, we asked 
respondents who indicated having intervened over social media in the past 18 months if they 
would be able and willing to share a screenshot of having done so. Pending a response of “yes,” 
the respondent was then able to upload a photo through Qualtrics’ file upload modality. The 
yes/no response to this question, in addition to whether the respondent actually uploaded a file 
(but not the content of the file), were both used to construct an index measuring this behavioral 
outcome.8
7 We did not define the specific social media venue to give the participants room for interpretation; however, the screenshots 
we received were overwhelmingly from Facebook. 
8 A substantial proportion of respondents who indicated willingness to upload a screenshot did not actually upload one (before 
continuing to finish the survey). The index thus ranges from 0-2, with a “2” indicating that a screenshot was actually uploaded, 
and a “1” indicating willingness without subsequent follow-up. Note that to respect participant privacy, we do not evaluate or 
share the specific contents of screenshots within this analysis. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 73
3. LONG TERM FOLLOW-UP METHODOLOGY
SAMPLE SIZE, DATA COLLECTION STRATEGY, TIMELINE, AND ANALYSIS
For evaluating program efficacy in the long-term follow-up, we maintained the same assumptions and 
desired sample sizes in order to compare efficacy of programming to placebo, and attempted to recruit as 
many former participants (including drop-outs) who comprised the population of those who submitted 
baseline survey data for programming. This amounted to a sample frame of approximately 2,212 individuals 
who were randomized into one of the four treatment statuses (i.e., the two facilitator variants of tolerance 
BIT programming, placebo, or no-program control). Of those 2,212 participants, approximately 1,305 
were program participants who were awarded a certificate of completion by Rupantar (the Bangladeshi 
civil society organization implementing the training) for having completed programming (treatment or 
placebo), 560 were participants who were randomized into a treatment group and submitted baseline data, 
but either never started programming or dropped out during programming, and 347 were no-program 
control participants (204 of whom submitted an endline survey). 
Data collection for the long-term follow-up was carried out from December 2021 through March 2022. 
The 2,212 individuals were first sent a brief opt-in intake form collecting updated contact information and 
interest in participating in the long-term follow-up survey. Existing email addresses and phone numbers 
collected during 2019-2021 program implementation were used to initiate contact for this purpose, with 
local assistance provided by the seven technical coordinators hired by NORC at the University of Chicago 
to facilitate local data collection efforts within/around each treatment site. A small incentive in the form of 
a mobile phone recharge (BDT 200, worth approximately $2.33) was offered for completing these data 
collection activities. This approach yielded intake form submissions from 1,229 of the sampling frame.9
Intake recruitment efforts spanned approximately two weeks in late December, 2021. 
Those who opted in via the intake form were later sent the survey starting in early January, 2022. 
Updated contact information was used to send periodic reminders via email, SMS, and phone calls, with 
a limit of three contact attempts per participant. The deadline for these data collection activities was 
extended through March, due to new Covid-related university closures in Bangladesh.10 The average 
completion time for long-term follow-up activities was approximately one hour, which was roughly 
about the same amount of time required to complete the endline survey. This amount of time was not 
thought to impact data quality, evidenced by low levels of "survey speeding,”11 and low rates of failure on 
four attention check/distraction questions.12
9 An additional 34 unique individuals submitted intake forms who were not associated with the programming/previous data 
collection efforts; presumably some participants shared the recruitment link with friends. Recruitment verbiage explained that 
anyone who submitted at least one prior survey during 2019-2020 was eligible to participate. 10 At the time, the Omicron variant of Covid-19 had begun circulating; while most participants were able to complete data 
collection activities remotely without issues, some had been relying on university WIFI connections. 
11 Completion times of less than 25 percent of the completion median (14 minutes versus 56 minutes) among those estimated 
to have completed the survey in fewer than three hours (a conservative proxy for completion in a single setting) were 
disqualified from analysis. From this measure, only 8 participants were classified as “speeders,” which was less than one 
percent of respondents. 
12 Respondents were asked to complete four simple arithmetic problems that served as both attention checks and distraction 
tasks while transitioning into new sections of the survey. Inclusion criteria for the analysis required passing at least three of four 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 74
All data were collected through Qualtrics and were stored on secure US-based servers. Participant 
personal identifiable information (PII, in the form of phone numbers) were used to match participants to 
de-identified participant numbers that were generated and assigned by the researchers during data 
processing of baseline/endline data during 2019-2020 programming. Data cleaning and translation took 
place during April, 2022. 
Table 1 summarizes the timing of long-term follow-up data collection for each treatment status and 
wave of the study. Data collection totals for combined endline activities are depicted for reference. 
Table 1: Long Term Follow-up Data Collection Timeline
Treatment Placebo Peer
Tolerance
Prof/
Tolerance Placebo Peer/
Tolerance
Prof/
Tolerance
Pure 
Control
Wave 1 1 1 2 2 2 2 
Endline 
(Combined) 
December 
2019/ January 
2020 
(N = 185)
December 
2019 /January 
2020 
(N= 190)
December 
2019/ January 
2020 
(N= 190) 
March 2020
(N=221) 
March 2020
(N=227) 
March 2020
(N=229) 
March 2020
(N=204) 
Long Term 
Follow-up 
January-March 
2022 
(N=130) 
January-March 
2022 
(N=136) 
January-March 
2022 
(N=139) 
January-March 
2022 
(N=171) 
January-March 
2022 
(N=168) 
January-March 
2022 
(N=181) 
January-March 
2022 
(N=139) 
BASELINE CHARACTERISTICS COVARIATE BALANCE BY LTF ATTRITION
Table 2 displays baseline covariate balance between those who submitted a long-term follow-up survey 
(complete or partial responses, N=1,064), and those who submitted an endline survey in 2019/2020, but 
did not complete any stage of the long-term follow-up survey (N=574).13 Imbalances between these 
groups were identified with respect to gender (LTF respondents were more likely to be male) and age 
(LTF respondents were more likely to be younger) than endline submitters who did not complete the 
LTF. In addition, minor differences were found with respect to baseline measures of economic welfare 
(LTF respondents were less economically well-off), and grit (LTF respondents had higher baseline levels 
of grit). Finally, LTF attrition status was not found to be correlated to treatment assignment; LTF 
respondents were well-balanced on treatment assignment when compared to endline submitters who 
of these attention check questions; only two of 1,064 complete or partial long-term follow-up participants failed these 
attention checks, which was less than one percent of respondents. 
13 Approximately 188 respondents completed a baseline survey and a long-term follow-up survey, but did NOT complete an 
endline survey. Only 42 of these respondents had been awarded completion certificates and are most likely to have failed to 
complete endline responses due to 2020 covid-related shutdowns. The bulk of these respondents (107) were drop-outs and 
never-starters who were assigned to treatment, but did not complete it. These respondents were presumably motivated by the 
small incentive offered for completing the long-term follow-up. In order to explore whether the addition of these long-term 
follow-up participants who did not complete endline activities may have impacted results, all model specifications were 
replicated for (1) only those endline participants who received a certificate (and no-program control), (2) re-classifying never￾starters assigned to treatment as no-program control, and (3) limiting the analysis only to those participants that also 
submitted an endline. Results were generally robust across these different scenarios (with minor changes in coefficients), albeit 
loss of statistical significance on some terms presumably due to smaller sample sizes. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 75
did not complete the LTF. Unbalanced demographics were included in the subsequent analysis as 
covariates. 
Table 2: Standardized Baseline Covariate Balance by Attrition: Long Term Follow-up vs 
Endline
Variable (Standardized)
LTF (1,064) 
Mean [SE]
No LTF Endline (574) 
Mean [SE]
Diff (t-test) 
LTF vs No LTF Endline
Age -0.05 [0.03] 0.06 [0.04] -0.12**
Male 0.63 [0.02] 0.55 [0.02] 0.08***
Sunni 0.77 [0.01] 0.79 [0.02] -0.03 
Economic Welfare -0.04 [0.03] 0.1 [0.04] -0.13**
Anger 0.01 [0.03] 0 [0.04] 0.01 
Grit 0.07 [0.03] -0.09 [0.04] 0.17***
Deliberativeness -0.01 [0.03] -0.02 [0.04] 0.01 
Mental Health 0.04 [0.03] -0.01 [0.04] 0.05 
Pro-sociality 0.02 [0.03] -0.03 [0.04] 0.05 
Social Media Use -0.05 [0.03] 0 [0.04] -0.05 
Family Relations 0.02 [0.03] 0 [0.04] 0.02 
Family Overseas 3.37 [0.07] 3.46 [0.09] -0.09 
Religiosity (Reversed) -0.04 [0.03] 0.02 [0.04] -0.06 
Diversity of Friends 0 [0.03] -0.01 [0.04] 0.01 
Negative Out-Group Interactions -0.13 [0.00] -0.13 [0.00] -0.00 
Critical Thinking Exposure -0.02 [0.03] -0.09 [0.04] 0.07 
Prior Programming Exposure -0.02 [0.04] 0.01 [0.05] -0.02 
Note: Robust standard errors in brackets. The value displayed for t-tests are the differences in the means across the groups. All estimation 
regressions include school and wave fixed effects. In the right-hand column, ***, **, and * indicate significance at the one, five, and ten 
percent critical level, respectively. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 76
DATA ANALYSIS
For the long-term follow-up, we replicate the same analysis framework articulated in the final endline 
report. Specifically, regarding the effect of the program on outcomes of interest, we report estimates 
from regression models specified as follows: 
Endline Outcome = a + b*BIT Curriculum + c*Baseline Outcome + University FE
+ Wave FE + Unbalanced Covariates + e (1)
where a is an intercept term that measures mean outcomes in the absence of the BIT intervention, b
measures the effect of the BIT intervention, c measures how the baseline measures of the outcomes 
correlate with endline measures, the University FE and Wave FE terms refer to indicator variables that 
control for variation at the level of universities and data collection waves, and the Unbalanced Covariates
are control variables determined on the basis of analysis of pre-treatment covariate data, and e is an 
error term that captures other idiosyncratic variation from individual to individual. We estimate the 
regression using weighted least squares, weighting by the inverse probability of treatment assignment. 
We use heteroskedasticity robust standard errors, given that the treatment assignment was at the level 
of individuals.14 
We report two types of effects. The first effect is based on the contrast between tolerance BIT program 
participants versus those in the no-program control condition. We do this by estimating equation (1) on 
a sample that includes program participants and those who were not engaged in any program at all, and 
excludes those who received the placebo curriculum. This estimates the gross impact of the program, 
including the effect of the BIT curriculum per se, as well as any effects attributable to getting people 
together in a novel social situation to think critically. The second effect isolates the effect of receiving 
the BIT curriculum as compared to receiving the placebo curriculum. This effectively controls for the 
aforementioned social interaction effects, which would be operative in both the BIT and placebo 
curricula (therefore enabling comparison of differences between the tolerance BIT and placebo). To do 
this, we estimate specification (1) on a sample that includes those in the BIT curriculum and those in the 
placebo curriculum, excluding those who were not engaged in any program at all. This estimates the 
added value of the curriculum per se, above and beyond the effect of the novel social interactions and 
critical thinking exercises that were also included in the placebo curriculum.15
We use a hierarchical testing and statistical significance strategy to first evaluate overall effects on sets of 
items, and then to evaluate item-level effects that are adjusted to account for the fact that we are doing 
many tests (i.e., to address the “multiple comparisons problem”). As in the final report, we have sets of 
items measuring our concepts. For example, for norms of intolerance, we have six items: a Religious 
Outgroup Disgust index, Hostile Sexism index, Benevolent Sexism index, Anti-LGBT Disgust index, 
Anti-Western Attitudes index, and Distrust of Foreigners index. To test overall effects, we use an 
omnibus “mean effect” test constructed as the average of z-score standardized effects for each item.16
14 As a technique of accounting for class cohort spillovers, we also ran specifications that clustered standard errors on 
facilitator class cohorts. Results were found to be negligibly different from those reported in the results section below. 
15 As a technicality, we estimate this curriculum effect using two approaches: one includes the no-program control group so 
only to get more precise estimates on the control variables, and the other excludes them. Both specifications target the same 
effect and yield essentially the same results. 
16 This follows the approach of Kling, J.R., Liebman, J.B. and Katz, L.F. (2007), Experimental Analysis of Neighborhood Effects.
Econometrica, 75: 83-119. For an explanation of hierarchical testing strategies for program evaluation, see also Casey, K., 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 77
We implement this using the mseffect command in Stata. Then, within each group of items, we report 
item-level effects and with multiple comparisons false discovery rate (FDR) p-value adjustment using the 
mhtreg command in Stata.17 Methodologists refer to such adjusted item-level p values as “q values.” The 
item-level results allow us to assess which items may be driving the overall omnibus effect, and the q
values allow us to test for statistical significance in a way that accounts for the multiple testing problem. 
In our discussion, we take effects with p or q values less than 0.05 are considered to be statistically 
significant, while when we have p or q values between 0.05 and 0.10, we refer to the finding as 
“suggestive.” 
4. LONG TERM FOLLOW-UP RESULTS
In this section, we report long-term results for the effects of the tolerance BIT programming. We 
present results according to the causal chain specified in Figure 1 in the final report (Figure 9 in 
appendices). Results are depicted as coefficient plots for all outcomes in this section. Each coefficient 
plot depicts differences between long term follow-up measures, first when comparing tolerance BIT 
program participants to those enrolled in no-program control (“gross effects of programming”), then 
when comparing tolerance BIT program participants to pooled no-program control and placebo (“effect 
of tolerance BIT curriculum”).18 Tables reporting coefficients, standard errors, p-values and FDR (q￾values) are presented in the appendices and referenced where appropriate. As discussed in the research 
design of the final endline report, outcomes are generally constructed as aggregated additive indices of 
Likert-scale survey and post-activity measures, which were then standardized using pooled group mean 
and standard deviation. Therefore, all effects that we report below are in pooled standard deviation 
units. Summary statistics for new outcomes added to the LTF are presented in Table 11 (appendices). 
We begin by reporting the program’s estimated effects on social norms of tolerance and rejection of 
violence. 
NORMS 
Per the research design, we consider two categories of social norms that the tolerance BIT 
programming was meant to promote: (1) norms of tolerance and (2) the rejection of violence. 
NORMS OF TOLERANCE AND INTOLERANCE: LONG TERM EFFECTS OF PROGRAMMING
Figure 1 depicts coefficient plots for the omnibus “mean effects” test for each category of outcomes 
(focusing on norms of tolerance), followed by item-level effects for each outcome index. All 
specifications include unbalanced covariates and wave/school fixed effects. Table 3 (appendices) reports 
coefficients, standard errors, p-values, and q-values. 
In terms of gross effects of programming, we find that intolerance was about one-fourth of a standard 
deviation lower among BIT program participants than those in no-program control (ca. 0.27 sd; Table 3, 
Glennerster, R., & Miguel, E. (2012). Reshaping institutions: Evidence on aid impacts using a preanalysis plan. The Quarterly 
Journal of Economics, 127(4), 1755-1812. 
17 This follows the approach presented in Michael L. Anderson (2008) Multiple Inference and Gender Differences in the Effects 
of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects, Journal of the American 
Statistical Association, 103:1481-1495. 18 Each coefficient plot depicts coefficients and their respective 95 percent confidence intervals versus reference points of zero. 
Coefficients whose confidence intervals do not include zero should be interpreted as statistically significant. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 78
column 1); this represents a statistically significant effect. Differences were most evident in anti-LGBT 
attitudes and hostile sexism, with differences also apparent in benevolent sexism and religious outgroup 
disgust. However, these differences virtually disappeared when examining the effect of the tolerance BIT 
program. In other words, the tolerance BIT curriculum did not seem to account for these differences in 
norms outcomes once accounting for programmatic effects. 
Figure 1: Long Term Effects of Programming and Tolerance BIT Curriculum on Norms of 
Tolerance
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 79
NORMS REGARDING THE USE AND JUSTIFIABILITY OF VIOLENCE: LONG TERM EFFECTS OF 
PROGRAMMING
Figure 2 depicts coefficient plots for the omnibus “mean effects” test for each category of outcomes 
(focusing on norms of anti-violence replicated from endline), followed by item-level effects for each 
outcome index. All specifications include unbalanced covariates and wave/school fixed effects. Table 4 
(appendices) reports coefficients, standard errors, p-values, and q-values. 
In terms of gross effects of programming, we find that no long-term differences were observable when 
comparing those assigned to tolerance BIT treatment to no-program control. Nor are apparent 
differences present when accounting for placebo to measure the impact of the tolerance BIT curriculum. 
Figure 2: Long Term Effects of Programming and Tolerance BIT Curriculum on Rejection 
of Violence and VEOs
Figures 3 and 4 depict coefficient plots for the omnibus “mean effects” test for each category of 
outcomes (focusing on justifiability of violence), followed by item-level effects for each outcome index. 
These measures include the three new indices measuring (1) generic justifiability of violence in defense 
of a cause, (2) justifiability of jihadist violence (asked only of Muslim respondents), and (3) justifiability of 
violence in defense of one’s religious in-group following the stimulus activity. In addition, collective 
narcissism and (reversed) worry toward a friend engaging in violent activities were included as well. 
Figure 4 includes all respondents (omitting the index on justifiability of jihadist violence). All 
specifications include unbalanced covariates and wave/school fixed effects. Tables 5 and 6 (appendices) 
report coefficients, standard errors, p-values, and q-values. 
In terms of gross effects of programming, we find that norms regarding the justifiability of violence were 
about one-third of a standard deviation lower among BIT program participants than those in no￾program control (ca. -0.35 sd; Table 5, column 1). Differences were most apparent within the two post￾stimulus question modules focusing on collective narcissism and justifiability of violence in defense of 
religious in-group. Collective narcissism was about one-half of a standard deviation lower among BIT 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 80
program participants than those in no-program control (ca. -0.53 sd; Table 5, column 1), while 
justifiability of violence in defense of religious in-group was about two-thirds of a standard deviation 
lower among BIT program participants than those in no-program control (ca. -0.67 sd; Table 5, column 
1). Results for the subset of respondents who identified as Muslim were nearly identical to those of the 
entire sample. However, while those enrolled in tolerance BIT programming were observed to report 
lower levels of jihadist violence justifiability, the differences were not statistically significant. Moreover, 
as with the norms of tolerance measures, these aforementioned differences in outcomes were not 
present when examining the effect of the tolerance BIT curriculum. 
Figure 3: Long Term Effects of Programming and Tolerance BIT Curriculum on Beliefs 
Regarding the Justifiability of Violence (All Respondents)
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 81
Figure 4: Long Term Effects of Programming and Tolerance BIT Curriculum on Beliefs 
Regarding the Justifiability of Violence (Justifiability of Jihadist Violence Module Included; 
Muslim Respondents Only)
Overall, we observe sustained differences in observed levels of tolerance and anti-violence norms when 
comparing BIT program participants to those enrolled in no-program control. However, we cannot 
attribute these enduring effects to the curriculum divorced from the remaining social interactions and 
general prompts to think critically that those enrolled in both BIT treatment programming and placebo 
likely experienced. 
SKILLS: LONG TERM EFFECTS OF PROGRAMMING
Figure 5 depicts results from the omnibus “mean effects” test for all skills outcomes followed by item￾level effects for each outcome index. All specifications include unbalanced covariates and wave/school 
fixed effects. Table 7 (appendices) reports coefficients, standard errors, p-values, and q-values. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 82
Figure 5: Long Term Effects of Programming and Tolerance BIT Curriculum on 
Internalized BIT Skills
Relative to no program control, we observe differences across the board on exercise of skills and 
judgment in scenarios of aggression (Table 7, column 1). Those enrolled in tolerance BIT programming 
were observed to have roughly one-third of a standard deviation lower rate of skill-based failures when 
compared to no-program control. However, the curriculum per se (Table 7, column 4) does not seem 
to account for these differences. 
CONFIDENCE AND SELF-ASSESSED COMPETENCE TO ACT
Figure 6 depicts results from the omnibus “mean effects” test for all confidence/competence outcomes 
followed by item-level effects for each outcome index. All specifications include unbalanced covariates 
and wave/school fixed effects. Table 8 (appendices) report coefficients, standard errors, p-values, and q￾values. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 83
Figure 6: Long Term Effects of Programming and Tolerance BIT Curriculum on 
Confidence Act
Regarding confidence to act, we found that the tolerance BIT curriculum (Figure 6, plot 2) appears to 
have an enduring impact on participants. Program participants (Figure 6, plot 1) appeared to exhibit 
more confidence to act across the board when presented with scenarios of religious intolerance, 
scenarios of violence, and scenarios of extremism. Nearly half of these effects are accounted for by the 
tolerance BIT curriculum (appendix Table 8, column 4), which presumably is attributable to having 
participants role play and practice the skills they learned as part of the training. It is at this stage in the 
causal chain that the BIT curriculum per se seems to have had a lasting impact on participants, and is 
consistent with endline findings (which identified clear differences attributable to treatment). 
WILLINGNESS TO INTERVENE
Figure 7 depicts results from the omnibus “mean effects” test for all willingness to intervene outcomes, 
followed by item-level effects for each outcome index. As a reminder, this is ultimately what the 
intervention was intended to inspire. All specifications include unbalanced covariates and wave/school 
fixed effects. Table 9 (appendices) report coefficients, standard errors, p-values, and q-values. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 84
Figure 7: Long Term Effects of Programming and Tolerance BIT Curriculum on 
Willingness to Intervene
We see that from the gross effects (Figure 7, plot 1) and then from the rest of the tolerance BIT 
curriculum against the placebo (Figure 7, plot 2) that there remain measurable differences between 
those enrolled in BIT treatment programming and no-program control, driven primarily by increased use 
of bystander intervention techniques in response to eve-teasing and anti-LGBT bullying scenarios. BIT 
trainees were also less likely to respond that they would “do nothing” in a situation characterized by a 
likely-radicalized peer. Some of these differences are attributable to the BIT curriculum when accounting 
for programmatic effects via placebo. Specifically, the curriculum is responsible for enduring willingness 
to intervene in religious discrimination and anti-LGBT bullying scenarios. 
Moreover, there is suggestive evidence that the BIT curriculum accounts for some (about one-third) of 
these differences. However, turning to the newly-added measures of retrospective BIT use (Figure 8), 
we did not observe differences between tolerance BIT program participants and no-program control. 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 85
Figure 8: Long Term Effects of Programming and Tolerance BIT Curriculum on 
Retrospective Bystander Intervention Behaviors
SUMMARY
In summary, the analysis suggests that the tolerance BIT curriculum had an enduring impact on 
participant confidence/competence to act, and suggestive evidence that this also resulted in enduring 
higher levels of willingness to intervene with respect to issue areas of religious discrimination and anti￾LGBT bullying. However, for other outcomes of interest, it was not possible to attribute differences to 
the curriculum rather than to gross effects of programming. This was particularly true regarding the 
tolerance and violence norms measures—there were stark differences between those enrolled in 
tolerance BIT programming and no-program control with respect to these measures, but those 
differences could not be attributed to the curriculum content, as opposed to the programmatic effects 
that were also present in the placebo. We discuss implications of these findings further in the 
discussions and concluding remarks. 
5. LONG TERM FOLLOW-UP DISCUSSION AND CONCLUSIONS
This report documents the long-term follow-up evaluation of a secondary preventing/countering violent 
extremism (P/CVE) strategy implemented in Bangladeshi universities from September 2019 through May 
2020. The strategy tested was a tolerance-focused bystander intervention training (BIT) program, which 
we chose due to its extensive prior use as a method of combating other hateful and antisocial behaviors 
among young people, and due to its potential viability in the challenging political and social context of 
Bangladesh. To our knowledge, this was the first program of its kind to be implemented in a non￾Western context as a randomized controlled trial. It is also the first such program of its kind for which a 
long-term impact evaluation (approximately 21-24 months after programming depending on program 
wave) was conducted. 
The preponderance of findings presented above suggest programming of this nature can have an 
enduring impact on the confidence and competence of participants to intervene to challenge intolerance 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 86
and hateful speech and actions that are the basis of violent extremism. Moreover, there remains some 
suggestive evidence that this programming can also increase effective intervention accordingly. 
As discussed in the endline report for this project, many of the intermediary outcomes we deemed 
prerequisite to willingness to intervene were impacted positively by both the tolerance BIT 
programming, and the placebo, when compared to participants enrolled in a no-program control group. 
We conjectured that these “virtuous spillovers” were potentially due to the programmatic effects of 
repeated social contact in mixed groups, in a setting where participants are working on civic-focused 
topics in ways that improve critical thinking skills. This suggests that programming of this nature—which 
did not directly target anti-social behaviors such as intolerance or sexism—can nevertheless be impactful 
at reducing the perceived acceptability of those behaviors. For many of the young participants (the 
preponderance of whom were in their first year of university), both the BIT programming and civic￾focused placebo were likely the first time they had interacted with a truly mixed19 group of their peers 
in a small-group setting. Given an educational context where large class sizes tend to be the norm, it is 
plausible that creating a space within which these young people could safely discuss challenges facing 
their country with those they might not otherwise have interacted with would have enduring benefits 
both to their general prosociality, as well as to their affirmation of the benefits of a complex and diverse 
society. These insights suggest a potentially viable path for designing programming to target anti-social 
behaviors in contexts where political and/or social realities constrain the thematic areas that can be 
explicitly addressed in program design. 
We find in this long-term follow-up that these differences apparent both in BIT treatment and placebo 
appeared to endure, thus obscuring whether the BIT curriculum had added long-term value in driving 
enduring change in these norms-related outcomes. Given that nearly two years have passed since this 
intervention, it is possible that many of the norms that the tolerance BIT programming sought to 
inculcate within participants have spread within their social climates – and that placebo participants have 
adopted them more apparently than those in no-program control. Given that the programming sought 
to train young “change-makers,” it would not be surprising if those within the participant’s social circles 
also adopted norms of tolerance and opposition to violence. 
19 Randomized.
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 87
APPENDICES
Figure 9: Pathways from BIT Programming to Willingness to Intervene
Table 3: Long Term Effects of Programming and Tolerance BIT Program on Norms of 
Tolerance20
Outcome Expectation
Gross Effect of Programming
(N=716)
Effect of Tolerance BIT 
Curriculum (N= 998)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Intolerance (ME) - -0.27 [0.06] <0.01*** 0 [0.04] 0.99 
Relig. Outgroup Disgust - -0.27 [0.1] 0.01** 0.07* -0.05 [0.07] 0.52 1 
Hostile Sexism - -0.3 [0.09] <0.01*** <0.01*** -0.01 [0.07] 0.94 1 
Benevolent Sexism - -0.27 [0.09] <0.01*** 0.02** 0.09 [0.07] 0.21 1 
Anti-LGBT Disgust - -0.39 [0.1] <0.01*** <0.01*** -0.11 [0.07] 0.1 0.62 
Anti-Western Attitudes - -0.24 [0.1] 0.02** 0.12 0.03 [0.07] 0.67 1 
Distrust of Foreigners - -0.21 [0.1] 0.04** 0.25 0.05 [0.07] 0.44 1 
20 For all tables, ***, **, and * indicate significance at the one, five, and ten percent critical level, respectively. 
Identification 
with Facilitator
and Program 
Goals
Internalize 
Norms
Tolerance BIT 
Curriculum
Facilitator
Identity
Internalize Skills
Confidence/ 
Competence to 
Act
Effective 
Intervention on 
Behalf of Victim
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 88
Table 4: Long Term Effects of Programming and Tolerance BIT Program on Rejection of 
Violence
Outcome Expectation
Gross Effect of 
Programming
(N=727)
Effect of Tolerance BIT 
Curriculum 
(N= 1,013)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects + -0.03 [0.04] 0.4 0.02 [0.02] 0.38
Friend Disapprove + 0.02 [0.11] 0.83 1 0.05 [0.07] 0.46 1
Friend Worry + -0.11 [0.1] 0.28 1 -0.01 [0.07] 0.91 1
VEO Rejection + 0.05 [0.11] 0.63 1 0.07 [0.07] 0.33 1
VEO Awareness + 0.03 [0.11] 0.79 1 0.02 [0.07] 0.79 1
Open to VEO 
Demobilization
+ -0.15 [0.11] 0.16 0.82 -0.02 [0.07] 0.76 1
Table 5: Long Term Effects of Programming and Tolerance BIT Program on Justifiability 
of Violence Beliefs (All Participants)
Outcome Expectation
Gross Effect of Programming 
(N=727)
Effect of Tolerance BIT 
Curriculum (N= 1,013)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects - -0.35 [0.06] <0.01*** -0.04 [0.04] 0.35
Violence Justified 
Index (Generic) - -0.24 [0.11] 0.03** 0.1 -0.08 [0.07] 0.28 1
Religious In-Group 
Collective 
Narcissism
- -0.53 [0.11] <0.01*** <0.01*** 0 [0.07] 0.99 1
Religious Violence 
Justified - -0.67 [0.1] <0.01*** <0.01*** -0.04 [0.07] 0.55 1
Acceptability of 
Violence: Friend 
Worry (Reversed)
- -0.02 [0.1] 0.83 1 -0.05 [0.07] 0.47 1
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 89
Table 6: Long Term Effects of Programming and Tolerance BIT Program on Dominant 
In-Group Justifiability of Violence Beliefs (Jihadist Violence Included; Muslim Participants 
Only)
Outcome Expectation
Gross Effect of Programming 
(N=567)
Effect of Tolerance BIT 
Curriculum (N= 792)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects - -0.35 [0.07] <0.01*** 0 [0.05] 0.95 
Violence Justified 
(Generic) - -0.24 [0.11] 0.02** 0.12 -0.08 [0.07] 0.28 1 
Violence Justified 
(Jihadist; Muslim Only) - -0.18 [0.13] 0.16 0.8 -0.03 [0.08] 0.67 1 
Religious In-Group 
Collective Narcissism - -0.53 [0.1] <0.01*** <0.01*** 0 [0.07] 0.99 1 
Religious Violence 
Justified - -0.67 [0.1] <0.01*** <0.01*** -0.04 [0.07] 0.56 1 
Friend Worry 
(Reversed) - -0.02 [0.11] 0.83 1 -0.05 [0.07] 0.46 1 
Table 7: Long Term Effects of Programming and Tolerance BIT Program on Internalized 
Skills
Outcome Expectation
Gross Effect of Programming 
(N=727)
Effect of Tolerance BIT Curriculum 
(N= 1,013)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects - -0.35 [0.07] <0.01*** 0.04 [0.05] 0.42 
No Noticing - -0.27 [0.11] 0.01** 0.07 0.05 [0.07] 0.49 1 
No ID Emergency - -0.36 [0.1] <0.01*** 0.01** -0.04 [0.07] 0.59 1 
No Responsibility - -0.46 [0.1] <0.01*** <0.01*** 0 [0.07] 0.95 1 
Skill Deficit - -0.31 [0.11] <0.01*** 0.02** 0.12 [0.07] 0.1 0.51 
Audience Inhibition - -0.35 [0.1] <0.01*** <0.01*** 0.08 [0.07] 0.26 1 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 90
Table 8: Long Term Effects of Programming and Tolerance BIT Program on Confidence
to Act
Outcome Expectation
Gross Effect of Programming 
(N=727)
Effect of Tolerance BIT 
Curriculum (N= 1,013)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects + 0.26 [0.07] <0.01*** 0.11 [0.05] 0.04**
Act Religious 
Intolerance 
+ 0.17 [0.07] 0.02** 0.09* 0.05 [0.05] 0.33 1 
Act vs Violence + 0.26 [0.09] <0.01*** 0.01** 0.1 [0.05] 0.06* 0.24 
Act vs Extremism + 0.3 [0.08] <0.01*** <0.01*** 0.1 [0.05] 0.08* 0.32 
Friends Act + 0.09 [0.11] 0.41 1 0.11 [0.08] 0.14 0.56 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 91
Table 9: Long Term Effects of Programming and Tolerance BIT Program on Willingness 
to Intervene
Outcome Expectation
Gross Effect of Programming 
(N=727)
Effect of Tolerance BIT 
Curriculum (N= 1,013)
(1)
Coeff 
[SE]
(2)
p-value
(3)
q-value
(4)
Coeff 
[SE]
(5)
p-value
(6)
q-value
Mean Effects + 0.16 [0.05] <0.01*** 0.06 [0.04] 0.09*
BIT in Religious Discrim. + 0.17 [0.11] 0.11 1 0.18 [0.07] 0.02** 0.21 
BIT in Eve Teasing + 0.33 [0.1] <0.01*** 0.02** 0.06 [0.07] 0.39 1 
BIT in LGBT + 0.3 [0.1] <0.01*** 0.04** 0.15 [0.07] 0.03** 0.36 
BIT in Radicalization + 0.03 [0.11] 0.78 1 0.12 [0.07] 0.12 1 
No Passivity Religious Discrim. + -0.03 [0.1] 0.8 1 0.02 [0.07] 0.8 1 
No Passivity Eve Teasing + 0.13 [0.1] 0.22 1 0.1 [0.07] 0.15 1 
No Passivity LGBT + 0.2 [0.1] 0.04** 0.52 0.02 [0.07] 0.78 1 
No Passivity VEO + 0.33 [0.11] <0.01*** 0.01** 0.1 [0.07] 0.16 1 
No Perpetrating Religious 
Discrim.
+ 0.04 [0.11] 0.72 1 0 [0.07] 1 1 
No Perpetrating Eve Teasing + 0.07 [0.1] 0.51 1 0.05 [0.07] 0.48 1 
No Perpetrating LGBT + 0.2 [0.1] 0.05 0.55 0.02 [0.07] 0.8 1 
No Perpetrating VEO + 0.11 [0.11] 0.31 1 -0.05 [0.07] 0.45 1 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 92
Table 10: Long Term Effects of Programming and Tolerance BIT Program on 
Retrospective Evaluations of BIT Use in Real Life and on Social Media
Outcome Expectation
Gross Effect of Programming 
(N=632)
Effect of Tolerance BIT 
Curriculum (N= 880)
(1)
Coeff [SE]
(2)
p-value
(3)
q-value
(4)
Coeff [SE]
(5)
p-value
(6)
q-value
Mean Effects BIT 
Retrospective Use Indices
+ 0.03 [0.09] 0.79 -0.03 [0.07] 0.65 
Counts of Retrospective 
BIT: Religious Intolerance
+ -0.07 [0.11] 0.58 1 -0.08 [0.08] 0.34 1 
Counts of Retrospective 
BIT: Sexism
+ 0.1 [0.11] 0.36 1 -0.06 [0.08] 0.44 1 
Counts of Retrospective 
BIT: Extremism
+ 0.12 [0.11] 0.28 1 -0.12 [0.09] 0.15 0.73 
Counts of Retrospective 
BIT: Violence
+ -0.05 [0.11] 0.64 1 -0.03 [0.08] 0.72 1 
Bystander Behaviors: 
Screenshot of BIT
+ 0.12 [0.09] 0.22 1 0.08 [0.06] 0.18 0.88 
CONTRACT No. GS-10F-0033M / ORDER No. AID-OAA-M-13-00013
USAID.GOV OBIRODH Impact Evaluation: Road to Tolerance Youth Leadership Training Program in Bangladeshi Universities | 93
Table 11: Summary Statistics for Long Term Follow-Up Additional Outcomes of 
Interest21
Outcome Count Mean St. Dev Min Max
Violence Justifiable (Generic; All) 1064 14.10 5.19 4 28 
Violence Justifiable (Jihadist; Muslim Only) 834 20.90 9.70 7 49 
Stimulus Activity: Collective Narcissism 1064 22.27 6.54 6 42 
Stimulus Activity: Religious Violence Justified 1064 28.76 8.85 7 49 
Amped Extremism: Friend Worry (Reversed) 1064 6.27 2.90 2 14 
Retrospective BIT: Religious Intolerance 853 6.30 3.53 1 20 
Retrospective BIT: Sexism 817 6.51 3.67 1 20 
Retrospective BIT: Extremism 774 6.07 3.57 1 20 
Retrospective BIT: Violence 846 6.57 3.52 1 20 
BIT on FB: Willing to Upload Screenshot 921 0.07 0.32 0 2 
21 Unstandardized values depicted for transparency of scaling; standardized values of additive indices were used for the analysis. 
United States Agency for International Development
1300 Pennsylvania Avenue NW
Washington, DC 20004