.
DHARTE
Desarrollando Habilidades Aprendiendo Responsabilidades
para Transformar el Entorno
Award: AID-523-A-15-00004
USAID México
FY14-18 Development Objective 1:
Crime and violence prevention models replicated by local stakeholders
Internal Midterm Evaluation
Submitted April 2017
Prepared by: 
Amy Black, Program Assistant for Monitoring and Evaluation 
Jaye Stapleton, Senior Program Officer for Monitoring and Evaluation
Acknowledgements 
The evaluators would like to thank the Heartland Alliance International Mexico country office, 
especially the DHARTE project team, for seamlessly managing field visit logistics and prioritizing all 
requests for information from the evaluation team. Additionally, the evaluators would like to thank 
Tarrajna Walsh for contributing to data collection, the HAI Latin America and Caribbean regional 
team for supporting evaluation planning, and most importantly, the DHARTE program students, 
parents, and school staff for generously giving their time to participate in this evaluation. 
3
Table of Contents 
ACKNOWLEDGEMENTS ......................................................................................................................................... 3
TABLE OF CONTENTS ............................................................................................................................................ 4
ACRONYMS AND ABBREVIATIONS ......................................................................................................................... 5
EXECUTIVE SUMMARY ........................................................................................................................................... 6
I. EVALUATION PURPOSE ................................................................................................................................. 8
II. PROJECT BACKGROUND ............................................................................................................................... 9
III. EVALUATION METHODS AND LIMITATIONS ................................................................................................ 11
A. EVALUATION METHODS .....................................................................................................................................11
B. EVALUATION LIMITATIONS ..................................................................................................................................13
IV. FINDINGS ................................................................................................................................................... 13
A. TO WHAT EXTENT IS THE PROJECT DESIGN LOGICAL, COHERENT, AND FEASIBLE?........................................................13
B. TO WHAT EXTENT ARE THE PROJECT INTERVENTIONS CONSISTENT WITH THE NEEDS OF KEY STAKEHOLDERS?.................14
C. HOW APPROPRIATE AND USEFUL ARE THE KEY PERFORMANCE INDICATORS IN ASSESSING THE PROJECT’S PROGRESS?....16
D. IS THE PROJECT ON TRACK TO ACHIEVE THE INDICATOR TARGETS ACCORDING TO SCHEDULE? IF NOT, WHAT HAVE BEEN THE 
OBSTACLES TO ACHIEVEMENT BOTH IN TERMS OF FACTORS THAT THE PROJECT IS ABLE TO INFLUENCE AND EXTERNAL FACTORS 
BEYOND ITS CONTROL?...............................................................................................................................................18
E. HOW EFFECTIVELY HAS THE PROJECT ENGAGED STAKEHOLDERS IN PROJECT IMPLEMENTATION? ..................................20
F. WHAT ARE THE KEY PROJECT ACHIEVEMENTS THUS FAR THAT COULD BE SUSTAINED BEYOND THE LIFE AND THE CONTEXT OF 
THE PROJECT? DOES THE PROJECT HAVE A STRATEGY IN PLACE TO SUSTAIN THESE ELEMENTS?.............................................21
G. WHAT EMERGING PROJECT PRACTICES OR EXPERIENCES HAVE POTENTIAL TO BECOME GOOD PRACTICES OR LESSONS 
LEARNED AT THE END OF PROJECT? ..............................................................................................................................21
V. CONCLUSIONS AND RECOMMENDATIONS ................................................................................................ 22
A. CONCLUSIONS .................................................................................................................................................22
B. RECOMMENDATIONS.........................................................................................................................................24
VI. ANNEX A: MATRIX OF EVALUATION QUESTIONS AND METHODS ............................................................... 25
VII. ANNEX B: DHARTE PROJECT RESULTS FRAMEWORK ................................................................................... 2
VIII. ANNEX C: PROJECT DOCUMENTS INCLUDED IN DESK REVIEW .................................................................... 2
IX. ANNEX D: ACTIVITY OBSERVATION GUIDE .................................................................................................... 3
X. ANNEX E: FOCUS GROUP GUIDES ................................................................................................................. 4
XI. ANNEX F: KEY INFORMANT INTERVIEW GUIDES ........................................................................................... 7
4
Acronyms and Abbreviations 
DQA Data Quality Audit
EC Espacio Creativo (Creative Space)
ECF Espacio Creativo Familia (Creative Family Space)
DHARTE Desarrollando Habilidades Aprendiendo Responsabilidades para 
Transformar el Entorno (Developing Skills and Learning Responsibilities to 
Transform the Environment)
FGD Focus Group Discussion
GOM Government of Mexico
HAI Heartland Alliance International
KII Key informant interview
RS Relaciones Saludables (Healthy Relationships)
SO Strategic Objective
SAVRY Structured Assessment of Violence Risk in Youth
5
Executive Summary 
Introduction
This report outlines the methodology and findings of an internal midterm evaluation for the USAID￾funded project, Desarrollando Habilidades Aprendiendo Responsabilidades para Transformar el 
Entorno (Developing Skills and Learning Responsibilities to Transform the Environment) (DHARTE). 
DHARTE, a three-year project that launched on October 1, 2015 in the Monterrey Metropolitan Area 
(MMA), Nuevo Leon, Mexico, aims to decrease the risk of youth participation in violent activity 
through primary and secondary prevention activities in high-risk communities. Specifically, DHARTE 
has three strategic objectives (SOs):
1. Improve civic participation in the community by promoting healthy relationships. 
2. Increase at-risk youth engagement in productive activities.
3. Strengthen leadership and entrepreneurial skills of at-risk youth.
At the time of the midterm DHARTE activities under SO 1 were being implemented in ten schools in 
the project area, targeting youth in 6th through 9th grade as well as parents of youth identified to be 
at medium- or high-risk. Under SO 2, activities for out-of-school youth in project communities were 
implemented by a local partner, Promocion de Paz (PPAZ) in the first year of the project. Due to 
concerns about financial mismanagement, HAI began an investigation into PPAZ in September 2016 
and soon after PPAZ withdrew from the DHARTE project. Heartland is currently working on a project 
modification with USAID under which Heartland staff will directly implement activities for out-of￾school youth. The bulk of SO 3 activities will not take place until the second half of the project. 
The overall purpose of this midterm evaluation was to assess the strengths of the DHARTE project, 
as well as identify areas for improvement in the latter half of the project. The evaluation largely 
focused on activities under SO 1 because, as described above, the majority of activities under SO 2 
and SO 3 were not ongoing at the time of the evaluation, although SO 2 and SO 3 were included 
when possible. In this report, the evaluators highlight (a) opportunities for mid-project adjustments 
and (b) best practices that should be reinforced and shared going forward. .
The midterm included the following evaluation questions:
1. To what extent is the project design logical, coherent, and feasible? 
2. To what extent are the project interventions consistent with the needs of key stakeholders? 
3. How appropriate and useful are the key performance indicators in assessing the project’s 
progress? 
4. Is the project on track to achieve the indicator targets according to schedule? If not, what 
have been the obstacles to achievement both in terms of factors that the project is able to 
influence and external factors beyond its control? 
5. How effectively has the project engaged stakeholders in project implementation? 
6. What are the key project achievements thus far that could be sustained beyond the life and 
the context of the project? Does the project have a strategy in place to sustain these 
elements?
7. What emerging project practices or experiences have potential to become good practices or 
lessons learned at the end of project?
Methodology 
The evaluation took place in March and April 2017 and included a desk review of all project 
documents as well as a site visit with activity observations, focus group discussions (FGDs), and key 
informant interviews (KIIs). The field visit took place in 7 out of the 10 DHARTE schools; 3 schools 
were not included due to time and logistical restrictions. KII respondents included school 
6
administrators (n=5), teachers (n=4), DHARTE curricula developers (n=2), project staff (n=8), and 
U.S.-based staff (n=3) from HAI’s parent company Heartland Alliance: two program managers for 
U.S.-based violence prevention programs and one specialist in clinical psychology for youth. Ten 
FGDs were conducted which included a total of 84 students and 10 parents.
The evaluation faced several limitations. First, it was not possible to randomly select either the 
schools or the participants that were part of the evaluation due to logistical constraints. Second, 
some FGDs exceeded or were less than the target number of 8 to 12 respondents. Third, evaluation 
participants may have biased their responses due to influence from their peers and/or fear that
negative feedback would result in punitive measures for facilitators or a loss of programming. The 
evaluation team made every effort to mitigate these biases through the following actions: including
the maximum possible number of schools, participants, and activity observations in order to ensure 
representative, quality data; triangulating findings from the qualitative data through analysis of 
quantitative data; and fully explaining the evaluation purpose to all participants. Finally, the
evaluators were not able to fully evaluate the Year 1 community-based component of the project due 
to a lack of data from PPAZ. As Heartland prepares to take over implementation of community-based 
activities it will be important to ensure strong monitoring and evaluation (M&E) systems so that this 
component of the project can be evaluated at the end of the project. 
Findings
Evaluators found the program design to be logical and coherent, with activities that are highly 
relevant to the needs identified by students, parents, and school staff. Key performance indicators 
were found to be useful and appropriate, although additional data collection tools should be 
considered in order to ensure that program impacts on life skills and emotional development are 
accurately captured. The project is largely on schedule to meet indicator targets, with the exception 
of two issues. First, parent participation has been lower than anticipated. Evaluation data shed light 
on the complexities of engaging parents, and this report includes recommendations on strategies 
that could be piloted to increase parent attendance. Second, as mentioned above, the community￾based component for out-of-school youth has not progressed as planned due to issues with PPAZ. A 
forthcoming project modification will detail the ways in which the project team will address indicator 
targets for SO 2 going forward. Finally, it should be noted that during an audit of the data used for 
measuring indicator progress evaluators discovered a high rate (50%) of data entry errors when 
comparing a sub-sample (n=60) of paper records to the electronic database. Given that there were 
no errors in the source documents, the data can be re-entered and re-analyzed in order to confirm 
progress against indicators to date.
Regarding stakeholder engagement, the project has successfully involved students by soliciting their 
suggestions through the use of anonymous feedback surveys. Stakeholders such as administrators 
and teachers were generally pleased with the project, but teachers reported some inconsistency in 
the extent to which facilitators allowed them to engage in the actual student workshops. 
Qualitative data from evaluation FGDs and KIIs indicated that activities implemented to date have 
contributed to positive changes in students. School personnel and project staff reported that 
students have greater self-awareness, emotional intelligence, and communication skills, which was 
supported by positive feedback from the majority of student participants. Students and school 
personnel also were highly satisfied with the performance of workshop facilitators. Project staff have 
created a sustainability plan in an effort to ensure these positive impacts will be carried forward after 
the project ends. However, the current plan relies on schools adopting some part of the intervention 
workshop into daily classes, which may not be feasible given the current level of teacher 
involvement. Additional work is needed with public, private, and civil society stakeholders in order to 
confirm an achievable sustainability plan is put in place. 
7
In terms of emerging good practices the project has two strengths: the development and use of 
evidence-based workshop curricula and the development and use of an evidence-based screening 
tool for at-risk youth, both of which are administered by skilled facilitators. Government and non￾government stakeholders have already expressed interest in adopting the screening tool. 
Recommendations to strengthen the project based on the evaluation findings are as follows:
 Correct data entry errors by re-entering paper records into the electronic database using
strict data entry protocols, free data entry software (e.g., Epi Info, CSPro), and regular data 
quality audits. 
 Strengthen project monitoring through systematic use of surprise facilitator monitoring visits.
 Collect and analyze data on student grades, in-school psychical altercations, dropouts, 
detentions, and suspensions.
 Identify an additional measurement tool that better captures outcomes such as resilience, 
emotional awareness, and/or life skills.
 Engage a gender expert in order to better understand the impact of gender on students and 
parents and determine whether a differential approach is needed. This expert could also 
further investigate the issue of stigma for both parents and students. 
 Develop and pilot strategies for increasing parent engagement such as holding workshops on 
the weekend or in the evening, inviting all parents to project activities rather than only 
parents of at-risk youth, providing child care during workshops, hosting joint workshops for 
parents and students, and adjusting ECF content to be more parent-driven (e.g., a parent 
support group).
 Create a policy for teacher participation in RS and EC workshops and confirm that all project 
and school staff understand the scope of teacher involvement. 
 Hold additional meetings and working sessions with teachers and school administrators in 
order to further define a feasible sustainability plan.
 Operationalize a learning system so that feedback from facilitators can inform real-time 
updates and modifications to the curricula in order to ensure that the final program model is
shaped by ongoing implementation.
 Develop a security plan for Heartland staff who will be conducting activities in communities 
for out-of-school youth. Provide training for all project staff on responding to security 
incidents (both in schools and out of schools). 
I. Evaluation Purpose 
The overall purpose of this midterm evaluation was to assess the strengths and weaknesses of the 
DHARTE (Desarrollando Habilidades Aprendiendo Responsabilidades para Transformar el Entorno) 
project design and implementation, review the progress made toward achieving project outcomes, 
and identify lessons learned to date. In this report, the evaluators identify (a) opportunities for mid￾project adjustments and (b) best practices that should be reinforced and shared going forward. 
The evaluation included a desk review of project documents, analysis of project data sets (including 
a data quality audit (DQA)), project activity observations, key informant interviews (KIIs), and focus 
group discussions (FGDs) with stakeholders. The primary audiences for this report are Heartland 
staff, USAID, and stakeholders from project communities (e.g., school administrators). 
The evaluation questions focus on the validity of the project’s design, the relevance of the project’s 
services to the target groups’ needs, the project’s efficiency and effectiveness, the impact of the 
results to date, and the potential for sustainability. Specifically, the evaluation questions are:
1. To what extent is the project design logical, coherent, and feasible? 
8
2. To what extent are the project interventions consistent with the needs of key stakeholders? 
3. How appropriate and useful are the key performance indicators in assessing the project’s 
progress? 
4. Is the project on track to achieve the indicator targets according to schedule? If not, what 
have been the obstacles to achievement both in terms of factors that the project is able to 
influence and external factors beyond its control? 
5. How effectively has the project engaged stakeholders in project implementation? 
6. What are the key project achievements thus far that could be sustained beyond the life and 
the context of the project? Does the project have a strategy in place to sustain these 
elements?
7. What emerging project practices or experiences have potential to become good practices or 
lessons learned at the end of project?
II. Project Background 
The DHARTE project was designed in direct response to the issues of youth involvement in crime and 
violence in the Monterrey Metropolitan Area (MMA), Nuevo Leon, Mexico. Youth exposure to criminal 
activity and violence is a major concern for the Government of Mexico (GOM), practitioners in the 
health, education, and juvenile justice systems, parents, and communities throughout the country. 
Youth as young as ten are recruited for minor roles in criminal activity and may become involved in 
more serious crimes, such as armed robbery or murder, during their teen years. 
Criminal violence is not the only threat facing Mexican youth. The acceptability and high prevalence 
of gender-based violence, including family violence, directly affects the wellbeing of many youth. This 
violence at home often translates to violence at school. According to the Organization for Economic 
Co-operation and Development (OECD), in 2013 Mexico had the highest rate of intimidation or verbal 
abuse among students (29.5%) as well as physical injury caused by violence among students 
(10.8%) compared to other OECD countries.1 Bullying and other violence in schools increases school 
dropout, crime, and gang involvement, particularly among youth who experience family and 
community violence outside of schools. Thus, violence and bullying at schools both reflect and 
influence crime rates, making schools an important space in which to foster change. 
Consistent with the Merida Initiative’s emphasis on reducing violence by creating healthy and 
resilient communities, the GOM is seeking new models tailored to specific needs at the state and 
municipal levels, explicitly recognizing that positive changes in children’s and adolescents’ lives are 
largely influenced by the institutions responsible for shaping their social development. In 
conjunction, HAI understands the right to security as a human right, and one that is of utmost 
importance in child and youth development.
DHARTE aims to address youth violence through a three-part approach to foster: 1) healthy 
relationships, 2) skills building, and 3) independence through social and economic entrepreneurship 
and empowerment. HAI believes that this approach will divert middle- and high-risk youth from 
criminal activity. The DHARTE program is implemented in five municipalities of the Monterrey 
Metropolitan Area in the state of Nuevo León in northern Mexico: Escobedo, Santa Catarina, 
Monterrey, Guadalupe and Juárez.
1 TALIS 2013 results: An International Perspective on Teaching and Learning p. 290
9
 
Building on lessons learned from HAI’s USAID-funded EJEMPLAR program, which leveraged
audiovisual tools and arts to promote youth development, DHARTE uses arts-based programming to 
offer alternatives to violence and prevent criminal activity. Arts programming has been used to 
address issues of community violence in the U.S.,2 and programs that use art and creative spaces 
have also been used to build resiliency.3 The National Endowment for the Arts has developed a 
handbook for using art as a form of violence prevention entitled “Creative Partnerships for 
Prevention: Using the Arts and Humanities to Build Resiliency in Youth, A Drug and Violence 
Prevention Resource for Schools, Cultural Organizations, and Others Working with Youth”, which 
contains information about several U.S. project sites that have used this framework successfully with 
at-risk youth.4 Art activities have also been used in the U.S. school context as a peacebuilding tool to 
help students heal from violence and build communities within their schools.5
DHARTE leverages the Structured Assessment of Violence Risk in Youth (SAVRY), a validated crime 
and violence assessment system, to identify at-risk children. The SAVRY “is composed of 24 items in 
three risk domains (Historical Risk Factors, Social/Contextual Risk Factors, and Individual/Clinical 
Factors), drawn from existing research and the professional literature on adolescent development as 
well as on violence and aggression in youth. Not designed to be a formal test or scale, there are no 
assigned numerical values or specified cutoff scores.”6 DHARTE staff used the SAVRY to create and 
test the Evaluación de Riesgo Individual (ERI), a tool to categorize students in primary and secondary 
schools as either low-, medium-, or high-risk. Additionally a modified version of the ERI was created 
to identify medium- and high-risk youth who are not in school or employed living in project 
communities. 
Interventions for in-school youth and their families are as follows:
 Relaciones Saludables (RS) workshops – These 45-minute student workshops include
primary prevention activities for all students in a project school once per week during school 
hours. Workshops take the place of regularly scheduled classes and are mandatory. 
Content focuses on identifying risky situations, healthy relationships, emotional awareness, 
etc. Each topic is addressed over the course of two classes: the first class directly addresses 
a topic related to violence prevention and the second class explores the same topic through 
art. RS workshops are offered to students in 6th through 9th grades.
 Espacio Creativo (EC) workshops – These 90-minute student workshops include secondary 
prevention activities specifically for medium- and high-risk youth in secondary schools only. 
Workshops are provided once per week during school hours and are optional for eligible 
students. Workshops are centered around art therapy activities that are inspired by the 
topics discussed in the RS workshops. EC workshops are offered to students in 7th through 
9th grades.
 Espacio Creativo Familia (ECF) workshops – These 90-minute workshops secondary 
prevention activities are optional for the parents/care givers of medium- and high-risk 
students who are invited to participate in the EC workshop. Workshops are provided once 
per week for seven weeks and focus on fostering positive parenting practices and 
communication among parents and youth. 
All workshops are taught by two Heartland facilitators, all of whom have experience working with 
youth and providing psychosocial services.
2 National Endowment for the Arts, 1998. 3 National Endowment for the Arts, 1998. 4 http://files.eric.ed.gov/fulltext/ED421449.pdf
5 Marshall, L. (2014). Art as peacebuilding. Art Education. 6 SAVRY website accessed on April 19, 2017. http://www4.parinc.com/Products/Product.aspx?ProductID=SAVRY
10
 
The DHARTE project also includes a community-based component that aims to target medium- and 
high-risk youth ages 16 to 22 who are not in school or employed. Under the original project design,
Heartland partnered with Promocion de Paz (PPAZ), a civil society organization based in Monterrey, 
Nuevo León, to implement these community-based services. In August of 2016, DHARTE staff 
uncovered inconsistencies in management of the funds that were awarded to PPAZ. As Heartland
was investigating these issues PPAZ informed Heartland of its intent to withdraw from the DHARTE 
project at the close of the annual subward agreement on September 30, 2016. Because Heartland
is no longer working with PPAZ, the DHARTE project will have to change its strategy to meet targets 
for the community-based component of the project. Heartland is in the process of working with 
USAID to finalize a project modification under which Heartland will implement the community-based 
portion of DHARTE. While PPAZ reports were included in the desk review of project documents, the 
evaluation team was unable to meet with PPAZ as part of this evaluation and was also unable to 
access PPAZ source data in order to evaluate the community-based component of the project. 
Evaluators also could not visit PPAZ communities during the field visit due to security concerns. 
Therefore, this evaluation focuses on project activities implemented by Heartland only under SO 1 
and does not include activities implemented by PPAZ under SO 2. 
III. Evaluation Methods and Limitations 
A. Evaluation Methods 
The evaluation was conducted in March and April 2017 by HAI monitoring and evaluation (M&E) staff 
from the headquarters office. The evaluation included the following methods: 1) desk review of
project documents, 2) analysis of existing project data, including a DQA, 3) observation of project 
activities, 4) key informant interviews (KIIs), 5) focus group discussions (FGDs). See Annex A for a 
matrix that maps evaluation methods to the relevant evaluation questions.
The evaluation utilized a non-experimental, three-phase design. During the first phase the evaluation 
team conducted background research and performed an analytical review of existing documentation 
and project data sets. During the second phase, the team conducted a field visit to the project sites 
in Monterrey, Mexico. Findings from the first phase informed the final design of data collection 
instruments used during the field visit for activity observations, KIIs, and FGDs. The third phase 
included both quantitative and qualitative data analysis and report writing. HAI’s nuanced approach 
to gender informed the evaluation design and implementation. 
Sample
The midterm evaluation field visit included 7 out of the 10 DHARTE program schools. The schools 
selected for FGDs, KIIs, and activity observations were purposely selected by the evaluation team, in 
conjunction with project staff, based on logistical restrictions. While the sample was non-random, the 
evaluation team made every effort to include the maximum possible number of schools, participants, 
and activity observations in order to ensure a representative sample.
Desk Review of Project Documents
The evaluation began with a desk review of project documents including the project proposal, 
quarterly reports, M&E plans, data collection tools, and curricula for project workshops. Please see 
Annex C for a full list of all documents reviewed. Evaluators used the documents to gain a thorough 
understanding of project’s goal, objectives, planned activities, and results to date. 
11
Analysis of Existing Data
Evaluators conducted secondary analysis of quantitative project data collected to date, including 
student attendance, student feedback surveys, and ERI data. The analysis used electronic 
databases created by DHARTE program staff based on paper source documents. During the field 
visit both hard and soft copies of data were assessed in a DQA. 
Activity Observations
Activity observations were conducted in a sub-sample of project schools by one member of the 
evaluation team. Observations were conducted for two and five sessions of the student-level EC and 
RS interventions, respectively (see Annex D for the Activity Observation Guide). Observations of the 
EC and RS workshops took place at project schools based on the normally scheduled time and place 
of each activity. No observations of the family-level ECF intervention were conducted. Due to 
logistical constraints, the evaluation team had only two opportunities to engage parents/caregivers
in the ECF workshops. The ECF workshops were launched at the end of February 2017, and to date 
attendance has varied considerably and has been unpredictable. The evaluation team opted to 
conduct two focus groups with families rather than one focus group and one ECF workshop
observation in order to maximize the amount of qualitative data collected and better understand 
factors driving parent engagement. 
Key Informant Interviews
Evaluators interviewed all project staff (n=8) as well as a sub-sample of teachers (n=5) and 
administrators (n=5) from project schools. KIIs were also conducted with the two individuals who 
created the curricula for the DHARTE workshops: a specialist in art therapy and a specialist in youth 
violence prevention. Finally, the evaluators interviewed three U.S.-based staff from HAI’s parent 
company Heartland Alliance: two program managers for U.S.-based violence prevention programs 
and one clinical psychologist that specializes in treating youth. 
DHARTE project staff scheduled KIIs with a sub-sample of school teachers and administrators. The 
KIIs were conducted by at least one member of the evaluation team and audio recorded for 
transcription and analysis. KIIs were scheduled with assistance from the DHARTE project team and 
took place at project schools and at the Heartland Monterrey office, for school staff and Heartland
staff, respectively. Interviews lasted approximately 30 to 60 minutes. The finalized KII guide can be 
found in Annex F. In total, five school administrators (principals or vice principals) were interviewed, 
four teachers were interviewed, the two individuals who developed the curricula were interviewed, 
and 8 staff members were interviewed (one staff member, the program coordinator, was interviewed 
several times during the week), for a total of 19 individuals interviewed. 
Focus Group Discussions 
FGDs were conducted with a sub-sample of participants from EC, RS, and ECF workshops. FGDs 
were conducted by two members of the evaluation team: One member led the focus group and one 
member acted as note taker and ensured the focus group audio was recorded. Audio recordings 
were transcribed and analyzed post-field visit. FGDs took place at project schools during the time 
that was normally designated for EC/RS/ECF workshops due to logistical limitations that prohibited 
participants from meeting at other times. FGDs lasted approximately 45 minutes. FGD participants 
were randomly chosen from the group of project participants that presented for the activity on the 
day of the focus group. The evaluation team aimed for FGDs to be comprised of approximately 8 to 
12 participants; however, in practice some FGDs exceeded 12 participants due to the inability to turn 
students away. The final FGD guide can be found in Annex E. There was one female-only focus group 
and one male-only focus group for RS participants to explore any gender dynamics. The number of 
focus groups with each population is as follows: two focus groups with parents, some who had 
attended ECF workshops and some who had been invited but had not attended; five focus groups 
12
with student participants from EC workshops; and three focus groups with student participants from 
RS workshops. Originally, four focus groups with student participants from RS workshops were 
planned, but due to lengthy travel time between schools and time restrictions of the RS workshops, 
one planned focus group had to be cancelled. In total, the evaluators conducted 10 focus groups 
with a total of 84 students and 10 parents.
B. Evaluation Limitations 
The evaluation faced several limitations. First, it was not possible to randomly select either the 
schools or the participants that were part of the evaluation due to logistical constraints. Therefore, it 
is possible that the qualitative data collected through FGDs and KIIs is not representative of all 
DHARTE participants. Since the DHARTE project staff assisted with participant selection, it is also 
possible that the team could have selected schools and participants in which they were more likely 
to get a positive review. The evaluation team made every effort to mitigate these sources of bias by 
using quantitative data to triangulate findings from qualitative data.
A second limitation was the inability to control the size of EC and ECF FGDs. Two FGDs for EC 
workshops included over 12 students; in one focus group the total number reached 18 students, 
which made it difficult to conduct the focus group effectively. Due to logistical limitations of student 
scheduling, evaluators were unable to turn away FGD participants for the EC workshops. Conversely, 
it was difficult to ensure an adequate number of parents would attend the ECF FGDs because parent 
involvement in project activities to date has been limited. It is possible that too many or too few 
participants limited the quality of data collected during FGDs. However, findings across FGDs were 
generally consistent, so evaluators believe that FGD size did not have a significant impact. 
Third, as described in the previous section, evaluators were not able to fully evaluate the community￾based component of the project. Although project reports from PPAZ, Heartland’s former 
implementing partner, were included in the evaluation desk review, there was generally a lack of 
data, and the evaluation team was unable to travel to the communities in which PPAZ worked due to 
security concerns. As Heartland prepares to take over implementation of community-based activities 
it will be important to ensure strong M&E systems are put in place so that this component of the 
project can be evaluated at the end of the project. 
Finally, there is the possibility that evaluation participants did not fully disclose problem areas of the 
project because of a fear that negative feedback would result in punitive measures for facilitators 
and/or a loss of programming. The FGD format also means that the evaluation team was unable to 
interact with student or parent participants on a one-on-one or anonymous basis, and participants’
feedback could have been biased by the presence of their peers. The evaluators tried to reduce
biases by ensuring that all evaluation participants understood the evaluation purpose and the ways 
in which FGD data would be used. 
IV. Findings 
A. To what extent is the project design logical, coherent, and feasible? 
Overall, the project design was found to be logical and coherent based on a review of the project 
Results Framework (Annex B). Activities are logically nested within intermediate results, however, 
the connection between intermediate results and strategic objectives (SOs) is less well defined. 
Specifically, the Results Framework contains three SOs:
1. Improve civic participation in the community by promoting healthy relationships. 
13
2. Increase at-risk youth engagement in productive activities.
3. Strengthen leadership and entrepreneurial skills of at-risk youth.
There is a clear logical connection between the activities and intermediate results under SO 1 and 
the promotion of healthy relationships, however, no activities under this SO specifically reference 
civic participation. The evaluators question the theory that healthier relationships result in improved 
civic engagement, especially in the absence of activities specific to civic participation. 
The project design builds on successes and lessons learned from EJEMPLAR, Heartland’s previous 
USAID-funded project that aimed to reduce youth violence. Information available at the time of 
project design indicated that the DHARTE project design was feasible; however, a key underlying 
assumption is that the project can engage parents of at-risk youth as well as local actors from the 
public and private sector. In practice, it has been challenging to engage parents in planned 
activities. (The reasons for limited parent engagement and potential recommendations for the latter 
half of the project are further detailed in the following sections.) Activities related to engaging the 
public and private sector to strengthen youth leadership and entrepreneurial skills are currently 
ramping up. Since these activities will take place in the latter half of the project, they were not 
included in this evaluation. 
B. To what extent are the project interventions consistent with the needs of key 
stakeholders? 
Throughout the KIIs and FGDs, several themes emerged on the needs of student participants their 
communities. The major themes were 1) a lack of involvement on behalf of adults in the lives of 
adolescents, often due to fragmented family structures, 2) a need to work with parents or guardians 
of at-risk youth in the program, 3) high levels of violence and insecurity in the project communities, 
and finally 4) a need for safe spaces in the community. Based on data collected through KIIS and 
FGDs, the interventions appear to be consistent with stakeholder needs, and no major changes have 
occurred during the first half of the project that would warrant alterations to the intervention. 
Fragmented Family Structures and Lack of Adult Involvement
When asked why youth may become involved in violence, the most common response was lack of 
parent engagement with children. Participants in the majority of student focus groups (n=8), parent 
focus groups (n=2), and KIIs (n=12) cited a need for parent involvement. Many informants believed 
that the cause of little parental involvement was fragmented family structures (n=137). Fragmented 
family structures were described primarily as single-parent homes, but references were also made to 
students who were living with grandparents or other guardians that were not their biological parents, 
as well as to describe families experiencing domestic violence, substance abuse issues, and other 
interfamilial violence. 
Additionally, a smaller amount of respondents (n=3) discussed a general ‘lack of things to do’ as a 
cause of youth involvement in violence; in relation to this subject, groups cited lack of parental 
involvement (n=13) and also economic concerns, such as lack of money and/or employment (n=8). 
High levels of Violence and Insecurity, Need for Safe Spaces
Almost all of the focus groups (n=9) identified the need for increased security in their neighborhoods; 
when asked about what students perceive in their environments that is insecure, students discussed 
violent events in the community. Some groups (n=5) mentioned shootings as a common occurrence; 
7 The number cited indicates the number of sessions (either FGD or KII) in which a theme was mentioned; it 
does not indicate the number of respondents. 
14
 
other events mentioned were kidnappings (n=2) and robbery (n=3) as well as other violent crimes. 
Individuals from the KIIs also mentioned security as a primary need (n=4) but focused more on 
family issues and substance abuse issues rather than violent crime. This may be due to the fact that 
the majority of individuals interviewed may not have resided in the communities (teachers and 
principals often do not live in the communities where they teach) and thus saw different sides of the 
issues. 
Insecure environments translated to a lack of ‘spaces’ that are safe and nonviolent for adolescents. 
Specifically, program staff in KIIs often stated (n=12) that a major contributing factor to violent 
behavior in youth was pervasive violence in their environments, including violence in the home, in 
their communities, and in the school, both in the form of crime and physical violence but also in 
terms of violence in relationships with others. 
When asked what things could be done to reduce violence, both FGD (n=6) and KII (n=4) 
participants spoke of the need for spaces – both physical, alluding to the lack of available safe 
space for gathering in communities, and otherwise – for adolescents to come together and be 
productive, whether that be through recreation, workshops, or other types of activities. 
Facilitators of both groups noted that the parents used the focus groups as a space to talk about 
their everyday struggles, mostly with their families (e.g., children and spouses). This may correspond 
to a need for a space for parents to discuss the difficulties and challenges they face in their daily 
lives rather than a need for a more regimented parent training curriculum. Furthermore, parents in 
both focus groups had difficulty identifying any topics that they were interested in learning about 
through the workshops. 
Need for a Gender-Informed Approach
Through a desk review of the curricula, the evaluators noted that some portions of the curricula 
explore gender differences and gender equality. Additionally, the quantitative data collected from 
the ERI indicates that, of the original sample, 70% of the students designated as high-risk were 
female, even though the total student population was comprised of about half male students and 
half female students. 
While the desk review indicated clear gender differences, qualitative feedback on the theme of 
gender was more mixed. In one focus group, some female student participants brought up the 
gender equality activity as one they liked, while the boys in the same group said they were not as 
interested in it. Conversely, some school staff that participated in KIIs (n=4) did not identify a 
difference in either how gender affected initial risk level or the ways in which students engaged with 
and benefitted from the DHARTE program. However, one staff member did mention s/he strongly 
believed that gender was a very important subject to speak about and that improvements could be 
made to the curricula to more deeply explore this issue. 
Although gender differences were not often referenced by stakeholders, the evaluators observed 
gender differences both in focus groups and in project activity observations. In activity observations, 
it was observed that regardless of the workshop type or gender of the facilitator, the majority of girls 
were more engaged and active in each workshop, while more boys tended to be less engaged, even 
sometimes not participating or focusing on the activity at all. Additionally, one round of focus groups 
was done in gender-specific groups: there was one group of only male students and one group of 
only female students. The female-only focus group produced a large amount of rich qualitative 
information: Students were eager to discuss issues and seemed to quickly demonstrate trust in the 
15
evaluators,8 talking about their personal lives and the ways in which they were impacted by the 
program much more explicitly compared to co-ed FGDs. Conversely, the male-only group that 
followed was very challenging; students were not engaged and spoke little about their personal 
experiences or even generally about the FGD topics. 
In sum, both quantitative and qualitative data indicates that gender does indeed play a role in the 
program and a gender-differential approach could be beneficial to stakeholders. 
Programmatic Capabilities to Address Stakeholder Needs
The DHARTE curricula directly speak to the needs identified above with modules on developing 
healthy relationships, recognizing violence in the lives of students, providing resources for 
understanding nonviolent options, and promoting safe decision-making. The RS and EC workshops 
also provide students with regular access to a nonviolent space in which they can freely express 
themselves. Student participants (n=3), as well as participants in KIIs (n=4), reported a strong 
relationship between program facilitators and students that allowed students to trust and confide in 
facilitators. 
While many of the identified environmental factors (shootings, robberies, lack of security, etc.) 
cannot be comprehensively addressed through the current scope of the program, the project 
activities related to engaging out-of-school youth (SO 2) as well as providing youth mentorship and 
entrepreneurial opportunities (SO 3) aim to address this need. It is also worth noting that the need 
for safe spaces may be especially relevant in the summer months when youth have more hours 
spent in the community rather than in school.
As mentioned above, a challenge to date is that parent participation in the ECF workshops has been 
low. This attendance issue also influenced the evaluation and made it difficult to recruit participants 
for the parent focus groups. While it has been difficult to engage parents, evaluation results indicate 
that the need to involve parents is clear, and therefore the DHARTE ECF intervention is consistent 
with stakeholder needs. As such, it is important for program staff to continue troubleshooting efforts 
to better engage adults in the second half of the project, including exploring the idea of a more 
support-oriented model for the workshops. Teachers can also serve as positive adult role models for 
students. Project staff stated that in Year 3 they plan to further engage teachers in programming to 
encourage them to reflect on how violence may be perpetuated in their interactions with students.
C. How appropriate and useful are the key performance indicators in assessing the 
project’s progress? 
Under SO 1, the SO-level outcome indicators9 for the school-based portion of programming are:
 % of youth in school-based program who develop at least two life skills according to baseline 
survey
 % decrease in participating youth who feel threatened or intimidated by fellow students or 
school staff
8 It should be noted that both evaluators were also female. While that may have contributed to the gender 
dynamics of the female-only and male-only FGDs, the gender patterns of engagement were also noted as a 
part of activity observations in which the evaluators were playing a passive role. 
9 A full monitoring and evaluation plan with additional project indicators and tracked progress towards each 
indicator can be found in the project PRIME dashboard, an internal Excel tool used by Heartland. A copy of this 
tool is available from the evaluation or project team upon request. 
16
 
 % of middle- and high-risk youth enrolled in after-school program who regularly attend school 
and complete the scholastic year. 
 % of families that feel their relationship with their child has improved
The evaluators deemed these indicators to be useful and appropriate based on a review of the 
project Results Framework and the intervention curricula. However, adjustments may be needed to 
the data collection tools used to track these indicators. The last outcome will be tracked through 
information collected during the ECF workshops. Since attendance has been low, it may be difficult 
to obtain representative data for this indicator, and if parent participation in ECF workshops does not 
increase, then the project team will need to identify a new strategy for collecting data for this 
indicator. Qualitative data suggests that students are learning about life skills, which supports the 
appropriateness of the first indicator. However, this indicator is currently measured with the ERI, and
the interviewed Heartland Alliance psychologist stated that the ERI may not be the most appropriate 
tool for measuring acquisition of life skills. Specifically, she stated that the ERI should be used as a 
risk assessment and screening tool and that a separate instrument(s) (e.g., the Ansell-Casey Life 
Skills Assessment, the Child and Youth Resilience Measure, the Problem Oriented Screening 
Instrument for Teenagers (POSIT) follow-up questionnaire) would better measure outcomes related 
to life skills, resiliency, etc. 
Although overall risk-level as determined by ERI scores is not listed as an indicator for SO 1, ERI 
scores have been used informally to inform programming and gauge project effectiveness. ERI data 
is collected via paper surveys which are then entered into an electronic database. The evaluation 
included a DQA that assessed the quality of paper surveys (i.e., source documents) as well as the 
electronic project database. The DQA revealed that paper surveys were clearly and completely filled 
out and stored appropriately. However, when a sub-sample of paper surveys from six schools (10 
surveys per school) from the first round of ERI data collection was checked against the electronic 
database, evaluators found that over 50% of surveys had at least one value erroneously entered in 
the electronic database (e.g., the source document list the response to a question as “A.” and the 
electronic database would list the response to the same questions as “D”). While the rate of data 
entry errors was higher than anticipated, it should be notes that these data entry errors may not 
ultimately affect the calculated risk level for a given student. For example, say that an ERI question 
has four response options, “A”, “B”, “C”, and “D”, and a student risk score increases if the select “C”
but not if they select “A”, “B”, or “D”. In that case, a data entry error in which “A” was selected on 
the paper form and “D” was recorded in the project database would not result in an overall change 
to student risk score. Factors that likely contributed to data entry errors include: lack of a data entry 
protocol, lack of data entry training, lack of data entry software, and lack of data quality audits to 
assess error rates as data was being entered. Given that that quality of source documents is high, 
the evaluators recommend that the paper surveys be re-entered using a data entry protocol and free 
data entry software (e.g., CSPro, Epi Info) by trained staff that undergo regular data audits. 
Additional analysis can then be done to assess the extent to which data entry errors influenced risk 
categorization. 
Indicators for SO 2 measure outcomes for community-based programming. While those indicators 
appear to be appropriate based on a review of the Results Framework, there was little evidence that 
evaluators could use to assess indicator usefulness because the dissolution of the partnership with 
PPAZ has necessitated changes to the way in which community-based activities will be
operationalized. Similarly, SO 3 indicators related to entrepreneurship and mentoring were deemed 
appropriate but could not be assessed for usability because those activities will occur in the coming 
months. 
The evaluators also assessed output-level indicators listed in the project PRIME dashboard, an 
internal Excel tool used by Heartland, and in quarterly reports. Output indicators under all SOs were 
17
found to be useful and appropriate, but the reporting format of the quarterly reports was somewhat 
difficult to interpret. The evaluators will work with the project team and USAID to modify the 
reporting form and ensure that all output indicators are reported clearly and accurately going 
forward. 
While the current key performance indicators are appropriate and useful, evaluators used qualitative 
data from the FGDs and KIIs to assess whether there were program results that were not captured 
by the existing indicators. In other words, the evaluators investigated whether additional key 
performance indicators were needed. Students were asked what, if anything, they learned from the 
workshops, and how, if at all, the workshops had changed their daily lives. The most common 
responses related to coexistence with others in school (in Spanish, convivencia). In the majority of 
student focus groups (n=6) respondents stated that the workshops helped them learn how to coexist 
better with other students, their friends, and others in the community. Other frequently mentioned 
topics included the consequences of substance abuse (n=4), decision-making (n=2), leadership 
(n=4), healthy relationships (n=7), and sexuality and dating (n=8). Many of these themes correspond 
to life skills highlighted in the DHARTE curricula, which supports the appropriateness of the SO 1 
indicator aimed at life skills that is listed above.10 Parents and school personnel also consistently 
mentioned that they had witnessed improvements in students’ social behavior. Some school 
personnel specifically mentioned that there was less fighting in their school. Heartland does not 
currently have access to quantitative data to verify this, but decreased rates of fighting in school 
could be used as a proxy indicator for measuring violent behavior outside of school. Therefore, it may 
be useful for the project team to ask schools for data on in-school altercations, if such data exists. 
D. Is the project on track to achieve the indicator targets according to schedule? If not, 
what have been the obstacles to achievement both in terms of factors that the project is able 
to influence and external factors beyond its control? 
Based on data from KIIs and FGDs, the project is on track to foster life skills, improve inter-student 
relationships, and increase student school attendance (see part G of this section for further 
discussion of qualitative data). However, ERI data indicates that risk levels have not decreased from 
Year 1 to Year 2 of the program. One reason for this may be that students under-reported in the first 
round of data collection, and that the second round of data collection provided a more accurate 
snapshot of risk levels for two reasons: 1) students were better able to identify risk factors after 
participating in a year of programming, and 2) students had more trust in the facilitators and 
program staff and therefore answered more truthfully. The Heartland Alliance psychologist 
confirmed that this phenomenon is not uncommon. Therefore, a more accurate assessment of 
program effectiveness (as measured by the ERI-informed indicators) can be completed once a third 
round of data is collected in June 2017. As mentioned above, it may also be appropriate to measure 
changes in life skills learned by using other validated tools in addition to the ERI, and track rates of 
in-school altercations, detention, and/or suspension as well. 
One potential obstacle to achieving SO 1 indicator targets is the low attendance of parents at ECF 
workshops. Further exploration of this topic was a key theme in all KIIs and in the parent FGDs. 
Parents generally expressed that it was difficult to come to the workshops because most of them 
worked and did not have much free time. Work was cited as a major barrier to parents attending 
workshops in both of the parent FGDs as well as by teachers (n=3) and administrators (n=3). The 
need to provide for the family, sometimes exacerbated by a family structure in which there is only 
10 Specifically, the RS workshops focus on five life skills: teamwork/cooperation, communication/healthy 
relationships, emotional control, building empathy, and decision making/problem solving.
18
 
one parent supporting the household, makes it very difficult for parents to be involved. The fact that 
lack of parental involvement was mentioned as a contributing factor to adolescent criminal activity
shows the compounded nature of this problem. Heartland contemplated providing parents with 
letters for their employers in order to try and help parents obtain permission to miss work. However, 
two parents mentioned that even if a letter facilitated permission from their employer that they 
would likely not get paid for days which they were absent, which is a prohibitive condition for most 
families. Because men are typically the breadwinners in two-parent households, the majority of 
participants in ECF workshops to date have been women. Therefore, further consideration should be 
given to a differential approach for mothers and fathers, and the ways in which this gender dynamic 
may affect students. 
Some parents (n=2) suggested that workshops should be later (e.g., after 6:00pm) or that 
workshops should be on weekends to get more participation. However, these suggestions were met 
with mixed feedback – one administrator mentioned that access to the school is difficult after hours. 
Additionally, security at night is a major risk and finding transportation back home can be difficult 
after nightfall as some taxis and other modes of transport will not travel to these areas after a 
certain time of day. Another teacher and parent mentioned that although weekends seemed like a 
good idea, many individuals would still not attend since it is their only rest time. One teacher even 
mentioned that recently another group had tried to organize an event for students and parents on a 
Saturday for a field trip and attendance was very low. 
One parent and several project staff alluded to another issue that may be affecting attendance: 
stigma.11 Evaluators witnessed the feeling of stigma firsthand during the FGDs. When parents who 
had attended an ECF workshop were asked to discuss what they felt they had learned, multiple 
parents responded that they liked the workshops but that they already communicate well with their 
children and do not have any issues interacting with their children. From these statements, it 
seemed parents felt that articulating lessons learned would imply they had difficulty with their 
children prior to attending the workshop and were not good parents. Another respondent mentioned 
that when parents are invited to the school for a meeting, as was the case with the ECF workshops, 
they fear a reprimand from school staff. 
These results indicate that the issues affecting parent participation are legitimate and complex. It 
may be beneficial to pilot different solutions (e.g., a weekend session, or a workshop for all parents 
rather than only parents for EC students) in one location to assess feasibility before implementing 
changes for all DHARTE schools. One such event is already planned by project staff in May 2017.
Indicator targets for SO 2 may need to be adjusted given the discontinued partnership with PPAZ. As 
mentioned above, HAI is in the process of submitting a project modification to USAID, and this 
modification will take into account any changes that need to be made to indicators or targets. 
11 The issue of stigma experienced by students was raised by project staff in KIIs and may warrant further 
investigation. Completion of the ERI and participation in RS classes are unlikely to result in stigma since these 
activities are for all students; however, EC students could potentially face stigma from other students as well 
as teachers. One staff member reported that some EC students were stigmatized by students and teachers in 
the first year of the project, despite mechanisms put in place by the DHARTE staff to reduce stigma (e.g., 
information packets for teachers, planned messaging with students). In Year 2, project staff talked to teachers 
individually in order to more carefully explain the DHARTE project, ensure that teachers messaged the project 
correctly to their classes, and prevent teachers from thinking negatively of EC students. The project staff have 
also tried to reduce stigma by letting EC students bring friends to workshops and avoiding the use of 
stigmatizing language (e.g., “at-risk”) when talking about the EC workshops. 
19
 
Similarly, there is no data to assess progress against S0 3 indicators because the bulk of activities 
are expected to occur during the summer of 2017.
E. How effectively has the project engaged stakeholders in project implementation? 
A key finding concerning stakeholder engagement is that various stakeholders, including student
FGDs (n=3), school personnel (n=4), and program staff (n=1), reported that the program staff has 
been able to build strong relationships with the student participants that allow the students to feel 
comfortable speaking to the facilitators about private issues, which suggests they likely also feel 
comfortable giving feedback on certain activities or topics. In the curricula, space is given for the 
facilitator to be flexible and make decisions regarding whether any particular activity needs to be 
altered given the mood and context of each group. The curricula also suggests alternatives to 
maintain consistency with the desired theme of each particular session. This allows facilitators to 
actively work with students in tailoring each session to the needs of the group.
Additionally, this year a program feedback survey was carried out shortly before the midterm 
evaluation in order to give student participants space to provide feedback anonymously. Based on 
the high response rate, these surveys were a successful vehicle for engaging student participants. 
Student feedback was generally positive: many students said nothing needed to be changed and 
students that did suggest changes listed ideas such as having activities outdoors and including 
music. The majority of students reported that workshop topics were interesting to them (85%), that 
they learned something new (89%), and that topics were useful to their lives (86%). Almost all 
students felt that facilitators were knowledgeable (91%) and respectful, professional, and kind 
(96%). Specifically, students had comments such as:
 “I like the workshops because I feel like I can be myself and no one criticizes me,”
 “[The facilitators] are very good and respectful and they teach the class well.”
 “The workshop is well structured […] it's the best class.”
When asked what kind of topics they would want to hear more about during FGDs, most groups 
(n=7) mentioned substance abuse as a primary area of interest, along with information about 
romantic relationships/sexuality (n=8), information about the risks of use of social media (n=2), and 
bullying (n=1). In the FGDs students were generally unable to articulate specific themes that they did 
not like or found to be not useful. Substance abuse and risky sexual behavior were identified as 
particularly important risk factors during the second application of the ERI (which took place in 
summer 2016), so these topics were emphasized in adjustments to the curricula for Year 2. 
Regarding the involvement of teachers and administrators, KIIs yielded mixed results. School staff 
did not have a uniform understanding of how they could or should be engaging in the program. Some 
facilitators (n=3) reported that they always invite teachers or that teachers are always welcome in 
the classrooms to observe the workshops; however, some school personnel (n=4) said that they 
never participate in or observe workshops. One administrator even said she was not allowed to 
observe the workshops. Conversely, three members of school personnel reported that they had 
observed workshops, with one teacher even stating that she incorporated material from the 
workshops in her own classes. Both school personnel and DHARTE project staff (n=3) noted that
there is a tension between the desire to involve teachers and the fear that teacher involvement in
workshops would cause students to be more guarded, thereby decreasing workshop effectiveness. 
Defining a standardized policy and clearly communicating this policy to all school and program staff 
is recommend to mitigate this issue. This policy should be complemented by teacher engagement 
outside the classroom, as discussed below. 
20
The DHARTE program engages teachers outside of the classroom through two workshops specifically 
for teachers and administrators from project schools in order to build teacher capacity for working 
with at-risk youth: 64 school personnel attended a May 2016 workshop, and an additional 52 
school personnel attended an October 2016 workshop. Although these workshops reached 116 
personnel, of the school personnel interviewed for this evaluation, none reported attending a 
workshop when asked about their involvement in the project. When a sub-section of interviewees 
were asked why they did not attend a DHARTE workshop, the most often cited reason was
scheduling conflicts. Because schools have both a morning and an afternoon cohort, it was reported 
that oftentimes teachers may have a second job at another school during the time they might 
otherwise be unoccupied. 
F. What are the key project achievements thus far that could be sustained beyond the 
life and the context of the project? Does the project have a strategy in place to sustain these 
elements? 
The project does have a sustainability plan in place, however, this plan relies on teachers and school 
staff being able to absorb and continue some portion of the DHARTE program activities. Based on 
the limited engagement of teachers thus far, including scheduling limitations, the feasibility of this 
plan should be reassessed to ensure feasibility. Project staff also identified the possibility of 
identifying public or private organizations as partners to continue all or part of the program activities 
when the DHARTE project ends. For example, the Secretaries of Prevention from the municipalities of 
Guadalupe and Escobeda have demonstrated interest in using the ERI to identify youth most at risk. 
The Integrated Attention Center for Adolescents, which currently provides youth services, has also 
expressed the need for an instrument will allow them to define the risk levels of their participants.
DHARTE staff reported that in Year 3 of the project more focus will be placed on sustainability and 
replication. For example, the content of the EC and RS workshops will be assessed and modified as 
necessary to be useable for teachers and other personnel that work with adolescents. One of the 
curricula creators also suggested that that project hold an art exhibit at the end of the 2016 school 
year in order to increase public awareness of the program. 
G. What emerging project practices or experiences have potential to become good 
practices or lessons learned at the end of project? 
Evidence-based In-School Intervention
The RS and EC workshop curricula were designed by an expert in art therapy and an expert in youth 
violence prevention. While additional quantitative data collection12 and analysis is needed, data from 
KIIs and FGDs indicates that the RS and EC workshops have positively impacted students. In most 
student FGDs (n=6), youth said they were able to use the things they learned in the workshops to 
their daily lives.13 Among the most useful topics were sexuality, consequences and risks of drug use, 
social media use, and dating. Students reported that they felt they got along better with their fellow 
students and even, in some schools, teachers. Some students reported that the workshops helped 
them with problems in their lives, such as problems in their families and with friends. Specifically for 
students in the EC workshops, some students reported that it was important and helpful to have a 
space where they could openly express themselves, which wasn’t always the available in school. All 
students agreed that they liked the workshops and they were good for them, the most common 
complaint being that they were too short. Specific quotes from the FGDs are as follows:
12 This includes a third wave of ERI data collection as well as data re-entry for the first two waves. 13 This finding is supported by quantitative data from the feedback surveys described in Part E of this section.
21
 
 “Before there were kids that did bad things but now with these talks they give us they are 
able to reflect and don’t do these things anymore.”
 “Mow we participate more in school….we aren’t afraid anymore…before we didn’t participate 
and we didn’t have confidence. Now, talking with [facilitator] they have taught us how to not 
be afraid, now we support each other.”
 “Now we understand that [bullying] feels bad and has consequences and can hurt people, 
and that they might hurt themselves, and that you will also feel bad.”
 “[We have learned] how to respect each other and get along.”
School personnel also reported positive changes in students, stating that students fought less14 and
had increased emotional control, confidence, and class participation. All the school personnel were 
supportive of the program continuing in their schools. Specifically, more than one respondent said 
that they thought these types of programs were needed in all schools, while another teacher said 
that the curricula should be integrated into all school subjects and throughout other school activities.
School personnel and DHARTE program staff also cited the ERI as an effective assessment tool to 
identify the students that needed extra attention through the EC workshops. 
Specific quotes from the KIIs are as follows:
 ‘What I have seen in students is that […] it has helped them have better relationships with 
each other, be more tolerant. Also I have noticed that their self-esteem has improved, the 
attitude they have when they relate with teachers, administrators, among themselves as 
students, there has been an improvement.”
 “I think [the students have learned] how to manage their emotions and how to manage 
certain things so that the situation does not take the student over.”
Given this feedback on effectiveness and the fact that the curricula were developed with a 
methodological and evidence-based approach, it is likely that the in-school intervention (i.e., the RS 
and EC workshops) should be shared widely with stakeholders as a good practice after project 
completion. Of note, in KIIs workshop facilitators reported that the curricula sometimes required 
small modifications (e.g., incorporating vocabulary commonly used by students, or adjusting art 
activities because some schools prohibit a material such as clay). These lessons learned should be 
incorporated into the curricula guides in order to ensure the best intervention model is available at 
the end of the project. 
Evidence-based Screening Tool for Assessing Youth Risk
The use of the ERI tool for assessing youth risk should also be highlighted as an emerging good 
project practice. The tool is based on a validated methodology, and is relatively easy to administer 
given that students can complete the survey themselves and protocols are in place for ERI 
implementation and scoring. Activities in the latter half of the project will contribute further 
information to help determine the ERI’s usefulness and effectiveness as a screening tool. 
Additionally, the tool will be used with a new population: out-of-school youth. 
V. Conclusions and Recommendations 
A. Conclusions 
This midterm evaluation assessed effectiveness, relevance, efficiency, and potential for sustainability and 
impact of the Heartland DHARTE project. The evaluation focused mainly on activities implemented 
14 As mentioned above, additional data on in-school altercations is needed to confirm this statement. 
22
 
under SO 1 because progress towards SO 2 was delayed due to operational issues with the local 
implementing partner and activities under SO 3 are scheduled largely in the latter half of the project. 
Based on evidence from KIIs and FGDs with school personnel, project staff, students, and parents, 
the activities implemented thus far under SO 1, and activities planned under SO 2 and SO 3, are 
highly relevant to meeting the identified needs in the project communities related to youth violence 
and engagement in criminal activity. The project activities also clearly and logically link to the 
expected intermediate results and SOs. 
Key performance indicators were found to be appropriate for assessing the project progress, 
although some program impacts may not be most accurately captured with current data collection 
tools (e.g., life skills). Results from analysis of the ERI data were not conclusive in determining 
whether the DHARTE program has reduced risks of violence to date. However, given that there were 
errors in data entry and that student responses in the first year may be unreliable due to lack of 
awareness or reluctance to truthfully report, no definitive conclusions can be made. The evaluators 
plan to conduct additional analysis after the June 2017 ERI data is collected and data from previous 
ERI waves is re-entered in order to further validate evaluation findings. 
The project is generally on schedule to achieve targets, with two exceptions. First, there has been 
some difficulty engaging the target number of parents in ECF workshops. Interviews with teachers 
and school administrators confirmed that it is difficult to engage parents and that even schools 
struggle with this issue. Recommendations to increase parent involvement are listed below and take 
into account feedback from parent FGDs. Second, the project has experienced issues implementing 
activities for out-of-school youth under SO 2 due to the dissolution of Heartland’s partnership with 
PPAZ. The project team is currently working on a modification to incorporate these activities into 
those run by Heartland in order to get SO 2 back on track. 
The project has been successful in engaging students and incorporating their feedback into project 
implementation through the use of anonymous student feedback surveys. The majority of students 
expressed satisfaction with the workshops, and the DHARTE team will continue to utilize student 
feedback when possible. Engagement of teachers has been less consistent: While some teachers 
sat in on workshops, others reported that they were relatively disconnected from the program. 
Results indicate that SO 1 activities implemented to date have contributed to positive changes in 
youth participants. School personnel and project staff reported that students appear to have greater 
self-awareness, emotional intelligence, and communication skills as a result of the DHARTE program. 
Positive feedback from the majority of student participants as measured by both qualitative (e.g., 
focus groups) and quantitative (e.g., feedback surveys) data further strengthened this conclusion. 
While additional tools may be needed to fully measure program impact, the development of the ERI 
as an evidence-based, objective tool to assess risk of violence in Latin American youth is a key 
achievement of the project to date. Although DQA results indicated that the electronic ERI database 
requires some correction, the quality of the paper source documents was high. Additionally, the 
project staff have a strong understanding of the need for strong data systems and were receptive to 
real-time feedback from the evaluation team. Therefore, evaluators believe the DHARTE project will 
ultimately result in a rich database that will document the effectiveness of the intervention model.
Multiple external stakeholders have also expressed interest in incorporating the ERI into their 
programs. 
High-quality workshop facilitation should also be seen as a strength that has contributed to the 
success of the project: The majority of school personnel, parents, and students were highly satisfied 
with quality of the DHARTE facilitators and the content of the workshops. Additionally, the workshops 
23
are based on comprehensive and standardized curricula designed by experts. While the exact effect
of the art therapy component cannot be separated from the violence prevention component, the 
violence prevention expert stated that the addition of art therapy was an innovative intervention that 
she had not encountered in other youth violence prevention programs. 
A sustainability plan is in place, but further work needs to be done to ensure these emerging good 
practices can be carried forward after the project ends. Both government and non-government 
stakeholders have expressed interest in using the ERI to identify at-risk youth, however, there is not 
a clear plan for ensuring that at least some portion of workshop activities can be maintained. The 
evaluation findings highlighted that while there are opportunities to engage teachers in program 
activities, additional investigation and planning will need to be done in Year 3 of the project in order 
to ensure that school personnel can feasibly implement all or part of the workshops. 
B. Recommendations 
Based on the midterm findings, the evaluators recommend the following actions be taken to 
strengthen the project going forward:
 Re-enter the paper ERI surveys into the electronic database (perhaps through hiring a 
temporary administrative assistant). Reduce the rate of data entry error through the 
development and strict implementation of data entry protocols, the use of free data entry 
software (e.g., Epi Info, CSPro), and application of regular data quality audits.
 Strengthen project monitoring through systematic use of surprise facilitator monitoring visits.
 Ask project schools to provide data each semester on student grades to assess the positive 
impacts the DHARTE might be having on academic performance. This analysis would also 
allow project staff to understand whether there are negative academic consequences to EC 
students missing class. Additional school data that would contribute to the assessment of 
program effectiveness includes data on the frequency of in-school psychical altercations (if 
available), as well as rates of student dropouts, detentions, and suspensions.
 Identify a measurement tool that better captures outcomes such as resilience, emotional 
awareness, and/or life skills to supplement the ERI. Potential tools include the Ansell-Casey 
Life Skills Assessment, the Child and Youth Resilience Measure, and the Problem Oriented 
Screening Instrument for Teenagers (POSIT) follow-up questionnaire.
 Engage a gender expert to assess workshop curricula content and project data and collect 
additional data, if needed, in order to better understand the impact of gender on students 
and parents and determine whether a differential approach is needed. This expert could 
also further investigate the issue of stigma for both parents and students. 
 Use evaluation data to develop and pilot strategies for increasing parent engagement such 
as holding workshops on the weekend or in the evening, inviting all parents to project
activities rather than only parents of at-risk youth, providing child care during workshops, 
hosting joint workshops for parents and students, and adjusting ECF content to be more 
parent-driven (e.g., a parent support group).
 Create a policy for teacher participation in RS and EC workshops and confirm that all project 
and school staff understand the scope of teacher involvement. 
 Hold additional meetings and working sessions with teachers and school administrators in 
order to further define a feasible sustainability plan.
 Operationalize a learning system so that feedback from facilitators can inform real-time 
updates and modifications to the curricula in order to ensure that the final program model is 
shaped by ongoing implementation.
 Develop a security plan for Heartland staff who will be conducting activities in communities 
for out-of-school youth. Provide training for all project staff on responding to security 
incidents (both in schools and out of schools). 
24
VI. Annex A: Matrix of Evaluation Questions and Methods 
Desk 
review
Analysis
of 
existing 
data
Activity 
observations
Focus 
groups
Key 
informant 
interview
1. To what extent is the project design 
logical and coherent? X X
2. To what extent are the project’s 
interventions consistent with the needs 
of key stakeholders? 
X X X
3. How appropriate and useful are the 
key performance indicators in 
assessing the project’s progress?
X X X X X
4. Is the project on track to achieve the 
indicator targets according to 
schedule? If not, what have been the 
obstacles to achievement both in terms 
of factors that the project is able to 
influence and external factors beyond 
its control? 
X X X X
5. How effectively has the project 
engaged stakeholders in project 
implementation? 
X X
6. What are the key project 
achievements thus far that could be 
sustained beyond the life and the 
context of the project? Does the project 
have a strategy in place to sustain 
these elements?
X X X X X
7. What emerging project practices or 
experiences are worth highlighting as 
holding potential to become good 
practices or lessons learned at the end 
of project?
X X X X X
 
25
VII. Annex B: DHARTE Project Results Framework 
Theory of Change: If children and youth from at-risk communities acquire positive life skills and are diverted from involvement in violence 
and crime, then violence and crime in targeted communities will be reduced.
Strategic Objective 1: Improve civic participation in the community by promoting 
healthy relationships
Strategic Objective 2: Increase at-risk 
youth engagement in productive 
activities
Strategic Objective 3: 
Strengthen leadership and 
entrepreneurial skills of at￾risk youth
Intermediate Result 
(IR) 1.1: Youth, 
families, and school 
personnel in 10 
schools participate in 
assessments to better 
define needs and 
design effective 
curricula.
IR 1.2: Youth within 10 
Monterrey schools develop life 
skills to protect them from 
participating in crime and 
violence. Youth identified as 
medium to high risk will be 
referred to an after-school 
program to receive specialized 
attention.
IR 1.3: Families actively 
participate in and promote 
healthier relationships with 
children to reduce risk of 
crime and violence 
involvement
IR 2.1: At-risk youth, who are neither in 
school or employed, reduce their 
probability to engage in criminal 
activities and violent behavior through 
vocational training and art-based 
interventions.
IR 3.1: At-risk youth 
develop leadership and 
entrepreneurship skills to 
form a leadership peer
network supported by local 
actors from the public and 
private sector
Activities Activities Activities
1.1.1:Design a tool 
that integrates SAVRY 
methods 
1.2.1: Develop a "Healthy 
Relationships" curricula using 
art, adapted both primary and 
secondary schools
1.3.1 Design the action 
plans for each 
youths/family in the after￾school program 
2.1.1: Assess the risk level of youth 
who are not in school or employed in 5 
communities with high indices of crime 
and/or violence.
3.1.1 Provide leadership 
and entrepreneurship 
training to at-risk youth and 
form sustainable 
leadership peer networks.
1.1.2 Assess de risk 
level of youth in 10 
target schools with the 
tool designed in 
activity 1.1.1
1.2.2 Implement the "Healthy 
Relationships" curricula for 
primary and secondary schools 
via weekly sessions
1.3.2 Adapt and implement 
modules from the "Healthy 
Relationships" curricula 
specifically for parents of 
after school program for 
youth at middle and high
risk. 
2.1.2: Organize arts-based events in 
target communities.
3.1.2 Organize a summer 
program to promote youth 
participation in private and 
public institutions.
1.1.3 Conduct an 
evaluation of the 
family profiles of 
participating medium 
to high risk youth in 
order to inform further 
activities
1.2.3 Design an after-school 
program for youth identified at 
middle risk and high risk. 
1.3.3 Provide monthly 
counseling session and two 
home visits per year to each 
interested family with a youth 
enrolled in schools programs 
to follow up on actions plans, 
provide coaching and 
determine next steps 
2.1.3: Implement and art-based 
curricula one afternoon per week at 
each neighborhood via its traveling 
caravan for youth aged 16-22 who are 
neither attending school nor employed.
3.1.3 Create a mentoring 
system for youth enrolled in 
leadership and 
entrepreneurship networks.
1.1.4: Conduct focus 
group session with key 
school staff in order to 
evaluate the working 
environment in the 
schools as well as the 
main risks faced by 
youth from the 
perspective of the 
workers in the school
1.2.4 Utilize assessment data to 
select and refer youth in to the 
after-school program. In 
conjunction with the schools 
and families, youth are officially 
enrolled in the program.
1.3.4 Organize an annual 
exhibition at the end of 
year to present results of 
the DHARTE program and 
invite families, key 
stakeholders, and 
communities.
2.1.4: Design a set of targeted 
vocational workshops for youth to 
complement the arts-based curricula.
3.1.4 Organize a public 
policy round table featuring 
at-risk youth proposals and 
presentations to promote 
positive participation 
opportunities for youth.
1.2.5 Implement after-school 
program for youth at middle risk 
and high risk with sessions held 
with session held 2 times per 
week for 2 hours per session.
1.3.5 Lead artistic 
activities engaging youth 
and their families as a form 
of art therapy.
2.1.5: Develop a Help Desk for target 
youth in 5 communities, facilitating 
their reintegration in educative and 
employees opportunities.
1.2.6 Implement a training of 
trainers program for staff in 6 
target schools in year 1 
covering violence and crime 
prevention at the school, 
appropriate disciplinary 
measures, how to asses risk 
and identity danger signs, and 
how to refer youth for 
appropriate services. 
2.1.6: Organize an annual exhibition at 
the end of each year to present results 
of the DHARTE program and invite 
families, key stakeholders, and 
communities.
3
1.2.7: Replicate the after-school 
program during Year 2 to 4 
additional schools
2.1.7: Train the youth who successfully 
completed at least 5 workshop units as 
local promotes called Líderes Urbanos 
(Urban Leaders) who serve as mentors 
to other youth participants and are 
spokespersons about their 
transformative process and the power 
of peacebuilding.
4
VIII. Annex C: Project Documents included in Desk Review 
 Proposal Documents
− All documents sent in relation to project proposal
− Original award agreement and attachments
 Design, Monitoring, Evaluation, and Learning (DMEL) documents
− Results framework
− M&E plan
− Theory of change
− M&E monitoring tools
 Prime dashboards for Y1 and Y2
 Indicator Monitoring Matrix
 Reports & other administrative docs
− Quarterly reports submitted to USAID
− Reports from PPAZ
− Field reports
− Monthly reports
 Country director monthly reports
− Yearly reports
− Activity schedules
− Work plans
 All protocols
− ERI application protocol
− ERI risk analysis protocol
− ECF instrument pilot protocol
− ECF instrument application and data management protocol
− Home visit protocol
 Blank versions of all questionnaires/tools used in project
− ECF instrument
− Program quality survey
− Population census for EC workshops
− Current version of the ERI
− Attendance sheets
− Instruments used by PPAZ
 All project data/databases
− Attendance data
− ERI data
− All consent documents
 Cooperative agreements/memos from partner organization
− Agreement with PPAZ
 Curricula for all workshops
− RS curriculum and attachments
− EC curriculum and attachments
− ECF curriculum and attachments
− Teacher workshop curriculum and attachments
− PPAZ curriculum and attachments
 Pictures
− Any existing pictures of project activities
IX. Annex D: Activity Observation Guide 
Name of observer: _________________ School: _____________ Date: _________ Workshop:_______
1. Does the workshop start on time? If no, what time did it start?
2. Do any students arrive late? If yes, how many?
3. Does the facilitator appear to have a thorough understanding of the workshop material? List 
specific examples to support your conclusion.
4. What materials, if any, are required for the workshop? Is the facilitator adequately prepared?
5. Does the facilitator appear to have control of the classroom throughout the duration of the 
workshop? List specific examples to support your conclusion.
6. Does the facilitator behave respectfully towards the students? List specific examples to 
support your conclusion.
7. Do students behave respectfully to the facilitator? List specific examples to support your 
conclusion.
8. Do students behave respectfully to one another? List specific examples to support your 
conclusion.
9. Do students fully participate in the workshop? Do they appear to be interested in the 
workshop material? List specific examples to support your conclusion.
10. How is gender manifested in the workshop? Are there differences between the behavior of 
boy students and girl student? Does the facilitator use gendered materials or language? 
Does the facilitator act the same towards boy and girl students? List specific examples to 
support your conclusion.
11. Please list any other observations you have about the workshop material, facilitator 
performance, student engagement, or general workshop dynamics. 
 
3
X. Annex E: Focus Group Guides 
Question for student participants – Focus Groups 
• Introduction 
o Hola, me llamo [nombre] y trabajo con Heartland Alliance International, parte de 
Alianza Heartland Mexico que es la organización a la cual pertenece el programa 
DHARTE. [Aquí deje que los otros miembros del personal se presentan]. Les voy a 
hacer unas preguntas mientras mis compañeras toman notas y graban esta sesión. 
Gracias por venir a ayudarnos y participar el día de hoy. Como los facilitadores del 
programa DHARTE les han dicho, hoy vamos a hablar sobre los talleres a los asisten 
como parte del programa DHARTE con la meta de encontrar información que nos 
puede ayudar mejorar y evaluar el programa. Recuerden que todo lo que dicen aquí 
hoy es confidencial, incluso su identidad y nombre. También quiero pedirles que 
ustedes respetan a la confidencialidad de los otros participantes y no repiten nada 
de que escuchan aquí hoy. Espero que todos se sientan cómodos participando hoy 
porque nos gustaría escuchar las opiniones de todos. Nos gustaría que solo un 
participante hable a la vez; si otro participante está hablando y quieres decir algo, 
favor de esperar a que terminen de hablar. Favor de hablar en voz alta para que 
todos pueden oír. Para confirmar, la participación de ustedes en esta sesión hoy es 
completamente voluntaria y tienen el derecho a no contestar cualquiera pregunta. 
¿Alguien tiene alguna pregunta antes de que empezamos?
o Para empezar, ¿cada quién puede decir su nombre y su tema favorito en la escuela?
• Warm up questions
o Antes de hablar del programa DHARTE específicamente, ¿me pueden decir que tipos 
de actividades les gusta hacer cuando no están en la escuela?
o ¿Qué imaginen que van a hacer ya cuando no estudian?
o ¿Qué te gusta acerca de la escuela/en su escuela? 
• DHARTE 
o ¿Qué hace el programa DHARTE en su escuela?
o ¿Cada cuándo participan en actividades que pertenecen al programa DHARTE?
o ¿Cómo te han afectado los talleres?
 ¿Cómo han cambiado sus vidas o te han ayudado en su vida los talleres?
o RS ONLY: ¿Que han aprendido en los talleres? 
o ¿Cómo piensan que el programa ha afectado el ambiente en su escuela?
o ¿Qué les gusta más acerca de los talleres de [nombre del taller aquí]?
 Si no lo mencionan, pregúnteles: ¿Cuáles son los temas más útiles para 
ustedes y porque?
o ¿Cuáles temas o actividades son menos útiles para ustedes y porque?
o ¿Qué opinan de la actividad del autorretrato? 
o RS ONLY ¿Qué opinan de la actividad del mural? 
o EC ONLY ¿Qué opinan de la actividad del termómetro? 
o ¿Cómo podría mejorar el programa?
4
 Si no lo mencionan, pregúnteles: ¿Sobre qué temas les gustaría aprender a 
través de los talleres y porque?
o ¿Alguna vez alguien se sintió incómodo al hablar de cierto tema durante un taller? 
En el caso de que sí, ¿cuál fue el tema y porque se sintió incomodo?
 En caso de que sí, ¿cómo manejaron sus sentimientos los facilitadores?
o ¿Qué opinan de los facilitadores de los talleres? ¿Hay algunas cosas que les 
gustaría que los facilitadores hicieran diferente?
o ¿Conocen de los otros talleres – que piensan u opinan de estos talleres?
o ¿Hablando específicamente de los talleres de Espacio Creativo, tienen amigos que
no asistan a los talleres de Espacio Creativo? En el caso de sí, ¿qué opinan estos 
amigos acerca de los talleres de Espacio Creativo?* 
• General Community Questions
o ¿Cuáles son los problemas o necesidades más grandes de las comunidades donde 
viven?
o Queremos aprender más acerca de la violencia y los jóvenes en la comunidad. 
¿Porque piensan que los jóvenes participan en violencia o delincuencia? 
o ¿Qué creen que se pueda hacer – como actividades, programas o estrategias - para 
que menos jóvenes se involucrarían en violencia? 
• Wrap-up question
o Muchísimas gracias por hablar conmigo hoy. ¿Hay algo más que les gustaría decirme 
acerca del programa DHARTE antes de que terminemos?
*solo se aplica durante grupos focales con estudiantes que asistan a los talleres de EC
Question for adult participants – Focus Groups 
• Introduction 
o Hola, me llamo [nombre] y trabajo con Heartland Alliance International, parte de 
Alianza Heartland Mexico, la organización a la cual pertenece el programa DHARTE. 
[Aquí deje que los otros miembros del personal se presentan]. Les voy a hacer unas 
preguntas mientras mis compañeras toman notas y graban esta sesión. Gracias por 
venir a ayudarnos y participar el día de hoy. Como los facilitadores del programa 
DHARTE les han dicho, hoy vamos a hablar de los talleres que ustedes asisten como 
parte del programa DHARTE con la meta de encontrar información que nos puede 
ayudar mejorar y evaluar el programa. Recuerden que todo lo que dicen aquí hoy es 
confidencial, incluso su identidad y nombre. También quiero pedirles que ustedes 
respetan a la confidencialidad de los otros participantes y no repiten nada de que 
escuchan aquí hoy. Espero que todos se sientan cómodos participando hoy porque 
nos gustaría escuchar las opiniones de todos. Recuerden también que todos son 
bienvenidos a expresar sus opiniones personales aunque sean diferentes, y que no 
hay respuesta correcta ni incorrecta. Nos gustaría que solo un participante hable a la 
vez; si otro participante está hablando y quieren decir algo, favor de esperar que 
terminen de hablar. Favor de hablar en voz alta para que todos pueden oír. Para 
confirmar, la participación de ustedes en esta sesión hoy es de manera voluntaria y 
tienen el derecho a no contestar a las preguntas. ¿Alguien tiene alguna pregunta
antes de que empezamos?
5
o Para empezar, cada quien puede decir su nombre y su pasatiempo favorito.
• General Community Questions
o Antes de hablar del programa DHARTE específicamente, ¿Cuáles son los problemas 
o necesidades más grandes de las comunidades donde viven?
o Queremos aprender más acerca de la violencia y los jóvenes en la comunidad. 
¿Porque piensan que los jóvenes participan en violencia o delincuencia? 
o ¿Qué creen que se pueda hacer para reducir el riesgo de que jóvenes se involucren 
en situaciones de violencia? 
• DHARTE 
o Muchas gracias. Ahora me gustaría hablar acerca del programa DHARTE. ¿Qué hace 
el programa DHARTE en su comunidad?
o Como saben, el programa DHARTE también realiza talleres para los jóvenes en la 
escuela. ¿Que han aprendido sus hijos a través de su participación en los talleres? 
o ¿Cuáles son algunos temas que les gustaría que se enseñaran a los estudiantes en 
los talleres?
o Si alguna vez han participado en una actividad del programa DHARTE, ¿Cada cuándo 
participan en estas actividades?
o En muchas de las escuelas donde realizamos el programa, la asistencia a los talleres 
de padres ha sido baja, y queremos pensar en cómo podemos involucrar a más 
padres. ¿Porque piensan que padres no asisten a estos talleres a veces?
 Si no lo mencionan, pregúnteles: ¿Cómo organización, que cosas podemos 
cambiar para que más padres asistan?
o ¿Sobre qué temas les gustaría aprender o hablar a través de estos talleres para 
padres?
o ¿Piensan que los talleres se tratan de temas relevantes a sus vidas? ¿Porque o 
porque no?
o Si han asistido a algún taller para padres, ¿qué le gustó más en los talleres?
o ¿Qué es lo que les gusta respecto a la forma en la que trabajan los facilitadores?
o ¿Hay algunas cosas que les gustaría los facilitadores hicieran diferente?
o ¿Qué han aprendido a través de las actividades del programa DHARTE?
• Wrap-up question
o Muchísimas gracias por hablar conmigo hoy. ¿Hay algo más que les gustaría decirme 
acerca del programa DHARTE antes de que terminemos?
 
6
XI. Annex F: Key Informant Interview Guides 
Question for Teachers – Individual Interviews 
• Introduction 
o Hola, me llamo [nombre] y trabajo con Heartland Alliance International, parte de 
Alianza Heartland Mexico que es la organización a que pertenece el programa 
DHARTE. [Aquí deje que los otros miembros del personal se presentan]. Le voy a 
hacer unas preguntas mientras mis compañeras toman notas y graban esta sesión. 
Gracias por su disponibilidad para hablar conmigo hoy. Como los facilitadores del 
programa DHARTE les han dicho, hoy vamos a hablar del programa DHARTE con la 
meta de encontrar información que nos puede ayudar mejorar y evaluar el programa. 
Recuerden que todo lo que dicen aquí hoy es confidencial, incluso su identidad y 
nombre. Para confirmar, la participación de usted en esta sesión hoy es de manera 
voluntaria y tiene el derecho a no contestar cualquiera pregunta. ¿Tiene alguna 
pregunta antes de que empezamos?
• Warm-up questions 
o ¿Cómo conoció al programa DHARTE?
o ¿Qué hace el programa DHARTE en su escuela?
o ¿Cada cuándo vienen a la escuela los facilitadores del programa DHARTE?
o ¿Cómo está involucrado en el programa DHARTE usted?
• Engagement questions 
o ¿Qué les gusta más acerca de los talleres del programa DHARTE?
o ¿Qué le gusta sobre la manera en la que trabajan las facilitadores?
o ¿Hay algunas cosas que les gustaría que los facilitadores hicieran diferente?
o ¿Qué han aprendido a través de las actividades del programa DHARTE?
o ¿Qué han aprendido los estudiantes a través de las actividades del programa 
DHARTE?
o ¿Cómo podría mejorar el programa?
o ¿Piensa que el programa afecta diferente a los chicos que a las chicas?
o Queremos aprender más acerca de la violencia y los jóvenes en la comunidad. 
¿Porque creen que los jóvenes participan en violencia o delincuencia? 
o ¿Piensa que el riesgo de estar involucrados en violencia es diferente para chicos y 
chicas?
o ¿Qué creen que se pueda hacer para reducir el riesgo de que jóvenes se involucren 
en situaciones de violencia?
o ¿Cómo ha afectado el programa DHARTE el riesgo de que los jóvenes se involucren 
en situaciones de violencia? 
o ¿Cuáles son los problemas o necesidades más grandes de esta comunidad?
o ¿Me puede contar acerca de las experiencias que ha tenido interactuando con los 
padres de sus estudiantes? ¿Cómo piensa que los padres deben involucrarse en el 
programa DHARTE? 
• Wrap-up questions 
o Muchísimas gracias por hablar conmigo hoy. ¿Hay algo más que les gustaría decirme 
acerca del programa DHARTE antes de que terminemos?
7
Question for Administrators - Individual Interviews 
• Introduction 
o Hola, me llamo [nombre] y trabajo con Heartland Alliance International, parte de 
Alianza Heartland Mexico que es la organización a que pertenece el programa 
DHARTE. [Aquí deje que los otros miembros del personal se presentan]. Le voy a 
hacer unas preguntas mientras mis compañeras toman notas y graban esta sesión. 
Gracias por su disponibilidad para hablar conmigo hoy. Como los facilitadores del 
programa DHARTE les han dicho, hoy vamos a hablar del programa DHARTE con la 
meta de encontrar información que nos puede ayudar mejorar y evaluar el programa. 
Recuerden que todo lo que dicen aquí hoy es confidencial, incluso su identidad y 
nombre. Para confirmar, la participación de usted en esta sesión hoy es de manera 
voluntaria y tiene el derecho a no contestar cualquiera pregunta. ¿Tiene alguna 
pregunta antes de que empezamos?
• Warm-up questions 
o ¿Cómo conoció al programa DHARTE?
o ¿Qué hace el programa DHARTE en su escuela?
o ¿Cada cuándo vienen a la escuela los facilitadores del programa DHARTE?
o ¿Cómo está involucrado en el programa DHARTE usted?
• Engagement questions 
o ¿Qué les gusta más acerca de los talleres del programa DHARTE?
o ¿Qué les gusta sobre la manera en la que trabajan las facilitadores?
o ¿Hay algunas cosas que les gustaría que los facilitadores hicieran diferente?
o ¿Qué han aprendido a través de las actividades del programa DHARTE?
o ¿Qué han aprendido los estudiantes a través de las actividades del programa 
DHARTE?
o ¿Cómo podría mejorar el programa?
o ¿Piensa que el programa afecta diferente a los chicos que a las chicas?
o Queremos aprender más acerca de la violencia y los jóvenes en la comunidad. 
¿Porque creen que los jóvenes participan en violencia o delincuencia? 
o ¿Piensa que el riesgo de estar involucrados en violencia es diferente para chicos y 
chicas?
o ¿Qué creen que se pueda hacer para reducir el riesgo de que jóvenes se involucren 
en situaciones de violencia?
o ¿Cómo ha afectado el programa DHARTE el riesgo de que los jóvenes se involucren 
en situaciones de violencia? 
o ¿Usted cree que hemos identificado los estudiantes correctos para participar en los 
talleres de Espacio Creativo? ¿Realmente hemos identificado los estudiantes que 
están en alto o medio riesgo para estar involucrados en violencia? 
o ¿Cuáles son los problemas o necesidades más grandes de esta comunidad?
o ¿Me puede contar acerca de las experiencias que ha tenido interactuando con los 
padres de sus estudiantes? ¿Cómo piensas que el programa DHARTE se debe 
involucrar los padres? 
• Wrap-up questions 
o Muchísimas gracias por hablar conmigo hoy. ¿Hay algo más que les gustaría decirme 
acerca del programa DHARTE antes de que terminemos?
8
Question for DHARTE Facilitators - Individual Interviews 
• Introduction 
o Hola, me llamo [nombre] y trabajo con Heartland Alliance International en Chicago. 
Le voy a hacer unas preguntas mientras mis compañeras toman notas y graban esta 
sesión. Gracias por su disponibilidad para hablar conmigo hoy. Hoy quiero hablar del 
programa DHARTE con la meta de encontrar información que nos puede ayudar 
mejorar y evaluar el programa. Recuerden que todo lo que dicen aquí hoy es 
confidencial, incluso su identidad y nombre. Para confirmar, la participación de usted 
en esta sesión hoy es de manera voluntaria y tiene el derecho a no contestar 
cualquiera pregunta. ¿Tiene alguna pregunta antes de que empezamos?
• Warm-up questions 
o ¿Cuál es su rol trabajando dentro de HAI el programa DHARTE?
o ¿Cuándo está haciendo su trabajo en el contexto del programa DHARTE, cuales 
experiencias del pasado le ayudan a apoyar su trabajo actual?
• Engagement questions –
o ¿Porque piensan que los jóvenes participan en violencia o delincuencia? 
o ¿Por qué razón o cómo es que los jóvenes se ven involucrados en situaciones de 
violencia? 
o ¿Qué son algunos factores y/o programas que pueden reducir el riesgo de jóvenes 
de estar involucrados en situaciones o comportamientos violentos? 
o ¿Cuáles son los problemas o necesidades más grandes de las comunidades donde 
trabaja el programa DHARTE?
o ¿Usted cree que hemos identificado los estudiantes correctos para participar en los 
talleres de Espacio Creativo? ¿Realmente hemos identificado los estudiantes que 
están en alto o medio riesgo para estar involucrados en violencia? 
o ¿Qué le gusta acerca del temario que usan en el programa?
o ¿Qué, si es que hay algo, piensa que se debe cambiar en el currículo?
o ¿Cuáles son los retos principales que encuentra al realizar su trabajo?
o ¿Cuáles son los éxitos más grandes que ha observado en su trabajo?
o ¿Qué, si es que hay algo, han aprendido los estudiantes a través de las actividades 
del programa DHARTE?
o ¿Cómo usted interactúa con los maestros y directores de las escuelas? 
o Entiendo que la asistencia a los talleres de ECF ha sido bajo. ¿Por qué razones cree 
que los padres no asisten a los talleres? ¿Cree que hay algunas razones/retos en 
específico?
o ¿Qué cambios cree podríamos hacer para que más padres asistan a los talleres? 
o ¿Cómo podría mejorar el programa?
• Wrap-up questions 
o Muchísimas gracias por hablar conmigo hoy. ¿Hay algo más que les gustaría decirme 
antes de que terminemos?
9