ARC TWC Evaluation Tanzania 
True Panacea, LLC i 
ARC TWC Evaluation:
Guyana and Tanzania
Comparative Report
Report Submitted 11 August 2010
True Panacea, LLC  
Kyung Endres, MPH
with
Pamela Rao, PhD
ARC TWC Evaluation Composite Report 
True Panacea, LLC  i
Acknowledgements
The final evaluation was commissioned by the American Red Cross to be conducted by True Panacea in
Guyana and Tanzania. This report could not have been possible without the assistance of the Guyana
and Tanzania Red Cross Societies. The researchers would like to thank the Red Cross Societies for their
participation in the evaluation of the Scaling‐Up Together We Can Project as key informants, and for
their logistical support. True Panacea is also thankful to the American Red Cross for the opportunity to
work with such an illustrious organization and for its valuable insights into the TWC project in both
countries. Most importantly, True Panacea would like to acknowledge and show its appreciation to the
many participants who helped shape the contents of this evaluation.
The evaluation was made possible by the generous support of the American people through the United
States Agency for International Development (USAID) under the terms of Cooperative Agreement No.
GPO‐A‐00‐04‐00005‐00. The contents are the responsibility of the Scaling Up Together We Can Program
and do not necessarily reflect the views of USAID or the United States Government.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  ii
TABLE OF CONTENTS
Acknowledgements........................................................................................................................................ i
TABLE OF CONTENTS..................................................................................................................................... ii
TABLES AND FIGURES................................................................................................................................... iii
ACRONYMS & ABBREVIATIONS..................................................................................................................... v
A. EXECUTIVE SUMMARY.............................................................................................................................. 7
B. INTRODUCTION......................................................................................................................................... 8
B.1. Together We Can History Before 2004.............................................................................................. 8
B.2. The Scaling‐Up Together We Can Project ......................................................................................... 8
B.2.a. SO 1:  Strengthen HIV‐prevention life skills of youth 10 to 24 years of age............................... 8
B.2.b. SO 2: Build capacity of National Societies to implement youth HIV‐prevention projects .......... 9
B.2.c. SO 3: Enhance community environment for adoption of safer sexual practices........................ 9
B.3. The American Red Cross and the National Societies in Guyana and Tanzania ................................. 9
C. EVALUATION DESCRIPTION..................................................................................................................... 10
C.1. Evaluation Objectives ...................................................................................................................... 10
C.2. Evaluation Methods......................................................................................................................... 10
C.2.a. Site Criteria & Selection............................................................................................................ 10
C.2.b. Respondent Criteria & Selection............................................................................................... 11
C.2.c. Data Collection ......................................................................................................................... 11
C.2.d. Data Analysis............................................................................................................................ 12
C.3. Evaluation Commentary.................................................................................................................. 12
D.  EVALUATION FINDINGS ......................................................................................................................... 13
D.1. Cross‐Cutting Issues for the Scaling‐Up Together We Can Project................................................. 13
D.1.a. Geographic Targeting .............................................................................................................. 13
D.1.b. Participant Targeting............................................................................................................... 13
D.1.c. Situational Environment........................................................................................................... 15
D.1.d. Volunteerism............................................................................................................................ 17
D.1.e. Monitoring and Evaluation ...................................................................................................... 18
D.2. HIV‐Prevention Education Activities Contributing to Measurable Changes................................... 19
D.2.a. Curriculum‐Based Interventions (CBI)...................................................................................... 19
D.2.b.  Follow‐Up Interventions (FUI) ................................................................................................. 22
D.2.c.  Adult‐Child Communication (ACC)........................................................................................... 24
D.3. HIV‐prevention Education Activities: Community Engagement ..................................................... 24
D.3.a. Town Halls and Community Councils....................................................................................... 24
D.3.b. Youth Clubs .............................................................................................................................. 26
D.3.c. Community‐Wide Events (CWE) ............................................................................................... 26
D.3.d. Mass Media.............................................................................................................................. 27
D.4. National Red Cross Society Capacity Building................................................................................. 28
D.4.a. Training.................................................................................................................................... 29
D.4.b. Mentoring ................................................................................................................................ 30
D.4.c. Develop Partnerships................................................................................................................ 31
D.4.d. Disseminate Best Practices ...................................................................................................... 32
E.  PROJECT IMPACT.................................................................................................................................... 33
E.1. Life Skills........................................................................................................................................... 33
E.2. HIV‐Prevention Knowledge.............................................................................................................. 35
E.3. Attitude Towards PLWHA ................................................................................................................ 36
E.4. Abstinence ....................................................................................................................................... 37
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  iii
E.5. Fidelity ............................................................................................................................................. 39
E.6. Condom Use .................................................................................................................................... 39
F.  OVERALL RECOMMENDATIONS ............................................................................................................. 42
F.1. Plan Collaboratively ......................................................................................................................... 42
F.2. Define Expectations for Target Populations and Engagement Strategies....................................... 43
F.3. Define and Prioritize Activities......................................................................................................... 44
F.4. Define Indicators and Link Measures to Use ................................................................................... 45
F.5. Document Progress.......................................................................................................................... 48
APPENDICES ................................................................................................................................................ 49
Appendix A: Evaluation Protocols .......................................................................................................... 49
Appendix B: Example of Pre‐ and Post‐Test Report ............................................................................... 61
Appendix C: Discussion Guides............................................................................................................... 65
Appendix D: Participatory Data Collection Methods............................................................................ 123
Appendix E: Evaluation Recommendations.......................................................................................... 125
TABLES AND FIGURES
Table 1: Field Interviews ............................................................................................................................. 11
Table 2: TWC Cost Per Youth Reached (FY07‐09) ....................................................................................... 14
Table 3: Number of Youth Reached through CBIs ...................................................................................... 20
Table 4: Percentage of YMs Responding Correctly to Select CBI Post‐Test Questions for Multiple Years. 20
Table 5: Percentage of YMs Responding Correctly to Select CBI and FUI Pre‐ and Post‐Test Composite of
Questions .................................................................................................................................................... 22
Table 6: Number of Youth Reached through FUIs ...................................................................................... 23
Table 7: Date Ranges for Pre‐ and Post‐Test Data...................................................................................... 33
Table 8: If my boyfriend asks me to have sex with him and I don't want to, I am confident I will not have
sex with him. (Female respondents only)................................................................................................... 34
Table 9: If my girlfriend wants me to have sex with her, I will say "Yes" because if I don't my friends will
say that I am just a small boy. (Male respondents only) ............................................................................ 34
Table 10: Can a person get AIDS because of witchcraft? ........................................................................... 35
Table 11: Comprehensive correct knowledge about AIDS (TWC) .............................................................. 35
Table 12: Specific HIV‐Prevention Knowledge............................................................................................ 36
Table 13: Accepting Attitudes Towards PLWHA ......................................................................................... 36
Table 14: If a member of your family became infected with the virus that caused AIDS, would you want it
to remain a secret? ..................................................................................................................................... 37
Table 15: A girl has the right to choose not to have sex with her boyfriend, even if he wants to............. 38
Table 16: By the time they are 16, all boys should have sex at least once................................................. 39
Table 17: If you're having sex, condoms are a good way to prevent pregnancy, HIV, and sexually
transmitted diseases................................................................................................................................... 40
Table 18: If I were to have sex, I could convince my partner to use a condom. (Female respondents only)
.................................................................................................................................................................... 40
Table 19: If I were to have sex, I could convince my partner to let me use a condom. (Male respondents
only) ............................................................................................................................................................ 41
Table 20: Illustrative TWC Results Framework ........................................................................................... 45
Table 21 Understanding the Operating Environment: Key Respondents................................................... 52
Table 22 Understanding the Operating Environment: Data Sought........................................................... 52
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  iv
Table 23 Understanding TWC Project Activities: Key Respondents ........................................................... 53
Table 24 Describing TWC Project Activities: Data Sought .......................................................................... 54
Table 25 Understanding NS Organizational Development: Key Respondents ........................................... 55
Table 26 Understanding NS Organizational Development: Data Sought ................................................... 55
Table 27 Illustrative Interview Schedule for One Site................................................................................. 57
Table 28 Evaluation Activities in Country ................................................................................................... 58
Figure 1: Community Mapping Diagram..................................................................................................... 42
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  v
ACRONYMS & ABBREVIATIONS
ABC Abstinence, Be Faithful, Consistent condom use
ACC Adult‐Child Communication
AI Appreciative Inquiry
AIDS Acquired Immune Deficiency Syndrome
ARC American Red Cross
CBI Curriculum‐Based Intervention
CWE Community‐Wide Event
FBO Faith‐Based Organization
FGD Focus Group Discussion
FM Field Manager
FUI Follow‐Up Intervention
FY Fiscal Year
GRCS Guyanese Red Cross Society
HRCS Haitian Red Cross
HRY High‐Risk Youth
HIV Human Immunodeficiency Virus
IDI In‐Depth Interview
IFRC International Federation of Red Cross & Red Crescent Societies
ISY In‐School Youth
M&E Monitoring & Evaluation
MOE Ministry of Education
MOH Ministry of Health
NGO Non‐Governmental Organization
n/a Not Applicable
NS National Society
OD Organizational Development
OSY Out‐of‐School Youth
PACA Participatory Appraisal for Community Action
PE Peer Educator
PEPFAR President’s Emergency Plan for AIDS Relief
PLWHA People Living With HIV/AIDS
PM Program Manager
PRA Participatory Rural Appraisal
RC Red Cross
SO Strategic Objective
TOT Training of Trainers
TRCS Tanzanian Red Cross Society
TWC Together We Can
UI Urban Institute
UNAIDS Joint United Nations Programme on HIV/AIDS
UNDP United Nations Development Programme
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  vi
UNICEF United Nations International Children’s Fund
U.S. United States
USAID United States Agency for International Development
VCT Voluntary Counseling and Testing
YM Youth Multiplier
YP Youth Participant
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 7
A. EXECUTIVE SUMMARY
The Scaling‐Up Together We Can (TWC) project was officially launched in February 2004, and is set to
close its doors in June of 2010. The American Red Cross (ARC) implemented activities in collaboration
with the Guyanese Red Cross Society (GRCS), Haitian Red Cross Society (HRCS), and Tanzanian Red Cross
Society (TRCS); however, this report covers only the Guyana and Tanzania projects. All of the TWC staff
and volunteers worked towards the common goal of reducing HIV incidence among youth 10 to 24 years
of age. The project had three strategic objectives (SO):
Strategic Objective 1: Strengthen HIV‐prevention life skills of youth 10 to 24 years of age
Strategic Objective 2: Build capacity of National Societies to implement youth HIV‐prevention projects
Strategic Objective 3: Enhance community environment for the adoption of safer sexual practices
The end‐of‐project evaluation involved four researchers collecting qualitative data using focus group
discussions and in‐depth interviews with National Society (NS) staff, youth, parents of youth, community
leaders, development partners, and other stakeholders. Key informants ranged from 10 years of age to
legal adults. Interviews were analyzed using a qualitative analysis tool, ATLAS.ti, in Washington, DC,
where the report writing also took place with contributions from four writers.
The evaluation investigated youth behavior change outcomes in addition to: 1) how well the project
responded to local needs by assessing its geographic and beneficiary targeting, as well as asking about
social and cultural influences on risky behaviors; 2) how well activities were implemented for each of the
beneficiary groups identified in the original project plans; and 3) the implementation of capacity building
efforts for the National Societies (NS) to perform their management duties. As there was no baseline
information, researchers on the ground asked evaluation participants if they experienced increases in
knowledge or changes in attitudes or behaviors as a result of participating in this project.
The TWC project was very dynamic. The NSs of both countries effectively transmitted messages using
rather different approaches. HIV‐prevention activities have a longer history with more development
funding in Tanzania than in Guyana. TRCS reached a many more people than did GRCS, but project
impacts were similar in both countries for those who received messages. Both countries had issues
targeting beneficiaries for activities other than those included in the TWC curriculum because they were
less well planned and attended to during the life of the activity. Because of the heavy focus on the
curriculum, other planned activities, e.g., mass media, youth groups, community‐wide events, town hall
meetings, community councils, and NS capacity building, received relatively less attention.
The overall recommendations include:
9 Collaboratively plan activities as intended using participatory community mapping;
9 Describe all participant targets in the same detail as for the Curriculum‐Based Intervention;
9 Define and prioritize activities in the same detail as for the Curriculum‐Based Intervention;
9 Link the monitoring and evaluation plan to project goals and objectives; and
9 Formalize and document project activities and management procedures in a ‘Scaling Up TWC’
project binder.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 8
B. INTRODUCTION
B.1. Together We Can History Before 2004
The Together We Can methodology was first developed and implemented in Jamaica in 1993, and ran to
1997 through a partnership between United States Agency for International Development (USAID), the
American Red Cross (ARC), and the Jamaican Red Cross. Since then it has been successfully replicated in
Africa, Central America, and throughout the Caribbean, including a limited roll‐out in Haiti and Guyana.
Together We Can (TWC) has been reviewed and occasionally modified over the years to incorporate
core elements that are known to have a positive impact on young peoples’ behavior. These include peer
learning, follow‐up contacts, participatory methods using two‐way communication, and a
comprehensive life‐skills approach. TWC has now been formally adopted by all 26 Caribbean Red Cross
Societies as the cornerstone of their ABC (Abstinence, Be faithful, Consistent condom use) HIV/AIDS
strategies, and is supported financially and technically by partners such as the Norwegian Agency for
Development Cooperation, UNICEF, and UNAIDS. TWC materials have been translated into five
languages, and in 2003 the International Federation of Red Cross & Red Crescent Societies (IFRC) printed
and distributed 150,000 TWC Activity Kits throughout the Caribbean.
B.2. The Scaling‐Up Together We Can Project
In Fiscal Year (FY) 2004, the ARC received a five‐year, three‐country PEPFAR (President’s Emergency Plan
for AIDS Relief) Track 1.0 grant from USAID to reduce incidence of HIV among youth 10 to 24 years old.
The project provided technical assistance to Guyanese, Haitian, and Tanzanian Red Cross (RC) National
Societies (NS) and was later extended through June of 2010. The project has exceeded targets, reaching
more than one million youth through interpersonal and participative approaches to relaying HIV‐
prevention messages, and many more through general diffusion “edutainment” events and mass media‐
based outreach. The project mobilizes youth and young adults to deliver HIV‐prevention messages, offer
life skills training, and provide education and support to youth to encourage them to reduce or eliminate
risky sexual behaviors. It also endeavors to change the environment in which youth are making decisions
about relationships and sexuality by building the capacity of parents and community stakeholders and
enhancing community resources to educate and support young people in making decisions that will
affect the rest of their lives. The project was to commence with a community mapping activity to
determine local resources and needs. It is guided by three strategic objectives (SO):
Strategic Objective 1: Strengthen HIV‐prevention life skills of youth 10 to 24 years of age
Strategic Objective 2: Build capacity of National Societies to implement youth HIV‐prevention projects
Strategic Objective 3: Enhance community environment for the adoption of safer sexual practices
B.2.a. SO 1:  Strengthen HIV‐prevention life skills of youth 10 to 24 years of age
Activities categorized under the first SO include a curriculum for youth that includes follow‐up visits,
“edutainment” events, youth clubs, and a mass media campaign. These interventions use peer‐to‐peer
outreach to transmit prevention messaging emphasizing abstinence (including secondary abstinence),
being faithful to one’s partner and reducing multiple partners (particularly overlapping or concurrent
multiple partners), and other healthy behaviors such as condom use, HIV testing, and accessing sexual
and reproductive health services.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 9
B.2.b. SO 2: Build capacity of National Societies to implement youth HIV‐prevention projects
Another critical strategy, improving the ability of NSs to manage and expand youth HIV‐prevention
programs, is accomplished through formal trainings, individual coaching, systems development, and the
dissemination of best practices. Capacity building of NSs includes organizational development training,
partnership building, and exchange workshops with other RC branches or RC Movement partners.
B.2.c. SO 3: Enhance community environment for adoption of safer sexual practices
In addition to working directly with youth, TWC creates an enabling community environment for youth
behavior change by soliciting the active involvement with TWC of youth gatekeepers such as parents,
teachers and religious leaders. Activities include holding town hall meetings with community
stakeholders and outreaching to new and existing community councils composed of host‐country
government officials, non‐governmental organization (NGO) staff, and other community leaders.
B.3. The American Red Cross and the National Societies in Guyana and Tanzania
ARC, in cooperation with host NSs in Haiti, Tanzania, and Guyana, has been implementing Together We
Can, a youth‐targeted HIV/AIDS peer education program since 2004. This program is currently funded by
PEPFAR’s Track 1.0, and has been extended through June 2010 with additional activities such as condom
distribution and implementation in new geographic areas in Haiti and Tanzania. As the program’s
primary recipient, ARC is responsible for providing funding and technical assistance to the program’s in‐
country implementers:  the Guyanese, Haitian, and Tanzanian Red Cross Societies.
Guyanese Red Cross Society (GRCS) has been supporting youth HIV‐prevention promotion activities
since 2000. GRCS prioritized HIV/AIDS as one of its central programs, focusing specifically on educating
youth and on supporting people living with or affected by HIV/AIDS. GRCS’s youth HIV program has been
supported in the recent past by UNICEF, UNAIDS, and UNDP. GRCS is an active partner of both the
central and local government Ministries of Health (MOH) and is considered a key partner in a country
where the presence of other community‐based NGOs and agencies is limited.
GRCS works in three regions, Regions 1, 4 and 9. Region 4 contains the largest urban area in the country,
but is the smallest in terms of territory. Region 4 is particularly important because it has the highest
incidence of AIDS. Region 1 is very similar to Region 9 in population density, that is, it is sparsely settled
yet has the third largest land area. Mobility is a challenge in Region 1, and as a result, it is not nearly as
well‐served by social programs as is Region 4.
Tanzanian Red Cross Society (TRCS) was founded in 1962. By the time the TWC program was initiated in
2004, there were 14 regional committees with 56 district committees or branches providing support to
more than 150 sub‐branch committees across the country. The total number of TRCS volunteers was
estimated at 7,000 active in first aid training, blood donor recruitment, disaster preparedness and
response, as well as providing support to MOH national immunization and health campaigns.
The current TRCS TWC project focuses on urban centers in the Shinyanga Region of Tanzania as well as
one urban site in Kigoma Region. TRCS began TWC implementation in Shinyanga only nine months
before the evaluation was conducted. It had previously been operating in Kigoma Region, but was
requested by USAID to move to Shinyanga due to the higher prevalence rates and lower levels of
knowledge of HIV‐related issues. Only one other international NGO was working in the region on youth‐
focused HIV‐prevention education.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 10
C. EVALUATION DESCRIPTION
This evaluation was conducted to assess the performance of the American Red Cross (ARC) in achieving
the program’s goals in Guyana and Tanzania as part of its close‐out process of the TWC program. Its
intent is to augment the TWC program’s internal monitoring and assessment of interventions that
promote self‐efficacy, abstinence, fidelity, condom use, and the uptake of reproductive health services
to reduce HIV transmission and unintended pregnancy in youth. Evaluators reviewed the mechanics of
the program and its effect on creating a community environment conducive to youth behavior change,
as well as its efforts to strengthen the NSs’ capacity to manage and expand their HIV‐prevention project
for youth.
Country evaluations were completed in both Guyana and Tanzania. This final comparative report looks
at what did and did not work in each country and draws generalizable conclusions across the two
countries regarding the best way to implement TWC projects in the future.
C.1. Evaluation Objectives
The evaluation was directed by three Strategic Objectives as agreed upon by the lead consultant and the
client, ARC:
SO1:   To evaluate the effectiveness and appropriateness of beneficiary and geographic targeting for
TWC activities within the environmental (including social and cultural) context.
SO2:   To evaluate the impact of TWC interventions in promoting abstinence, fidelity, condom use,
testing, and uptake of reproductive health services.
SO3:   To evaluate the efficacy of TWC’s efforts to build the capacity of each country’s Red Cross
National Society to implement youth HIV‐prevention projects and development projects in
general.
C.2. Evaluation Methods
The evaluation protocols are summarized below. See Appendix A: Evaluation Protocols for details.
C.2.a. Site Criteria & Selection
Local field offices were asked to select two sites, one urban and one rural, where the evaluation could
be conducted. They were also asked to select a trial site where the discussion guides could be pilot
tested. The sites were to be accessible within one day’s travel from their headquarters. Each site was to
have key informants who would be able to represent views from the perspective of all of the TWC
targeted populations. For the purposes of the evaluation, a “site” is defined as a geographic location
where program activities are centered. A site could be a school, a youth group venue, a religious center,
a neighborhood in an urban setting, or an entire small village.
In Guyana, the evaluation focused on GRCS sites in Regions 1 and 4. Unfortunately, Region 9 could not
be included in this evaluation. Sites from within Regions 1 and 4 were included and provided some
diversity in terms of target population, risk conditions and youth behavior. Most of the peer‐to‐peer
outreach in the areas occurs through schools which were the sites selected for the evaluation.
In Tanzania, the evaluation sites were similar highly‐populated towns on major trucking routes. TRCS
and ARC/Tanzania selected Ushirombo and Kahama Districts, along with Muhongolo town within
Kahama, as testing sites for the discussion guides. TRCS operates in schools, but makes a concerted
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 11
effort to reach youth through venues outside of schools. Interviews were held both with school children
and with participants who had either completed or were no longer enrolled in school.  
C.2.b. Respondent Criteria & Selection
In order to collect data relating to implementation and efficacy of TWC, all the major organizations and
beneficiaries in the program were included in the evaluation. Interviews were conducted with United
States Government donor representatives, ARC headquarters and field staff, NS staff, stakeholder
representatives, and project participants. Table 1: Field Interviews lists categories of respondents and
the number of focus group discussions (FGD) and in‐depth interviews (IDI) conducted with each type of
respondent. The number in parentheses indicates the total number of FGD interviewees.    
Table 1: Field Interviews
Each NS was asked to identify key
informants and arrange FGDs and
IDIs. The sampling frame was
designed to encompass the age
range of the participating youth,
both boys and girls, and to obtain
data from students as well as youth
reached in out‐of‐school settings.
It was originally planned that the
same number of interviews would
be conducted in each country, but
due to the short turnaround time to
organize the evaluation, logistical
constraints, and the GRCS’s lack of
prior experience with project
evaluations, the number of interviews in Guyana was smaller than expected.
C.2.c. Data Collection
The research team in the field consisted of the team leader, three research assistants, and in Tanzania, a
translator. The research assistants had good experience in research methods through other projects and
were able to work independently after two days of training, thus forming a strong team.
Secondary data. The ARC headquarters in Washington, DC, provided more than 70 documents relating
to the TWC program, ranging from planning documents to project reports. These included monitoring
and evaluation reports, descriptions of data collected in the field, logic models, and other project
materials. Researchers reviewed all this literature prior to developing the research protocol and
discussion guides, and returned to it throughout the data analysis process. In the final analysis pre‐ and
post‐ test intervention data, as gathered by project staff was looked at. A sample report from the Access
data management program is in Appendix B: Example of Pre‐ and Post‐Test Report.
Primary data. The team leader and a Washington, DC‐based anthropologist created a series of
discussion guides for collecting data. (See Appendix C: Discussion Guides for guides.) Since the type and
level of project participation differed greatly between, for example, community leaders and peer
educators, each discussion guide was tailored to the respondent’s role. Including the guides for
respondents representing organizations, 17 distinct guides were created. Based on the number of
interviews to be conducted, a period of approximately 30 days in each country was allocated for primary
Guyana Tanzania
Respondent Category FGD IDI FGD IDI
Youth Participants  ‐‐  5 3 (22) 11
Youth Multipliers 4 (23) 14 4 (30) 15
Peer Educators 3 (20) 15 2 (10) 10
Parents 1 (5) 7 2 (9) 12
Community Council n/a * 3 n/a * 4
Town Hall Participants 3 (11)  ‐‐  2 (13) n/a
Community Stakeholders  ‐‐  5 1 (5) 9
COMMUNITY TOTAL 11 (59) 47 14 (89) 61
National Stakeholders n/a 5 n/a  ‐‐
GRCS Staff 1 (6)  7 1(6) 9
ARC Staff n/a n/a n/a 2
NATIONAL TOTAL 1 (6) 12 1 (6) 11
COUNTRY TOTAL 12 (65) 59 15 (95) 72
* n/a = not applicable
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 12
data collection. There was no significant difference between the selected sites within each country in
terms of the actual number of interviews and groups.
The time allotted for IDIs was one hour, while the FGDs of 4‐6 people were expected to last 90 minutes.
All the interviews were recorded, and the audio files and notes retained for analysis. The privacy of
respondents was protected by creating a coding structure that allowed each interview to be logged and
personal information deleted. A discussion about the project goals and activities preceded each
interview and focus group so the participants could provide informed consent. Both male and female
respondents were comfortable with the researchers and responded to questions relatively openly and
freely.
C.2.d. Data Analysis
The data analysis process followed the original research plan approved by the ARC prior to fieldwork.
Data analysis was iterative, beginning in the field after each IDI or FGD. The researchers used a “rolling
debrief form” tailored to capture key learning, new questions, developing issues, and potential
implications. The field researchers would regularly discuss what they were learning so themes could be
fleshed out in subsequent discussions. By examining the data immediately after collection, researchers
were able to begin the process of refining questions and identifying emerging patterns and anomalies.
These preliminary analyses were used to develop substantial presentations to share early findings and
recommendations with NS management and field staff, ARC/Tanzania and ARC headquarters.
A four‐person DC‐based team conducted further analysis on the data using ATLAS.ti to code the
information and identify overarching themes for more in‐depth analysis. Each researcher took
responsibility for analyzing and writing portions of the evaluation. The team met and communicated
regularly to confirm and ensure consistency of findings.  
C.3. Evaluation Commentary
After the primary data collection was completed in Guyana, GRCS staff expressed concern that the
evaluation approach left important gaps in understanding, and that the data gathered did not present a
complete and accurate picture of the TWC project. They felt that the inclusion of Region 9 would have
led to far different conclusions.
TRCS and ARC/Tanzania were well situated to manage the evaluation. They were enthusiastic about the
evaluation, but would have preferred that it be conducted midway through the project so that any
recommendations could have been implemented. They also felt that only evaluating Shinyanga did not
provide a fair representation of the project since the bulk of the activities took place in Kigoma during
the first five years.
It was very helpful to meet with the entire NS TWC teams to discuss and clarify the preliminary findings
and receive the team’s impressions of the data collected and ideas for improving the evaluation. It was
also helpful that researchers who were not also data collectors analyzed the data, as they were able to
draw completely objective conclusions and bring balance to the report.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 13
D.  EVALUATION FINDINGS
This section compares the main findings of each country’s evaluation and offers recommendations on
cross‐cutting issues, activities contributing to measurable changes, community engagement
interventions and NS capacity building for future programming.
D.1. Cross‐Cutting Issues for the Scaling‐Up Together We Can Project
This section explores issues that are not activity specific, but which affect the entire project, namely,
geographic and beneficiary targeting strategies, the situational environment, volunteerism and
monitoring and evaluation (M&E).
D.1.a. Geographic Targeting
The catchment area covered by TWC was a function of population density, need, the number of
volunteers, and the transportation situation. Geographic reach was mainly constrained by the number
of Field Managers (FM) (referred to as Coaches in Tanzania) hired to work with volunteers and by the
transportation budget. Although project participants in both countries requested an enlarged catchment
area, planners had to prioritize where and how extensively they could work.
The evaluation did not find any standardized TWC approach to determining geographic targeting or
priority setting for site selection. Because the activities could have worked anywhere, TRCS chose their
first site, Kigoma Region, because of existing infrastructure i.e., National Society branch office presence
and strength and later moved to the higher HIV prevalence region of Shinyanga. In Kigoma Region they
chose to target the generalized population, while in Shinyanga they focused on high‐risk urban centers.
GRCS started working in Region 4 because they already had an office in Georgetown, but the criteria
selecting Regions 1 and 9 were not specified.
It is clear that activities must be organized differently in urban versus rural areas. Rural areas are more
difficult to access and have fewer resources such as youth groups, services for referrals and partner
organizations, for planned activities. Because of the small populations and inaccessibility of many rural
areas in Guyana, time‐consuming and costly travel was required to organize and reach small groups of
people. GRCS adapted outreach activities to meet the needs of the population while simultaneously
minimizing time and financial outlays. The sites evaluated were found to be appropriate locations for an
HIV‐prevention project as they were commercial transit hubs, and, with the exception of Georgetown,
were receiving messages from few other trusted sources.
Recommendations. When determining the appropriate TWC sites, as with all projects, it is advisable to
explicitly set priorities in a list of site selection criteria and clearly document why sites were selected.
All activities may not be feasible in all settings. When considering site selection, evidence of need should
certainly be high on the list. In addition, the NSs’ comparative advantage in implementing in a given site
would also be a strong argument for reduced start‐up times and spending.
D.1.b. Participant Targeting
The three NS TWC projects targeted 766,000, but reached over one million youth with interpersonal
communication‐based outreach. Table 2: TWC Cost Per Youth Reached (FY07‐09) – not including the
many more youth reached with general diffusion and mass media events ‐ shows that the average cost
per youth reached at US$ 9.00 for FY07‐FY09. Expenditures are significantly higher in Guyana where
transportation is a major cost factor.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 14
Table 2: TWC Cost Per Youth Reached (FY07‐09)
FY 07, 08 ,09 ARC GRCS HRCS TRCS TOTAL
3‐year budget $2,317,186 $491,028 $979,669 $1,355,848   $5,143,731
% of total 3‐year budget 45% 10% 19% 26% 100%
Youth reached with community outreach 556,052 32,418 262,888 260,746 556,052
% of total youth reached 100% 6% 47% 47% 100%
Cost/youth reached with peer‐to‐peer contact $4 $15 $4 $5   $9
[Source: ARC Annual reports; FY07, FY08 and FY09]
The primary beneficiary target group, youth between 10 and 24 years of age, was the most clearly
defined TWC cohort, and the project most definitely succeeded in reaching them. Youth were then
further stratified by gender, age group, in‐school youth (ISY), out‐of‐school youth (OSY), and high risk
youth (HRY). The term OSY may not accurately reflect this participant category, as some people
categorized as such were students who happened to participate in an activity outside of the school
setting. In Tanzania, HRY were youth who did not complete the entire curriculum.
Overall, Guyana had more difficulty engaging males, while Tanzania found that fewer females were
receiving direct messaging. In Tanzania, ARC led the charge to remedy inequities in peer‐to‐peer
outreach. They worked with TRCS to hold special events targeting girls and worked with PEs to reach
OSY. Messages delivered to youth through out‐of‐school activities were adapted by the PEs depending
on the time available and group dynamics. GRCS was less successful in reaching males and OSY through
direct interpersonal interactions. Staff expressed dismay over their attempted failures to reach more
OSY and were at a loss as to how to proceed. This would have been an excellent opportunity for
technical assistance from ARC.
Many adults in both countries expressed concerns that condom and reproductive health messages were
too mature for youth under 14 years of age. The two NSs were careful to exclude condom
demonstrations when public opinion dictated.  
Significant energy went into identifying, counting, documenting and assessing the youth who were
reached by TWC under SO1. The same level of effort was not seen for adult or NS participants whose
activities fell under SOs 2 and 3. It was unclear from the project planning documents exactly who was
being targeting by which activities and what measurements would indicate that these people had been
successfully engaged. Furthermore, adult beneficiary targeting was never clarified during the life of the
project. Part of this was due to the concentration of ARC and the NSs on delivering five separate youth‐
focused activities, and project administration and management consumed much staff time.
Interviewees asked for greater engagement by the project for adults and expressed a desire to expand
the age range to 35 years of age. The definition of ‘youth’ apparently varies among cultures. Everyone –
local leaders, teachers and parents – wanted to be trained on HIV prevention, and in some cases, to
transfer messages to their peers. Although parents requested HIV‐prevention information in Guyana
and Tanzania, they wanted it directly from the RC, not from their children. TRCS and ARC/Tanzania
responded to the demand for parent education by developing and rolling out a trusted adult‐youth
communications (ACC) curriculum, discussed in more detail below. The request by adults for a TWC‐like
project targeting adults is unrealistic under the current TWC format. It would require a great deal of
effort to develop a separate curriculum for adults who are not intended to be a part the peer‐to‐peer
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 15
outreach. On the other hand, this would be an appropriate point at which to partner with organizations
that can deliver messages to adults. Specific organizations should be identified during the community
mapping process.
That being said, adults are intended to support the NSs’ efforts by allowing access to their children and
offering opinions about the project for improvements. SO3 was devised on the assumption that adults
would already have basic HIV‐prevention knowledge and would immediately understand what the
project was attempting to accomplish. The first statement is somewhat true and the second was found
not to be true. The adults interviewed were pleased by the positive changes they saw in their children’s
everyday attitudes and behaviors, but did not necessarily know the details of what they were being
taught. Parents and teachers felt the youth were being taught something that they should know about
in order to adequately support their children’s learning. The local leaders in Guyana wanted more
information about the project, and in both locations asked for training on program management to
adequately support TWC.
ARC generally planned that NS TWC staff would receive capacity building to effectively manage TWC.
The approach varied greatly between Guyana, where there was no ARC presence, and Tanzania, where
ARC was imparting their expertise daily. Although the NSs were dealing with widely different situations,
both required capacity building at all administrative levels. It was assumed that the NSs would have
some level of in‐house skills to run programs, and ARC would only be required to train and mentor local
staff on the specifics of running the project. Instead of assuming, it would have been helpful to take the
first steps to assess with the NSs who needed assistance and in which technical areas, what resources
would be required, and the steps needed to be ready to run the program, and to agree on expected
results and a timeline by which results would be met.
Recommendations. Continue targeting youth with TWC, but consider that age‐appropriate information
should be explicitly outlined for PEs if future projects are to pursue youth through out‐of‐school
activities and children in each of the three age cohorts. Changes to the curriculum could be specified in
a Training of Trainers (TOT) manual for FMs to reduce additional reprinting costs for PE and participant
manuals. In support of TWC, adult targeting should be clearly stated whilst adaptations to the program
will be required depending on the situational environment and the outcome of the community mapping
process. These adults include parents, government administrators, referral service providers, leaders of
faith‐based organizations, FBOs, school officials and teachers. ARC could, for example, develop guidance
specifically for sessions targeting parents, not just of basic facts, using select TWC activities but also an
orientation to the program with actual materials used on display for viewing. The NS targets also need
to be clearly identified and agreed upon. Initial assessments of staff readiness for effectively handling
both programmatic and administrative duties would have gone a long way.  
D.1.c. Situational Environment
Some opportunities and threats posed by the operating environment were discussed in project
documents reviewed by the evaluators. They play a pivotal role in the uptake and adoption of health
promotion information. The physical, political, and socio‐cultural environments mentioned during the
evaluation are addressed below. FMs were very adept at making existing materials relevant to the local
social and cultural situation. In Tanzania, the project took this one step further and developed a
brochure and a magazine that talks about pertinent high risk situations. An opportunity neither NS
picked up on were recent Ministry of Education (MOE) mandates to include HIV education in schools.
The MOEs documented their need for teacher training to fulfill the directives. NSs could have used this
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 16
as an opportunity to reopen discussions with MOE officials to formally promote and further the TWC
curriculum in schools.  
One of the roadblocks to successful implementation relates to promoting condoms in societies where
religion dictates that birth control is unacceptable. The NSs managed this issue by trying to explain the
facts‐based methodology to persuade a change of opinion or, failing that, removing condom
demonstrations from presentations, but keeping it as an option for those who wanted the information.
The second major roadblock was adults not being comfortable with discussing sex and sexuality openly
and directly with younger children. Both societies felt that reproductive health should be broached after
children reach puberty. PEs in Guyana were sensitive to this fact and focused on good decision making,
self‐efficacy, and HIV‐prevention games for the youngest participants. In some instances, GRCS staff
used the local value system to persuade parents that youth needed more information in order to
abstain from sex and avoid pregnancy. Evaluation respondents expressed more concerned over
abstinence and unintended pregnancy than HIV transmission. So, taking advantage of local values was a
very clever TWC marketing strategy. There is also great concern over intergenerational and transactional
sex in both locations. This value was not used as effectively as it could in promoting the ABCs of HIV
prevention to communities.
Evaluation participants in both countries stated that there was significant peer pressure to have sex,
particularly for boys. TWC life skills training addressed the ability of youth to refuse sex which appeared
to be more effective in Tanzania than Guyana based on interviews with youth. Because youth in both
locales reported learning, appreciating and using the TWC decision‐making model the difference is most
likely due to cultural influences. Adults in rural and urban locations of Guyana discussed the inroads
popular culture, especially music and dance, had on encouraging sex while it was not brought up in
Tanzania by anyone.
The two cultures value fidelity more for women than men to the point that it is expected that men cheat
on their partners. Fidelity was more accepted in Guyana by interviewees as a life goal than in Tanzania.
It was more acceptable in both countries for men to purchase and have condoms than women, but this
attitude seemed to be changing, more so in Guyana. Social issues that arose in Guyana were sexual
abuse and inequitable sexual partnerships, whereas witchcraft was distinct to Tanzania. Myths
surrounding witchcraft were methodically and deliberately dispelled during TWC sessions with
Tanzanian youth but the Guyanese concerns were not addressed head‐on in the same way.
The political environments in Guyana and Tanzania were very different. Guyana had strong but
disengaged local leadership, while Tanzania had the polar opposite. The Tanzanian officials were very
interested in being paid for their involvement with TWC, whereas the motivating factor for Guyanese
officials was time well spent. TRCS actively promoted TWC to local leaders by keeping them informed of
their activities through conversations and submitting reports. GRCS did not capitalize on the capacity
present in their officials by engaging them mainly because leadership was already busy and did not
make TWC meetings a priority.
The undeveloped physical environment made everything just a little bit more difficult for the Guyanese
branches outside of Georgetown. Transportation, telecommunications and power were all prohibiting
factors to doing more. Rural sites in both countries may have no cell phone service and certainly no
power by which to run electronics. This affects the ability of NSs to organize activities with communities,
share documentation between sites, and develop and reproduce materials locally. FMs in Guyana rely
on public transportation to reach rural sites, which is costly and may not be completely reliable.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 17
Recommendations. During project planning, community mapping and implementation, deliberately take
into consideration the strengths and weaknesses of the operating environment. Use this information
to set realistic activities and targets. TWC may look very different in rural areas versus urban ones.  
D.1.d. Volunteerism
The RC is a volunteer‐based organization for which voluntary service is a fundamental principle. The
Urban Institute (UI) states that, “Volunteer management capacity is a function of two things. One is staff
support. The other is the adoption of relevant administrative practices necessary for the effective
management of volunteers.” [Source: Hager & Brudney, The Urban Institute. (2004) Volunteer
Management Practices and Retention of Volunteers.] Although the study was about United States (U.S.)
‐based charities, universal lessons can be learned for application to international settings. Good
management practices and valuing volunteers had the most significant positive influences on retention,
an issue that has been problematic for both NSs.
NSs recognize that without volunteers, TWC would be a much smaller and less successful program. The
results of the UI study positively correlates gratitude, including evidence of tangible achievements, to
higher retention. GRCS staff discussed the desire to hold recognition ceremonies for their volunteers to
show their appreciation of the Peer Educators (PE). Even though ARC gave NSs the opportunity to
increase their budgets, NS staff mentioned monetary constraints as the reason for few formal volunteer
acknowledgment activities. ARC and the NSs brainstormed how to motivate volunteers which to date
have included; giving allowances, a t‐shirt and a vest and loaning them bicycles in Tanzania; and in
Guyana they have provided snacks during PE activities and in Regions 1 and 9 FMs went to great lengths
to organize fun outings for the PEs and their friends and family members. PE evaluation participants
requested certificates as proof that they had been trained by the RC. In Guyana, rural volunteers
requested more social activities and snacks. Tanzanian volunteers felt that recognition of their services
should come in the form of cash payments and bicycle donations.  
Despite desires for more tangible rewards for services, youth volunteers were highly motivated. The
greatest motivating factor reported by evaluation respondents was that they were learning new skills,
seeing results, and saving lives. PEs in both countries said that the strength of personal character and
charisma of the FMs was a key motivator for them to join and stay. Their reasons for leaving are
generally not because they no longer want to be a part of the project. In fact, in Guyana PEs return to
the RC offices when they have free time to see how they can volunteer. They have competing interests
which means the way they are able to volunteer is more of an issue than whether or not they are
provided t‐shirts.  
The UI data show that long‐term retention is also influenced by volunteers recruiting other volunteers
and very negatively associated by a dependence on volunteers less than 24 years of age. Some GRCS PEs
were encouraged by friends to volunteer for TWC, but most volunteers, especially in Guyana, are under
the age of 24. The TWC methods for recruitment were slightly different in each country. Both NSs
screened PEs and provided well organized training to deliver Curriculum‐Based Intervention (CBI)
materials. TRCS involved local leaders and imposed strict criteria related to one’s potential to deliver the
Activity Kit, which ARC believes has served them well. GRCS was assisted by school teachers. A hybrid
recruitment strategy could be developed whereby current volunteers recommend people they know,
with the final selections made by the NSs and the local leaders. PEs were also expected to volunteer for
other TWC activities such as organizing and participating in Community‐Wide Events (CWE) and
maintaining communication with local leaders.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 18
Recruiting PEs is one example of how local leaders were asked to participate in TWC, in addition to
contributing ideas for project improvement. Branch committees were also expected to volunteer for the
advancement of NS objectives, including HIV prevention. Staff in NS Branches in Tanzania received some
limited training, but other than that, these branch volunteers were not actively managed by NSs who did
not apply the best practices already in use with PEs.
Recommendations: NSs are implementing many proven best practices for the recruitment and retention
of PEs but a formal recognition structure should be put in place. The management of adult volunteers
should be as deliberate as that used with PEs.
D.1.e. Monitoring and Evaluation
An enormous amount of effort was put into designing, testing, rolling out and adapting the M&E system.
In an effort to compare Track 1.0 projects, USAID mandated that they perform a centrally‐managed
baseline study and mid‐term evaluation. The baseline was never completed and the mid‐term
evaluation, while giving USAID a good idea of progress, was less helpful to individual county‐level
projects on specific recommendations for improvements or changes. The lack of baseline data was
ultimately problematic. Neither ARC nor the NSs knew what their starting points were so measuring
progress was somewhat circumspect. A well‐done baseline study could have also assisted ARC and the
NSs to develop meaningful outcome indicators to track. The absence of TWC evaluations deprived NSs
of the opportunity to step back and objectively look at how well the project was or was not meeting
objectives through the eyes of a third party.
Perhaps because ARC changed the system nearly annually, NSs never quite caught up or on. As both
NSs’ staff stated during interviews, the M&E system should be clear from the beginning of the project.
Having a defined M&E system from the beginning would have allowed it to be integrated into activities
rather than ending up as an afterthought for data collectors.
There was an M&E plan finalized in May of 2009 composed of two diagrams, a description of each form
and the actual forms to be filled out. There was no linkage made between the data collected and project
management or toward reaching intended results. In fact, according to the M&E plan, the only
expectation for the project is to train and educate people, and hold meetings and events. If this is the
measure of success, then the implementers should consider the project a resounding success. In project
documents and conversations with ARC staff, however, more lofty goals were espoused. For the
evaluation it was assumed, due to the nature of the project that TWC was hoping to impact abstinence,
fidelity, and condom use even though these ideas are not explicitly in the M&E plan.
ARC trained NS staff on how to use the forms which were reported to be onerous and difficult for
illiterate or semi‐literate people to fill out. Data collection was a part of training for FMs and PEs, but not
for Youth Multipliers (YM). PEs and YMs randomly filled in CBI rosters. Some PEs, YMs and FMs reported
knowing of YMs who made names up. An appreciation of the importance of good data was not
systematically instilled in volunteers. In addition to collecting names and personal information, each
form had spaces where volunteers and coaches could write down their reflections on the activity which
proved to be the burdensome task. This qualitative information was to inform supervisors of areas
needing follow‐up and improvements and to document progress. Even though the forms were not used
as intended, YMs spoke with PEs who spoke with FMs to resolve problems. The downside to this
strategy is that commentary that could be useful to others has been lost. This section could be simplified
to three questions what worked, what did not work, and what is recommended.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 19
In an effort to look at end point outcomes, ARC implemented a pre‐  and post‐test for YMs who
participated in CBIs and Follow‐Up Interventions (FUI). This test covered HIV‐prevention knowledge,
attitudes towards People Living with HIV/AIDS (PLWHA), and changes in cultural values regarding the
right to refuse sex and insist on condom use. TRCS attempted a short pre‐ and post‐test at CWEs which
they found unwieldy and unusable. Other than the TWC Activity Kit, no other SOs have outcome data to
show progress. An annual or bi‐annual information collection exercise could be conducted to obtain a
better‐rounded picture of overall TWC achievements, not just accomplishments of the CBI and FUIs. The
regular review of the entire SO1 would inform decision makers of other areas needing improvement
such as CWEs and mass media. If this approach was used, the pre‐  and post‐test could become
redundant and is discussed more below.   
By project end, both NSs were using targeting data and the results from the pre‐  and post‐tests to
inform CBI programming decisions. It would have been helpful to look at the quality of the interventions
for each SO, such as the degree to which activities were youth‐, adult‐  or NS‐initiated and ‐directed,
access to and utilization of referral services, and level of community engagement. Countless lists exist
that could help in the development of more useful indicators and suggestions are included below.
Recommendations: Each NS should come up with their own M&E plan and include all fo the requisite
components. When developing the M&E framework, consider key decisions that will need to be made to
determine whether activities are being implemented as intended. Develop indicators and systems
which track outcomes to help with decision making, not just output from the start of the project.
Simplify the data collection forms. Train staff and volunteers on M&E basics and explicitly tie data
collected to individual and project achievements and decision making. Complement or replace the pre‐ 
and post‐test with a more comprehensive evaluation tool to measure outcomes resulting from all
interventions. This is further described in F.4. Define Indicators and Link Measures to Use below. (See
Appendix D: Participatory Data Collection Methods for ideas.)
D.2. HIV‐Prevention Education Activities Contributing to Measurable Changes
TWC targeted youth individually with direct peer‐to‐peer interactions through CBIs, FUIs and the ACC
curriculum. For each of these three activities, evaluators inquired about beneficiary targeting strategies,
message delivery, activity monitoring, volunteer supervision and sustainability.
D.2.a. Curriculum‐Based Interventions (CBI)
The curriculum was the part of the project most developed and focused upon by ARC and the NSs. It was
well received by most stakeholders, especially volunteers and participants, in both countries. Many of
the stories, games, and activities were singled out for praise by evaluation participants. PEs in Guyana
noted that the content was most appropriate for youth between the ages of 15 to 19 years, a much
narrower range than the targeted 10 to 24 years of age. In particular, the parts of the curriculum that
included depictions of reproductive anatomy and discussions of condoms were problematic in both
countries. Parents of youth in the younger age groups felt they were too young to be exposed to those
topics and speaking about sex would encourage experimentation. On the other hand, for youth aged 19
to 24, some of the TWC Activity Kit games and activities were considered juvenile by the Guyanese
participants. FBO objected to the presentations about condoms on religious grounds. These
circumstances are unlikely to change any time soon, and were taken into consideration by PEs who
presented the curriculum without condom demonstrations.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 20
Table 3: Number of Youth Reached through CBIs
During the six years of implementation, TWC reached nearly
a half‐million youth through CBI peer‐to‐peer outreach in
the two countries. Although the evaluation found very little
difference in terms of self‐reported HIV‐related outcomes
between the highly stylized Tanzanian curriculum and the
relatively unmodified Guyana one, TRCS reached a great
many more youth through CBI activities as presented in
Table 3: Number of Youth Reached through CBIs.  
Multi‐year efforts were put into adapting the CBI curriculum to the local environment in Tanzania by
ARC. GRCS chose to remain largely faithful to the original IFRC curriculum. A greater fraction of YMs
tested in Guyana than Tanzania were found to have comprehensive knowledge about AIDS per the TWC
project’s definition – responding correctly to all five HIV knowledge‐related questions. But a far larger
percentage of Tanzanian YMs held accepting attitudes towards PLWHA. The results of the other
composite indicators showed similar ability levels to negotiate abstinence and condom use between
groups of YMs over time, as illustrated in Table 4: Percentage of YMs Responding Correctly to Select CBI
Post‐Test Questions for Multiple Years.  
Table 4: Percentage of YMs Responding Correctly to Select CBI Post‐Test Questions for Multiple Years
Guyana Tanzania
Data Period
Select Composite Indicators
10/07
to
09/08
10/07
to 9/09
10/07
to
05/10
06/09
to
09/09
05/09
to
03/10
05/09
to
04/10
Number of post‐tests 607 749 1018 317 1770 2402
Comprehensive correct knowledge about AIDS 43% 55% 57% 11% 48% 48%
Accepting attitudes towards PLHIV 34% 33% 35% 61% 62% 63%
Ability to negotiate abstinence 59% 59% 64% 64% 67% 65%
Ability to negotiate condom use 86% 86% 87% 92% 90% 91%
[Source: GRCS and TRCS Pre‐ and Post‐Test CBI Databases]
Although over 80% of people responded accurately to nearly all of the individual questions about HIV
transmission, the low comprehensive score is likely due to individuals missing one or more correct
answers from the group of five applicable questions. The large difference of stigma towards PLHIV
between Guyanese and Tanzanian test takers is probably related to the lack of personal exposure in
Guyana to PLWHA and not the curriculum. It is important to note that the absolute increases in people
with comprehensive knowledge and decreases in stigma towards PLWHA were greater in Guyana (85%
and 95%, respectively) than in Tanzania (78% and 62%, respectively). These data would indicate that the
extensive changes to the CBI curriculum in Tanzania may not have had the expected results for more
efficacious information transfers.  
Tanzanian PEs were older and worked more independently in indentifying, contacting and educating
YMs. TRCS PEs were given a quota of one school per month, and there was a more concerted effort to
target youth through out‐of‐school activities in market places, workplaces and gathering places for
youth. In the two Guyana evaluation sites, the FMs led, identified, and organized every TWC outreach
session to YMs and YM to Youth Participant (YP) outreach was stressed much less than in Tanzania.
FY Guyana Tanzania TOTAL
FY04 300 0 300
FY05 6,529 111,135 117,664
FY06 7,881 96,991 104,872
FY07 5,447 95,986 101,433
FY08 6,393 81,434 87,827
FY09 3,688 47,894 51,582
LOA 30,238 433,440 463,678
[Source: ARC APRs]
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 21
In short, the success of the CBIs, as measured by TWC, is very influenced by effective targeting. CBI
activities were constrained by such factors as the number of volunteers the NS is equipped to manage,
the numbers of youth available to participate, and as described in the section on Geographic Targeting,
by challenges of geography and transportation. The last two issues are very applicable in Guyana. GRCS
trained 298 staff and volunteers under SO1 over the life of the activity and TRCS trained 935. Although
younger Tanzanian YMs more vehemently reported having a harder time passing messages on to YPs,
staff and PEs trained by TRCS reached an average of 463.6 people each through the CBI peer‐to‐peer
cascade, whereas each person trained by GRCS reached approximately 101.5 YMs and YPs. The number
of people reached is most likely considerably higher than what was reported, since YMs did not keep
good records of who they spoke with, and YPs transferred messages as well, especially in Tanzania.  
One of the reasons that monitoring YM activity fell short may have been that concept of “Take Home
Assignments” is not a very appealing one. It might be appropriate for school children, but what really
motivated youth to pass on messages was that they felt they were saving lives. A better way to frame
the activities might be something such as “Spread the Word.” An additional PE responsibility could be to
record the number of people YMs spoke with in an additional column on their forms, instead of having
separate YM rosters. Given the context of low literacy levels in both countries, oral data collection
occurs quite frequently already. PEs already ask YMs to describe the impediments they encountered in
spreading information and could record this as well in bullet points for the group of YMs.
The CBI targeted individuals and institutions. TWC used three main institutions to reach youth: schools,
FBOs, and youth centers. Schools were the most successful due to the high degree of central level
controls and structure at the grassroots level. Both NSs used the formal hierarchical administrative
structures of the MOE to gain entry into schools. This proved effective in persuading school principles to
allow the RC regular access to students in Tanzania and rural Guyana. In Georgetown, where the RC is
competing with numerous other organizations for public schools’ time and attention, entrée was more
labor intensive, while private schools turned out to be more flexible in incorporating TWC into their
schedules.
FBOs and youth clubs were venues by which the project planned to reach OSY. Although many FBOs
have a rigid hierarchy, such as the Catholic Church with arch‐bishops, bishops and priests, it was not
used in the same way as the MOE’s structure. Youth clubs are relatively unorganized where they do exist
so difficult to identify and engage. ARC and the NSs identified constraints in accessing both of these
organizations. Neither type of organization is explicitly set up to engage youth as they are in schools.
Even though identifying groups and FBOs was not as straight forward as finding schools, TRCS staff were
able to work with a church choir, a madersa (Islamic school) and a football team in Shinyanga. TWC staff
maintained that gaining entry was the larger problem with religious organizations who opposed any
conversation related to birth control. Youth clubs were difficult to locate and often times disinterested
when found. Instead, GRCS and TRCS used organized sporting events to engage OSY.
In addition to the importance of targeting for CBI activities, another consideration is the use and
promotion of referrals. As it was operationalized, it was more a directory of services than a referral
system. The need for referrals should reflect the goals of the project. Certainly Voluntary Counseling and
Testing (VCT), condoms, and reproductive health should be a part of this system. Both countries had
resource lists that were created by the NSs and presented to YMs by the PEs in the form of posters. TRCS
was more methodical on keeping the list updated through the participation of YMs and YPs. Youth were
not systematically referred for services in either country. Although Tanzania authorities reported a
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 22
greater uptake of VCT services and felt that TWC contributed to the increase in demand for services, it
was not possible to verify the linkage.
Recommendations. The content and delivery mechanisms for HIV‐prevention messages are liked and
effective. Additional efforts do not necessarily need to go into adapting the curriculum as CBI testing
data does not support great increases in knowledge or skills because of the changes. It may be helpful to
have a TOT manual for facilitators that contains more specific directions for M&E, age‐appropriate
messages, and referral resources, which can be determined during the community mapping process.
Promote spreading the word instead of giving homework assignments. Referrals require more thought
on involving the service providers. It would be advisable to provide YMs with referral forms that are
signed upon receipt of services and returned to the NS staff for tracking.
NSs should continue to strategize on effective targeting. A more methodical approach, such as the one
used with MOE, for gaining entrée with FBOs might be more effective for obtaining buy‐in from religious
leaders. Another option could be to develop a written presentation specifically targeting religious
leaders that FMs and volunteers would be trained to deliver. Some people did manage to connect with
youth through FBOs in Tanzania. It would be appropriate to share these strategies among the staff and
volunteers. But in the final analysis, if the FBO gatekeepers continue to decline to allow TWC to be
presented to the youth in their charge, recruitment efforts should be redirected towards more
amenable entry points.
D.2.b.  Follow‐Up Interventions (FUI)
The FUI Toolkit developed by ARC and finalized in 2009, provided a step‐by‐step explanation of how to
conduct an FUI. The first step is to gather information to determine which topics would be covered in
the upcoming FUI. The document then outlines a syllabus to present key messages and facts, to discuss
managing peer pressure, to allow for a question and answer session and to reinforce the use of youth‐
friendly services. Portions of the guide are to be used by PEs as they see fit, based upon the analysis of
data assembled by NS staff and PEs.  
Implementation of FUIs began in FY07 in Tanzania and FY08 in Guyana. They may or may not have
occurred with YMs who had received the full curriculum. YMs and YPs interviewed did not differentiate
FUI sessions from the CBI session, so the evaluation could not pick up nuances on the value of its
content. YMs were able to state whether or not RC had returned after the original sessions. Participants
liked this opportunity to refresh their information, ask questions, and learn more from PEs and
requested more return visits. They felt PEs did a good job transferring appropriate knowledge. YMs
stated that they learned new information through the FUIs, but when pressed for details, they spoke in
general terms about HIV prevention. PEs mentioned FUIs in Guyana when probed and to a slightly
greater extent in Tanzania. They did not talk about following the FUI Toolkit but explained FUIs as an
opportunity for YMs to ask and get answers to their questions.  
Table 5: Percentage of YMs Responding Correctly to Select CBI and FUI Pre‐  and Post‐Test Composite of
Questions
Guyana CBI 10/07 – 05/10 Data FUI 06/09 – 05/10 Data
Select Composite Indicators Pre‐Test Post‐Test Change Pre‐Test Post‐Test Change
Number of tests 1068 1018   693 638  
Comprehensive correct knowledge about AIDS 26% 46% 84% 41% 56% 37%
Accepting attitudes towards PLHIV 18% 35% 94% 28% 36% 29%
Ability to negotiate abstinence 57% 64% 12% 57% 63% 11%
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 23
Ability to negotiate condom use 79% 87% 10% 89% 89% 0%
Tanzania CBI 05/09 – 04/10 Data FUI 06/09 – 05/10 Data
Select Composite Indicators Pre‐Test Post‐Test Change Pre‐Test Post‐Test Change
Number of tests 2483 2402   1732 1689  
Comprehensive correct knowledge about AIDS 27% 48% 78% 40% 59% 48%
Accepting attitudes towards PLHIV 39% 63% 62% 56% 66% 18%
Ability to negotiate abstinence 55% 65% 18% 63% 70% 11%
Ability to negotiate condom use 82% 91% 11% 87% 92% 6%
[Source: GRCS and TRCS CBI and FUI Pre‐ and Post‐Test Databases]
The Toolkit did offer guidance and a semblance of structure to the FUIs but the influencing factor of the
activity seems to be related to effective targeting. Table 5: Percentage of YMs Responding Correctly to
Select CBI and FUI Pre‐ and Post‐Test Composite of Questions shows that much of the post‐test data
were nearly identical after FUIs and CBIs in Guyana. Gains in knowledge from pre‐ to post‐test were not
as huge after the FUI as after the entire CBI so some information had been retained. More YMs in
Tanzania responded correctly to FUI post‐tests than to CBI post‐test, if even slightly. These observations
may be of no consequence as YMs attending the FUIs did not necessarily sit in on the CBI. Where
numbers do not change significantly the question for TWC program managers is to determine if
information exchanged through an FUI format is just as effective as through the more labor intensive
and time consuming CBI. If this is the case this would have very positive implications for reaching OSY
and school children with little time allotted to RC where a shorter presentation would elicit the same
results.
TRCS put more effort into methodically reaching out to the same youth who had participated in the CBI
whereas the YMs in Guyana were not as diligently targeted and may not have taken part in the original
CBI. So it is understandable that FUI‐related gains would be more pronounced in Tanzania. These data
would indicate that FUIs are more valuable for YMs who have already had exposure to interpersonal
information from other sources. As with CBIs, it was relatively easy to locate and recontact youth in
schools if the FUIs were conducted within the same school year. Finding youth who had been reached
through out‐of‐school CBIs proved very time consuming for TRCS PEs. It was decided in Tanzania to
focus FUIs on ISY, which may be a better strategy as the younger the YM age cohort, the lower the
baseline knowledge and information retention was shown to be from the CBI pre‐test to the FUI post‐
test.
Table 6: Number of Youth Reached through FUIs
Once again, by focusing on highly populated areas, TRCS
was able to reach a greater number of youth with FUIs, as
illustrated in Table 6: Number of Youth Reached through
FUIs. The percentage of YMs reached through FUIs in
Guyana and Tanzania was 7% and 5%, respectively, of the
number of youth reached through the CBI in FYs 07‐09. In
FY09 when FUIs were fully up and running a greater
proportion of all youth reached through CBIs participated in FUIs, 18% and 8%, respectively for Guyana
and Tanzania. FUIs did not include reporting on YPs, so the actual number of youth reached is likely
underreported.
Recommendations. Stay the course of using FUIs to reinforce knowledge of YMs who had participated in
CBIs.
FY Guyana Tanzania TOTAL
FY07 0 1,921 1,921
FY08 419 5,997 6,416
FY09 669 4,064 4,733
LOA 1,088 11,982 13,070
[Source: ARC Annual Performance Reports]
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 24
D.2.c.  Adult‐Child Communication (ACC)
GRCS staff reported having educational sessions with YMs’ parents, but the ACC curriculum is unique to
Tanzania. In its first year of operations, FY09, TRCS reached 30 trusted adults and 30 youth. Although the
curriculum, as originally planned, did not end up working with parents of YMs, it was very warmly
received. The content was found by participants to be appropriate, well presented, and helped parents
to communicate better with their children. The substance of this activity does not need modification.
Once again the issue that needs more consideration is participant targeting.
Recommendations. Continue with this activity but develop a clear targeting strategy. Determine
whether the YMs or the parents will be used as NSs’ point of entry, or both. Once the entry point has
been established, work with NS staff to market the activity and recruit participants. This could be
accomplished though community leaders such as the community‐based advisory board or religious
leaders and/ or a concerted effort to engage YMs in the recruitment process of their trusted adult.
D.3. HIV‐prevention Education Activities: Community Engagement
The TWC methodology integrates several approaches for communicating with, reaching out to, and
drawing upon the community as a full partner in the process. Town hall meetings and community
councils are used to garner and engage community support. Youth clubs, community‐wide events, and
mass media are means for delivering HIV awareness messages to those not reached by the one‐on‐one
interventions. These TWC components are distinguished from those in the previous section in that it is
not possible to measure their individual or even group impact, but they contribute to the success of the
overall program by creating a welcoming environment for TWC messages. The evaluation asked relevant
project participants and NS staff about their perception of the goals of each activity, the degree to which
they had been carried out and their level of sustainability after the RC no longer sponsors TWC.
D.3.a. Town Halls and Community Councils
Many TWC stakeholders were unclear on the distinction between Town Halls and Community Councils.
Although their purposes were ostensibly different (town halls were to be community information
events; community councils were to act as advisory bodies), that distinction was lost on staff and
participants and in practice. This lack of clarity extends to the TWC documentation for community
engagement activities, e.g., “Town Hall and Community Council Operational Plan for Haiti June‐
December 2006,” which does not disambiguate the two types of meetings. It seems to distinguish
between them temporally – the first meeting in any given site is a town hall, subsequent meetings are
community councils. However, a note within this document defines them in functional terms,
introductory versus activity planning.
1 This section will discuss the evaluation findings with respect to
both types of meetings together, but provide separate recommendations.
The Town Hall meetings were intended to be a marketing tool for the NSs to present and garner
community support for their programs with the general public, and obtain permission to conduct the
TWC project in the community. However, the concept of the “town hall” was unclear in both countries,
and frequently conflated with the Community Council. Evaluation participants reported either not being
sure of the purpose of the town hall, or not even being aware that any such meetings took place,
                                                           
1 “N.B. In some cases a community council meeting may occur immediately after a town hall meeting (that is introductory and
activity planning topics occur during same meeting…).”
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 25
probably due to the uncertainty surrounding the term. They were simply termed by beneficiaries as
community meetings or RC meetings.
In Guyana, attendees found the meetings to be useful for learning about the program and the content of
TWC. However, the bad experiences of most attendees with poorly run Parent‐Teacher Association and
other community meetings led them to be wary of getting involved, and attendance soon fell off despite
the efforts of the TWC staff to promote them. Respondents also noted that townspeople might be
reluctant to attend meetings where they might look ignorant in front of their peers. In Tanzania,
stakeholders recalled a sustainability meeting, which was favorably received. Instead of relying on town
hall meetings to garner community support, TRCS staff and volunteers were able to do so through the
existing community government structure, which, while valuable for gaining community entrée,
neglected many other stakeholders.
GRCS and TRCS both referred to the existing community governing structure as their Community
Council. Between two and 24 councils were reported as ‘operational’ each year, which, practically
speaking, translated into government leaders being informed of activities. It turned out to be very
difficult for the NSs to establish community councils to provide ongoing advisory support. The concept
of a supportive leadership is an important one, but its functionality was constrained by the local
operating environments. Reporting progress to local leaders turned out to be a workable solution,
especially in light of Tanzania’s “sitting fee” system, for holding meetings with government officials,
which were avoided by getting on the agenda of already‐scheduled meetings to present progress
reports.
Recommendations. The actual form either type of meeting will take will vary depending on the local
situation, e.g., community interest, school system and government structure. Distinguishing more
clearly between the two types of community engagement events would make it easier for TWC staff
and volunteers to promote them to potential attendees, as well as simplify record keeping. Town halls
should be formalized in the TWC documentation as the venue for reaching out to and informing the
community, especially stakeholders not otherwise involved with the program. Community councils
should be advisory bodies that take an active role in planning and decision‐making. This distinction
should be clearly spelled out in TWC documentation, and assumed for the recommendations provided
next.
Town Halls. Every community reached by TWC should have one entrée meeting with gatekeepers to
request permission to operate in the area and periodic meetings where new information is presented,
progress is documented, and questions and concerns are addressed. The public should be motivated to
attend by offering education, information, or other incentives that directly benefit participants. These
incentives should be selected by assessing local needs during the initial set up and community mapping
process. As noted in the Tanzania report, these meetings do not necessarily have to be called “town
halls,” nor do they need to follow that format. The important point is to ensure that the community is
informed, engaged, and invested in the TWC process.
Community Councils. Rather than relying on local government officials and other community leaders, the
community‐based advisory board could be made up of RC branch volunteers, with ARC providing
technical assistance, and coordinated by a TWC staff member. Local government should be kept
informed but should not be solely relied on for regular advice. In Tanzania, the assigned NS coordinator
might be the branch development manager; in Guyana, this might be handled by a qualified volunteer,
e.g., Peace Corps, or it may be necessary to hire someone for the role. Board activities might include
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 26
holding the town hall meetings, and members might attend TWC staff meetings and should participate
in the community mapping process. This will require that ARC invest in developing the advisory board’s
capacity and provide clear guidance.
D.3.b. Youth Clubs
Youth clubs are not an activity per se, but rather a means of accessing youth through non‐school based
activities, especially OSY. The plan was to target existing youth clubs. [Source: TWC FY09‐10 Workplan].
Previously organizing youth clubs proved to be fairly labor intensive and required a high degree of
oversight and involvement by RC field staff. As reported by evaluation participants, lack of clarity in the
TWC documentation on how to go about setting up a club, establish its purpose, and monitor and
evaluate its effectiveness hindered implementation of this component of the project.  
GRCS was significantly more successful in accessing youth via youth clubs, mainly in the urban Region 9,
where such youth venues already existed. Youth clubs are not plentiful in rural settings, and they are not
always identified with a “bricks‐and‐mortar” location. They may be informal and lack official leadership.
They most likely do not meet regularly, and may only exist for a limited amount of time. In short, they
have limited utility as recruiting sites over the lifetime of the CBI. On the other hand, PE youth groups
have played an important role in rural Guyana as a kind of support group for volunteers, by providing
them with a setting in which to gather outside of school to work on TWC planning and preparation, do
homework, or just congregate socially.
Recommendations.  TWC should establish guidelines for working with youth groups that make clear the
target population and the process for engaging them, as with any other group such as church groups of
school children. Youth groups should be identified in the initial stages of establishing TWC in a region
during community mapping and targeted for gaining access to youth in the same way as any other
organization. As appropriate, other locations or contexts in which youth gather could be identified by
YMs and YPs and added to the referral list as part of the TWC take home assignments. The concept
could be recast as “youth groups” rather than clubs in order to avoid the connotation of a specific venue
like a club house.
D.3.c. Community‐Wide Events (CWE)
The purpose of and approach to community‐wide events changed considerably over the course of the
project based on direction received from USAID. Initially, CWEs were conceptualized as small‐scale (less
than 500 reached) versus large‐scale (over 500) community mobilization events, and there was an
expectation that TWC staff and volunteers would be instrumental in organizing or staging them. Over
time, it became clear that “mobilization” was not an accurate description of the events and that the
depth and level of interaction in the intervention was more important than size, so the guidance was
amended accordingly. The differentiation was now between interpersonal and general diffusion events,
and any given CWE could be both at once. This distinction completely changed the character of the
concept. CWEs now needed to be tracked in two separate ways, by the number of individual
conversations of three or more minutes, and as general diffusion events by number of attendees who
only engaged in uni‐directional communication with limited participation of recipients. The complexity
of the tracking system is evidenced by the TWC document, “Guidance on Community‐Wide Events
(CWE) Terminology,” which includes three pages of instruction with 15 examples.  
The result was considerable confusion among TWC evaluation participants regarding recordkeeping and
reporting for CWEs. Since any given CWE could include both types of events the number of attendees
way usually not known, and, the FMs would often just guess on the paperwork. The reported number of
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 27
CWE participants varied widely from year to year, ranging from 4,500 to nearly 75,000 youth. For the
purposes of reporting targets to USAID, only small‐scale CMEs and later interpersonal CWEs were
counted towards the strategic indicator of youth reached.
Both NSs participated in CWEs in conjunction with already‐planned events such as health fairs, festivals,
vocational camps, and sporting events, i.e., they were “piggy‐backed” on existing events of other
organizations. The “piggy‐backing” strategy used by NSs is a good way to conserve resources and
volunteers’ time. One popular activity in Guyana was to facilitate discussions in the form of a quiz show
and the PEs in Region 1 used the opportunity to raise funds to purchase items for the program.
Community leaders in Guyana were very impressed with the PE‐run booths and RC is now expected by
the community to participate in large gatherings. In Tanzania, only one CWE was reported by evaluation
participants for the nine‐month period covered by this evaluation in Shinyanga. Relocating and starting
work in a new region superseded organizing CWEs. Implementing CWEs was constrained in both
countries by issues of geography and resources, that is, the existence of events in the local area and the
time available to volunteers to participate in them.  
Recommendations. Community‐based advisory board members and YMs could be charged with
identifying and alerting NS and PEs of upcoming events, and with promoting TWC’s upcoming
participation to their constituents. General diffusion CWEs could also be used for recruiting branch
members and PEs and for fundraising. For example, PEs could have a booth with signup sheets for youth
and groups that would like more information, or hold drawings for prizes to be announced at a town hall
meeting to encourage attendance.  
While USAID’s purpose in creating the distinction between interpersonal and general diffusion CWEs is
appreciated, the purpose of the general diffusion CWE needs to be clarified in terms of TWC. Activities
involving direct one‐to‐one communication make more sense if incorporated into the CBI or FUI peer‐to‐
peer outreach. Regardless, ARC needs to clarify exactly what is to be gained from three‐minute
conversations and then communicate that to the TWC volunteers.  Monitoring success of the events will
be easier if the rationale for conducting the events is well established.
D.3.d. Mass Media
The use of mass media to educate youth missed by all the other outreach efforts is a commendable
idea, but may not be the best use of scarce TWC resources. The idea is to use radio, television,
magazines, etc., as marketing tools to bring NS programs to the public’s attention and to spread the
TWC message to a wider audience. The difficulty with this approach is that it requires a level of
expertise, resources, time, and commitment that may not exist in the NSs or ARC in these countries.
Although the amount of funding devoted to this effort is minimal (less than 10% of the budget), if not
done well, it could actually detract from the strength of the project. Mass media HIV‐prevention
campaigns are already being implemented by other organizations in both Tanzania and Guyana, and
TWC’s effort may well be unnecessary duplication of effort.
Evaluation participants reported “hearing messages,” but usually could not say whether they were
produced by TWC or some other organization. A few noted that while they had heard the messages,
they did not really apply them to their own situations until they heard the TWC message directly. This
speaks well for the value of the flagship TWC approach – one‐to‐one education – as more likely to affect
attitudes, and eventually behavior.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 28
Recommendations. The Tanzania and Guyana National Societies and the ARC should limit time and
resources spent on mass media, if at all, to partnering with other organizations or government to
conduct mass media campaigns. This would most likely be an activity for TWC management rather than
the volunteers and the approach will be different in each country.
D.4. National Red Cross Society Capacity Building
The discussion on ARC and NS capacity to manage development projects should not be confused with
their management of activities within their core set of competencies. This evaluation only considers the
capacity building that occurred between ARC and the NSs to manage development activities, as stated in
the project documents. It should be noted however that the NSs’ ability to manage staff and its human
resources policies sets the stage for any future capacity building exercises. GRCS staff, for example, had
good skill sets but weak leadership, resulting in them not reaching their full potential. TRCS is hampered
by low salaries and high turnover, especially in the finance department, which resulted in personnel with
fewer, notably computer, skills and made it difficult to raise standards. Staff roles are dependent on
organizational structure, but should be clearly defined, including specifying the final decision maker.
ARC proposed to develop the capacity of NSs through four activities: training, mentoring, developing
partnerships, and exchange activities among NSs, within NSs, and with other organizations. Capacity
building was intended to help strengthen NSs’ abilities to manage development projects in general and
TWC specifically. This required institutional strengthening as well as skills building for the TWC project
staff. IFRC was to assist with the institutional strengthening, but was not as engaged as intended due to
competing demands, staff turnover and reorganization within the coordinating body.  However, it did
carry out some training sessions in collaboration with ARC and the NSs. Because the institutional
management of development projects was weak in some areas such as leadership, planning, monitoring
and evaluation, and financial stewardship, ARC did not have the required foundation upon which to
build project management skills. Ultimately, ARC took on the IFRC responsibility for institutional capacity
building by identifying and organizing training sessions in some of the above areas for TRCS.
There was a palpable difference in the way the NSs interacted with ARC and their understanding of what
to expect from them in terms of capacity building. GRCS capacity building was constrained by
management’s initial resistance to outside assistance coupled with a lack of ARC presence in Guyana.
Conversely, TRCS had the great advantage of having two fulltime ARC staff members based in their
headquarters and/or field offices. The relationship between the two organizations was a continuous
work in progress where ARC was an appreciated resource.
The areas of focus for capacity building included: project planning, tools development, activity
implementation, volunteer management, financial management, donor compliance, M&E, branch
leadership development and sustainability planning. It may be assumed that these activities targeted all
relevant TWC staff, but this was not specified, which left TRCS feeling that management should have
been targeted for training and mentoring rather than the field staff who would be leaving the NS after
the end of the project.
To measure success, this SO counted number of people trained, the number of partnerships in existence
and the number of existing national task forces. Based on these monitoring data, it cannot be
determined whether or not the SO achieved its desired outcome of building capacity. It would have
been helpful to track standard project management indicators such as volunteer and staff retention,
percentage of on‐time reports, and branch performance. Volunteer retention is tracked internally and
staff retention may not have been recorded because there was little turnover at the field level in
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 29
Tanzania and none in Guyana. Headquarters‐based management in Tanzania were promoted to new
positions within TRCS but remained involved in TWC. The turnover in the finance department, although
not directly assigned to TWC, did affect the ability of this division to focus on the project as it was busy
getting staff up to speed. GRCS headquarters staff, but for the finance manager, were present for the
life of the project. The high staff retention says a lot about the projects in the highly competitive world
of development where it is not uncommon for staff to completely turn over before the completion of a
five year project.
Recommendations: Working through the planning process with NS staff, including choosing indicators
and setting targets for the SO, would have been a valuable lesson and ensured that both parties held
similar expectations for the upcoming year. This joint planning document could have served as an
agreement between ARC and the NS on roles and responsibilities including who is ultimately
accountable for decision making. Joint work plan development and developing usable indicators would
have helped both NSs. For example, the GRCS management style was to focus on achieving targets. Had
the indicators been developed such that there were targets for outcomes other than the number of
organizations involved and people reached and trained, FMs would have had more direction on what
they were trying to achieve for the overall project.
D.4.a. Training
Between one and 46 NS staff were trained annually on Organizational Development (OD). As stated
above, plans for who would be trained, on what, when, and by whom were not explicit. Perhaps due to
the lack of planning for training activities, there was a disconnect between training and practice. ARC
felt that NSs could have done a better job of proactively managing TWC, whereas NSs felt they were
doing what was expected of them.
TRCS staff relied heavily on the institutional development support they were receiving from ARC through
the TWC project and other funding sources. ARC supported branch leadership training in Tanzania. TRCS,
in collaboration with ARC has developed a training schedule for field staff to facilitate the project close
out where they already completed group organization and computer skills training. TRCS management
received training on reporting, communications and finances. They have been and continue to work
with TRCS to complete a programmatic audit. The Guyanese NS has requested assistance in completing
an audit as well.  
Tanzanian staff felt they had developed some strong project management skills through ARC‐supported
training, but that it was not enough. During the evaluation both senior management and field staff
specifically requested that more emphasis be placed on planning training sessions and developing a
schedule. GRCS field staff could not speak to ARC capacity building efforts, but some had attended
training that they thought may have been sponsored by ARC. TRCS staff were very grateful for the
opportunity to learn new skills and felt they had gained valuable skills at both the management and field
levels.  
Although NS staff were asked during the evaluation what they knew now because of ARC training that
they did not know before, most of the TRCS field staff responses came back to HIV‐prevention
knowledge and delivering HIV‐prevention materials. ARC trained NSs on how to implement the CBIs,
FUIs and worked in Tanzania on ACC and how to better reach HRY and OSY. They brought a BCC trainer
in from Trinidad to assist GRCS on developing messages in the last months of the project. It may be that
evaluation participants were not cognizant of their new abilities or that the training they received was
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 30
not applicable. Whatever the reason, there was little feedback during the evaluation on the training that
ARC provided.
It is quite possible that the NSs did not have a good idea of what training they needed. Leadership in this
area is incumbent upon ARC and NS management. ARC headquarters has OD training materials
developed in many technical areas that were not fully utilized even though they identified project
management as problematic. ARC ended up spending most of its time developing HIV‐prevention
activities and responding to urgencies as they arose. This left them with limited time to focus on OD.
Since it is often easier to “do” than “teach,” ARC took over some of the project management functions
that should have been completed by the NSs.
NSs need to remember field staff are employees of the NS and not of ARC. Ultimately it is the NS’s
responsibility to ensure that activities, including project management, are completed satisfactorily. ARC
and IFRC can conduct as many training sessions as possible without effecting any real change in the way
things are done. At this point, the NS needs to make some difficult decisions about how to maintain
staff. If training objectives are clear, then failure to meet those objectives during implementation
requires action. It would be effective to associate training objectives to human resources management
policies where the endpoints for success and failure and time limits are clearly delineated.
This SO is very important for sustainability, so staff training should be planned with as much enthusiasm
and structure as the volunteer training. ARC assumes that beneficiaries are blank slates when planning
the HIV‐prevention activities, but presupposes that NSs come with enough skills to independently
determine what they need to learn. Not knowing where to begin is likely even more pronounced with
Branch management committees.
Recommendations. ARC should take leadership by providing their available modules for OD training to
NS management. ARC and NSs should collaboratively plan sessions, develop a training schedule, and
come to a mutual understanding of how training will occur. NSs can support training and mentoring
goals by linking staff outcomes to human resource management and use the performance review of
staff and volunteers that occurred at the beginning of the project.
D.4.b. Mentoring
Mentoring is a wonderful concept, but very difficult to measure without becoming onerous, which is
most likely why there was no data reported on it. In Guyana mentoring was focused on the Program
Manager (PM) through telephone conversations, email and site visits by ARC headquarters project
manager and technical staff. In Tanzania, mentoring was a daily occurrence, especially when the ARC
health delegate was positioned in the field, and seems to have been focused on telling TRCS staff what
and how to do things better. All NS project managers participated in TWC international conference calls
with ARC HQ to share best practices and discuss the project in general.
Mentoring is the management equivalent of FUIs. The process refreshes and helps to solidify
information in the participants’ minds. It would be quite unheard of for project staff to put condoms on
beneficiaries to ensure it was done correctly. The same is true of project management. If the project
goals are clear, the NSs should be encouraged to develop their own way of reaching those targets and
supported by ARC through resource allocations and services such as training, advice and technical
support.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 31
Because ARC was not present in Guyana, this is what GRCS did. They made they own mistakes and
learned from them. They were fully responsible for all aspects of project management and
implementation. TRCS was not allowed to make mistakes and was not given the leeway to proceed as
they saw fit. They were not the primary decision makers for TWC; ARC was. This resulted in TRCS taking
a back seat to ARC and most likely contributed to them not taking initiative. Balancing between the need
to get the job done up to donor standards and building local capacity to achieve these standards is a
process which requires sincere and focused efforts on the part of both organizations.
ARC’s mentoring role should be to follow‐up on the training sessions, respond to questions that arise,
and work with NS staff to implement policies and procedures. The NSs’ role is to utilize training
materials and accept that they will make mistakes and that answers may be outside of their realm of
knowledge, at which point they should actively seek outside assistance. ARC has multi‐country
experience implementing TWC and can provide helpful advice that should be considered. NSs have
intimate country knowledge. The two should work together to establish and implement appropriate
policies and procedures.
Recommendations. Mentoring should occur, and as with training, should be planned with the NSs and
reassessed on an on‐going basis. ARC’s global TWC experience is the value they bring to the table, both
in terms of technical activities, and more importantly, in setting processes, procedures and standards
and documenting them in a TWC Project Handbook.
D.4.c. Develop Partnerships
The primary partnership to develop for the TWC project should be between the NS and ARC.
Partnerships were intended to be leveraged as learning opportunities and to develop common goals and
duplication of efforts. [Source: Workplan for Fiscal Years 2009 and 2010.] With the exception of
developing the ACC curriculum with FHI and publishing a brochure with Femina in Tanzania and with
Peace Corps in Guyana, this did not occur. This is not surprising since it is unlikely that other
organizations have a mandate to teach RC NSs what to do. Both NSs and ARC proceeded to implement
without developing synergistic partnerships with other players in the field. The project tracked “active
partnerships”. They were not necessarily durable relationships but the one‐off CWE opportunities.
Some partnerships did occur informally for CWEs with small CBOs, schools and the MOH where the FMs
and PEs would have set‐up an information booth at community health fairs, schools and other
community events. The Peace Corps partnership was not as effective as hoped because GRCS staff did
not necessarily have the skills to manage fulltime volunteers. It may be worth looking into ways of using
Peace Corps Volunteers to assist with the development of NS M&E systems, new business strategy,
proposal writing, branch development and strategic partnerships. Other potential partnerships could be
identified during the community mapping process with local stakeholders for all aspects of TWC.
The NSs should be encouraged to engage in the national HIV‐prevention dialogue. Liaising with other
implementing partners was a part of the job description of the PM. The project relied on national task
forces such as National AIDS Councils, the Global Fund Country Coordinating Mechanisms and USAID
implementing partner groups. The national task force is a nice idea, but really not the responsibility of a
NS in either evaluation countries. In both of these countries, government claims that information
sharing between programs is their role. Neither government was directing this and unfortunately many
of these groups were not very active, so NGOs were finding creative ways of getting around the
leadership vacuum. Instead of establishing and strengthening new task forces, NSs should participate in
these informal groups. A more strategic and proactive approach to networking with clear goals for
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 32
creating active and official alliances in specific technical areas will require significant investments in time
and effort to meet with other program leaders on an individual basis to negotiate the specific terms of
the association.
Recommendations. Activity‐based partnerships should be informed by the community mapping process
during which gaps are identified between what TWC can offer and what the community wants and
needs. The mapping activity will help is identifying potential partners and the gaps between community
needs and what the TWC project is offering. Creating partnerships is aligned with all the SOs, not just
SO2. Ensure that management is responsible for identifying and developing formal partnerships with
written agreements to assist in reaching both project and community objectives.
D.4.d. Disseminate Best Practices
NS capacity was to be built by sharing project information between NS staff and volunteers within
Guyana and Tanzania, between NSs outside of the country of origin and at international conferences
and workshops. This activity is not crucial for the success of the TWC project, but did provide added
programmatic value and an incentive for potential participants.
Identifying and organizing sharing opportunities between NSs around the world is probably the
responsibility of a coordinating organization like ARC, which is how they occurred. This activity will most
likely not happen once ARC no longer has funding to support HIV prevention. Staff from both NSs were
enthusiastic at the opportunity to attend international conferences. When asked how ARC had built NS
capacity these were mentioned first and foremost.  
GRCS staff participated in a national youth leadership workshop to present on the TWC life skills
approach. A GRCS volunteer presented the project and its activities at the World Bank on World AIDS
Day and GRCS staff offered technical assistance to Miami chapter launching a TWC program. Project
managers from all three countries participated in regular conference four‐way calls with ARC HQ that
served as both a mentoring opportunity and a venue for sharing best practices.
Both NSs were sharing information between project staff and volunteers, mainly during staff meetings.
Face‐to‐face meetings in Guyana were constrained by the very heavy burden of travel between regions.
Tanzania did not have this problem as the staff were concentrated in one region. These internal
exchanges are most likely to be sustained and perhaps more applicable than international dissemination
strategies due to a presumed similarity of environments and shared issues; they also require less
funding to organize than international visits.
Recommendations. Disseminating best practices is clearly a valuable exercise. NSs should continue to
hold staff and volunteer meetings. NSs should make serious efforts to participate in the national
dialogue through established coalitions and by PMs targeting individual organizations identified during
community mapping to plan complementary activities and develop operating agreements. NSs could
hold annual dissemination and feedback meetings at national, regional, district and local levels which
could be part of the definition of Town Hall meetings. International conferences and workshops are a
nice, but will be difficult to finance without significant amounts of external funding unless NSs are willing
to fundraise for them.  This is most likely a lower priority for the NSs than delivering services.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 33
E.  PROJECT IMPACT
The overall project impacts for youth were attributable to a combination of the knowledge and skills
learned through CBI and FUIs. Rural‐based Guyanese adults also benefited from town hall meetings and
CWEs. In Tanzania adults felt the ACC training not only taught them new information about HIV
prevention, but more importantly, it helped them become closer to their children by practicing verbal
communication.
An unintended result of TWC’s implementation was that RC has become better known in both countries,
although participants were not exactly sure what RC had to offer them. In Guyana, RC was thought to be
doing good and helping people. In Tanzania, because of the TWC and Home‐Based Care projects,
communities trusted RC more in Tanzania and came to view them as an organization that also works in
HIV whereas it was formally known for “sucking blood” and distributing relief supplies.
Table 7: Date Ranges for Pre‐ and Post‐Test Data
Included in the discussion of impacts is
data from the GRCS and TRCS Access
databases. The database is an
interactive knowledge management
tool which allows for reports to be run
on disaggregated data for a specific
time period by a single branch, school, lead peer educator name, gender, age cohort, ISY and OSY,
school level of the or PE experience. The data in each report are organized by pre/post‐test question
and provide the number tests taken, the percentage of correct pre‐  and post‐test responses and the
percent change in the number of correct responses from pre‐ to post‐ test. The peer‐to‐peer testing
activities did not begin at project inception but much later. The data used in the analyses below are from
2007 in Guyana and 2009 in Tanzania, as outlined in Table 7: Date Ranges for Pre‐ and Post‐Test Data.
An example of the output is presented in Appendix B: Example of Pre‐  and Post‐Test Report. When
commenting on the country reports, the ARC headquarters monitoring and evaluation office requested
confidence intervals for these data, but this is not available when running the standardized reports in
the Access database, and the data are not accessible for manipulation in statistical programs.  
E.1. Life Skills
The most mentioned favorite part of TWC was Tafakari in Tanzania and SOCA in Guyana. Two pre‐ and
post‐test questions, shown in Table 8 and Table 9 assessed the uptake of these self‐efficacy lessons.
One question was asked only of girls and the other strictly of boys. Generally speaking, the older the
participants, the greater was the likelihood of them giving the desired answer, with Tanzanian females
being the exception to the rule. Although those examined may not have used this life skill to refuse sex,
as these questions imply, this activity permeated their daily decision making and made youth feel more
self‐confident in all aspects of their lives.
An unintended result of the self‐efficacy message was reported changes in social behaviors such as more
attention being paid to school work and more respect being shown towards adults and other youth.
Youth used the critical thinking process they learned to question common beliefs and their own values.
Youth reported a greater tendency towards monogamous relationships with the opposite sex. Local
leaders, teachers and parents appreciated the maturity exhibited by youth who had participated in TWC.
DATE RANGES FOR DATA
Activity Type Guyana Tanzania
CBI 10/2007‐05/2010 05/2009‐04/2010
FUI 06/2009‐05/2010 06/2009‐05/2010
[Source: GRCS and TRCS CBI and FUI Pre‐ and Post‐Test Databases]
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 34
These two indicators plus the indicators to negotiate condom use seem to be the most relevant
measures for whether or not SO1, Strengthen HIV‐prevention life skills of youth 10 to 24 years of age,
has been achieved. Where fewer than four‐fifths, the TWC threshold for success, of tests came back
with the correct answer for the post‐test, numbers in tables have been outlined in yellow. Three of the
10 branches had greater than 80% of girls’ stating that they could refuse sex and no branches for boys.
For having reached nearly a half‐a‐million people the sample sizes seem smaller than needed to draw
any statistically significant conclusions but the numbers do give us an indication of how difficult it is to
change social and cultural norms and where need for additional messaging remains.
Table 8: If my boyfriend asks me to have sex with him and I don't want to, I am confident I will not have sex with
him. (Female respondents only)
Guyana CBI Data Tanzania CBI Data
Age Cohort # Tests
(pre/post)
Pre‐test Post‐test Age Cohort # Tests
(pre/post)
Pre‐test Post‐test
10‐14 379/376 61% 66% 10‐14 760/742 55% 73%
15‐19 196/187 69% 74% 15‐19 371/379 62% 71%
20‐24 118/12 78% 100% 20‐24 70/62 66% 65%
Branch # Tests
(pre/post)
Pre‐test Post‐test Branch # Tests
(pre/post)
Pre‐test Post‐test
Georgetown 133/107 85% 86% Bukombe 20/20 95% 100%
Letham 388/388 60% 69% Kahama 505/498 57% 73%
Mabaruma 49/40 43% 40% Kigoma Urban 85/73 49% 74%
Santa Rosa 24/41 62% 68% Kishapu 234/213 63% 73%
    Shinyanga Rural 35/74 80% 65%
    Shinyanga Urban 20/20 95% 100%
[Source: GRCS and TRCS CBI Pre‐ and Post‐Test Databases]
Table 9: If my girlfriend wants me to have sex with her, I will say "Yes" because if I don't my friends will say that
I am just a small boy. (Male respondents only)
Guyana CBI Data Tanzania CBI Data
Age Cohort # Tests
(pre/post)
Pre‐test Post‐test Age Cohort # Tests
(pre/post)
Pre‐test Post‐test
10‐14 296/283 51% 55% 10‐14 673/634 47% 56%
15‐19 159/141 42% 56% 15‐19 458/448 56% 59%
20‐24 19/18 47% 72% 20‐24 115/109 69% 76%
Branch # Tests
(pre/post)
Pre‐test Post‐test Branch # Tests
(pre/post)
Pre‐test Post‐test
Georgetown 136/97 57% 69% Bukombe 357/342 53% 59%
Letham 289/281 45% 50% Kahama 492/472 51% 57%
Mabaruma 33/32 48% 56% Kigoma Urban 132/122 52% 64%
Santa Rosa 16/32 25% 69% Kishapu 214/172 55% 66%
   Shinyanga Rural 51/81 67% 63%
   Shinyanga Urban 19/20 0% 0%
[Source: GRCS and TRCS CBI Pre‐ and Post‐Test Databases]
Youth felt they were able to identify risky situations after participating in TWC. Evaluation interviewees
in both countries reported reductions in teen pregnancies and substance abuse. The decrease in teen
pregnancies was verified by health workers, school teachers, and community leaders. In Tanzania
participants said cultural myths were dispelled, such as traditional birth control and abortion methods
and the ability of witch doctors to cause and cure AIDS. No witch doctors were asked if their business
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 35
had decreased since the inception of TWC which could be used to triangulate data. Pre‐ and post‐tests
listed in Table 10: Can a person get AIDS because of witchcraft? show that a high proportion of YMs
already felt that AIDS is not the result of having a spell cast. The older the test‐taker, the higher were the
pre‐ and post‐test knowledge scores.
Table 10: Can a person get AIDS because of witchcraft?
Volunteers, both PEs and
YMs, reported gaining
organizational and
communication life skills.
Guyanese leaders
recognized the value of
TWC in creating tomorrow’s
leaders. Self‐reported
changes in self‐efficacy
were very similar in both
Tanzania and Guyana.
E.2. HIV‐Prevention
Knowledge
Respondents in both
countries were able to list ways to prevent HIV and pregnancy, including abstinence, fidelity, and
condom use. They also mentioned avoiding sharing sharp objects with people who were HIV positive. In
Tanzania, interviewees discussed HIV testing and wearing gloves while caring for an HIV positive person.  
Table 11: Comprehensive correct knowledge about AIDS (TWC) indicates that YMs started with similar
amounts of HIV prevention knowledge in both countries for the CBIs and the FUIs and knowledge
increased after the interventions. The leap from one quarter of people having comprehensive HIV‐
prevention knowledge to near double the numbers is stunning even though the overall scores appears
to be low. The fact that overall correct and comprehensive HIV prevention knowledge is lost with time is
a strong argument for the continuation and importance of FUIs.  
Table 11: Comprehensive correct knowledge about AIDS (TWC)
Test Guyana Tanzania
# Tests
(pre/post) Pre‐test Post‐test Change # Tests
(pre/post) Pre‐test Post‐test Change
CBI Data 1680/1018 26% 46% 84% 2483/2402 27% 48% 78%
FUI Data 693/638 41% 56% 37% 1732/1689 40% 59% 48%
[Source: TRCS and  GRCS CBI and FUI Pre‐ and Post‐Test Databases]
The only two CBI post‐test questions where fewer than 80% of people responded incorrectly are shown
in Table 12: Specific HIV‐Prevention Knowledge. The data also show that after the FUI over 80% of
people taking the test provided correct responses. Both countries’ YMs retained these pieces of
knowledge from the CBI to the FUIs. The Guyanese CBI data were similar across branch and gender.
Younger aged cohorts scored lower than did their more aged counterparts after the CBI but this evened
out with the FUI. Correspondingly fewer ISY knew about fidelity as an HIV prevention method than OSY
GRCS CBI Data FUI Data
Branches # Tests
(pre/post)
Pre‐
test
Post‐
test
# Tests
(pre/post)
Pre‐
test
Post‐
test
Georgetown 269/204 75% 80% 221/155 85% 78%
Letham 677/699 76% 89% 453/466 77% 85%
Mabaruma 82/72 62% 81% 19/17 79% 94%
Santa Rosa 70/73 35% 88% No data    
TRCS CBI Data FUI Data
Branches # Tests
(pre/post)
Pre‐
test
Post‐
test
# Tests
(pre/post)
Pre‐
test
Post‐
test
Bukombe 696/657 80% 87% 433/415 86% 92%
Kahama 997/970 77% 89% 503/503 82% 88%
Kigoma Urban 217/195 55% 72% 172/169 71% 67%
Kishapu 448/385 87% 92% 302/302 94% 96%
Shinyanga Rural 86/155 85% 81% 137/131 88% 84%
Shinyanga Urban 39/40 92% 100% 18/18 89% 94%
[Source: TRCS and  GRCS  CBI and FUI Pre‐ and Post‐Test Databases]  
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 36
and with the additional FUI training, 84% of ISY as compared to 82% of OSY learned that maintaining a
faithful HIV negative partner could prevent contraction of the disease.
Table 12: Specific HIV‐Prevention Knowledge
Test
Guyana: Can people protect themselves from
the AIDS virus by having one uninfected
faithful sex partner?
Tanzania: Can the AIDS virus be transmitted from
mother to child?
# Tests
(pre/post)
Pre‐
test
Post‐
test Change # Tests
(pre/post) Pre‐test Post‐test Change
CBI Data 1680/1018 62% 73% 18% 2483/2402 72% 76% 6%
FUI Data 693/638 75% 84% 2% 1732/1689 75% 81% 8%
[Source: TRCS and  GRCS CBI and FUI Pre‐ and Post‐Test Databases]
In Tanzania after CBIs a greater proportion of school children knew that HIV could be transmitted from
mother giving birth to her child than did OSY both after the CBI and the FUI. The older the YM, the less
likely they were to have this information. Across branches only 43% of respondents in Shinyanga rural
knew about mother‐to‐child transmission after the CBI whereas greater than three‐fourths of YMs
situated at the other branches were aware of this fact and the level of belief fell yet further after the FUI
to 39%. Females (78%) held a slight advantage over males (74%) for this question after the CBI and kept
the lead after the FUI at 83% as compared to the males’ 78%.
E.3. Attitude Towards PLWHA
Significantly more time and effort has been put into reducing stigma of and discrimination toward
PLWHA in Tanzania than in Guyana by a host of partners. It is therefore expected that accepting
attitudes would be greater in Tanzania than in Guyana. Few evaluation participants in Guyana had any
personal contact with PLWHA, but some had heard gossip and witnessed discrimination against
suspected individuals. Evaluation participants in both countries felt that improvements had been made,
but there were still people who were afraid of infected persons. More people interviewed in Tanzania
were HIV positive and/or had personal relationships with HIV positive people, and so tended to have
greater empathy. Tanzanian respondents said the greatest discriminatory hurdle was the self‐imposed
segregation that people carrying the virus put on themselves.  
The gains made in changing attitudes towards PLWHA were not altogether attributable to TRCS efforts
since there had been an inundation of messages before TWC arrived. In Guyana, however, GRCS was
one of the only sources of this message in the rural sites, and so presumably played a greater direct role
in the improvements, especially in rural areas where they may have been the only organization based in
the field.
Table 13: Accepting Attitudes Towards PLWHA
Activity Type Guyana Tanzania
# Tests
(pre/post) Pre‐test Post‐test Change # Tests
(pre/post) Pre‐test Post‐test Change
CBI 1680/1018 18% 35% 94% 2483/2402 39% 63% 62%
FUI 693/638 28% 36% 29% 1732/1689 56% 66% 18%
[Source: TRCS and  GRCS CBI and FUI Pre‐ and Post‐Test Databases]
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 37
The qualitative data supports pre and post‐test findings listed in Table 13: Accepting Attitudes Towards
PLWHA. Gains were made in both countries, and YMs tested higher in overall acceptance levels of
PLWHA after TWC exposure than before the original CBI. Where FUI post‐test levels in Tanzania were
slightly higher than CBI post‐test numbers, in Guyana accepting attitudes increased for a greater
proportion of the people and remained at the same level after the FUI.
The CBI responses for one of the four questions asked regarding attitudes towards PLHIV are
disaggregated in Table 14 by gender, age and NS branch location. Once again age appears to play
somewhat of a factor in influencing key decisions. Georgetown and Santa Rosa YMs in Guyana and
Kigoma urban YMs in Tanzania were particularly concerned with people knowing that their family
member was HIV positive.  
Table 14: If a member of your family became infected with the virus that caused AIDS, would you want it to
remain a secret?
Guyana CBI Data Tanzania CBI Data
Gender # Tests
(pre/post) Pre‐test Post‐test Gender # Tests
(pre/post) Pre‐test Post‐test
Female 594/576 56% 63% Female 1218/1193 71% 83%
Male 474/442 52% 57% Male 1265/1209 69% 82%
Age Cohort # Tests
(pre/post) Pre‐test Post‐test Age Cohort # Tests
(pre/post) Pre‐test Post‐test
10‐14 675/659 53% 58% 10‐14 1433/1376 68% 81%
15‐19 355/328 57% 64% 15‐19 829/827 71% 82%
20‐24 37/30 43% 60% 20‐24 185/171 84% 96%
Branch # Tests
(pre/post) Pre‐test Post‐test Branch # Tests
(pre/post) Pre‐test Post‐test
Georgetown 269/204 36% 43% Bukombe 696/657 69% 76%
Letham 677/669 64% 65% Kahama 997/970 70% 86%
Mabaruma 82/72 48% 69% Kigoma Urban 217/195 50% 56%
Santa Rosa 40/73 32% 49% Kishapu 448/385 76% 89%
   Shinyanga Rural 86/155 92% 94%
   Shinyanga Urban 39/40 77% 100%
[Source: GRCS and TRCS CBI Pre‐ and Post‐Test Databases]
The test also contained the question, “If a member of your family became sick with the AIDS virus,
would you be willing to care for him or her in your household?” Despite many YMs wanting the
information to remain a secret, a great majority said they would care for them, 94% in Tanzania and 90%
in Guyana. 73% of Guyanese YMs said a teacher with AIDS should be allowed to teach in school and 79%
would buy fresh vegetables from an HIV positive shopkeeper. Response rates in Tanzania were 89% and
85% for teachers and shopkeepers, respectively. So it appears that the question that drove down the
overall acceptance rate of PLWHA in Guyana was the question of secrecy which is not necessarily a
reflection of their views but of their perception of how society would critique their family if they were
aware of a member’s HIV status.
E.4. Abstinence
Evaluators found that abstinence was a very popular message for young school children, and students in
both countries touted abstinence as appropriate for them. The rationale was that abstinence was a sure
way to not get pregnant and to avoid HIV so they could reach their life goals. Adults felt by that once
children reached puberty, abstinence was not realistic, but they still promoted the idea until graduation.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 38
In Tanzania, evaluation participants of all ages attributed some of the drop in teen pregnancy to
abstinence. Rural Guyanese nearing or at the age of puberty were viewed as more innocent and naive
than those in Georgetown and in the Tanzania evaluation sites. They were not even interested in sex
yet, whereas Tanzanian school children were caught “playing” sex. Abstaining from sex for the very
young in Guyana was seen as problematic where rape and incest were involved and out of the control of
the youth. Abstinence was felt to be more difficult for boys than girls in both countries due strong peer
pressure on boys to engage in sexual activity.
The test question highlighted in Table 15 attempts to ascertain whether YMs feel that girls have control
over their sexual decisions. Choosing not to have sex does not immediately translate into abstinence but
does give an indication as to the level of difficulty, due to social pressures, that a girl may face when not
having sex. It seems that for both males and females, women in Tanzania had been given more power
by society to refuse sex which was reinforced by the TWC project. Age is once again a factor in the
responses where the older the respondent, the greater the numbers who bestow women with the right
to choose not to have sex.  
Table 15: A girl has the right to choose not to have sex with her boyfriend, even if he wants to.
Guyana CBI Data Tanzania CBI Data
Gender # Tests
(pre/post) Pre‐test Post‐test Gender # Tests
(pre/post) Pre‐test Post‐test
Female 594/576 68% 78% Female 1218/1193 78% 91%
Male 474/442 66% 74% Male 1265/1209 77% 88%
Age Cohort # Tests
(pre/post) Pre‐test Post‐test Age Cohort # Tests
(pre/post) Pre‐test Post‐test
10‐14 675/659 66% 75% 10‐14 1433/1376 74% 88%
15‐19 355/328 69% 78% 15‐19 829/827 82% 91%
20‐24 37/30 78% 90% 20‐24 185/171 84% 95%
Branch # Tests
(pre/post) Pre‐test Post‐test Branch # Tests
(pre/post) Pre‐test Post‐test
Georgetown 269/204 87% 91% Bukombe 696/657 77% 87%
Letham 677/669 61% 71% Kahama 997/970 74% 90%
Mabaruma 82/72 59% 68% Kigoma Urban 217/195 75% 89%
Santa Rosa 40/73 60% 92% Kishapu 448/385 84% 91%
   Shinyanga Rural 86/155 88% 88%
   Shinyanga Urban 39/40 97% 100%
[Source: GRCS and TRCS CBI Pre‐ and Post‐Test Databases]
In Guyana reactions to this idea ended up where Tanzania started and are very mixed by branch.
Georgetown and Santa Rosa numbers were on par with the Tanzanian post‐test figures whereas the
attitudes at the remaining two branches were well below the targeted 80%.  Age appears to influence
responses but the sample size for the 20‐24 year old cohort is relatively small so may not be reflective of
other YMs of the same peer group.
Pressure to have sex seems to be very pronounced for boys in both countries. As indicated in Table 16,
females seem to be more open to the idea that young boys do not need to have sex by the age of 16.
Younger aged participant responses indicate that males feel less able to say no to sex than girls. Only in
Santa Rosa in Guyana and Shinyanga urban in Tanzania did greater than 80% of YMs tested feel that
boys under 16 do not have to be sexually active and that abstinence is a possibility.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 39
Table 16: By the time they are 16, all boys should have sex at least once.
Guyana CBI Data Tanzania CBI Data
Gender # Tests
(pre/post) Pre‐test Post‐test Gender # Tests
(pre/post) Pre‐test Post‐test
Female 594/576 60% 67% Female 1218/1193 53% 67%
Male 474/442 50% 57% Male 1265/1209 52% 65%
Age Cohort # Tests
(pre/post) Pre‐test Post‐test Age Cohort # Tests
(pre/post) Pre‐test Post‐test
10‐14 675/659 56% 60% 10‐14 1433/1376 50% 67%
15‐19 355/328 57% 66% 15‐19 829/827 55% 65%
20‐24 37/30 43% 83% 20‐24 185/171 57% 71%
Branch # Tests
(pre/post) Pre‐test Post‐test Branch # Tests
(pre/post) Pre‐test Post‐test
Georgetown 269/204 72% 78% Bukombe 696/657 49% 63%
Letham 677/669 51% 56% Kahama 997/970 57% 75%
Mabaruma 82/72 51% 60% Kigoma Urban 217/195 44% 57%
Santa Rosa 40/73 35% 86% Kishapu 448/385 58% 66%
   Shinyanga Rural 86/155 22% 29%
   Shinyanga Urban 39/40 44% 95%
[Source: GRCS and TRCS CBI Pre‐ and Post‐Test Databases]
E.5. Fidelity
Fidelity was thought to be more important for women than men in both countries. Interviewees
reported little change after TWC. Adults were skeptical that within the general population people were
more faithful due to the project. TWC messages did cause some Tanzanian participants, both males and
females, to report behavior changes, specifically, swearing to be faithful to their partners for fear of
contracting HIV. There were no test questions relating to fidelity.
E.6. Condom Use
National governments have made a push in the promotion and availability of condoms in both Guyana
and Tanzania. ‘Condomize’ has become a verb in Guyana. While unsystematic, GRCS helps transport
government stock to health centers. Health centers in both countries provide free condoms and shops
sell them. In Tanzania organizations distribute condoms at motels and bars. Adult and youth self‐
reported greater acceptance and use of condoms since the inception of TWC. Males generally are the
ones to acquire condoms as girls who obtain or carry condoms are perceived as immoral, although this is
less pronounced in Guyana. Condom use between faithful partners is acceptable in Tanzania, but not in
Guyana. This could be related to the acknowledgement in Tanzania that discordant couples exist and
HIV prevention messaging unambiguously promotes condom use for couples through health centers and
development partners.  
There was more public resistance voiced during the evaluation to promoting condoms in Guyana than
Tanzania which is reflected in Table 17 data. Adults in both countries were opposed to discussing
condoms with children who had not yet reached puberty but Tanzanian respondents of all ages said that
condoms were a life saving device and should be encouraged for adults. Although Guyanese YMs had
overall lower test scores, the amount of the gains were similar between the two countries but for the
youngest aged cohort in Guyana. This may be due to the absence of condom messages for these pre‐
pubescent youth due to social resistance. Age once again appears to play a role in the proportion of
older YMs believing in greater numbers than the younger ones that condoms are an HIV prevention tool.  
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 40
Table 17: If you're having sex, condoms are a good way to prevent pregnancy, HIV, and sexually transmitted
diseases.
Guyana CBI Data Tanzania CBI Data
Gender # Tests
(pre/post) Pre‐test Post‐test Gender # Tests
(pre/post) Pre‐test Post‐test
Female 594/576 72 79 Female 1218/1193 76 89
Male 474/442 76 75 Male 1265/1209 79 89
Age Cohort # Tests
(pre/post) Pre‐test Post‐test Age Cohort # Tests
(pre/post) Pre‐test Post‐test
10‐14 675/659 75 75 10‐14 1433/1376 74 88
15‐19 355/328 69 78 15‐19 829/827 79 89
20‐24 37/30 78 90 20‐24 185/171 91 93
Branch # Tests
(pre/post) Pre‐test Post‐test Branch # Tests
(pre/post) Pre‐test Post‐test
Georgetown 269/204 83 83 Bukombe 696/657 73 89
Letham 677/669 69 75 Kahama 997/970 76 90
Mabaruma 82/72 80 74 Kigoma Urban 217/195 77 83
Santa Rosa 40/73 70 89 Kishapu 448/385 83 88
   Shinyanga Rural 86/155 91 92
   Shinyanga Urban 39/40 95 100
[Source: GRCS and TRCS CBI Pre‐ and Post‐Test Databases]
During the pre‐  and post‐tests female YMs were asked if they could convince their partners to use a
condom. In Table 18 it show that older girls felt more confident about their ability to persuade their
partner to use a condom. Tanzanian females say with greater frequency than the Guyanese that they
would be able to negotiate the use of prophylaxis. Although 79% of females in Guyana and 89% in Santa
Rosa understand that condoms are an effective means of birth control and barrier to contracting STIs, a
smaller proportion feel they could actually use a condom with their partner.
Table 18: If I were to have sex, I could convince my partner to use a condom. (Female respondents only)
Guyana CBI Data Tanzania CBI Data
Age Cohort # Tests
(pre/post) Pre‐test Post‐test Age Cohort # Tests
(pre/post) Pre‐test Post‐test
10‐14 379/376 61% 66% 10‐14 760/742 78% 88%
15‐19 196/187 69% 74% 15‐19 371/379 82% 94%
20‐24 18/12 78% 100% 20‐24 70/62 90% 98%
Branch # Tests
(pre/post) Pre‐test Post‐test Branch # Tests
(pre/post) Pre‐test Post‐test
Georgetown 133/107 96% 91% Bukombe 339/315 77% 86%
Letham 388/388 74% 86% Kahama 505/498 79% 93%
Mabaruma 49/40 78% 92% Kigoma Urban 85/73 66% 81%
Santa Rosa 24/41 75% 59% Kishapu 234/213 91% 92%
   Shinyanga Rural 35/74 91% 99%
   Shinyanga Urban 20/20 95% 100%
[Source: GRCS and TRCS CBI Pre‐ and Post‐Test Databases]
Males were similarly asked if they could influence their partners to allow them to use a condom. Most of
them felt they could within all aged cohorts and across all branches. Even the youngest boys were
confident they could sway their partners. When comparing these data in Table 19 with those of Table
18, there does not appear to be any strong correlation between the male and female responses within
age or branch groupings.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 41
Table 19: If I were to have sex, I could convince my partner to let me use a condom. (Male respondents only)
Guyana CBI Data Tanzania CBI Data
Age Cohort # Tests
(pre/post) Pre‐test Post‐test Age Cohort # Tests
(pre/post) Pre‐test Post‐test
10‐14 296/283 79% 90% 10‐14 673/634 82% 90%
15‐19 159/141 77% 87% 15‐19 458/448 85% 90%
20‐24 19/18 79% 89% 20‐24 115/109 89% 94%
Branch # Tests
(pre/post) Pre‐test Post‐test Branch # Tests
(pre/post) Pre‐test Post‐test
Georgetown 136/97 83% 82% Bukombe 357/342 85% 94%
Letham 289/281 78% 92% Kahama 492/472 80% 87%
Mabaruma 32/33 76% 88% Kigoma Urban 132/122 84% 93%
Santa Rosa 16/32 62% 84% Kishapu 214/172 87% 88%
   Shinyanga Rural 51/81 82% 94%
   Shinyanga Urban 19/20 100% 100%
[Source: GRCS and TRCS CBI Pre‐ and Post‐Test Databases]
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 42
F.  OVERALL RECOMMENDATIONS
ARC and NSs should work through the program management process together. Overall TWC project
evaluation recommendations therefore fall into five categories; plan collaboratively using participatory
community mapping, define intended outcomes for each target population with strategies for engaging
them, define and prioritize activities, define measures and data use, and document progress. Specific
recommendations for the evaluation findings are contained in Appendix E: Evaluation
Recommendations.
F.1. Plan Collaboratively
The community mapping idea was a fabulous one. Unfortunately it did not occur as a participatory
process but is highly recommended and necessary for greater success of all future projects. Mapping will
most likely be a two step process: with NS TWC staff and with community representatives. The three
main players are represented by circles in Figure 1: Community Mapping Diagram; the NS, volunteers
and the community. The diagram is a pictorial representation of desired outputs and outcomes (wants
and needs). Each box characterizes individual or shared wants or needs. The area under Box A
symbolizes mutual interest by all three parties. Boxes B, C and D signify that only two organizations have
a shared goal and boxes E, F and G embody unshared goals. Each entity also brings skills and resources
which can be used to achieve the wants and needs.
Figure 1: Community Mapping Diagram
The idea of the mapping exercise was to
work with community representatives
to first identify already existing ties. NSs
have the benefit of branch management
committees who can act as
representatives and/or organize these
sessions. At the same time mapping can
be used to prioritize activities and
ascertain the best way forward based on
the available skills and resources each
group is willing to contribute towards
meeting an end goal.
Participatory methods such as
Participatory Rural Appraisal (PRA),
Appreciative Inquiry (AI), and
Participatory Appraisal for Community
Action (PACA) need only be selected by
the NS to start down this path. NS staff
would need to be trained in the chosen
approach which could be an ARC role.
Whichever technique used, the goal is the same, to begin developing long‐term, collaborative
partnerships with communities NSs serve and to develop community action plans which specify the
activities to be implements, responsible parties and timelines. Pieces of the process are already being
implemented, i.e. convincing communities that condom promotion is wise. The topic of condom
NS
Community Volunteers 
Community Mapping 
A
C 
D
E
F G 
B
-Wants -Skills 
-Needs -Resources 
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 43
promotion fell squarely in areas represented by boxes E and C in Figure 1. GRCS used Town Hall
meetings to move the condom discussion into zone A, a shared aspiration.
This process will also help in more clearly identifying youth‐friendly services such as youth groups,
community organizations, youth gatekeepers – especially for FBOs – and organized events where CWEs
can be held as well as what resources each special interest is willing to bring to the table. For each
targets’ influencing social and cultural factors would be discovered and appropriate incentives can be
identified to promote active engagement along with resources the community already posses which can
act as encouragement. Opportunities for partnerships should also be classified for unmet target
populations (Boxes F and G) such as adult HIV prevention education and continuing education for youth.
Specific efforts should be made to learn more about how to target volunteer and their motivations.
The end product of community mapping is a plan for the prioritized activities to be undertaken, by
whom and under what time frame. Within the community action plan clear lines of communication and
oversight should be established. The action plans should be written by community representatives to
ensure local ownership with TWC staff contributing to the final product. The approach generally requires
that a given community have their action plans written before any development assistance is provided.  
F.2. Define Expectations for Target Populations and Engagement Strategies
TWC defined youth, ages 10 to 24 years old, parents, community leaders, the general community,
partners and NSs as targets. These are large and disparate groups for which exact targets should be
massaged during the community mapping exercise. Expected outcomes for each group evolved over the
life of the project for some and are still a work in progress for others. It was clear from the inception
that youth would be expected to gain knowledge, improve self‐efficacy, become more accepting of
PLWHA and adopt healthy behaviors. The strategies for engaging youth were to work with schools,
youth groups and FBOs. NSs continue to refine their efforts to reach youth through groups and FBOs.
The target population, what they were to achieve and strategies for engagement were heavily focused
on for the duration of TWC.  
For the other target populations there were breakdowns in defining expected outcomes which resulted
in unfocused targeting strategies or no strategy at all. There appeared to be considerable confusion
between the planned activities and where the activities would lead to. Because of this disconnect,
activities were diluted. Part of the problem may have been the one‐size fits all approach to community
councils and town hall meetings. The project planned for activities and then retrofitted the target
populations into those activities. It may make more sense for developers to first think about who needs
to be involved in the project to reach its objectives, secondly to determine what each target group is
expected to achieve, and lastly to consider the activities which will lead to realizing these outcomes or
intermediate results.
As an example of this approach one can look at RC branch management committees whose involvement
could contribute to all of the SOs. There were never explicitly part of TWC but should be. They could be
used to provide advice and guidance to project staff regarding the identification of volunteers, where to
access youth for outreach activities and to get permission from gatekeepers of youth to operate within
their community, religious establishments, workplaces, etc. the management committees can be
targeted by staff either by calling a special session or by getting on the agenda of an existing meeting.
Once these first two steps have been clearly elucidated planners of TWC can design activities in support
of the committee to reach their desired goals. These may include but should not be restricted to,
developing and documenting outreach processes, training committee members on topics identified
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 44
during community mapping and setting‐yup systems for conducting feedback sessions. The feedback
session and trainings can also be considered a part of the motivational strategy for these volunteers.
F.3. Define and Prioritize Activities  
All TWC activities should be defined to the same degree as the CBI by ARC and their NS partner
organizations during the planning phase and though out the life of the activity. The large number of
activities combined with low NS capacity and challenging operating environments necessitated
concessions to be made regarding the degree to which some activities were implemented. Included in
the project were activities which resulted in directly attributable results to the NSs’ actions such as the
CBI, FUI and ACC which could engage youth clubs, those which reinforced youth taking steps to prevent
HIV – ACC, mass media, CWEs, town hall meetings, community councils, and partnerships  ‐ and others
which ensured that the NSs’ could implement the aforementioned interventions, i.e., training,
mentoring, information exchanges and tools development.
The menu of TWC activities could be presented to each NS and their ability, national need and resource
availability would determine which to pursue and which to disregard. This would result in individualized
TWC projects which could become confusing for ARC but better for each country in which TWC is
implemented. Some of the pursuits will remain central to the project such as all of the peer‐to‐peer
curricular exercises (CBI, FUI, ACC), engaging the community through meetings and NS capacity building.  
Youth groups should be used as a targeting strategy, not an additional activity. These could be formal or
informal. This changes the focus of what is expected by engaging them in TWC. Instead of counting them
as a separate group of youth, they would be included in those who participated in the CBIs and FUIs.  
Activities that contributed less to reducing incidence of HIV in youth ages 10 to 24 should be reassessed.
For example mass media. Youth respondents said that they had heard HIV prevention public service
announcements – from unknown sources  ‐ on television and radio but they did not influence them to
change risky behaviors. They said that their own behavior changed only after personal interactions with
a FM, PE or YM. TWC’s strength lies in their peer‐to‐peer outreach and not in creating and promoting
through mass media. For these three reasons, this activity could be removed from a given NS’s TWC
project without sacrificing the project’s outcomes. If mass media is included in the project its outcomes
should be clear and measurable.
Holding CWEs was found to be most valuable in marketing the NSs and transferring some knowledge but
no one interviewed during the evaluation said it caused them to change what they were doing thus
avoiding new infections. They were also useful in building PE life skills of interpersonal communications,
organizing events and leadership. If CWEs are chosen as a part of the overall project, what they are able
to achieve should be realistic and assessed.
Town hall meetings and community‐based advisory boards should be better defined so all staff
understand and are able to achieve their objectives. Project documents were clear in stating that town
halls were to engage the community’s approval and disseminate information. How this was to be done
was not documented which resulted in confusing during implementation. The same is true of
community councils. They were to act as an advisory board to the project but the methods were
unclear. By using representative members of the community who are already engaged with the RC and
willing to contribute may be a more realistic option than asking that government officials serve in this
capacity.
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 45
Developing partnerships was designed to be cross‐cutting activity and could remain so if, during the
joint planning process NSs and ARC deemed them useful in achieving TWC objectives; building youths’
life skills, supporting the uptake of HIV prevention messages and/or building the NS capacity. This
activity became a bit muddled in execution because of the multi‐faceted aspects it could potentially take
on. It should be clearly understood and documented where partnerships would add value to the NS and
what would be expected of each partnership.  
Ongoing planning with the NS is a critical process and should be completed for each proposed activity
which will not only end up as a list of prioritized activities but will also help in defining what each activity
looks like. Priorities set by ARC and the NS will be further massaged during community mapping. This
process will provide each country program with more details concerning how TWC will be implemented.
For example, take the town hall meetings. Perhaps the idea of holding meeting falls outside of the
community set of priorities. It would then be incumbent on the NS staff, the TWC volunteers and the
advisor board to come up with a workable solution. The end result may not be a town hall. It may end
up that announcements are made at other community events such as village meetings or RC volunteers
going door to door to inform people of TWC activities. The end goal is to market the project and garner
approval for its implementation.
F.4. Define Indicators and Link Measures to Use
As stated above, each activity should be explicitly linked to project objectives and the TWC goal. A
conceptual framework for the entire project should be developed. It needs to be clear how actual
progress towards reaching each SO will be measured and documented in the M&E Plan. Each NS's M&E
Plan should feed into the overall TWC M&E Plan. This discussion will focus on the ARC document.
Detailed instructions for how to write a complete M&E plan and the components to incorporate can be
found on any donor website, including the USAID TIPS Preparing a Performance Monitoring Plan
(Number 7, 1996). A starting point for ARC and the NSs would be to rethink the intermediate results and
replace the activities listed as intermediate results in the proposal with what effects the project seeks to
achieve. An illustrative results framework is offered in Table 20 which provides some ideas for
structuring future TWC projects. Additional columns could include target numbers or percentages, a
short description of how data will be collected, the data source, who is responsible for collecting the
data. Each indicator should be carefully chosen so that it will be helpful in making informed
programming decisions. Developers should ask such questions as, “What does an enhanced community
look like?” and “What could we identify as being different in an enhanced community as compared to
today?” In choosing capacity development measures NSs could reference the USAID TIPS Measuring
Institutional Capacity (Number 15, 2000).
Table 20: Illustrative TWC Results Framework
Project Goal, Strategic Objectives (SO), Intermediate Results
(IR) and Activities
Indicators
Goal: Increase HIV prevention practices among youth 10‐24
years old
% of youth reached through peer‐to‐peer outreach who
report abstaining
% of youth reached through peer‐to‐peer outreach who
report having one faithful partner
% of youth reached through peer‐to‐peer outreach who
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 46
consistently use condom  
SO1: Strengthen HIV‐prevention life skills of youth 10 to 24
years of age
Life effectiveness scores2
IR1: built correct comprehensive   HIV prevention
knowledge in 80% of youth reached through peer‐to‐peer
activities
% of youth with correct comprehensive HIV prevention
knowledge
Activities: CBI, FUI, ACC  
IR2: improved the uptake of HIV preventive services for
80% of youth reached through peer‐to‐peer outreach
% of youth tested for HIV
Activities: CBI, FUI, ACC  
SO2: Build capacity of National Societies to implement youth
HIV‐prevention projects
Level of institutional self‐reliance
Level of project sustainability
IR1: enhanced NS leadership Leadership index score3
  
Activities: training, mentoring, information exchanges
and tools development
IR2: improved financial management % on‐time complete financial reports
Activities: training, mentoring, information exchanges
and tools development
IR3: improved management of human resources   % staff turnover
Activities: training, mentoring, information exchanges
and tools development
% volunteer turnover
IR4: improved staff technical skills   % on‐time complete project reports
Activities: training, mentoring, information exchanges
and tools development
SO3: Enhance community environment for the adoption of
HIV prevention practices
% of communities mapped with workplan
% of community workplans implemented as planned for the
specified time period
IR: garner broad based support for TWC & RC % of PE, YM and YP parents expressing supportive attitudes of
TWC & RC
Activities: ACC, mass media, CWEs, town hall meetings,
community‐based advisory boards, and partnerships
IR: garner community input towards TWC programming % of planned advisory board with meeting minutes
% of partner MOUs fulfilled
Activities: town hall meetings, community‐based
advisory boards, and partnerships
Even if indicators are not reported on to the donor, additional indicators are needed for sound program
management. The pre‐ and post‐test approach is useful but the approach should be modified to only
collect data really needed. As a stop‐gap measure in the absence of any baseline data, the pre‐ and post‐
test did provide an indication for progress made. If a baseline study were conducted, this leaves one to
ponder the real value of the tests. The information collected in the post‐test is used to determine what
topics need to be included in future FUIs. When avoidable, excess data not used for decision making
should be averted. It would therefore be recommended to continue with the post‐test after CBIs and
determine whether or not FUI post‐tests add value to the information PEs are already supposed to be
collecting.  
A bit of tangential discussion on the pre‐ and post‐test Access database may help improve that bit of
data management and evaluation. The evaluator questions how these comprehensive numbers were
                                                           
2 Resource : http://wilderdom.com/tools/ToolsSummaries.html 3 Resources: http://www.greatsystems.com/leadindex.htm, http://www.dau.mil/doddacm/alei/index.htm
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 47
determined. The report calculations for the comprehensive numbers is not apparent when running them
through the Access database so it is puzzling how overall correct knowledge could be so low when the
individual questions for both the CBI and the FUI which make up the total figures were all near or above
80%. Computations for the composite indicators should be described in the M&E Plan. The reports
should contain additional labels ‐ name of the NS and type of activity for which test results are provided
(CBI or FUI). Another limitation regarding the Access database report format is that the evaluator is
assuming the heading, “pre‐test %” and “post‐test %” indicate the proportion of respondents who
provided correct responses. If this is the case, it would be helpful for this to be explicitly stated in the
report and explained in the M&E Plan. The label “% Increase” would be more accurate as “% Change”.  
Other test‐related ideas to consider are the consequences of not having unique identifiers for individual
test takers and the level of ease by which data can be manipulated and analyzed. The reporting function
for the database is the end reason for its existence. There is the option to export the data into a
spreadsheet which did not work at the time of the evaluation. This would have been very helpful and
reduced cutting and pasting information from the word processing document into a format where data
could be readjusted and studied. Although the project has invested a lot into the Access database, it
may be overkill for what decision makers need. By directly using a spreadsheet for data entry,
management and analysis, NS project staff will gain valuable computer skills. This would be an important
capacity building activity as it is much more likely that they will need to use spreadsheets over Access in
the future. All staff should receive training and mentoring on the filling in spreadsheet and how to
manipulate the data. Instructions should be contained in the TWC Project Binder.
Comparing pre‐ and post‐test data for CBIs and FUIs over time is crude if what the project is attempting
to do is measure longitudinal progress which corresponds to curriculum changes, for example. This is
where the issue of unique identifiers comes in for each person tested. If the NSs and ARC felt it was
important to collect longitudinal data, each person tested should have an identification number to
associate with the tests they take. Perhaps more skilled PEs, or some who volunteer superficially as part
of this pilot could choose one of the groups of YMs they are working with to follow over time. One of
the insights this type of monitoring would supply is a percentage of youth lost to follow‐up after CBIs
and FUIs. It would also impart how knowledge and its self‐reported use grew with each peer‐to‐peer
session.
The post‐tests provided an indication of HIV prevention knowledge gained and applied. Some of the
same indicators could be used for measuring overall progress. Similar proxy indicators should also be
devised for SO2 and SO3 and for each piece of information captured it should be stated in the M&E Plan
how often the data is collected and what decisions could be made using these data. Like with the pre‐ 
and post‐tests currently used, outcome data does not need to be collected at every contact. Enough
data should be provided to give an indication of the efficacy of the activity and determine next steps.  
Regular surveys could be used to collect outcome level data for indicators such as those outlined in
Table 20, depending on the foreseen need of program adjustments. Many projects use full baseline,
mid‐term and final evaluations as the standard by which to collect these level of data but instead of one
mid‐term evaluation the NSs and ARC may decide that smaller annual surveys would be more
appropriate. The sampling framework of the surveys can be structured so project managers will be
better able to determine the source of prevention knowledge or the impetus to test for HIV.  
For future projects the monitoring, baseline and other internal evaluations should be a part of activity
implementation, not an add‐on activity. When PEs are trained on delivering CBIs, they should have a
 ARC TWC Evaluation Composite Report 
True Panacea, LLC  Page 48
thorough training on the M&E Plan, as contained in their TWC Project Binder. RC staff and branch
volunteers who are part of the management committee can all contribute to data collection and
analysis. As stated above, one of the greatest motivating factors for volunteers is seeing the fruits of
their labor. What better way than experiencing it first hand?  
F.5. Document Progress
Great efforts were make during the life of the activity, especially starting in FY08 after which multiple
documents were finalized in an attempt to better describe component pieces of the TWC project and to
document each activity. The evaluators reviewed behavior‐determinant logic models to assist with CBI
and ACC curriculum modifications, guide for CWEs, community councils, town halls and FUIs, the TWC
M&E plan including a logframe for scaling‐up activities, strategies for PE retention and reaching OSY and
HRY.
It would be helpful to have a TWC Project Binder containing all policies, procedures, and materials for
each TWC staff member and volunteer. This should be a reference in both paper and electronic formats
for all management, project staff and branches. As information is updated new pages could replace
outdated ones. Included in the binder should be the strategic framework, activity implementation
documents – such as those already developed, financial management processes and forms, the M&E
plan, etc. By having these information written down and all in one place and all associated staff knowing
that they are responsible for whatever is contained in the TWC Project Binder will facilitate
communication between ARC and the NSs as to what expectations are and what they are agreeing to do.
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 49 
APPENDICES
Appendix A: Evaluation Protocols
AMERICAN RED CROSS  
“TOGETHER WE CAN”
PROGRAM EVALUATION
December 2009 – June 2010
Developed for activities in Guyana, Haiti, Tanzania
Author: Kyung Endres 
Submitted: January 8, 2010 
Revised: March 8, 2010 
RESEARCH
PROTOCOLS
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 50 
This publication was made possible through funding from the U.S. Agency for International Development 
(USAID) to the American Red Cross (ARC) “Together We Can” (TWC) program. This publication is an 
external evaluation of a three country, six year youth HIV prevention program. This publication is funded 
in part through the USAID Cooperative Agreement with ARC for TWC, No. GP0-A-00-04-00005-00 
True Panacea is an independent international consulting firm dedicated to responsible international 
development programming. This document has been authorized by ARC under a contract signed 24 
December 2009 and ARC’s purchase order number 442682. 
The opinions expressed in this document do not necessarily reflect the policies of USAID or ARC. 
True Panacea, LLC 
1445 Otis Place NW Ste 329 
Washington, DC 20010 USA 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 51 
Table of Contents
I. Background .................................................................................................................................... 52
II. Research Objectives....................................................................................................................... 52
III. Evaluation Audience & Use............................................................................................................ 52
IV. Specific Research Topics ................................................................................................................ 52
V. Methods......................................................................................................................................... 55
VI. Ethical Considerations.................................................................................................................... 57
VII. Project Management ..................................................................................................................... 57
VIII. Timetable ....................................................................................................................................... 58
IX. Report Outline ............................................................................................................................... 58
Acronyms
ABY Abstinence Be Faithful Youth 
ARC American Red Cross 
CB Capacity Building 
CC Community Council 
CL Community Leader 
DC District of Columbia 
FG Focus Group 
FM Field Manager 
HIV Human immuno-deficiency virus 
HQ Headquarters 
HRC Haiti Red Cross 
IDI Individual Interview 
NS National Society 
PC Program Coordinator 
PE Peer Educator 
RC Red Cross 
STI Sexually Transmitted Infection 
TWC Together We Can 
USAID United States Agency for International Development 
YM Youth Multiplier 
YP Youth Participant 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 52 
I. Background  
A. Description of project
1. TWC logic model
II. II. Research Objectives
A. To evaluate outreach affecting the community environment in TWC implementation communities
B. To evaluate the efficacy of TWC interventions in promoting abstinence, fidelity, condom use and uptake
of reproductive health services towards reducing HIV transmission and unintended pregnancy in youth
C. To evaluate the efficacy of TWC’s capacity building (CB) of National Societies in Guyana, Haiti and
Tanzania to implement youth HIV prevention projects and development projects in general
III. Evaluation Audience & Use
A. ARC headquarters ABY program stakeholders
1. improve TWC program models using best practices, lessons learned and recommendations
from final report
2. improve NS capacity building efforts and NS program implementation capacity using best
practices, lessons learned and recommendations from final report
B. NS staff program managers, health directors and leadership
1. improve NS project implementation and management
IV. Specific Research Topics
A. To evaluate outreach affecting the community environment in TWC implementation communities
1. describe environmental and cultural elements of the project
2. discuss informant perceptions of contextual influences on youth in contracting HIV
Table 21 Understanding the Operating Environment: Key Respondents
1. Description 2. Outcomes 
Geographic Targeting Documentation, ARC HQ, ARC 
Country, NS Program 
Coordinator (NS PC), NS 
Assistant PC, NS Health 
Director, NS Field Managers 
(FM) 
ARC Country, NS 
Beneficiary Targeting Documentation, ARC HQ, ARC 
Country, NS PC, NS Health 
Director, NS Assistant PC, NS 
FM 
ARC Country, NS, PE, YM, YP, 
parents, Community Council 
Environmental & Cultural 
Context 
Documentation, ARC HQ, ARC 
Country, NS Health Director, NS 
PC, NS Assistant, NS FM 
NS FM, PE, YM, YP, parents, 
Community Council 
Table 22 Understanding the Operating Environment: Data Sought
1. Description 2. Outcomes 
Geographic Targeting Describe geographic location of 
project; national, regional, 
district, sites 
Discuss appropriateness of 
geographic location of activities 
Beneficiary Targeting Describe the project participants 
by age group, gender, risk 
category 
Discuss appropriateness of 
beneficiaries for TWC activities 
Environmental & Cultural Describe local risk factors to Discuss appropriateness of TWC 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 53 
1. Description 2. Outcomes 
Context youth for HIV transmission activities in addressing HIV 
prevention in youth 
B. To evaluate the efficacy of TWC interventions in promoting abstinence, fidelity, condom use and uptake
of reproductive health services towards reducing HIV transmission and unintended pregnancy in youth
1. describe TWC interventions  
2. discuss how activities contributed to changes in youth knowledge regarding sexuality and
STI/HIV and in community and parental knowledge of risk factors for youth
3. discuss the attitudes and behaviors of youth to avoid HIV and of parents and communities
to support their children’s behavior choices (abstinence, fidelity, partner reduction,
protected sex, use of reproductive health services)
4. discuss changes in curriculum delivery and target audiences over time
Table 23 Understanding TWC Project Activities: Key Respondents
1. Description 2. Knowledge 3. Attitudes & Behavior 
Curriculum 
Based 
Interventions 
Documentation, ARC HQ, 
ARC Country 
Documentation, NS PC, NS 
FM, PE, YM, YP 
PE, YM, YP 
Follow Up 
Interventions 
(FUIs) 
Documentation, ARC HQ, 
ARC Country 
Documentation, NS PC, NS 
FM, PE, YM, 
PE, YM 
Peer 
Educator 
training and 
support 
Documentation, ARC HQ, 
ARC Country 
Documentation, NS PC, NS 
FM, PE, YM, 
PE, YM 
Youth Clubs Documentation, ARC HQ, 
ARC Country 
Documentation, NS PC, NS 
FM, PE, YM, YP 
PE, YM, YP 
Mass Media Documentation, ARC HQ, 
ARC Country 
Documentation, NS PC, NS 
FM, PE, YM, YP 
PE, YM, YP, parents 
Referrals and 
condom 
distribution 
Documentation, ARC HQ, 
ARC Country 
Documentation, NS PC, NS 
FM, PE, YM, YP, Partner 
YM, YP, Partner 
Parental 
Involvement 
Documentation, ARC HQ, 
ARC Country 
Documentation, NS PC, NS 
FM, PE, YM, Parents 
PE, YM, YP, Parents 
Community 
Wide Events 
(general 
diffusion) 
Documentation, ARC HQ, 
ARC Country 
Documentation, NS PC, NS 
FM, Parents, Community 
Council, PE, YM, YP 
PE, YM, YP, parents, CC 
members 
Community 
Wide Events 
(participative) 
Documentation, ARC HQ, 
ARC Country 
Documentation, NS PC, NS 
FM, Parents, Community 
Council, PE, YM, YP 
PE, YM, YP, parents, CC 
members 
Town Hall 
Meeting 
Documentation, ARC HQ, 
ARC Country 
Documentation, NS FM, 
Community Council, Town 
Hall Participants 
Town Hall Participants 
Community 
Council 
Documentation, ARC HQ, 
ARC Country 
Documentation, ARC 
Country, NS PC, NS FM, 
CC members 
CC members 
a. USAID Washington & Missions will be questioned for outcome and impact comparisons
to other projects + management efficiency
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 54 
Table 24 Describing TWC Project Activities: Data Sought
1. Description 2. Knowledge 3. Attitudes & Behavior 
Curriculum 
Based 
Interventions 
Describe activity; process 
of cascade training, number 
of people trained, revisions 
to curriculum, homework 
assignments, pre and post 
test scores 
Discuss knowledge gained 
through activity 
Discuss activity influences 
on participant attitudes & 
behavior 
Follow Up 
Interventions 
(FUIs) 
Describe activity; who 
attends, meeting 
frequency, meeting 
activities, knowledge 
gained 
Discuss knowledge gained 
through activity 
Discuss activity influences 
on participant attitudes & 
behavior 
Peer 
Educator 
training and 
support 
Describe activity; media 
activities, population 
reached, messages 
disseminated 
Discuss knowledge gained 
through activity 
Discuss activity influenced 
on participant attitudes & 
behavior 
Youth Clubs Describe activity; who 
attends, meeting frequency, 
meeting activities, 
knowledge gained 
Discuss knowledge gained 
through activity 
Discuss activity influences 
on participant attitudes & 
behavior 
Mass Media Describe activity; media 
activities, population 
reached, messages 
disseminated 
Discuss knowledge gained 
through activity 
Discuss activity influences 
on participant attitudes & 
behavior 
Referrals and 
condom 
distribution 
Describe activity; 
community mapping & use 
of resource map 
Discuss knowledge gained 
through activity 
Discuss activity influences 
on participant attitudes & 
behavior 
Parental 
Involvement 
Describe activity; parental 
activities (organized and 
self-initiated), 
communication with 
children, 
Discuss knowledge gained 
through activity 
Discuss activity influences 
on participant attitudes & 
behavior 
Community 
Wide Events 
(general 
diffusion) 
Describe activity; 
attendants, messages, 
frequency 
Discuss knowledge gained 
through activity 
Discuss activity influences 
on participant attitudes & 
behavior 
Community 
Wide Events 
(participative) 
Describe activity; 
attendants, messages, 
frequency 
Discuss knowledge gained 
through activity 
Discuss activity influences 
on participant attitudes & 
behavior 
Town Hall 
Meeting 
Describe activity; 
attendants, frequency, 
discussion points, key 
concerns 
Discuss knowledge gained 
through activity 
Discuss activity influences 
on participant attitudes & 
behavior 
Community 
Council 
Describe activity; 
attendants, frequency of 
meetings, decisions and 
action taken, leadership 
characteristics 
Discuss knowledge gained 
through activity 
Discuss activity influences 
on participant attitudes & 
behavior 
C. To evaluate the efficacy of TWC’s capacity building (CB) of National Societies in Guyana, Haiti and
Tanzania to implement youth HIV prevention projects and development projects in general
1. describe capacity building activities
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 55 
2. discuss the success levels of each type of activity   and why or why it was not optimal in
ensuring the sustainability of NS in administering and managing youth HIV prevention
projects
Table 25 Understanding NS Organizational Development: Key Respondents
1. Description 2. Outcomes (Planning, Organizing, 
Staffing, Leading, Monitoring, 
Motivating)
TWC 
supported 
CB Activities
ARC HQ, ARC Country, NS PC, NS FM, 
NS Branch, PE 
ARC HQ, ARC Country, NS PC, NS FM, 
NS Branch 
Intra-Country 
Fertilization 
ARC HQ, ARC Country, NS PC, NS FM, 
NS Branch 
ARC HQ, ARC Country, NS PC, NS FM, 
NS Branch 
Cross￾Country 
Fertilization 
ARC HQ, ARC Country, NS PC, NS FM, 
NS Branch 
ARC HQ, ARC Country, NS PC, NS FM, 
NS Branch 
Table 26 Understanding NS Organizational Development: Data Sought
1. Description 2. Outcomes (Planning, Organizing, 
Staffing, Leading, Monitoring, 
Motivating)
TWC 
supported 
CB Activities
Describe activities; country visits, 
information shared, source of information, 
frequency of visits 
Discuss how learned skills have been used
Intra-Country 
Fertilization 
Describe activities; NS branches, outreach 
and cooperative efforts with other relevant 
organizations/other Track 1.0 
implementers, Red Cross information 
sharing events, information shared 
Discuss how learned skills have been used
Cross￾Country 
Fertilization 
Describe activities, regional information 
sharing events, information shared 
Discuss how learned skills have been used
V. Methods
A. Document Review
1. Monitoring data
2. TWC semi‐annual and annual reports
3. other TWC program documents
4. literature review of ABY programs & organizational management
B. Sampling
1. site selection (2 sites per country)
a. 1 urban or peri‐urban per country
b. 1 rural
c. research team able to travel to site in one day
d. new curriculum has been utilized
e. have had access to community‐wide events
f. beneficiaries are willing to participate in the evaluation
g. some level of community participation ideal
2. respondent selection
a. ARC HQ (recommendations from Lindsay helpful here)
b. ARC Country Office (Head of Programs, Health delegate, others?)
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 56 
c. National Society (PC & Assistant, FM, Health Director)
d. Youth Beneficiaries (PE, YM, YP)
e. Community Members (Community Council (CC), Community Leaders (CL = nurse,
teacher, government official, traditional leader), Town Hall Participants, Parents)
f. USAID Washington & Missions
C. Primary Data Collection
1. Semi‐structured focus groups  
a. 6 participants per group unless the group size is smaller (example: community council
may have only 3 members)
b. Estimated 1 hour 30 minutes in duration
c. Person participated in the program as appropriate for the focus group per Table 27
d. Youth participants (PE, TM, YP) with the maturity to respond to questions in a group
setting of mixed ages
2. Key informant interviews
a. 1 participant per interview to follow‐up on gaps in information from focus groups
b. Estimated 1 hour in duration
c. Key respondents will be selected from the focus group members by the interviewer who
will make an appointment for individual interviews on day 3
3. Questionnaires
a. ARC HQ (recommendations from Lindsay helpful here)
b. ARC Country Office (Head of Programs, Health delegate, others?)
c. National Society (PC & Assistant, FM, Health Director)
d. PE
e. YM
f. YP
g. Community Council (CC),  
h. Community Leaders (CL = nurse, teacher, government official, traditional leader)
i. Town Hall Participants
j. Parents
k. USAID Washington & Missions
4. Data collection calendar per site
a. 2 sites
b. 6 days per site (1 day travel to site + 3 days data collection + 1 day return travel + 1 day
data entry & analysis)
D. Data Management
1. data collection using paper forms
2. data entry into digital format to improve legibility  
E. Data Analysis
1. preliminary findings for debrief without in‐depth analysis
a. Evaluation objectives
b. Evaluation methods
c. Site 1 preliminary findings
d. Site 2 preliminary findings
e. Next steps
2. organize data by topic in word processing program
3. discuss findings by topic per Tables 2, 4 and 6
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 57 
Table 27 Illustrative Interview Schedule for One Site
Start Day 1 Day 2 Day 3
Time PI RA1 RA2 PI RA1 RA2 PI RA1 RA2 
0800 YM FG PE FG CC FG1 YP FG Parent YM IDI- PE IDI-1 CL IDI-1
0830 male male male FG 1 
0900 male debrief debrief debrief 
0930 debrief debrief debrief debrief debrief debrief 
1000 YM IDI- PE IDI-2 CL IDI-2
1030 2 
1100 debrief debrief debrief 
1130 YM FG PE FG Town YP FG Parent YP IDI-1 Parent CC IDI 
1200 female female Hall female FG IDI-1 
1230 FG1 female debrief debrief debrief 
1300 lunch lunch lunch lunch lunch lunch lunch lunch lunch 
1330 
1400 debrief debrief debrief debrief debrief debrief YP IDI-2 Parent Gov’t 
1430 IDI-2 IDI 
1500 debrief debrief debrief 
1530 
1600 travel from site travel from site travel from site 
1630 
1700 data entry data entry data entry 
VI. Ethical Considerations
A. Informed consent
1. written statement with verbal agreement
B. Privacy protection procedures for data collection and management
1. no names, use codes to represent type of respondent
VII. Project Management
A. Staffing
1. principle investigator
2. 1 DC‐based researcher
3. 2 research assistants per country
4. 1 translator per country
5. 3 note takers (2 if translator can also take notes)
6. local staff as logistician
B. Materials
1. questionnaires
2. laptop computers for 2 in‐country research assistants
3. flip charts & markers
4. printer
5. training room
C. Data collection logistics
1. organize respondents for interviews
2. meeting space for interviews
3. transportation to and from site visits
4. lodging in field for principle investigator, 2 research assistants, translator, RC staff, driver
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 58 
VIII. Timetable
A. See work plan for details
1. Table 28 illustrates how time in country will be spent each day
Table 28 Evaluation Activities in Country
Day No. Activity 
1 a) Individual Interviews with RC staff 
b) Finalize travel and sub-contractor logistics 
2 c) Train research assistants and RC staff 
3 d) Translate field guide questionnaires 
4 e) Field test field guides 
5 f) Revise field guides 
g) Reproduce field guides 
6-14 h) Primary data collection 
15-17 i) Preliminary data analysis 
j) RC staff interviews & follow-up 
k) USAID meeting 
18 l) Consultant debrief with RC 
IX. Report Outline (65 PAGES)
A. Executive Summary – 1 PAGE
B. Introduction – 3 PAGES
1. Describe Country Context & local HIV history
2. TWC conceptual framework
3. TWC program description
C. Evaluation Objectives – 1 PAGE
1. To evaluate outreach affecting the community environment in TWC implementation
communities
2. To evaluate the efficacy of TWC interventions in promoting abstinence, fidelity, condom use
and uptake of reproductive health services towards reducing HIV transmission and
unintended pregnancy in youth
3. To evaluate the efficacy of TWC’s capacity building (CB) of National Societies in Guyana,
Haiti and Tanzania to implement youth HIV prevention projects and development projects in
general
D. Evaluation Methodologies – 2 PAGES
1. Site Selection and Criteria
2. Respondent Sample and Criteria
3. Data Collection
4. Data Analysis
a. Delivered per protocols and discussion guides delivered previously to ARC
E. Program Description/Results/Best Practices/Lessons Learned/Recommendations
1. Targeting – 5 PAGES
a. Geographic
b. Beneficiary
c. Socio‐Cultural
2. Activities by Target Group
a. General Population – 3 PAGES
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 59 
i. Town Hall Meetings
ii. Garner Community Support
iii. Community‐Wide Events (general diffusion)
iv. Mass Media Campaign
b. Stakeholder/Community Leadership – 4 PAGES
i. Community Council Support (volunteer, financial)
ii. Competence Building, Training & Support (Coordination)
c. Stakeholder/Partner Organizations
i. Garner Community Support
ii. Competence Building, Training & Support (Coordination)
iii. Complementary Coverage
iv. Referrals to other health & social services
v. Condom distribution
d. Guardian/Schools (Administration & Teachers) – 6 PAGES
i. Garner Community Support (volunteer, financial)
ii. Competence Building, Training & Support (Coordination)
e. Guardian/Youth Guardians
i. Garner Community Support (volunteer, financial)
ii. Parental Involvement
f. Youth/PE – 20 PAGES
i. Garner Community Support
ii. Competence Building, Training & Support
iii. Intra‐Country Fertilization
iv. Cross‐Country Fertilization
v. Curriculum‐Based Interventions
vi. Follow‐up Interventions
vii. Referrals to other health & social services
viii. Condom distribution
ix. CWE (general diffusion)  
x. CWE (interpersonal sharing)  
xi. Youth Clubs
g. Youth/YM  
i. Garner Community Support
ii. Competence Building, Training & Support
iii. Curriculum‐Based Interventions
iv. Follow‐up Interventions
v. Referrals to other health & social services
vi. Condom distribution
vii. CWE (interpersonal sharing)  
viii. Youth Clubs
h. Youth/YP (Ruth/Pamela‐coding)
i. Curriculum‐Based Interventions
ii. Follow‐up Interventions
iii. Referrals to other health & social services
iv. Condom distribution
v. CWE (interpersonal sharing)  
vi. Youth Clubs
3. National Society Competency Built – 20 PAGES
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix A: Page 60 
a. Program Management
i. Program planning  
ii. Sustainability planning & implementation
iii. Staff & volunteer recruitment, training, retention, management
iv. M&E
v. Budgeting & financial management  
b. Technical Knowledge
i. BCC for HIV prevention
ii. Community mobilization & organization
iii. HIV prevention‐specific programming
F. Appendices
1. list of interviews
2. dissemination strategies
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix B: Page 61
Appendix B: Example of Pre‐ and Post‐Test Report
Analysis by Question Surveys from 10‐May‐09 to 28‐Apr‐10
Crosstab by Age Cohort
Question 5. Do you think a healthy-looking person can be infected with HIV, the virus that causes 
AIDS?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 65 84 29%
Age 15-19 829/827 71 81 14%
Age 20-24 185/171 75 88 17%
Age Out of Range 36/28 78 75 -4%
Question 6. Can people protect themselves from the AIDS virus by using a condom correctly every
 time they have sex?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 82 94 15%
Age 15-19 829/827 84 93 11%
Age 20-24 185/171 89 96 8%
Age Out of Range 36/28 83 86 4%
Question 7. Can a person get the AIDS virus from mosquito bites?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 76 87 14%
Age 15-19 829/827 82 90 10%
Age 20-24 185/171 75 93 24%
Age Out of Range 36/28 89 100 12%
Question 8. Can people protect themselves from the AIDS virus by having one uninfected faithful 
sex partner?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 66 82 24%
Age 15-19 829/827 69 83 20%
Age 20-24 185/171 73 86 18%
Age Out of Range 36/28 72 71 -1%
Question 9. Can people protect themselves from the AIDS virus by not having sexual intercourse?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 74 87 18%
Age 15-19 829/827 79 89 13%
Age 20-24 185/171 86 94 9%
Age Out of Range 36/28 89 82 -8%
Thursday, July 15, 2010 Question and Crosstab by Age Cohort Page 1 of 4
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix B: Page 62 
Question 10. Can a person get AIDS because of witchcraft?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 75 85 13%
Age 15-19 829/827 84 89 6%
Age 20-24 185/171 78 92 18%
Age Out of Range 36/28 75 96 28%
Question 11. Can a person get AIDS virus by getting injections with a needle that was already 
used by someone else?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 86 93 8%
Age 15-19 829/827 88 95 8%
Age 20-24 185/171 90 96 7%
Age Out of Range 36/28 83 89 7%
Question 12. Can the AIDS virus be transmitted from mother to child?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 76 80 5%
Age 15-19 829/827 69 72 4%
Age 20-24 185/171 56 63 12%
Age Out of Range 36/28 64 68 6%
Question 13. If a Female teacher has the AIDS virus but is not sick, should she be allowed to 
continue teaching in school?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 67 88 31%
Age 15-19 829/827 77 89 16%
Age 20-24 185/171 88 94 7%
Age Out of Range 36/28 64 96 50%
Question 14. If you knew that a shopkeeper or food seller had the AIDS virus, would you buy fresh 
vegetables from him or her?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 64 84 31%
Age 15-19 829/827 75 86 15%
Age 20-24 185/171 82 95 16%
Age Out of Range 36/28 61 89 46%
Thursday, July 15, 2010 Question and Crosstab by Age Cohort Page 2 of 4
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix B: Page 63 
Question 15. If a member of your family became sick with the AIDS virus, would you be willing to 
care for him or her in your household?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 83 93 12%
Age 15-19 829/827 90 94 4%
Age 20-24 185/171 94 96 2%
Age Out of Range 36/28 89 93 4%
Question 16. If a member of your family became infected with the virus that causes AIDS, would 
you want it to remain a secret?
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 68 81 19%
Age 15-19 829/827 71 82 15%
Age 20-24 185/171 84 96 14%
Age Out of Range 36/28 72 82 14%
Question 17. Postponing sex until later is a wise decision for teenagers.
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 78 92 18%
Age 15-19 829/827 86 95 10%
Age 20-24 185/171 90 95 6%
Age Out of Range 36/28 83 93 12%
Question 18. A girl has the right to choose not to have sex with her boyfriend, even if he wants to.
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 74 88 19%
Age 15-19 829/827 82 91 11%
Age 20-24 185/171 84 95 13%
Age Out of Range 36/28 78 75 -4%
Question 19. By the time they are 16, all boys should have had sex at least once.
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 50 67 34%
Age 15-19 829/827 55 65 18%
Age 20-24 185/171 57 71 25%
Age Out of Range 36/28 42 54 29%
Question 20. If you're having sex, condoms are a good way to prevent pregnancy, HIV, and 
sexually transmitted diseases.
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 1433/1376 74 88 19%
Age 15-19 829/827 79 89 13%
Age 20-24 185/171 91 93 2%
Age Out of Range 36/28 78 96 23%
Thursday, July 15, 2010 Question and Crosstab by Age Cohort Page 3 of 4
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix B: Page 64 
Analysis by Question Surveys from 10‐May‐09 to 28‐Apr‐10
Crosstab by Age Cohort‐ Girls
Question 21. (girls) If my boyfriend asks me to have sex with him and I don't want to, I am 
confident I will not have sex with him.
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 760/742 55 73 33%
Age 15-19 371/379 62 71 15%
Age 20-24 70/62 66 65 -2%
Age Out of Range 17/10 41 60 46%
Question 22. (girls) If I were to have sex, I could convince my partner to use a condom.
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 760/742 78 88 13%
Age 15-19 371/379 82 94 15%
Age 20-24 70/62 90 98 9%
Age Out of Range 17/10 88 100 14%
Analysis by Question Surveys from 10‐May‐09 to 28‐Apr‐10
Crosstab by Age Cohort‐ Boys
Question 23. (boys) If my girlfriend wants me to have sex with her, I will say "Yes" because if I 
don't my friends will say that I am just a small boy.
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 673/634 47 56 19%
Age 15-19 458/448 56 59 5%
Age 20-24 115/109 69 76 10%
Age Out of Range 19/18 37 72 95%
Question 24. (boys) If I were to have sex, I could convince my partner to let me use a condom.
Total Number 
Age Cohort Pre-test % Post-test % % Increase
Age 10-14 673/634 82 90 10%
Age 15-19 458/448 85 90 6%
Age 20-24 115/109 89 94 6%
Age Out of Range 19/18 79 89 13%
Thursday, July 15, 2010 Question and Crosstab by Age Cohort Page 3 of 4
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 65
Appendix C: Discussion Guides
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 66 
USAID Washington Discussion Guide 
Interviewer:   Guide No.:  
Respondent:   Date:  
Country:   Time In:  
City/Town:     Time Out:  
I. Introduction 
* Describe evaluation content and process
* Informed consent
* Note setting
II. Respondent Characteristics 
A. What has your role been with the TWC program?
1. When did you begin to work on the TWC program?  
2. What were you responsible for? What is your role?
3. Who did you work most closely with at ARC?
4. What was your level of involvement?
5. Have you ever visited any of the implementation sites? Where did you go? What did you
see? What were your impressions?
III. Targeting 
A. How appropriate do you feel the selected countries and regions within those countries were?
B. How well do you feel that the appropriate beneficiaries have been targeted?
C. What external factors played a role in how well the project rolled out?  
1. Please describe the political context of the implementation of the TWC program in terms of
PEPFAR start‐up. How has the operating environment changed and what influence has that
had on program implementation?
2. What affect did the operating environment have on the decision to withdraw the originally
planned baseline study? How do you think the lack of the USAID centrally‐managed baseline
study impacted roll‐out?
IV. Monitoring & Reporting 
A. Given that no baseline existed, how have you monitored progress?
1. What feedback have you provided and to whom?
2. How have the USAID centrally‐funded program evaluations assisted in program
management? How well do you think ARC has responded to Measure Evaluation’s
recommendations?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 67 
a. Measure Evaluation. (March 2007) Field Assessment of Emergency Plan Centrally‐
Funded HIV Prevention Programs for Youth.  
i. Use a clear programmatic framework that specifies how resources are applied to
activities with key populations to achieve outcomes and potential impacts (a logic
model) at the planning phase to ensure that program strategies are directly related
to program goals and objectives.
ii. Conduct a needs assessment to improve the relevance and potential impact of the
program activities.
iii. Secure community buy‐in for program activities.
b. Measure Evaluation. (May 2009) Curricula Review of Emergency Plan Centrally‐Funded
HIV Prevention Programs for Youth.
c. GHTech. (July 2009) Preventing HIV Infections in Youth Aged 10‐24 Years: Report of the
USAID Track 1.0 ABY Program Review.
V. Tiered Programming 
A. Both the Measure Evaluation curricula review and GH Tech ABY program review included the
TWC program in their studies. What indication did you get from the evaluation teams as to the
efficacy of the ARC to NS tiered system of implementation?
1. What is your understanding of how ARC works with the NRCS?
2. How comfortable do you feel that ARC has built the capacity of the National Red Cross
Societies?
3. What would you recommend to improve capacity building of NS and program sustainability?
VI. TWC Activity Impact 
  
A. Given the nature of Track 1.0 programs and OGAC’s original intention to use centrally funded
projects as models by making comparisons across organizations and countries, how would you
grade the TWC projects in terms of meeting their objectives?
1. How effective do you think ARC has been at disseminating messages? Improving
knowledge? Changing attitudes? Changing behaviors? Building capacity?
2. How would you compare the ARC program to other Track 1.0 ABY projects in terms of
meeting their objectives? How inclined would you be in supporting ARC‐ type strategies to
reducing HIV prevalence in the future?
3. What are your impressions of project sustainability?
4. What could improve the project? What do you think USAID should have done to improve
program outcomes and impacts?
5. What did you like about the program?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 68 
USAID Mission Discussion Guide
I. Introduction 
* Describe evaluation content and process
* Informed consent
* Note setting
II. Respondent Characteristics 
A. What has your role been with the TWC program?
1. When did you begin to work on the TWC program? Are you still working with TWC?
2. What were you responsible for? What is your role? What was your level of involvement?
3. Who did you work most closely with at ARC?
4. Have you ever visited any of the implementation sites? Where did you go? What did you
see? What were your impressions?
5. What does the Mission relationship look like with USAID/Washington? How do you work
together to manage the TWC program?
III. Targeting 
A. How appropriate do you feel this country and regions within this country were?
B. How well do you feel that the appropriate beneficiaries have been targeted?
C. What external factors played a role in how well the project rolled out?  
1. Please describe the political context of the implementation of the TWC program in terms of
PEPFAR start‐up. How has the operating environment changed and what influence has that
had on program implementation?
2. What affect did the operating environment have on the decision to withdraw the originally
planned baseline study? How do you think the lack of the USAID centrally‐managed baseline
study impacted roll‐out?
IV. Capacity Building & Program Management Activities 
A. What would your recommendations be to improve program management?
B. What cross‐country and in‐country fertilization activities do you think have been effective within
the RC program offices and between Track 1.0 partners.
1. Do you have recommendations for how these objectives could have been reached more
efficiently or thoroughly?
2. How well do you feel that collaboration between Track 1.0 and other ABY projects has been
handled? How could collaboration been improved?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 69 
C. What is your understanding of the Red Cross – National Society relationship?
1. How well do you feel that ARC has built the capacity of the National Red Cross Societies?
2. Do you think this has been a sustainable approach to involving additional local organizations
in PEPFAR?
D. How well has ARC been providing timely, accurate and useful reports?
1. How do you monitor the program?
2. What feedback have you provided and to whom?
3. In your opinion, what is the purpose of the USAID centrally funded evaluations and how
successful have they been?
V. Behavior Change Activities 
A. List the activities you used in your country to enhance the community environment
a. Community Wide Events (general diffusion)     g. Curriculum‐based program
b. Community Wide Events (inter‐personal)     h. Follow‐up intervention
c. Mass Media   i. Condom distribution
d. Community involvement     j. Youth Clubs
e. Town hall meetings     k. Other
f. Referrals to other services      
1. Have you attended any community‐based activities for TWC? Explain.
2. What were your impressions of them?
3. What recommendations would you provide to improve the community environment for
uptake of TWC messages? (probe on each type of activity)
B. List the activities you used in your country to strengthen youths’ life skills
1. Have you reviewed or attended any of the youth‐focused activities?
2. What are your impressions of the work being done? (probe on each type of activity)
3. What recommendations would you provide to strengthen youths’ life skills to prevent HIV
and unintended pregnancy?
VI. General Impressions 
A. How effective do you think ARC has been at disseminating messages? Improving knowledge?
Changing attitudes? Changing behaviors?
1. How would you compare the ARC program to other Track 1.0 ABY projects in terms of
meeting their objectives?
2. What do you think ARC should have done to improve program outcomes and impacts?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 70 
3. What do you think USAID should have done to improve program outcomes and impacts?
4. What did you like about the program?
5. What would you change?
6. What are your impressions of project sustainability?
7. How inclined would you be to supporting TWC strategies to reducing HIV prevalence in the
future?
B. Do you have any questions or comments?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 71 
American Red Cross HQ Discussion Guide 
Interviewer:   Guide No.:  
Respondent:   Date:  
Country:   Time In:  
City/Town:     Time Out:  
I. Introduction 
* Describe evaluation content and process
* Informed consent
* Note setting
II. Respondent Characteristics 
A. What has your role been with the TWC program?
1. When did you begin to work on the TWC program?  
2. Are you still working with the TWC program? If not, when did you finish?
3. What were you responsible for?
4. To what degree did you interact with the donor?
a. Describe your relationship to USAID/Washington? How was USAID involved in the TWC
program? How did that differ from the original expectations?
b. How did USAID’s involvement levels affect the usefulness of the program baseline
studies and evaluations?
5. What assistance did you offer the field? To whom?  
a. How frequently did you interact with the field?
b. What was the nature of your interactions?
c. Did you ever make any visits to the field? Where did you go? When? What struck you?
B. What resources do you believe ARC brings to a program like the TWC?
1. How could the program have been better structured to reach its objectives?
III. Program Implementation 
* determine whether or not changes from the original plan were intentional based on monitoring or
reactionary to which environmental situations
* Use follow‐up questions appropriate to their current and past roles
A. What changes to the program did you influence? What do you feel were necessary changes
from the original plan to the actual program implementation? (Do you believe that the TWC
program was implemented in all three countries with reasonable fidelity to the original plan?)
Why did these changes need to occur?
B. Is there a consensus that the program was implemented in the right countries?  Right
communities? Where would you recommend the project have been implemented?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 72 
1. How did ARC choose which countries to work in?
2. Define “high‐risk groups.”  What were your strategies for working with them?
3. How does ARC promote gender equity in these TWC programs?
4. How could beneficiary and geographic targeting been better?
C. What was the environmental context that influenced how, when and where the program was
implemented? Please describe the political context of the implementation of the TWC program
with USAID, within ARC, with NS’s, with host country governments.
1. Describe any impediments within the operating environment to program implementation.
2. Describe any catalysts within the operating environment to program implementation.
D. Please discuss the capacity building for the National RC’s.  What was effective?  What was
lacking? How could it have been improved?
1. Management (Is the leadership in place strong enough and willing to continue?)
2. Governance
3. Policy development
4. Organizational development tools
5. Planning
6. Volunteer management
7. Technical assistance  
8. Participatory methodologies
9. Program monitoring
E. What was ARC HQ’s role in ensuring that information was shared between national branches?
1. Describe ARC and the Federation’s effort to work with the three Red Cross National
Societies to share lessons learned and facilitate regional youth and HIV/AIDS networks that
include both Red Cross and non‐Red Cross actors.  
2. How did ARC’s staff work with the Federation’s staff to facilitate branch exchanges so that
stronger branches shared lessons learned and best practices with their peers?
3. Did the country programs have the right leaders?  Did they have effective and locally
appropriate materials and activities?
4. What cross‐country fertilization efforts were carried out?  How many and how effective
were they?
a. What happened to the Red Cross National Youth HIV Prevention Task Force & 
Community and Social Mobilization listserve? 
b. Would the inclusion of the these activities been fruitful? 
5. How could information sharing have been improved? 
F. What are your impressions of the efficacy of the activities that were implemented?
1. Parental Involvement
a. How would you describe the perfectly involved parent?
2. Community Wide Events (general diffusion)
a. What general diffusion community‐wide events has ARC promoted?
3. Community Wide Events (participative)
a. What interactive community‐wide events has ARC promoted?
4. Town Hall Meeting
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 73 
a. How has ARC promoted town hall meetings?
5. Community Council
a. What resources has ARC provided to promote the development and management of
community councils?
6. Curriculum Based Interventions
a. Where did the original TWC curriculum come from? Was the TWC program pilot tested?  
In all three countries?  What was the outcome?
b. How do you feel about the current curriculum being used?
c. How did you determine that it wasn’t effective? What were the major problems? How
did you react to the problems identified?
7. Follow Up Interventions
a. What was the impetus for FUIs? What were you trying to achieve with FUIs?
b. Did you achieve what you had intended to?
8. Peer Educator support
a. Describe the ideal peer support scenario that ARC has promoted.
9. Youth Clubs
a. What is the ARC strategy for creating or using existing youth clubs?
10. Mass Media
a. What tools has ARC provided for the use of mass media outreach and education?
11. Referrals
a. Describe the ARC model for referrals.
12. Condom distribution
a. How has ARC been involved in condom distribution?
13. Did you achieve what you had intended to?
14. How did you determine that it wasn’t effective?
15. What were the major problems?
16. How could the project have been improved?
G. Describe your role in program monitoring?
1. What and how did you provide feedback to the field offices?
2. How could monitoring be improved?
H. Did you receive any feedback from USAID?
1. How was USAID involved in the TWC program? How did that differ from the original
expectations?
2. How did this affect the usefulness of the program evaluations and baseline studies?
3. How could the ARC’s relationship to USAID have been improved?
IV. Resource Management 
A. How did changes in the operating environment and program affect resource management?
1. How were resources managed for the program?
2. Where could money have been used more wisely?
3. Did other programs suffer because funding or resources were diverted to TWC?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 74 
V. TWC Sustainability 
A. Will there be long‐term funding?
1. Is there evidence that funding from other (e.g. European‐based) Red Cross Societies
currently working in the region will be contributed?
2. Is there evidence that external sources of funding (e.g., multilateral, international sources)
will contribute long‐term?
3. Have the local TWC programs shown evidence of being able to generate some income on
their own?  
4. What is the fundraising capacity of each program?
5. How is the outreach to recipients of the CDC‐funded “Hinterland Initiative” progressing?  
Does that appear to be a viable source of support for continued TWC outreach?
B. Do you believe the TWC program is sustainable in all three countries?
1. What does sustainability mean in the context of TWC?
C. What has ARC done to ensure sustainability?
D. Is the leadership in place strong enough and willing to continue?
E. Please discuss community and local political support for the continuation of the program in
each country.
F. What are the impediments to long‐term sustainability?
G. What are the lessons that have been learned about building sustainable programs?
H. Do you believe the TWC program is sustainable in all three countries?
VI. TWC Activity Impact 
A. What lessons have been learned about working on HIV/AIDS issues specifically with young
people?
1. What would you do differently? / What do you think needed to be changed but was not?
(Was TWC implemented too quickly?  Too slowly?)
2. How would you use abstinence and fidelity in future programs?
B. What would you list as ARC best practices? What are the positive results that you see coming
from the program? Why is this positive?
1. What lessons have been learned about working on HIV/AIDS issues specifically with young
people?
2. How would you use abstinence and fidelity in future programs?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 75 
American Red Cross Field Office Discussion Guide
Interviewer:     Guide No.:  
Respondent:   Date:  
Country:   Time In:  
City/Town:     Time Out:  
I. Introduction 
* Describe evaluation content and process
* Informed consent
* Note setting
II. Respondent Characteristics 
A. What has your role been with the TWC program?
1. When did you begin to work on the TWC program?  
2. What were you responsible for?  
a. What makes a good program coordinator?
3. Who did you work most closely with in the NS?
a. What makes a good field manager?
III. Capacity Building & Targeting 
* determine whether or not changes from the original plan were intentional based on monitoring or
reactionary to which environmental situations
* Use follow‐up questions appropriate to their current and past roles
A. How do you fit into the overall management structure of the program? (Please describe the
quality of the management of the TWC program.) How could it be improved?
1. Organizational structure, including relationship with national society and the Federation
2. Staffing and training
3. Resources, including money and staff
a. Did the TWC program drain resources from other programs?  
b. What was the impact on those programs?   
c. What advice would you have to prevent that from happening in other locations?
4. decision making for impediments and resolutions
B. How has the reporting structure helped you in decision making? How did the reports and
evaluations contribute to the program? Please describe the reporting structure that was set up.
(Participants to field personnel to national office to ARC)
1. How did the data collected for the reports evolve over time?
2. How did you use the data collected?
a. Which reports were the most helpful to you? What do you think of the annual reports?
3. What feedback did you receive and from whom?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 76 
4. What feedback did you provide and to whom?
5. What proportion of staff time do you estimate was spent on reporting requirements? Do
you think the reporting requirements are too onerous?
6. What methods were set up to verify data?
7. Did you try to verify any of the figures you received from PE’s or YM?
a. If not, why not.
b. What did you find?
c. How accurate do you think the figures generally were?
8. What were the limitations in data collection?  
9. What problem areas developed in the reporting process and how were those addressed?
10. What do you think of the data collection and reporting requirements? How could the
system be improved? What resources would those improvements require?
C. How did the data from the monitoring system impact geographic and beneficiary targeting?
1. How appropriate do you feel your country is for TWC programming?
2. Is there a consensus that the program was implemented in the right communities?
3. Do you think the TWC program is targeting the right population to reduce HIV incidence and
unwanted pregnancies?
4. What other groups that should be targeted? Why?
5. How have you handled gender inequality issues?
a. Are both genders equally represented as PE’s and YM’s?
b. Do they have the same types of responsibilities?
c. How have gender inequality issues been addressed in the program?
d. Do you have all‐female youth groups?
D. What guidance did ARC/HQ provide you on geographic and beneficiary targeting?  
1. In what other ways did ARC/HQ support your office? How did you interact with ARC HQ?
a. Administrative
i. Was the administrative and logistical support ARC provided adequate?
b. Managerial
c. Financial  
d. Technical
e. Reporting
f. Communications
2. How has the role of ARC evolved over the life of the activities? What role did ARC play at the
beginning?
3. Was the communication from ARC sufficient?
4. What would have improved ARC/HQ support to your office?
E. How about support from your office to the NS? How did your office support the NS?  
1. Administrative
2. Managerial
3. Financial  
4. Technical
5. Reporting
6. Communications
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 77 
7. How did you communicate with the NS?
8. How capable do you feel the NS is in program management? What skills are strong? What is
weak?
9. Did the country programs have the right leaders?  Did they have effective and locally
appropriate materials and activities?
10. What would have improved ARC/field support to NS’s?
F. What activities have you held to exchange information between NS branch offices? Describe
your participation.
1. How did ARC’s field‐based staff also work with the Federation’s in‐country staff to facilitate
branch exchanges so that stronger branches shared lessons learned and best practices with
their peers? Has the inclusion of these activities been fruitful?
a. What was the outcome?
b. What it repeated?
2. What happened to the Red Cross National Youth HIV Prevention Task Force and the
Community and Social Mobilization Listserve? How were the original goals of Task Force and
List serve realized without these tools?
a. assist the Red Cross in developing the TWC project action plan, recruiting and 
orienting project staff, and providing technical guidance and oversight to country 
implementation 
b. capacity building 
c. formally discuss youth peer education and share valuable TWC lessons learned 
d. Share experiences with other Societies in the regions to further expand the 
project’s impact 
3. How did you communicate with the planned participants of the Task Force? (National 
AIDS Committee, relevant government ministry representatives, youth leaders, other 
key youth and HIV/AIDS stakeholders) 
4. How could information sharing been improved?
G. What activities have you held to exchange information between different countries’ NS
offices? Describe your country’s participation.
1. What was the outcome?
2. What it repeated?
3. How could information sharing been improved?
H. As a Track 1.0 program one of the USG intended benefits was the diffusion of information across
country programs. Describe your relationship to the USAID/Mission and how this relationship
promoted information sharing across various Track 1.0 programs. To what degree did you
interact with the donor?
1. How was USAID involved in the TWC program? How did that differ from the original
expectations?
2. How did USAID’S involvement levels affect program outcomes?
a. Lack of program baseline studies and usefulness of evaluations?
3. What would you have liked the ARC/field relationship to look like with the USAID/Mission?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 78 
I. With all this support going to building that capacity of the NS, describe the quality of the
management of the TWC program and how it could be better.
IV. Environmental Context 
A. How well has the NS taken all this support and translated that into culturally appropriate
program improvements?
1. Describe cultural issues that needed to be taken into consideration for the program to be
effective.
2. Describe political issues that needed to be taken into consideration for the program to be
effective. (USAID, Federation, ARC, NS, host‐country government, community)
B. Given the local context, What changes to the original TWC program were necessary?
(*Directions: Use a visual aid of the program components and ask respondent(s) to discuss them
and the changes to them.)
1. How were the messages and tools adapted for your program?
2. How effective were the adaptations in meeting the needs of the target audiences?
3. What changes are still ongoing ?
V. Behavior Change Activities 
A. I have cards listing all the TWC activities used to promote positive behaviors. Rank order the
activities in order of perceived efficacy in meeting program objectives.
1. What community interactions have been the most effective in garnering community
support?
a. Community council (Is there a functioning governing committee in each 
community? What work has it done to-date? How was the relationship handled?) 
b. Community wide events (general diffusion) 
c. Community wide events (participatory) 
d. Condom distribution 
e. Curriculum Based Interventions 
f. Follow Up Interventions 
g. Mass media (Please discuss the development of media campaigns. 
Have you gotten any well-known figures as media sponsors?) 
h. Parental involvement 
i. Peer Educator support 
j. Referral system (What organizations did you begin to work with for the first 
time? How was the relationship handled? What did those organizations do to 
advance or work against the program? What were issues in those relationships?) 
k. Schools or other existing community structures (Did you develop good 
working relationships with community structures? How was the relationship 
handled? Did they provide support, resources, and or information to you?) 
l. Town hall meeting 
m. Youth Clubs 
2. Discuss why they were and were not effective in enhancing the community environment
a. Did outreach to adults support outreach to youths?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 79 
b. Which outreach programs best complimented the curriculum‐based approach?
c. Was anything missing?  What should have been different?
d. How were the messages and tools adapted for your community?
e. How effective were the adaptations in meeting the needs of the target audiences?
f. Ongoing changes
B. How did relationships with communities affect direct outreach activities towards youth?  
1. List the activities you used in your country to strengthen youths’ life skills. Rank order the
activities in order of perceived efficacy. Discuss why they were and were not effective in
strengthening youths’ life skills. How could the activities been improved?
a. Community council 
b. Community wide events (general diffusion) 
c. Community wide events (participatory) 
d. Condom distribution 
e. Curriculum Based Interventions (Please discuss the participant materials. 
What significant changes were made over time? Were they culturally appropriate? 
Sufficiently age appropriate?) 
f. Follow Up Interventions 
g. Mass media 
h. Parental involvement 
i. Peer Educator support 
j. Referral system 
k. Schools or other existing community 
l. Town hall meeting 
m. Youth Clubs 
2. Which outreach programs best complimented the curriculum‐based approach?
3. Was anything missing?  What should have been different?
4. Was the Youth Participant outreach effective in spreading awareness? Why?
5. How were the messages and tools adapted for your community?
6. How effective were the adaptations in meeting the needs of the target audiences?
7. Ongoing changes
VI. TWC Sustainability 
A. Given that changes are still happening to the program, do you feel the program has been given
enough time?  Too much?
1. Did the program have enough resources throughout?  
a. Do you think it was given more resources than it should have been?
b. Did you find ways to generate income for the program?  How significant was that?
c. Did you obtain support from local authorities?
d. Resource challenges and impacts
e. Biggest sources of support
B. How has ARC’s involvement contributed to overall sustainability?  
1. What does sustainability mean in the context of TWC?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 80 
2. Are there parts of the program that are more likely to be sustained than others?   
3. What has ARC done to ensure sustainability? Which and why. What resources do you
believe ARC brings to a program like the TWC?
4. Do you have a closeout plan? How well is the close‐out plan working?  
C. Do you think the TWC program can realistically carry on after the support of ARC is removed?
1. How could the program have been better structured to reach its objectives?
2. What are the impediments to long‐term sustainability?
3. Do you believe the TWC program is sustainable in this country?  Please discuss community
and local political support for the continuation of the program. How did implementation
through the NS support of hinder program implementation?
4. What are the lessons that have been learned about building sustainable programs? What
advice would you give to the head of a local Red Cross that is starting a TWC program?
VII. TWC Activity Impact 
A. Given your response to questions regarding TWC sustainability, do you think the TWC
program here has been a success?  How do you describe that success?
B. What lessons have been learned about working on HIV/AIDS issues specifically with young
people?
1. How would you use abstinence and fidelity in future programs?
2. What would you do differently?
3. What has been the impact of the TWC program on the community?
4. Has the community changed in any ways due to the program?
a. What has been the impact on young people?
b. What has been the impact on parents?
c. What has been the impact on schools and local health providers?
C. What have been the health impacts of the program?
1. Do you believe that the program has had an impact on unplanned pregnancies?
2. Has the rate of new HIV cases changed?
3. Has there been an increase in testing?
4. Do young people who are sexually active use condoms more often?
5. Has the age of first sexual experience changed?
D. What have been the effects on young people?
1. Sympathy to those with AIDS & PLWH. How has your staff advocated on behalf of the rights
of people with AIDS & PLWH?
2. Ability to deal with sexual pressure
3. Understanding of HIV and transmission  
4. Openness to discussions of HIV with peers.
5. Openness to discussion of HIV with parents or at school.
6. Ability and willingness to use condoms
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 81 
E. What would you list as ARC best practices? What are the crucial elements in the success of the
program?
1. People
2. Organizations
3. Knowledge/materials/resources
F. What role do you think your office will have in future youth HIV prevention programs?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 82 
National Society Red Cross Manager In-Depth Interview
I. General TWC Program Questions 
A. Personal Information
1. Please describe your role in the TWC program.
a. How long have you been a manager?
b. Who do you receive directions from?
2. Do you have the resources you need to do your job well?
3. Are you able to make changes to the program that you deliver?
a. If yes, what changes have you made?  Why?
b. How do you think your role contributes to the success of the TWC program?
B. Program Management
1. Please describe the management of the TWC program.
a. Organizational structure, including relationship with national society and the Federation
b. Staffing
c. Resources (money, equipment, building,…)
d. Impediments and resolutions to project implementation
C. Geographic Targeting
1. How appropriate do you feel your country & sites are for TWC programming?
2. Is there a consensus that the program was implemented in the right communities?
a. Where else should the program be conducting interventions?  Why?
b. Where should you not be working?
D. Beneficiary Targeting
1. Do you think the TWC program is targeting the right population to reduce HIV incidence
and unplanned pregnancy?
2. What other groups that should be targeted? Why?
a. How well have you been able to reach out‐of‐school youth?  Were you able to resolve
any of the challenges?
3. Do you think that boys and girls have both benefited from the program?  Equally?
4. Do you think the program failed anyone?
E. Environmental & Cultural Context
1. Describe cultural & political issues that needed to be taken into consideration for the
program to be effective.
2. Please discuss community and local political support for the continuation of the program.
3. Do you think this project has been sensitive to the local cultural context? Give examples.
II. Strengthen National Society Project Management Capacity 
A. International Learning Opportunities
1. What international learning activities have you or others from your office participated in?
Describe the participation and what value it brought to TRCS.
2. What did your organization gain from attending these activities?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 83 
3. What would you recommend for future international learning efforts?
B. In‐Country Learning Opportunities
1. What in‐country information sharing activities have you participated in? Describe your
participation.  
2. What did your organization gain from attending these activities?
3. What would you recommend for future in‐country information sharing efforts?
C. TWC supported CB Activities
1. What support did ARC provide to the NS & TWC project? Please describe the training and
mentoring you received. How was the support from ARC/field to your office
adequate/inadequate?  
a. Administrative
b. Managerial
c. Financial  
d. Logistical
e. Technical
f. Reporting
g. Communications/ feedback
h. Branch development
2. What would have improved ARC/field support to your office?
3. What would have improved the support you provided to the branch offices? What did you
like about the training? Were there any weaknesses in the training?
4. How capable do you feel your office is in general program management? What skills are
strong? What is weak?
5. What would you recommendation give for capacity building if asked?
D. Monitoring & Reporting  
1. Please describe the reporting structure that was set up. How was the information used?
(Participants to field personnel to national office to ARC.)
a. Quarterly FM Observation Form
b. PE agreements
c. Pre/Post test database
d. PE reports
e. Follow up Interventions
f. Community Council Roster
g. Town Hall Roster
h. Condom Inventory Management Form
2. How much has the reporting work changed over time?
3. How much time do you currently spend every month working on reports?
4. Do you regularly review the?  How often? What methods were set up to verify data?
5. What feedback do you provide and to whom?
a. When you review the FM Reports every month, why is the information important?
6. What feedback have you receive and from whom?
7. Which reports were the most helpful to you? What do you think of the annual reports? How
did you use the data collected?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 84 
8. What were the limitations in data collection? What problem areas developed in the
reporting process and how were those addressed?
9. What do you think of the data collection and reporting requirements? What could be done
to improve the reporting structure?  The quality of the reports?
III. Enhanced Community Environment 
A. In your experience, was the community positive toward the program?  How did they actively
support it?
1. Discuss why they were and were not effective in enhancing the community environment.  
IV. Strengthen Youths’ Life Skills 
A. List the activities you used in your country to strengthen youths’ life skills
1. Discuss the strengths and weaknesses of strengthening youths’ life skills.
a. Curriculum Based Interventions
b. Follow Up Interventions
c. Volunteer support
d. Youth Clubs
e. Mass Media
f. Referrals
g. Condom distribution
2. Explain how the TWC messages and tools were adapted for your community?
a. When did it happen? In your opinion, how well did the adaptation meet the needs of
the target community?
3. What are the outreach approaches that best complemented the curriculum‐based
approach? Were they sufficient? How could they have been different? What was missing?
V. TWC Activity Impact 
A. Impact
1. What have been the effects the program has had on HIV transmission?  What are the
biggest contributors to that outcome?
2. What have been the main obstacles to success for the program?
B. Sustainability
1. What activities do you think the NS RC will continue to implement after current program
funding stops?
2. How will you finance the continuation of these activities?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 85 
National Society Field Staff IDI
I. General TWC Program Questions 
A. Personal Information
1. Please describe your role in the TWC program.
a. How long have you been a staff member?
b. Who do you receive directions from?
c. Is the feedback you receive fair and helpful?
2. Do you have the resources you need to do your job well?
3. Please describe the training you received.
4. Are you able to make changes to the program that you deliver?
a. If yes, what changes have you made?  Why?
b. How do you think your role contributes to the success of the TWC program?
B. Program Management
1. What advice would you give to improve management at this point in time?
2. How has the program management evolved over time?
C. Geographic Targeting
1. When did activities begin in your area?
a. How did the project start here?
2. Why do you think your geographic area is appropriate for TWC activities?
a. What locations would you add or subtract?
D. Beneficiary Targeting
1. Do you think the TWC program is targeting the right population to reduce HIV incidence
and unplanned pregnancy?
2. How are the people who attend your program recruited?  Are the right people attending?
3. What other groups that should be targeted? Why?
4. Do you think that boys and girls have both benefited from the program?  Equally?  Do you
think the program failed anyone?
E. Environmental & Cultural Context
1. Describe cultural & political issues that needed to be taken into consideration for the
program to be effective.
2. Please discuss community and local political support for the continuation of the program.
3. Do you think this project has been sensitive to the local cultural context? Give examples.
II. Strengthen National Society Project Management Capacity 
A. International Learning Opportunities
1. What international learning activities have you or others from your office participated in?
Describe the participation and what value it brought to TRCS.
2. What did your organization gain from attending these activities?
3. What would you recommend for future international learning efforts?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 86 
B. In‐Country Learning Opportunities
1. What in‐country learning activities have you participated in? Describe your participation.  
2. What did your organization gain from attending these activities?
3. What would you recommend for future in‐country learning efforts?
C. TWC supported CB Activities
1. How has ARC supported your office? Tools? Visits? Mentoring? Training?
a. Administrative
b. Managerial
c. Financial  
d. Logistical
e. Technical
f. Reporting
g. Communications/ feedback
h. Branch development
2. What would have improved ARC support to your office?
3. How capable do you feel your office is in general program management? What skills are
strong? What is weak?
4. What would you recommendation for capacity building if asked?
D. Monitoring & Reporting (Directions: Ask for last month’s completed reports)
1. Please discuss the reports that you create  
a. Which, how often, how time consuming
2. Please discuss reports that you are responsible for reviewing
a. Who creates, which ones, how often, how time consuming
3. How much has the reporting work changed over time?
4. Which reports are the most useful?
5. How do you use the reports to improve programming?  Give an example.
6. What could be done to improve the reporting structure?  The quality of the reports?
7. Have you ever discovered PE’s or YM’s fill out false information on their reports?  What
happened?
  
III. Enhanced Community Environment 
A. List the activities you used in your country to enhance the community environment
1. In your experience, was the community positive toward the program?  Did they actively
support it?
2. How do you involve the community in TWC? Please discuss the identification and
recruitment of key stakeholders.
  
B. Parental Involvement
1. How have you involved parents?
2. How have the parents responded to your programs?   
C. Community Wide Events
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 87 
1. List activities have you held? Explain the difference between general diffusion and
interpersonal events.
2. What are the most effective edutainment programs?  Why?
D. Town Hall Meeting
1. Describe how you conduct town hall meetings and who is involved?
2. How effective do you feel Town Hall meetings were at meeting their objectives?  
3. How could they be improved?
E. Community Council
1. Describe how you work with community councils.
2. Discuss the quality of the relationship with the Community Councils/Town Councils
3. What were some of the positive and negative issues over time with local leaders?
4. What were the lessons learned with the community councils?  
F. Partnerships
1. Please discuss the identification and recruitment of and synergies with key stakeholders.
a. What role do you play?  Just referrals?
b. What is your impression about the relationship between the TWC program and health
practitioners?
2. What have you learned about working effectively with local ministries, schools, health
facilities and leaders?
IV. Strengthen Youths’ Life Skills 
A. List the activities you used in your country to strengthen youths’ life skills
1. How have the youth responded to your programs?
2. Describe how well the materials and activities work well together.
B. Curriculum Based Interventions
1. Has the content of the curriculum changed over time? Why?
2. Is it more important for the curriculum to be presented exactly as it was developed or that it
is appropriate for each audience?
3. Have you been able to attend the TWC sessions at least once a month to observe PE
outreach of activities?  Do you check the roster against the participants?
4. Was the PE‐YM‐YP outreach effective in spreading awareness? Why or why not?
C. Follow‐Up Interventions
1. Did the Follow‐Up Interventions have an effect? How? How could they have been better?
D. Peer Educator support
1. Describe how your office supports peer educators.
2. Have you conducted any PE exit interviews?  What were some of the reasons for leaving?
3. How do you motivate PEs? How long do PE’s and YM’s stay with the program?
a. Why do they leave?
b. What could be done to improve retention?
c. What sorts of incentives have you provided? In what situations? How often?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 88 
4. What are the main issues you have had concerning PE identification, selection, motivation
& retention?
E. Youth Clubs
1. Discuss the development of the Youth Clubs.
a. If none, why were they not developed?   
b. What is the value of the clubs?
2. What do the clubs do?
3. What determines whether a club is successful or not?
  
F. Mass Media
1. Describe the mass media campaigns you have launched in your country.
a. Timing
b. Messages
c. Type
d. Targeted audience
e. Significant points (celebrity, interactive, discontinued because, collaborated with other
organization, …)
2. What do you think are the most effective media campaigns?  Why?
G. Referrals
1. Please describe how TWC works with other organizations. For how long? Who from your
organization in involved? How often?
2. Describe how the NS coordinates the TWC program with other HIV/youth programs &
services?
H. Condom distribution
1. What constraints have you experienced?
2. Where? Number of sites?
3. Who manages the stock?
4. Will this continue post‐USAID funding?
V. TWC Activity Impact 
A. Best Practices
1. What have been the effects the programs has had on HIV transmission?  What are the
biggest contributors to that outcome?
2. In your opinion, has the type of high‐risk behavior changed among participants in this
program?
3. Do you think that the program has affected people’s use of condoms?  How?  Are condoms
the same price they were at the beginning of the program?  Are they more or fewer places
to get them?  Are young people better informed about how and why to use condoms?
4. Do you think that unintended pregnancies are fewer than they were before?  Why is that?  
Do you have any examples of people taking steps to avoid pregnancy that they didn’t
before?
5. Do you think that some young people have different ideas about fidelity than they did
before the program?  Does this apply to both boys and girls?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 89 
6. Do you think that some people have changed their ideas about abstinence because of the
program?  Does this include both boys and girls?
7. Have you seen any difference in the way that people with AIDS are treated in the
community?
B. Lessons Learned
1. What have been the main obstacles to success for the program?
2. What are the difficulties in measuring behavioral change?  How have you addressed this?
C. Sustainability
1. Do you think the NS RC will continue to implement some TWC activities after current
program funding stops?
2. What activities do you think will continue?
3. How will you finance the continuation of these activities?
4. Who will continue to work on future TWC projects?
5. Where will you work post‐USAID funding?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 90 
Community Council In-Depth Interview
I. Introduction 
* Describe evaluation content and process. Provide informed consent. 
* Enter interview information & respondent names on Respondent Sign In sheet 
* Do you have any questions for me before we begin?
II. Respondent Characteristics 
A. Describe the community council. 
1. Who is on the council? How many members? What are the leadership positions? 
2. How did you become a council member? Why were you chosen? 
3. Who formed it? When was the council formed? When did the Council become 
involved in the TWC program? 
4. How long is your term on the council? 
a. Discuss turnover in council members. 
b. Who has turned over and why? How often? 
5. How often do you meet? 
B. How is the council involved with the TWC program? 
1. Describe the relationship between the community council and the Red Cross. 
2. Why does this council exist? What are you trying to accomplish? What does the 
Council see as its goals in involvement with the TWC program? 
3. What Community Council support was provided to TWC? 
4. What are your roles on the council? 
5. What records do you keep? 
a. What do you do with the records? 
6. Has the Council faced any opposition in its work in the TWC program? 
a. From which people or groups? 
b. How did it develop? 
c. How was it resolved? 
III. Capacity Building 
A. How has the Red Cross supported the Council? 
1. How often have you communicated with them? 
a. Who do you communicate with? 
b. What is the nature of these communications? 
2. What kind of support have they provided? How often? Has it been helpful? 
3. What support would you like to receive but you have not? 
IV. Prevention Activities 
A. What are some examples of TWC activities that the council is responsible for? 
1. Who organized the activity? What was the Council’s role? 
2. What was your impression? What did you like/dislike about the event(s)? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 91 
 
B. Did the TWC program address a real community need? In what way? 
1. Do you think your community is appropriate for the TWC program? 
2. Describe who TWC targeted and whether the right people were included. 
3. Have you seen changes in the TWC program over time? What were they and what 
outcome did they produce? 
V. Sustainability 
A. How will TWC sustain itself after the Red Cross no longer funds the project? 
1. What are some of the handicaps that the community may have in continuing with the 
work of the TWC program? (Probe for non-financial issues) 
2. Do you think there is long-term local support for it? 
3. What changes should be made to it? 
4. Will the Council continue as an organization long-term? Why or why not? 
VI. Impact 
A. Overall, what changes in the community would you attribute directly to the TWC 
program? 
1. Since the TWC program started here, have you seen any change in the behavior of 
young people due to its work? 
a. Have you seen any change in their attitudes? 
b. Toward sexual activity, abstinence, fidelity, contraception 
2. Do you see any differences in girls’ and young women’s ability to handle 
relationships with boys and men? 
3. Do you think that young people are communicating more with their parents, teachers 
and each other about HIV prevention issues? 
4. Are young people better able to deal with people who have AIDS and PLHA? 
5. What changes have you seen in rates of people getting HIV tests? 
VII. Recommendations 
A. What recommendations do you have to improve the project? 
B. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 92 
Peer Educator Focus Group Discussion 
I. Introduction 
* Describe evaluation content and process. Provide informed consent. 
* Enter interview information & respondent names on Respondent Sign In sheet 
* Do you have any questions for me before we begin?
II. Respondent Characteristics 
A. Tell me the story of how you learned about and joined this program. 
1. How did you learn about the program? Were you a Youth Multiplier before becoming 
a Peer Educator? 
2. Why did you decide to join? 
3. How were you recruited? What were the selection processes and selection criteria? 
4. What motivates you to continue with your work? 
III. Prevention Activities 
A. Peer Educator Training 
1. Describe the training you received. (CBI, facilitation, organizing events, …) 
a. When were you trained? How long did the training last? 
b. Where did the training take place? 
c. Who trained you? 
d. How many people were you trained with? 
2. What topics did you learn about? 
a. Was there any difference in the emphasis on condoms, abstinence and fidelity? 
b. What facilitation/communication techniques did you find useful? 
c. What community organizational techniques did you find useful? 
3. Are there activities that you like better than others? Do you think the purpose of 
each activity is clear? What is the most memorable part of your training? 
4. What other training did you receive or participate in other than for the curriculum? 
Any follow-up interventions, national or international workshops or conferences? 
Please describe. 
a. What are your thoughts on that training? 
b. How has this information helped you to do your job better? 
c. If you did not participate, do you know of any other PE’s who have participated in 
international conferences, workshops or information sharing events? Describe 
what these people learned and how you heard about it. 
d. Do you wish you had more or different training? 
B. Curriculum Participants 
1. Then you finished your training and went to work. Where did you work? 
a. How did you know where to do your work? How were the sites chosen? 
b. How many locations did you work concurrently? Consecutively? 
2. How much time did you spend with each group? Did the amount of time vary? 
According to what factors? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 93 
3. Do you think the YMs you have instructed are the appropriate people to be speaking 
with? Why or why not? 
C. Curriculum Outreach Activities 
1. Describe what you did during a typical training course for youth multipliers. 
2. What were some of the more prevalent myths that you dispelled? How successful do 
you think you were? 
3. Does the training differ between different target groups? What changes between 
target groups? 
4. Did you have the materials and resources you needed? Do you know where these 
materials came from? 
a. Do the Peer Educator’s Handbook and the Activity Kit work well together? Have 
you had any difficulty in understanding the relationship between material in 
them? 
b. Do you think the curriculum sounds foreign, or appears to be something 
developed here? Why? 
5. Do you think the curriculum gives Youth Multipliers the tools to be effective? 
6. Did you ever go back to these sites? To do what? Why not? (FUI) 
D. Other Community-Based Extension Activities 
1. What was your role in other community-based extension activities? (Planning, 
implementing, recruiting participants, participating, monitoring,…) 
 a. Community Wide Events (general diffusion) e. Condom distribution 
 b. Community Wide Events (inter-personal) f. School activities 
 c. Mass Media g. Youth Clubs 
 d. Referrals to other services h. other 
2. Who did you invite to participate in these activities? 
3. What did your guests think of these events? 
4. Which activities would you recommend more or less or? Why? 
a. Which activities were most effective in helping you to reach TWC goals? 
b. What approaches best complement the curriculum? 
IV. Project Management 
A. Reporting (PE & YM) 
1. What reports were filled out and by whom? 
2. How often were reports filled out and submitted? 
3. How thoroughly and accurately were reports filled out? 
4. What did the PEs and YMs do with the reports after they had been filled out? 
5. What kind of feedback did you receive on the reports you filled out? From whom? 
How was the feedback helpful to you? 
6. How did you know if you were doing a good job or not with these training sessions? 
(supervision) 
7. What feedback did you provide to the YMs on their reports? 
8. Why are these reports important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 94 
B. Volunteer Motivation 
1. How long do you think you will continue being a peer educator? 
2. Why do you think people stay or leave? What lowers a person’s commitment? 
3. How did you motivate the YMs to participate? 
4. How did the YMs know if they were doing a good job or not? 
5. What did you do with completely uninterested people? 
6. What proportion of the people you attempted to contact were uninterested? How did 
you know? 
C. Volunteer Support 
1. While organizing different activities, whom have you turned to with questions or 
problems? Do you think that you have had the support you needed? 
2. How often did your FM visit you? Did you receive helpful feedback?
a. Has your effectiveness as Peer Educators changed over time? Why is that? 
3. Do you think the Peer Educators have the support and resources to be effective? 
a. What could the program do that would help Peer Educators be more effective? 
b. Describe your incentives and why you received them. 
4. Do Peer Educators help each other? How do they resolve any differences that 
develop? 
5. What was the reaction of your friends, schoolmates and family? How important is 
their support in your decision to be a Peer Educator or not? 
6. How have your parents attitudes changed with time? Did your parents understand 
the program at the beginning? Do they understand what you do now? 
7. Are your parents supportive of your work? Do you think they should be part of the 
program? Why or why not? 
8. What do they do to support you as a Peer Educator? What role could parents have in 
this program? 
9. Would your parents want your sibling to be a PE? Would you want your brother or 
sister to be one? 
D. Partnerships 
1. What institutions or organizations did you partner with? List the supportive services 
in your community that you collaborate with. 
2. Describe the partnership. 
V. Targeting 
A. Geographic Targeting 
1. What constitutes your community? (education, town, religion, geographic 
boundaries, tribe, etc) 
2. Do you think that your community needs a TWC program? Why? 
a. Why do you think these activities you just described are appropriate for your 
community? 
3. Do you think this project has been sensitive to the local cultural context? Give 
examples. 
a. What cultural, political or social issues have impacted your work? How have you 
dealt with those issues? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 95 
4. In your experience, was the community positive toward the program? What are 
some examples of community support? 
a. In what ways has the community resisted this activity? 
B. Beneficiary Targeting 
1. Within this larger community, who should the program be targeting for messages but 
are not? 
a. Do you think the program failed anyone? 
b. Do you think that boys and girls have both benefited from the program? Equally? 
VI. Sustainability 
A. What activities will you continue after the Red Cross no longer funds TWC 
activities? 
1. Please discuss community and local political support for the continuation of the 
program. What needs to happen for the project activities to be able to continue? 
2. What was the biggest obstacle that you were aware of? 
B. Would you like the program to continue? 
1. Do you think this program should continue? Who should it be targeting? Where? 
2. What could the program do that would help it be more sustainable? 
VII. Program Impact 
A. General 
1. What have you gotten out of being a Peer Educator? How has being a Peer 
Educator affected you? 
2. Have any of your ideas changed as a result of your work? Describe the old and new 
ideas. 
3. Have you observed changes in behavior among other Peer Educators? What is 
different? 
4. Has any of your own behavior changed because of your work? Describe what you 
do differently now. 
B. Changes in Knowledge 
1. What did you learn that you didn’t know before? 
2. What myths have been dispelled due to TWC? 
C. Changes in Attitudes 
1. How have people changed their ideas about abstinence because of the program? 
Does this include both boys and girls? 
2. How is being faithful viewed amongst your peers? How are ideas about fidelity 
amongst young people different than before the program? Does this apply to both 
boys and girls? 
3. What adjectives would you use to describe condom use? How has the program 
affected people’s use of condoms? Price? Where to buy? How and why to use 
condoms? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 96 
4. What about interacting with people living with HIV or people living with AIDS, have 
you seen any difference in the way they are treated in the community? Describe 
what you have seen. 
D. Changes in Behavior 
1. What have been the effects the programs has had on HIV transmission? What are 
the biggest contributors to these outcomes? 
2. In your opinion, what high-risk behaviors have changed among people who have 
heard TWC prevention messages? Please provide examples. 
3. What changes have you seen in unintended pregnancies for those who have 
received TWC messages? Please provide examples. 
4. What changes have you seen in STIs, including HIV transmission? What do you 
think caused the change? Please provide examples. 
5. What have been the effects the TWC project has had on HIV testing? What are the 
biggest contributors to that outcome? Please provide examples. 
6. What other behaviors have changed as a result of TWC? Please provide examples. 
VIII. Recommendations 
A. What are some of the best things about being a Peer Educator? 
1. In what ways has your work gotten easier or harder? 
2. What are some of the challenges you have in your role? What don’t you like? 
B. Overall impressions of the program 
1. What is your opinion about the TWC program? 
2. Is there anything you would like to see that you didn’t? What recommendations do 
you have for improving the project? (PE training, curriculum, management, volunteer 
support, …) 
C. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 97 
Peer Educator In-Depth Interview 
I. Introduction 
* Describe evaluation content and process. Provide informed consent. 
* Enter interview information & respondent names on Respondent Sign In sheet 
* Do you have any questions for me before we begin?
II. Respondent Characteristics 
A. Tell me how you joined TWC. 
1. Why did you decide to become a PE? 
2. What motivates you to continue with your work? 
III. Prevention Activities 
A. Training 
1. What did you like about the training you received from Red Cross? 
2. What were some of the ideas that were unclear during training? How did you get 
answers to your questions? 
3. Did you participate in any national or international workshops or conferences? 
a. How has this information helped you to do your job better? 
b. If you did not participate, describe what others learned and how you heard about 
it. 
B. Curriculum Participants 
1. Where did you work? How did you decide where to do your work? 
a. How long did you interact with each site? 
2. How much time did you spend with each group? Did the amount of time vary? 
According to what factors? 
3. Do you think the YMs you have instructed are the appropriate people to be speaking 
with? Why or why not? 
C. Curriculum Outreach Activities 
1. What activities do you think were more popular with the youth? Was there a 
difference in the popularity of activities by cohort? 
2. Does the training differ between different target groups? What changes between 
target groups? 
3. Did you have the materials and resources you needed? Do you know where these 
materials came from? 
4. Did you ever go back to these sites for follow-up interventions? Why or why not? 
D. Other Community-Based Extension Activities 
1. What was your role in other community-based extension activities? (Planning, 
implementing, recruiting participants, participating, monitoring,…) 
 a. Community Wide Events (general diffusion) e. Condom distribution 
 b. Community Wide Events (inter-personal) f. School activities 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 98 
 c. Mass Media g. Youth Clubs 
 d. Referrals to other services h. other 
2. Which activities did you enjoy the most? 
3. Which activities helped you to do your work better? 
IV. Project Management 
A. Reporting (PE & YM) 
1. What was involved in the reporting? What reports did you have to fill out? 
2. How often did you fill out and submit reports? 
3. When did you fill these reports out? How thoroughly and accurately did you fill out 
the reports? 
4. What did you do with the reports after they had been filled out? 
5. What kind of feedback did you receive on the reports you filled out? From whom? 
How was the feedback helpful to you? 
6. Why are these reports important? How did you use the data? 
7. How did you know if you were doing a good job or not with these training sessions? 
(supervision) 
B. Volunteer Motivation 
1. How did the YMs know if they were doing a good job or not? 
2. How did you motivate the YMs to participate? 
C. Volunteer Support 
1. While organizing different activities, whom have you turned to with questions or 
problems? Do you think that you have had the support you needed? 
2. How often did your FM visit you? Did you receive helpful feedback?
a. Has your effectiveness as Peer Educators changed over time? Why is that? 
3. Do you think the Peer Educators have the support and resources to be effective? 
a. What could the program do that would help Peer Educators be more effective? 
b. Describe your incentives and why you received them. 
4. Do Peer Educators help each other? How do they resolve any differences that 
develop? 
D. Partnerships 
1. Where did you work? With what institutions (schools, churches, workplaces, …)? 
2. Describe the partnership. 
V. Targeting 
A. Geographic Targeting 
1. Do you think that your community needs a TWC program? Why? 
a. Why do you think these activities you just described are appropriate for your 
community? 
2. In your experience, was the community positive toward the program? What are 
some examples of community support? 
a. In what ways has the community resisted this activity? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 99 
B. Beneficiary Targeting 
1. Within this larger community, who should the program be targeting for messages but 
are not? 
a. Do you think the program failed anyone? 
b. Do you think that boys and girls have both benefited from the program? Equally? 
VI. Sustainability 
A. Community Support 
1. What activities will you continue after the Red Cross no longer funds TWC activities? 
2. Please discuss community and local political support for the continuation of the 
program. What needs to happen for the project activities to be able to continue? 
3. What was the biggest obstacle that you were aware of? (If something that the 
program could affect) was anything done that you are aware of to overcome that 
obstacle? 
4. What was the reaction of your friends, schoolmates and family? How important is 
their support in your decision to be a Peer Educator or not? 
5. Are your parents supportive of your work? Do you think they should be part of the 
program? Why or why not? 
6. What do they do to support you as a Peer Educator? What role could parents have in 
this program? 
B. Volunteer Retention 
1. What are some of the best things about being a Peer Educator? 
2. In what ways has your work gotten easier or harder? 
3. What are some of the challenges you have in your role? What don’t you like? 
4. How long do you think you will continue being a peer educator? 
5. Why do you think people stay or leave? What lowers a person’s commitment? 
VII. Program Impact 
A. General 
1. What have you gotten out of being a Peer Educator? How has being a Peer 
Educator affected you? 
2. Have any of your ideas changed as a result of your work? Describe the old and new 
ideas. 
3. Have you observed changes in behavior among other Peer Educators? What is 
different? 
4. What have been the effects the programs has had on HIV and STI transmission, 
unintended pregnancy and HIV testing? What are the biggest contributors to these 
outcomes? 
B. Changes in Knowledge 
1. What did you learn that you didn’t know before? 
2. What myths have been dispelled due to TWC? How successful do you think you 
were? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 100 
C. Changes in Attitudes 
1. How have you changed your ideas about abstinence, fidelity or condoms because of 
the program? 
2. What about interacting with people living with HIV or people living with AIDS, have 
you seen any difference in the way they are treated in the community? Describe 
what you have seen. 
D. Changes in Behavior 
1. What high-risk behaviors have you changed? Please provide examples. 
2. What other behaviors have changed as a result of TWC? Please provide examples. 
VIII. Recommendations 
A. What are some of the best things about being a Peer Educator? 
1. In what ways has your work gotten easier or harder? 
2. What are some of the challenges you have in your role? What don’t you like? 
B. Overall impressions of the program 
1. What is your opinion about the TWC program? 
2. Is there anything you would like to see that you didn’t? What recommendations do 
you have for improving the project? (PE training, curriculum, management, volunteer 
support, …) 
C. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 101 
Parents of Participating Youth Focus Group Discussion
I. Introduction 
* Describe evaluation content and process. Provide informed consent. 
* Enter interview information & respondent names on Respondent Sign In sheet 
* Do you have any questions for me before we begin?
II. Respondent Characteristics 
A. Personal Information 
1. What do you know about the Red Cross “Together We Can” program? 
2. How did you learn about the TWC program? By whom, where, what? 
3. Were you more or less open to the project as a result of this interaction? 
4. What has your involvement been with this project? 
III. Prevention Activities 
A. TWC Curriculum 
1. What do you know about your child’s involvement with the program? Do you recall 
any TWC activities your child was involved with? 
a. How are your children involved with the project? 
b. For how long? 
c. What did your children learn from the project? 
2. Did the children discuss it with you? When? What did they say about it? 
a. How old was he/she? 
b. At that time were you worried about any sexual pressure on your child? 
c. Were you worried about HIV? 
d. Did you have any concerns about the program? What were they? 
3. What do you think about the Red Cross project? How has you attitude towards the 
project changed over time? 
a. What did you like/dislike about the activity? 
B. Non-curriculum Based Activities 
1. What other Red Cross activities have occurred in your community? 
2. What Red Cross activities have you participated in? 
 a. Community Wide Events (general diffusion) e. Condom distribution 
 b. Community Wide Events (inter-personal) f. School activities 
 c. Mass Media g. Youth Clubs 
 d. Referrals to other services h. other 
3. Which activities did you enjoy the most? 
IV. Targeting 
A. Geographic & Beneficiary Targeting 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 102 
1. Do you think your children should be participating in an HIV prevention program? If 
not, who should the project target? If so, why do you think this program is 
appropriate for your children? 
2. Do you think HIV prevention and sex education programs should be in your 
community? What makes your community appropriate or inappropriate? 
3. Do you think the community has supported the program? What could have been 
done to increase community support? (recommendations) 
4. Who would you not recommend it to? Why? 
B. Targeting Parents 
1. Did parents become involved in the TWC program at all? In what way? 
2. In what ways have parents supported their children’s participation in the project? 
V. Sustainability 
A. Community Support 
1. Do you think TWC activities should continue in your community? 
2. What should that continuation look like? 
3. How would you support it if it did continue? 
VI. Impact 
A. Changes in Knowledge 
1. In general, where have you learned about HIV, AIDS and STIs? If it was at an event, 
who sponsored the activities? 
2. What did you learn at TWC events that you didn’t know before? 
B. Impressions of attitudes and behavior change 
1. Describe any changes you have seen in peoples’ attitudes and behaviors, in your 
community, as a result of the TWC activities. (HIV transmission, high-risk behavior, 
use of condoms, unintended pregnancies, abstinence, fidelity, the way that people 
with AIDS and PLHIV are treated in the community) 
VII. Recommendations 
A. Overall Impressions of TWC 
1. What would you like changed in the project? 
B. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 103 
Parents of Participating Youth In-Depth Interview
I. Introduction 
* Describe evaluation content and process 
* Informed consent 
* Enter interview information & respondent names on Respondent Sign In sheet 
II. Respondent Characteristics 
A. Personal Information 
1. What do you know about the Red Cross “Together We Can” program? 
2. How did you learn about the TWC program? By whom, where, what? 
3. What do you know about your child’s involvement with the program? Do you recall 
any TWC activities your child was involved with? 
4. Did the children discuss it with you? When? What did they say about it? 
III. Prevention Activities 
A. TWC Curriculum 
1. What do you know about your child’s involvement with the program? Do you recall 
any TWC activities your child was involved with? 
2. Did the children discuss it with you? When? What did they say about it? 
3. What do you think about the Red Cross project? How has you attitude towards the 
project changed over time? 
B. Non-curriculum Based Activities 
1. What other Red Cross activities have occurred in your community? 
2. What Red Cross activities have you participated in? 
 a. Community Wide Events (general diffusion) e. Condom distribution 
 b. Community Wide Events (inter-personal) f. School activities 
 c. Mass Media g. Youth Clubs 
 d. Referrals to other services h. other 
3. Which activities did you enjoy the most? 
4. Which events did you find most useful? 
IV. Targeting 
A. Geographic & Beneficiary Targeting 
1. Do you think your children should be participating in an HIV prevention program? If 
not, who should the project target? If so, why do you think this program is 
appropriate for your children? 
2. Do you think HIV prevention and sex education programs should be in your 
community? What makes your community appropriate or inappropriate? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 104 
3. Do you think the community has supported the program? What could have been 
done to increase community support? (recommendations) 
4. Who would you not recommend it to? Why? 
5. What have you said to other people about TWC? To whom? 
V. Sustainability 
A. Community Support 
1. Do you think TWC activities should continue in your community? 
2. What should that continuation look like? 
3. How would you support it if it did continue? 
VI. Impact 
A. Changes in Knowledge 
1. What did you learn at TWC events that you didn’t know before? 
2. What new information has your child learned from TWC? 
B. Changes in Attitudes 
1. Have you changed any of your attitudes or behaviors as a result of the project? Give 
examples. 
2. Have you ever cared for someone HIV positive or has AIDS? What did that consist 
of? 
3. If you knew someone was HIV positive, how would you treat them as compared to 
someone who is HIV negative? 
4. How has the attitude of your child changed due to their involvement in TWC? 
C. Changes in Behavior 
1. Has your child brought home new information about HIV, STI and pregnancy 
prevention that you didn’t know about before? What was it? Where did s/he learn it? 
2. How has your child’s participation in TWC changed your communication about HIV 
and sexuality? 
a. Do you speak with your children about sex? Describe how and why that 
occurred? (is it because of TWC?) At what age do you first initiate these 
conversations? Who initiated the conversation? What do you say? 
3. How do you feel about those discussions? Had the program changed how your feel? 
4. Has the program made your child more comfortable with those topics? You? 
5. What changes have you noticed in your child’s behavior that you think are due to 
TWC? 
a. Is your child sexually active? 
b. Has he or she been tested for AIDS? When? 
c. Have you been tested? How did you feel? 
6. Does he or she practice any preventive measures to prevent pregnancy, HIV or 
STIs? 
a. What measure? 
b. Has this changed since participating in the program? 
c. Did he or she see a demonstration of condom use? 
7. Has TWC information changed any of your behaviors? Describe. 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 105 
VII. Recommendations 
A. Overall Impressions of TWC 
1. What advice would you give to the Red Cross if they were to have another HIV 
prevention program in your community? What would you do differently? 
B. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 106 
Stakeholder In-Depth Interview
I. Introduction 
* Describe evaluation content and process. Provide informed consent. 
* Enter interview information & respondent names on Respondent Sign In sheet 
* Do you have any questions for me before we begin?
II. Respondent Characteristics 
A. TWC Participation 
1. What is your position in the local community? 
2. How long have you held this position? 
3. What responsibilities to you have? 
B. Describe your relationship to the TWC program. 
1. Do you deal with other organizations with a similar mandate to the TWC program? 
a. Which ones? 
2. What do you see as the main reason you are involved with the TWC program? 
3. Have you faced any opposition in your work in the TWC program? 
a. From which people or groups? 
b. How did it develop? 
c. How was it resolved? 
III. Capacity Building 
A. What capacity building support have you seen the Red Cross provide? 
1. Have you been the recipient of any such support? 
a. If so, how often have you communicated with them? 
b. Who do you communicate with? 
c. What is the nature of these communications? 
2. What kind of support have they provided? How often? Has it been helpful? 
3. What support would you like to receive but you have not? 
IV. Prevention Activities 
A. What are some examples of TWC activities that you have been involved with? 
1. Who organized the activity? What was your role? 
a. facilitated referrals between complementary services 
b. condom distribution 
c. community events 
d. working with the youth in schools and other 
e. town hall meetings 
f. mass media 
2. What was your impression? What did you like/dislike about the event? 
a. Were they engaging to the participants? 
b. Were the messages given clear to the people attending? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 107 
c. Were they appropriate? 
d. Do you have any opinion as to their effectiveness? 
 
B. Did the TWC program address a real community need? In what way? 
1. Do you think your community is appropriate for the TWC program? 
2. Describe who TWC targeted and whether the right people were included. 
3. Have you seen changes in the TWC program over time? What were they and what 
outcome did they produce? 
V. Sustainability 
A. How will TWC sustain itself after the Red Cross no longer funds the project? 
1. What are some of the handicaps that the community may have in continuing with the 
work of the TWC program? (Probe for non-financial issues) 
2. Do you think there is long-term local support for it? 
3. What changes should be made to it? 
4. Will you continue to support HIV prevention in the community even though the TWC 
program is ending? 
VI. Impact 
A. Overall, what changes in the community would you attribute directly to the TWC 
program? 
1. Since the TWC program started here, have you seen any change in the behavior of 
young people due to its work? 
a. Have you seen any change in their attitudes? 
b. Toward sexual activity, abstinence, fidelity, contraception 
2. Do you see any differences in girls’ and young women’s ability to handle 
relationships with boys and men? 
3. Do you think that young people are communicating more with their parents, teachers 
and each other about HIV prevention issues? 
4. Are young people better able to deal with people who have AIDS and PLHA? 
5. What changes have you seen in rates of people getting HIV tests? 
VII. Recommendations 
A. What recommendations do you have to improve the project? 
B. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 108 
Town Hall Participants Focus Group Discussion
I. Introduction 
* Describe evaluation content and process. Provide informed consent. 
* Enter interview information & respondent names on Respondent Sign In sheet 
* Do you have any questions for me before we begin?
II. Respondent Characteristics 
A. TWC Participation 
1. Have you heard of the Red Cross “Together We Can” Program? 
a. What do you know about the program? 
b. How do you come to know this information? 
2. Describe what happened leading up to and at the Town Hall meeting you attended. 
a. How were you notified? When was it? Where? Who hosted it? Who attended? 
What the purpose was? What was discussed? What did you learn? Where 
materials handed out? 
b. Did you give your name at the meeting? Did they write it down? 
3. Why did you decide to go to this meeting? Why was this meeting important? 
4. Do you think that was a good forum to accomplish that purpose? What did you 
like/dislike about the Town Hall meeting? 
III. Targeting 
A. Geographic & Beneficiary Targeting 
1. Do you think your community is appropriate for the TWC program? Why or why not? 
2. Describe who participates in TWC. Are they targeting the youth who need to be 
targeted? If not, who should be the participants? 
3. Do you think that boys and girls have both benefited from the program? Equally? 
Do you think the program failed anyone? 
IV. Prevention Activities 
A. TWC Curriculum 
1. Do your children participate in the program? What is their role? 
2. What do you think about their participation? Is this a good use of their time? Why or 
why not? 
B. Non-Curriculum Based Activities 
1. Have you had any exposure to the project since the town hall meeting? What has 
that been? 
2. Describe the HIV prevention activities you attended? What happened? Where? 
When? Sponsored by whom? 
a. What did you think of the event? What did you learn that you didn’t know before? 
Is there anything they said that you don’t believe to be true? Would you go 
again? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 109 
3. Did participating in the town hall meeting make a difference in whether you attended 
any community wide events or not? 
4. What are some ideas or practices that you have changed because of your 
participation in the town hall? 
V. Impact 
A. Motivations 
1. In your experience, was the community positive toward the program? How did they 
actively support it? 
2. What are some reasons that people do and do not support the program? 
B. Impressions of attitudes and behavior change 
1. Do you think the TWC program has made a difference? How has the TWC program 
affected your community? 
2. Describe any changes you have seen in peoples’ attitudes and behaviors, in your 
community, as a result of the TWC activities. (probe on HIV transmission, high-risk 
behavior, condom use, unintended pregnancies, abstinence, fidelity, the way that 
people with AIDS and PLHIV are treated in the community) 
VI. Sustainability 
1. What do you think about the program? Do you think TWC activities should continue 
in your community? What should that continuation look like? How would you support 
it if it did continue? 
2. Who would you not recommend it to? Why? 
VII. Recommendations 
A. Overall Impressions of TWC 
1. What is your current opinion about the TWC program? 
a. What aspects of the program did you like/dislike? 
2. Is there anything you would like changed? What do you disapprove of? 
B. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 110 
Youth Multiplier Focus Group Discussion 
I. Introduction 
* Describe evaluation content and process. Provide informed consent. 
* Enter interview information & respondent names on Respondent Sign In sheet 
* Do you have any questions for me before we begin?
II. Respondent Characteristics 
A. TWC Participation 
1. What can you tell me about the Red Cross? How did you learn about the Red Cross? 
Who provided the information? 
2. How did you learn about the TWC program? Who told you about the TWC program? 
a. Where did the Peer Educator find you? 
b. How many times did interact with the Peer Educator? (probe for # of meetings 
with PE) 
3. Describe how you became involved in the program. 
4. What can you tell me about the Together We Can project? 
III. Prevention Activities 
A. TWC Curriculum 
1. Content
a. Tell us what you learned from the Red Cross volunteer about HIV prevention.
What information was new? 
b. Which activities did you like better/less than others? Explain. (Discuss why the 
topics are important.) 
c. What did you think of the homework assignments? What instructions did you 
get? Did you understand what you were supposed to do? Is there any 
information that was unclear? Did your trainer do a good job of helping you 
learn?
2. Targeting Peers
a. Describe the peers you spoke with. How did you identify these people? Describe 
the process you used to find people to talk to.
b. How many people did you talk to? What did they think about the messages?
c. When you started, how hard was it to reach out to 10 other people? Did it get 
easier for you? How did that happen? 
d. Describe types of people who you did not speak with but you think should hear 
what you have to say. Do you think the program failed anyone? (beneficiary 
targeting)
3. Motivation 
a. What motivates/demotivates you to continue promoting HIV and pregnancy 
prevention messages? 
b. What lowers your commitment? What are some of the challenges you have in 
your role? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 111 
c. Why do you think Youth Multipliers continue or stop to volunteer? 
4. Activities with Peers
a. After you found people to talk to what did you talk about? Describe the activities 
you did with them. How long did each homework assignment take? Did you have 
enough information to pass on messages accurately?
b. Were you comfortable conducting them? Which were your favorites? Why? 
What materials did you use most often? Or very little?
c. What referrals did you make to YPs? What referrals did they act upon?
5. Record Keeping 
a. What records did you keep of your homework? What did you do with the 
records? What did the person you gave the records do with them? (feedback) 
What is the information used for? Did you understand the forms? What was 
hard?
b. What do you think about the reports and the work involved? Do you think they 
should be continued?
6. Supervision
a. If you have any questions or problems, whom have you turned to? What support 
and resources have you received to be effective?
b. Do you think you have the support and resources to be effective without Red 
Cross support? (sustainability)
7. Follow-up Interventions
a. Did you have any additional training? What did it consist of? Who trained you?
b. Did you enjoy the Follow-Up Intervention? What did you like/dislike about them? 
Did you get any new information? Were you reminded of anything that you had 
forgotten?
B. Non-Curriculum Based Activities 
1. What other HIV prevention activities you have participated in by yourself, with your 
family or with friends? (youth club, community-wide event, mass media, town hall)
2. Describe the activity or event in order of interest to you. Why was it interesting? Why 
was it not interesting? What feedback did you get from your friends who also 
participated?
3. Who did you hear this information from? Do you know who sponsored ____ activity?
4. Did you help to organize or promote any of the events? What did you do?
C. What are other sources of information for HIV prevention? 
1. Did they tell you the same information as you heard from TWC? 
2. What other projects are working on HIV prevention in your area? 
IV. Targeting 
A. Geographic & Beneficiary Targeting 
1. Are these messages appropriate for your community? (geographic targeting) 
2. Where do you think the TWC project should be working but is not? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 112 
B. YM Support 
1. What do people (churches, school teachers, parents) think about TWC in your 
community? 
2. What did you tell your parents about these home work assignments? What did your 
parents think about your participation, especially talking to people about HIV and 
sexuality? 
3. In your experience, was the community positive toward the program? Did they 
actively support it? How? 
V. Program Impact 
A. Impressions of attitude and behavior changes since 2004 
1. What have been the effects of the program on YMs? YPs? What are the biggest 
contributors to that outcome? 
B. Changes in Knowledge 
1. See section III. Prevention Activities; B. What did you learn that you didn’t know 
before? 
2. What myths have been dispelled due to TWC? 
C. Changes in Attitudes 
1. How have people changed their ideas about abstinence because of the program? 
Does this include both boys and girls? 
2. How is being faithful viewed amongst your peers? How are ideas about fidelity 
amongst young people different than before the program? Does this apply to both 
boys and girls? 
3. What adjectives would you use to describe condom use? How has the program 
affected people’s use of condoms? Price? Where to buy? How and why to use 
condoms? 
4. What about interacting with people living with HIV or people living with AIDS, have 
you seen any difference in the way they are treated in the community? Describe 
what you have seen. 
D. Changes in Behavior 
1. What have been the effects the programs has had on HIV transmission? What are 
the biggest contributors to these outcomes? 
2. In your opinion, what high-risk behaviors have changed among people who have 
heard TWC prevention messages? Please provide examples. 
3. What changes have you seen in unintended pregnancies for those who have 
received TWC messages? Please provide examples. 
4. What changes have you seen in STIs, including HIV transmission? What do you 
think caused the change? Please provide examples. 
5. What have been the effects the TWC project has had on HIV testing? What are the 
biggest contributors to that outcome? Please provide examples. 
6. What other behaviors have changed as a result of TWC? Please provide examples. 
VI. Sustainability 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 113 
A. Would you like the program to continue? 
1. Do you think this program should continue? Who should it be targeting? Where? 
2. What could the program do that would help it be more sustainable? 
VII. Recommendations 
A. Overall impressions of the program 
1. What is your opinion about the TWC program? 
2. Is there anything you would like to see that you didn’t? What recommendations do 
you have for improving the project? 
B. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 114 
Youth Multiplier In-Depth Interview 
I. Introduction 
* Describe evaluation content and process. Provide informed consent. 
* Enter interview information & respondent names on Respondent Sign In sheet 
* Do you have any questions for me before we begin?
II. Respondent Characteristics 
A. TWC Participation 
1. Describe how you became involved in the program. 
a. Where did the Peer Educator find you? 
b. How many times did interact with the Peer Educator? (probe for # of meetings 
with PE) 
2. What can you tell me about the Together We Can project? 
III. Prevention Activities 
A. TWC Curriculum 
1. Content
a. What new information did you learn from TWC? 
b. Which activities did you like better/less than others? Explain.
c. What did you think of the homework assignments? What instructions did you 
get? Did you understand what you were supposed to do? Is there any 
information that was unclear? 
d. Did your trainer do a good job of helping you learn?
2. Targeting Peers
a. How many people did you talk to? 
b. Did you speak with anyone that you know someone else spoke with? Did you 
both report that person? Did this happen often?
c. If you didn’t speak with 10 people, why not? What are some common reasons 
that people give you for not participating? How do you respond? What did you 
do if you did not speak with 10 people?
3. Motivation 
a. What motivates/demotivates you to continue promoting HIV and pregnancy 
prevention messages? 
b. What lowers your commitment? What are some of the challenges you have in 
your role? 
c. Why do you think Youth Multipliers continue or stop to volunteer? 
4. Activities with Peers
a. Did you have enough information to pass on messages accurately?
b. What referrals did you make to YPs? What referrals did they act upon?
c. Have you had other young people independently approach you for information 
about HIV, birth control, etc.?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 115 
5. Record Keeping 
a. What records did you keep of your homework? What did you do with the 
records? What did the person you gave the records do with them? (feedback) 
What is the information used for? Did you understand the forms? What was 
hard?
6. Supervision
a. Who supervised your homework assignments? Describe the supervision. (how 
often, by whom, what happened,…)
b. Was this feedback helpful to you?
7. Follow-up Interventions
a. Did the FUIs cause you to speak with your friends again about HIV prevention 
and avoiding pregnancy? What did you talk about? What did the YPs think about 
the FUIs?
B. Non-Curriculum Based Activities 
1. What other TWC activities did you participate in? (youth club, community-wide event, 
mass media, town hall)
2. Did your participation in these other events help you to do your homework 
assignments better? Which ones? Or do you think the curriculum alone gave you the 
tools to be effective?
IV. Targeting 
A. Geographic & Beneficiary Targeting 
1. Are these messages appropriate for your community? (geographic targeting) 
2. Where do you think the TWC project should be working but is not? 
B. YM Support 
1. What did you tell your parents about these home work assignments? What did your 
parents think about your participation, especially talking to people about HIV and 
sexuality? 
2. Have your parents been supportive? What do they do to support your activities? 
3. What have you hear the community saying about TWC? Did they actively support it? 
How? 
C. Targeting Youth Multipliers 
1. What do you think about your role in this program? Why are you important? What 
qualities do you have that makes you good at educating others? 
2. Do you think both boys and girls make good educators? All age groups? Why? 
V. Program Impact 
A. Impressions of attitude and behavior changes since 2004 
1. How has participating in TWC affected you? 
2. Has it changed your relationship with your friends? Positively or negatively? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 116 
3. Do you think that you have more self-confidence? In what way? To do what? How 
did that develop? 
B. Changes in Knowledge 
1. What (myths) do you no longer believe as a result of TWC? 
C. Changes in Attitudes 
1. Have any of your ideas changed as a result of your participation? Describe the old 
and new ideas. 
2. Do you know anyone who is HIV positive? How do you feel about interacting with 
someone who is HIV positive? What if they have AIDS? What sources of information 
made you feel this way? 
D. Changes in Behavior 
1. Has any of your own behavior changed because of your participation? What 
changes have you made in your lifestyle? Describe what you do differently now. 
2. How do you prevent the transmission of HIV and other STIs? (probe on HIV testing, 
abstinence, fidelity, condoms) What made you decide to use this prevention method? 
a. Do you know your HIV status? Have you ever had an HIV test? What made you 
decide to get tested? 
b. Do you have a boy/girlfriend? How do you know that you are safe from HIV, STIs 
or getting pregnant? 
c. How do you use condoms? Where do you get condoms? Where did you get this 
information? Have you ever applied what you learned? 
VI. Sustainability 
A. Would you like the program to continue? 
1. What activities will you continue you continued after you graduate from the project? 
2. What could the program do that would help it be more sustainable? 
VII. Recommendations 
A. Overall impressions of the program 
1. What is your opinion about the TWC program? 
2. If someone from the Red Cross asked you how the program could be improved, what 
would you say? 
B. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 117 
Youth Participant Focus Group Discussion 
I. Introduction 
* Describe evaluation content and process. Provide informed consent. 
* Enter interview information & respondent names on Respondent Sign In sheet 
* Do you have any questions for me before we begin?
II. Respondent Characteristics 
A. TWC Participation 
1. What can you tell me about the Red Cross? How did you learn about the Red Cross? 
2. Who told you about the TWC program? What is the Together We Can project about? 
III. Prevention Activities 
A. How did you participate in the TWC project? (probe on the list below for 
involvement) 
1. TWC curriculum/ Follow-up interventions 
a. How many times did you meet with the YM? (probe on # of meetings with YM) 
b. Where did the Youth Multiplier find you? 
2. Referrals 
a. What services did the Youth Multiplier tell you about? 
b. What services did the Youth Multiplier refer you to? 
c. What services did you use? 
3. Youth club 
a. Did you participate in a youth club? How was/is the club involved in the TWC project? 
4. Community-wide event 
a. Did you participate in any community-wide events? Please describe. 
5. Mass media 
a. Did you hear TWC provide HIV messages on the radio or TV? 
b. What did they talk about? 
c. What did you think of the announcement? 
6. Town hall meeting 
a. Did you attend any meetings where the Red Cross gave a presentation? 
b. What did they talk about? How did this information apply to you?
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 118 
B. What did you learn that you didn’t know before? Which activity did you learn this 
information from? 
1. What did you like about _____ activities? What didn’t you like? 
2. Which activity gave you the most useful information? Why was it useful? 
3. Were there any messages that were unclear? What is still difficult about the topic? 
C. Where (else) have you heard HIV prevention messages? 
1. Did they tell you the same information as you heard from TWC? 
2. What other projects are working on HIV prevention in your area? 
IV. Targeting 
A. Geographic & Beneficiary Targeting 
1. Why do you think your community is an appropriate place for TWC? 
2. What do people (churches, school teachers, parents) think about TWC in your 
community? 
3. Where do you think the TWC project should be working but is not? 
4. Who do you think needs to hear these messages? Where do you find these people? 
What makes you choose that group of people? 
B. YP Support 
1. Who do you talk to for information about HIV, STIs, sex or pregnancy prevention? 
The YM? Your parents? 
2. Who do you think would be the best source of information? Have you spoken with 
that person/those people? What made you decide to (not) have that conversation? 
3. Do you feel more, less or equally comfortable talking and hearing about these issues 
than you were before speaking with the YM? 
V. Program Impact 
A. Changes in Knowledge 
1. See section III. Prevention Activities; B. What did you learn that you didn’t know 
before? 
2. What myths have been dispelled due to TWC? 
B. Changes in Attitudes 
1. How have people changed their ideas about abstinence because of the program? 
Does this include both boys and girls? 
2. How is being faithful viewed amongst your peers? How are ideas about fidelity 
amongst young people different than before the program? Does this apply to both 
boys and girls? 
3. What adjectives would you use to describe condom use? How has the program 
affected people’s use of condoms? Price? Where to buy? How and why to use 
condoms? 
4. What about interacting with people living with HIV or people living with AIDS, have 
you seen any difference in the way they are treated in the community? Describe 
what you have seen. 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 119 
C. Changes in Behavior 
1. What have been the effects the programs has had on HIV transmission? What are 
the biggest contributors to these outcomes? 
2. In your opinion, what high-risk behaviors have changed among people who have 
heard TWC prevention messages? Please provide examples. 
3. What changes have you seen in unintended pregnancies for those who have 
received TWC messages? Please provide examples. 
4. What changes have you seen in STIs, including HIV transmission? What do you 
think caused the change? Please provide examples. 
5. What have been the effects the TWC project has had on HIV testing? What are the 
biggest contributors to that outcome? Please provide examples. 
6. What other behaviors have changed as a result of TWC? Please provide examples. 
VI. Recommendations 
A. Overall impressions of the program 
1. What is your opinion about the TWC program? 
2. If someone from the Red Cross asked you how the program could be improved, what 
would you say? 
B. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 120 
Youth Participant In-Depth Interview 
I. Introduction 
* Describe evaluation content and process. Provide informed consent. 
* Enter interview information & respondent names on Respondent Sign In sheet 
* Do you have any questions for me before we begin?
II. Respondent Characteristics 
A. TWC Participation 
1. What can you tell me about the Red Cross? How did you learn about the Red Cross? 
2. Who told you about the TWC program? What is the Together We Can project about? 
B. Where (else) have you heard HIV prevention messages? 
1. Did they tell you the same information as you heard from TWC? 
2. What other projects are working on HIV prevention in your area? 
 
III. Prevention Activities 
A. How did you participate in the TWC project? (probe on the list below for 
involvement) 
1. TWC curriculum 
2. Follow-up interventions 
3. Referrals 
4. Youth club 
5. Community-wide event 
6. Mass media 
7. Town hall meeting 
B. What did you learn that you didn’t know before? 
1. What was your favorite activity? Why? 
2. What was your least favorite activity? Why? 
3. Were there any messages that were unclear? What is still difficult about the topic? 
 
IV. Targeting 
A. Geographic & Beneficiary Targeting for Messages 
1. Is this the right place to have a project like TWC? Why? 
2. Do you think that you are the right person to hear HIV prevention messages? Why 
do you think the Youth Multiplier spoke with you? 
3. Who do you think needs to hear these messages? Where do you find these people? 
What makes you choose that group of people? 
B. YP Support 
1. Who do you think could help you discuss HIV prevention & unwanted pregnancy 
when you need to? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 121 
a. Have you ever had a discussion with this person? 
b. What did you talk about? 
c. Would you have gone to that same person before you joined the program? 
2. Who would you not tell what you learned? What are some reasons you would not tell 
people about what you learned? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix C: Page 122 
V. Program Impact 
A. Observed Changes 
1. What have been the effects the programs has had on HIV transmission? What are 
the biggest contributors to these outcomes? 
2. In your opinion, what high-risk behaviors have changed among people who have 
heard TWC prevention messages? Please provide examples. 
3. What changes have you seen in unintended pregnancies for those who have 
received TWC messages? Please provide examples. 
4. What have been the effects the programs has had on HIV transmission? What are 
the biggest contributors to these outcomes? 
B. Personal Changes 
1. How have you changed as a result of TWC? Explain with examples. 
2. What (myths) do you no longer believe as a result of TWC? 
3. Do you know anyone who is HIV positive? How do you feel about interacting with 
someone who is HIV positive? What if they have AIDS? What sources of information 
made you feel this way? 
4. How do you prevent the transmission of HIV and other STIs? (probe on HIV testing, 
abstinence, fidelity, condoms) What made you decide to use this prevention method? 
a. Have you ever had an HIV test? What made you decide to get tested? 
b. Do you have a boy/girlfriend? How do you know that you are safe from HIV, STIs 
or getting pregnant? 
c. How do you use condoms? Where do you get condoms? Where did you get this 
information? Have you ever applied what you learned? 
VI. Recommendations 
A. Overall impressions of the program 
1. What is your opinion about the TWC program? Is there anything you would like to 
see that you didn’t? 
B. Do you have any question or comments about things we didn’t talk about that you 
think are important? 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix D: Page 123 
Appendix D: Participatory Data Collection Methods
Data
Collection
Method
Type of Data Constituency Group Purpose of data
   
Community 
Surveys 
Quick  and  dirty 
information  on  how  girls 
feel  about  a  particular 
issue 
Girls in  the community or 
in programs 
Parents,  gatekeepers, 
service  providers  in  the 
community 
Get  an  idea  of  general 
attitudes  and  beliefs  of 
girls  and  other 
community members w/o 
filter 
   
Interviews  More  in  depth 
understanding  of  girl’s 
experiences  with 
particular issue within the 
context  of  their 
experience  and  in  their 
own words 
Girls  who  have  some 
personal  experience  that 
has relevance  to  the issue 
you want to know about 
Use  in  development  of 
educational  campaign  or 
activities/development  of 
engagement  strategies/ 
development of services 
   
Focus 
Groups 
Feedback  on  perspectives 
of  individuals  within  a 
group  context  that  leads 
to  deeper  discussion  and 
illustrative interaction 
Groups  of  people  with 
shared  characteristic  or 
experience  (i.e.  girls  in  a 
particular  community, 
parents  of  girls  in 
program, teachers) 
Provides insight into  how 
your  program/issue  is 
perceived  by  a  particular 
group of constituents  that 
can  help  shape  program 
activities  or  educational 
approaches 
   
Video‐
graphy/ 
Ethno‐
graphic 
observation 
Gives  a  visual  or 
descriptive  look  at  the 
issue/program ‐framing it 
within  an    environmental 
context 
Gives  information  about 
how  the culture of  group 
works 
Participants  in  activities, 
people in the community 
Provides  an 
understanding  of  the 
frame in which an issue or 
program functions 
   
Community 
Mapping/ 
Photo 
Mapping 
Maps  the  terrain  and 
interactive  patterns  of  a 
community  or  group  in 
word  and  with  pictures 
including  transportation 
patterns,    congregate 
settings, public behaviors, 
safe  and  unsafe  spaces, 
information  sites 
Gatekeepers  and 
stakeholders,  people  in 
community 
Provides  a  cultural 
blueprint  for 
understanding  a  group  or 
community  that  informs 
outreach,  organizing  and 
educational  or  program 
services 
   
Relation‐ Provides  information  Girls in programs, parents  Create  a  visual 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix D: Page 124 
Data
Collection
Method
Type of Data Constituency Group Purpose of data
ship 
Mapping 
about  relationships  and 
patterns  of  relationships 
as they are constructed in 
a  person’s  life  either  by 
intent or by circumstance 
and other extended family 
members 
perspective of how people 
see themselves in relation 
to others – what they give 
and what  they look  for in 
others  that  can  influence 
choices,  values,  decision 
making and behaviors 
   
Journaling  Provides  a  structured 
means  for  expression  of 
feeling  and  ideas  and  can 
be  constructed  in  ways 
that  address  particular 
questions  to  draw  out 
thinking  that  is  not 
influenced by others 
Girls in programs, parents 
and other extended family 
and  social  network 
members 
Brings  a  personal 
perspective  regarding  the 
effects  of  an  issue  on  an 
individual  inside  their 
inner  dramatization  of 
their  lives  and  all  the 
attendant players  
   
Photo 
Journaling 
Provides  a  visual  and 
accompanying story to  
express  the  ideas  and 
meaning  that  girls  get 
from  their  relationships 
and surroundings 
Girls in programs, parents 
and other extended family 
and  social  network 
members 
Provides  a  visual 
depiction  of  life  through 
the  eyes  of  the 
participants  which  is 
defining  and  described  in 
their own words 
[Source: P. Catlin Fullwood (2005) Girl’s Best Friend Foundation] 
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix E: Page 125 
Appendix E: Evaluation Recommendations
Cross‐Cutting Issues for the Scaling‐Up Together We Can Project
Geographic Targeting
When determining the appropriate TWC sites, as with all projects, it is advisable to explicitly set
priorities in a list of site selection criteria and clearly document why sites were selected. All activities
may not be feasible in all settings. When considering site selection, evidence of need should certainly be
high on the list. In addition, the NSs’ comparative advantage in implementing in a given site would also
be a strong argument for reduced start‐up times and spending.
Participant Targeting
Continue targeting youth with TWC, but consider that age‐appropriate information should be explicitly
outlined for PEs if future projects are to pursue youth through out‐of‐school activities and children in
each of the three age cohorts. Changes to the curriculum could be specified in a Training of Trainers
(TOT) manual for FMs to reduce additional reprinting costs for PE and participant manuals. In support of
TWC, adult targeting should be clearly stated whilst adaptations to the program will be required
depending on the situational environment and the outcome of the community mapping process. These
adults include parents, government administrators, referral service providers, leaders of faith‐based
organizations, FBOs, school officials and teachers. ARC could, for example, develop guidance specifically
for sessions targeting parents, not just of basic facts, using select TWC activities but also an orientation
to the program with actual materials used on display for viewing. The NS targets also need to be clearly
identified and agreed upon. Initial assessments of staff readiness for effectively handling both
programmatic and administrative duties would have gone a long way.
Situational Environment
During project planning, community mapping and implementation, deliberately take into consideration
the strengths and weaknesses of the operating environment. Use this information to set realistic
activities and targets. TWC may look very different in rural areas versus urban ones.
Volunteerism
NSs are implementing many proven best practices for the recruitment and retention of PEs but a formal
recognition structure should be put in place. The management of adult volunteers should be as
deliberate as that used with PEs.
Monitoring and Evaluation
Each NS should come up with their own M&E plan and include all fo the requisite components. When
developing the M&E framework, consider key decisions that will need to be made to determine whether
activities are being implemented as intended. Develop indicators and systems which track outcomes to
help with decision making, not just output from the start of the project. Simplify the data collection
forms. Train staff and volunteers on M&E basics and explicitly tie data collected to individual and
project achievements and decision making. Complement or replace the pre‐ and post‐test with a more
comprehensive evaluation tool to measure outcomes resulting from all interventions. This is further
described in F.4. Define Indicators and Link Measures to Use below. (See Appendix D: Participatory
Data Collection Methods for ideas.)
HIV‐Prevention Education Activities Contributing to Measurable Changes
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix E: Page 126 
Curriculum‐Based Interventions (CBI)
The content and delivery mechanisms for HIV‐prevention messages are liked and effective. Additional
efforts do not necessarily need to go into adapting the curriculum as CBI testing data does not support
great increases in knowledge or skills because of the changes. It may be helpful to have a TOT manual
for facilitators that contains more specific directions for M&E, age‐appropriate messages, and referral
resources, which can be determined during the community mapping process. Promote spreading the
word instead of giving homework assignments. Referrals require more thought on involving the service
providers. It would be advisable to provide YMs with referral forms that are signed upon receipt of
services and returned to the NS staff for tracking.  
NSs should continue to strategize on effective targeting. A more methodical approach, such as the one
used with MOE, for gaining entrée with FBOs might be more effective for obtaining buy‐in from religious
leaders. Another option could be to develop a written presentation specifically targeting religious
leaders that FMs and volunteers would be trained to deliver. Some people did manage to connect with
youth through FBOs in Tanzania. It would be appropriate to share these strategies among the staff and
volunteers. But in the final analysis, if the FBO gatekeepers continue to decline to allow TWC to be
presented to the youth in their charge, recruitment efforts should be redirected towards more
amenable entry points.
Follow‐Up Interventions (FUI)
Stay the course of using FUIs to reinforce knowledge of YMs who had participated in CBIs.
Adult‐Child Communication (ACC)
Continue with this activity but develop a clear targeting strategy. Determine whether the YMs or the
parents will be used as NSs’ point of entry, or both. Once the entry point has been established, work
with NS staff to market the activity and recruit participants. This could be accomplished though
community leaders such as the community‐based advisory board or religious leaders and/ or a
concerted effort to engage YMs in the recruitment process of their trusted adult.
HIV‐prevention Education Activities: Community Engagement
Town Halls and Community Councils
The actual form either type of meeting will take will vary depending on the local situation, e.g.,
community interest, school system and government structure. Distinguishing more clearly between the
two types of community engagement events would make it easier for TWC staff and volunteers to
promote them to potential attendees, as well as simplify record keeping. Town halls should be
formalized in the TWC documentation as the venue for reaching out to and informing the community,
especially stakeholders not otherwise involved with the program. Community councils should be
advisory bodies that take an active role in planning and decision‐making. This distinction should be
clearly spelled out in TWC documentation, and assumed for the recommendations provided next.
  
Town Halls. Every community reached by TWC should have one entrée meeting with gatekeepers to
request permission to operate in the area and periodic meetings where new information is presented,
progress is documented, and questions and concerns are addressed. The public should be motivated to
attend by offering education, information, or other incentives that directly benefit participants. These
incentives should be selected by assessing local needs during the initial set up and community mapping
process. As noted in the Tanzania report, these meetings do not necessarily have to be called “town
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix E: Page 127 
halls,” nor do they need to follow that format. The important point is to ensure that the community is
informed, engaged, and invested in the TWC process.
Community Councils. Rather than relying on local government officials and other community leaders, the
community‐based advisory board could be made up of RC branch volunteers, with ARC providing
technical assistance, and coordinated by a TWC staff member. Local government should be kept
informed but should not be solely relied on for regular advice. In Tanzania, the assigned NS coordinator
might be the branch development manager; in Guyana, this might be handled by a qualified volunteer,
e.g., Peace Corps, or it may be necessary to hire someone for the role. Board activities might include
holding the town hall meetings, and members might attend TWC staff meetings and should participate
in the community mapping process. This will require that ARC invest in developing the advisory board’s
capacity and provide clear guidance.
Youth Clubs
TWC should establish guidelines for working with youth groups that make clear the target population
and the process for engaging them, as with any other group such as church groups of school children.
Youth groups should be identified in the initial stages of establishing TWC in a region during community
mapping and targeted for gaining access to youth in the same way as any other organization. As
appropriate, other locations or contexts in which youth gather could be identified by YMs and YPs and
added to the referral list as part of the TWC take home assignments. The concept could be recast as
“youth groups” rather than clubs in order to avoid the connotation of a specific venue like a club house.
Community‐Wide Events (CWE)
Community‐based advisory board members and YMs could be charged with identifying and alerting NS
and PEs of upcoming events, and with promoting TWC’s upcoming participation to their constituents.
General diffusion CWEs could also be used for recruiting branch members and PEs and for fundraising.
For example, PEs could have a booth with signup sheets for youth and groups that would like more
information, or hold drawings for prizes to be announced at a town hall meeting to encourage
attendance.  
While USAID’s purpose in creating the distinction between interpersonal and general diffusion CWEs is
appreciated, the purpose of the general diffusion CWE needs to be clarified in terms of TWC. Activities
involving direct one‐to‐one communication make more sense if incorporated into the CBI or FUI peer‐to‐
peer outreach. Regardless, ARC needs to clarify exactly what is to be gained from three‐minute
conversations and then communicate that to the TWC volunteers.  Monitoring success of the events will
be easier if the rationale for conducting the events is well established.
Mass Media
The Tanzania and Guyana National Societies and the ARC should limit time and resources spent on mass
media, if at all, to partnering with other organizations or government to conduct mass media
campaigns. This would most likely be an activity for TWC management rather than the volunteers and
the approach will be different in each country.
National Red Cross Society Capacity Building
Working through the planning process with NS staff, including choosing indicators and setting targets
for the SO, would have been a valuable lesson and ensured that both parties held similar expectations
for the upcoming year. This joint planning document could have served as an agreement between ARC
and the NS on roles and responsibilities including who is ultimately accountable for decision making.
ARC TWC Evaluation Composite Report 
True Panacea, LLC  Appendix E: Page 128 
Joint work plan development and developing usable indicators would have helped both NSs. For
example, the GRCS management style was to focus on achieving targets. Had the indicators been
developed such that there were targets for outcomes other than the number of organizations involved
and people reached and trained, FMs would have had more direction on what they were trying to
achieve for the overall project.
Training
ARC should take leadership by providing their available modules for OD training to NS management.
ARC and NSs should collaboratively plan sessions, develop a training schedule, and come to a mutual
understanding of how training will occur. NSs can support training and mentoring goals by linking staff
outcomes to human resource management and use the performance review of staff and volunteers
that occurred at the beginning of the project.
Mentoring
Mentoring should occur, and as with training, should be planned with the NSs and reassessed on an on‐
going basis. ARC’s global TWC experience is the value they bring to the table, both in terms of technical
activities, and more importantly, in setting processes, procedures and standards and documenting them
in a TWC Project Handbook.
Develop Partnerships
Activity‐based partnerships should be informed by the community mapping process during which gaps
are identified between what TWC can offer and what the community wants and needs. The mapping
activity will help is identifying potential partners and the gaps between community needs and what the
TWC project is offering. Creating partnerships is aligned with all the SOs, not just SO2. Ensure that
management is responsible for identifying and developing formal partnerships with written agreements
to assist in reaching both project and community objectives.
Disseminate Best Practices
Disseminating best practices is clearly a valuable exercise. NSs should continue to hold staff and
volunteer meetings. NSs should make serious efforts to participate in the national dialogue through
established coalitions and by PMs targeting individual organizations identified during community
mapping to plan complementary activities and develop operating agreements. NSs could hold annual
dissemination and feedback meetings at national, regional, district and local levels which could be part
of the definition of Town Hall meetings. International conferences and workshops are a nice, but will be
difficult to finance without significant amounts of external funding unless NSs are willing to fundraise for
them.  This is most likely a lower priority for the NSs than delivering services.