Strengthening Relationships and Resilience among HIV affected Adolescents and their Caregivers: A Pilot Study of Let's Talk in South Africa (TALC) Final Evaluation Report March 2021 Support for this project is provided by the United States Agency for International Development (USAID) Southern Africa under the President’s Emergency Plan for AIDS Relief (PEPFAR) through Cooperative Agreement No. AID-674-A-12-0002 awarded to Tulane University. The views expressed in this document do not necessarily reflect those of USAID or the United States government. i Project Title: “Strengthening Relationships and Resilience among HIV affected Adolescents and their Caregivers: A Pilot Study of Let's Talk in South Africa (TALC)” Contractor: Tulane University (Highly Vulnerable Children’s Research Center (HVC-RC), School of Social Work) Award No: AID-674-A-12-00002 Project Start Date: January, 2015 Project End Date: January, 2021 Persons responsible for compiling the report: ii Tonya Renee Thurman Country Director Postnet Suite 13, Private Bax X 80 Bergvliet, Cape Town 7864 South Africa Tel: (+27) 71 801 -0102 Email: tthurma@tulane.edu Tory Taylor M&E specialist 1440 Canal Street, New Orleans, LA 70112 U.S.A. Tel: (504) 988 -4027 Email: ttaylor@tulane.edu Johanna Nice Senior Program Manager 127 Elk Place New Orleans, LA 70112 U.S.A. Tel: (504) 862 -3476 Email: jnice@tulane.edu Brian G. Luckett Senior Research Scientist & Statistician 1440 Canal St. New Orleans, LA 70112 U.S.A. Tel: (504) 669 -7044 Email: bluckett@tulane.edu Maretha Visser Professor of Psychology Humanities Building, University of Pretoria cnr Lynnwood Road and Roper Street Hatfield, 0028 Pretoria iii Tel: (+27)12 420-2549 Email: maretha.visser@up.ac.za Table of Contents Acknowledgments ................................................................................................................................................ v List of Acronyms................................................................................................................................................... vi List of Tables, Figures and Appendices............................................................................................................... vii 1. Executive Summary....................................................................................................................................... 1 1.1 Introduction ................................................................................................................................................ 1 1.2 Methodology............................................................................................................................................... 1 1.3 Results......................................................................................................................................................... 3 1.4 Limitations .................................................................................................................................................. 4 1.5 Conclusions and Recommendations........................................................................................................... 4 2. Project Background........................................................................................................................................... 6 3. Methodology..................................................................................................................................................... 7 3.1 Evaluation purpose and questions.............................................................................................................. 7 3.2 Program Design........................................................................................................................................... 8 3.2.1 Program Description............................................................................................................................ 8 3.2.2 Program structure and materials.......................................................................................................10 3.3 Program evaluation...................................................................................................................................12 3.3.1 Description of the setting and population.........................................................................................12 3.3.2 Implementing partners......................................................................................................................14 3.3.3 A pre-test post-test pilot study design ..............................................................................................15 3.3.4 Pathways to better intervention outcomes: structural equation models.........................................18 3.3.5 Focus Group Discussions (FGD)..........................................................................................................18 3.3.6 Caregiver incentives...........................................................................................................................20 3.3.7 Costing of the program......................................................................................................................20 3.4 Stakeholder engagement..........................................................................................................................21 3.5 Ethical considerations and assurances .....................................................................................................21 4. Findings and Limitations................................................................................................................................. 23 4.1 Pilot survey ...............................................................................................................................................23 4.2 Process and qualitative data.....................................................................................................................26 4.3 Pathways analysis.....................................................................................................................................28 4.4 Caregiver incentives..................................................................................................................................29 iv 4.5 Costing analysis.........................................................................................................................................29 4.6 Overall study limitations...........................................................................................................................29 5 Conclusions and Recommendations................................................................................................................31 5.1 Conclusion.................................................................................................................................................31 5.2 Recommendations and future studies......................................................................................................31 6. Dissemination .................................................................................................................................................33 7. References......................................................................................................................................................35 v Acknowledgments Many people were instrumental to realizing the evaluation of Strengthening Relationships and Resilience among HIV affected Adolescents and their Caregivers: A Pilot Study of Let's Talk in South Africa (TALC) project. The study team is particularly grateful for the support provided from personnel at USAID Southern Africa, particularly Anita Sampson and Naletsana Masango for their support of this study and the development of the Let’s Talk program. We also extend our appreciation to the partner organizations involved in the pilot, Children in Distress Network (CINDI) and HIV South Africa (HIVSA), for their leadership and support throughout the pilot process. We are grateful to the facilitators and supervisors from these organizations for their dedication, hard work, and for often exceeding expectations in their commitment to the project. Finally, we extend our gratitude to the program and study participants for sharing their time and personal experiences to help us understand the potential of this intervention to mitigate challenges faced by vulnerable families. We sincerely hope that these individuals and others in similar circumstances are the ultimate beneficiaries of this work, through improved programming. vi List of Acronyms ACASI Audio computer-assisted self-interview AIDS Acquired Immune Deficiency Syndrome CBO Community Based Organization CINDI Children in Distress Network CLH Caregivers living with HIV FGD Focus Group Discussions HCT HIV Counseling and Testing HIV Human Immunodeficiency Virus HIVSA HIV South Africa HVC-RC Highly Vulnerable Children Research Center KZN KwaZulu-Natal NGO Non-governmental Organization OVC Orphans and Vulnerable Children OVCY Orphans and Vulnerable Children and Youth PEPFAR The United States President‘s Emergency Plan for AIDS Relief RCT Randomized Control Trial RSA Republic of South Africa TALC Teens and Adults Learning to Communicate USAID United States Agency for International Development vii List of Tables, Figures and Appendices Tables Table 1. Pilot program session outline Table 2. Eligibility criteria for quantitative pilot study Table 3. Eligibility criteria for qualitative study Table 4. Baseline characteristics of adolescents and their caregivers Figures Figure 1. Let’s Talk Theory of Change Figure 2. Adolescent Sex Refusal Self-Efficacy Figure 3. Adolescent HIV Transmission Knowledge Figure 4. Adolescent Condom Knowledge Figure 5. Adolescent Condom Negotiations Figure 6. Adolescent DASS 21 Figure 7. Caregiver-Adolescent Sexual Communication Figure 8. Caregiver-Adolescent Connection Figure 9. Caregiver DASS 21 Figure 10 Caregiver HIV Knowledge Figure 11. Pilot program session attendance among caregivers Figure 12. Pilot program session attendance among adolescents Figure 13. Attendance of joint sessions by caregiver-adolescent dyads Figure 14. Self-rated facilitator preparedness and competencies Appendices (in separate attachment) Appendix A: Study Protocol Appendix B: Informed Consent Forms (English) Caregiver Consent Form Household Tracking Consent Form Appendix C: Data Collection Instruments Adolescent Baseline Survey Caregiver Baseline Survey viii Adolescent Follow-up Survey Caregiver Follow Up Survey Household Tracking Form Appendix D: Brief bios of evaluation team 1 1. Executive Summary 1.1 Introduction This evaluation report presents details of the study investigating the ability of a structured, evidence￾based and family-centered program, Let’s Talk, to address behavioral and mental health outcomes among orphaned and vulnerable adolescents and their caregivers in South Africa. The program was modelled on an existing evidence-based program for HIV-affected families in the US, Teenagers and Adults Learning to Communicate (TALC), with content also derived from evidence informed HIV prevention and parenting curriculums in South Africa. It is designed to address risk factors common among orphaned and vulnerable adolescents, including elevated risk for poor psychological health and sexual risk behavior. These efforts are accentuated through parallel programming to support caregivers’ mental health and parenting skills. Let’s Talk was developed in South Africa by the Highly Vulnerable Children Research Center (HVC-RC) at Tulane University in collaboration with the University of Pretoria, with funding from the President’s Emergency Plan for AIDS Relief (PEPFAR) via the United States Agency for International Development (USAID) Southern Africa. Let’s Talk is a structured, manualized HIV prevention intervention offered to adolescents aged 13 years and older and their primary caregiver, and it consists of weekly cognitive￾behavioral based group sessions targeting emotional distress and conflict within families affected by HIV and AIDS. Importantly, it is designed to address risk factors common among orphaned and vulnerable adolescents, including elevated risk for poor psychological health and sexual risk behavior. These efforts are accentuated through parallel programming to support caregivers’ mental health and parenting skills. The key hypothesis underlying this pilot study was that Let’s Talk would improve HIV prevention knowledge and behavioral skills, caregiver and adolescent mental health, and family dynamics such as the adolescent-caregiver relationship and sexual health communication. 1.2 Methodology A mixed methods evaluation of the Let’s Talk intervention was employed to assess the effectiveness and processes of the program over the course of two years (2015-2016). A pre-test post-test pilot study design was utilized to assess program effectiveness among caregivers and their adolescents, aged 13-19 years, who participated in the initial implementation of the program. In addition, focus group discussions (FGD) were conducted among 24 groups to complement the quantitative study. A total of 78 caregivers and 92 adolescents that participated in subsequent program offerings at 2 of Children in Distress Network’s (CINDI) community-based organizations (CBOs) in Umgungundlovu district, KwaZulu-Natal (KZN) were included. The content of the FGDs was divided into two themes among both caregivers and adolescents – some groups focused on mental health outcomes, while the others focused on sexual health, behaviour and communication. Participants were recruited from the list of beneficiaries that had attended at least 50% of the program sessions. FGDs were 2 conducted in isiZulu by trained fieldworkers not affiliated with the program, transcribed and translated into English for analysis. Caregiver-adolescent pairs were recruited for intervention participation by two South African non-profit organizations focused on capacity building of CBOs that serve orphans and vulnerable children and youth (OVCY) and their families in KZN and Gauteng Province. Prior to intervention initiation, a baseline survey was administered to eligible caregivers and the adolescents in their care in June and July 2015. A follow-up survey was administered three months following the intervention period (nine months after the baseline assessment) to enable comparison of the magnitude and direction of change in short-term outcomes among those with exposure to the intervention. Details of the full pre-test post-test pilot evaluation, including the study design and sample details, are reported elsewhere (Thurman et al., 2018). Let’s Talk was piloted by CINDI, located in KZN, South Africa in 2015 and implemented again among new participants in 2016. In addition, several analyses were conducted to investigate different aspects of the intervention. 1) Pilot evaluation: This was a pilot survey study adopting quantitative one-group pre- and post￾design. Twelve Let’s Talk groups, each serving approximately 10 families, were piloted by two local community-based organizations in Gauteng and KZN provinces, South Africa, HIV South Africa (HIVSA) and CINDI. Face-to face interviews were administered at baseline in June and July 2015, prior to intervention. Post-test surveys were conducted three months after the final session (nine months after baseline assessment) in March 2016. The pilot study included adolescents aged 13-17 years and their caregivers. The analysis was restricted to the 95 female caregivers and 105 adolescents who participated in Let’s Talk and completed both surveys. Face-to-face interviews were administered to participants in their homes using portable electronic data collection devices, with adolescents using audio computer-assisted self-interviewing (ACASI) technology to respond to portions of the interview addressing sexuality topics). All interviews were delivered in the preferred language of the participant, namely English, isiZulu or Sesotho. Data analyses were completed with statistical programs SAS 9.3 and STATA 14. Generalized estimation equations were used to estimate average change on priority outcomes. 2) Process evaluation: As a supplement to the Let’s Talk quantitative evaluation, a series of focus group discussions (FGDs) with adolescents and caregivers who took part in 2016 Let’s Talk sessions were conducted. FGDs constituted a qualitative research technique that allowed for in-depth exploration of issues with selected participants in a small group setting. FGDs were conducted in October 2016 among program participants in Umgungundlovu District, KZN. The primary aim of these FGDs was to assess perceived intervention impact on the adolescent-caregiver relationship and sexual health communication. FGD eligibility was limited to Let’s Talk participants with at least 50% attendance from three CBOs in KZN Province who implemented the program in 2016. Discussions were audio recorded, transcribed and translated by the fieldwork team. A thematic analysis was performed with the aim to understand findings meaningfully across individual respondents and respondent groups. 3 3) Pathways analysis: In addition, a post hoc path analysis was performed using structural equation modeling to test whether changes in key outcomes were related to improvements in caregiver/adolescent communication on sexual health. This analysis was limited to data from the 64 female adolescents ages 13–17 and their caregivers who attended at least one Let's Talk session and completed both the pre- and post-intervention surveys. 4) Incentivizing participation: Caregiver participation was identified as a significant implementation challenge to the Let’s Talk program. To inform future intervention offerings, the impact of varying incentives on caregivers’ attendance was investigated. Four CBOs in KZN were randomly assigned to give one of four caregiver incentives. 5) Costing of the program: KPMG Services (Pty) Ltd (KPMG) was engaged by Tulane University (Tulane) to investigate the cost of implementing the Let’s Talk program for adolescents and caregivers. The objective of the study was to generate estimates and conclusions regarding the cost per participant as well as cost implications for expansion, replication, and modifiable implementation characteristics. Caregivers provided written informed consent for their own and the adolescent’s participation, and adolescents provided written informed assent. Ethical approvals were granted by the Tulane University Institutional Review Board, USA and the Faculty of Health Sciences Research Ethics Committee at the University of Pretoria, South Africa. 1.3 Results • Statistically significant improvements were found on all adolescent and caregiver outcomes measured in the pre-post survey, except for adolescent reported self-efficacy to refuse sex. Large improvements were seen in terms of adolescent mental health (26%), communication with caregivers about risk (21%) and condom knowledge (17%). Caregivers also exhibited a large improvement in their mental health (22%). Moderate improvements were seen with regard to adolescents’ self-efficacy for condom negotiation and use (12%) and caregiver-adolescent connectedness (11%). Smaller, albeit significant improvements were also seen in adolescent (7%) and caregiver (3%) HIV transmission knowledge. • The path analysis illustrated the indirect effect of caregiver mental health and the direct effect of adolescent mental health on dyad connectedness, which directly influenced the level of open communication between adolescents and their caregivers about sexual health. Findings demonstrate that building a closer relationship with improved communication among caregivers and adolescents is contingent on improved mental health, a conclusion which is particularly important in South Africa given the demonstrated higher rates of psychological distress experienced by vulnerable children and their caregivers. • Qualitative findings showed that increased communication skills and emotional control improved the caregiver-adolescent relationship, thereby opening the door for more sensitive conversations. However, facilitators and participants noted that frank discussions were still difficult due to cultural taboos and that messaging around relationships was often indirect and negative. 4 • Increasing adolescent-caregiver communication about sex and sexual health matters is possible, despite the perceived barriers to doing so. This is a critical stepping stone for ensuring long lasting prevention effects. Through communication about sex and sexual risks, caregivers can provide ongoing information and can shape the beliefs, attitudes and values of young people’s that can ultimately affect their behaviour. • In addition, offering significant incentives may be crucial to sustaining caregiver participation in the program. Higher incentive packages resulted in better completion rates. • While the total cost of implementing Let’s Talk was high given serving one group of adolescents required investment in two separate groups (adolescents and caregivers) as well as preparation costs including training of facilitators and printing of curricula and participant workbooks. Due to economies of scale, the average cost per participant reduced significantly the more participants there were. • Results from the qualitative component of the study were used to contextualize quantitative findings in journal articles and presentations. In particular, these findings are helpful for understanding the mechanism by which Let’s Talk may improve adolescent and caregiver outcomes. Key results were made available through peer reviewed journals and incorporated into the Let’s Talk policy brief for use by programmers and policy-makers alike. 1.4 Limitations Several limitations should be considered when interpreting evaluation results: The pilot study had no comparison group and a small sample size. The study population was limited to the beneficiaries enrolled by community-based organizations into their programs and may not be generalizable to other groups. The intervention was offered to 75% of the caregivers and 56% of the adolescents enrolled with program implementing partners. All focus group participants had attended at least half of the intervention session which could result in more positive affirmations. 1.5 Conclusions and Recommendations Results from this study suggest that Let’s Talk holds significant potential and helps to fill the gap in the South Africa SA for family centered programs. Based on the findings of Let’s Talk program evaluation, a number of recommendations have been compiled for further research and program scale-up: Recommendation 1: Family-centered program models hold promise to mitigate adolescent OVC risk. Findings from the pilot study highlight the significant potential of interventions that engage adolescents alongside their caregivers. Efforts to improve caregivers’ mental health and parenting skills can mitigate adolescent risk behavior. Recommendation 2: A streamlined shortened programme model and quality control is necessary. Participant attendance challenges identified led to a shortened and simplified session set. Findings were used to finalize materials and guidance for a revised 14-week program released in 2017 consisting of individual and joint sessions for adolescents and their caregivers. A 10-week version is also being 5 developed and will be released in 2021. Tulane continues to work with Let’s Talk implementers to support ongoing program quality control, including training enhancements and efforts to maximize participant attendance at sessions. Recommendation 3: A randomized controlled trial aiming to provide more robust evidence of the program’s effectiveness should be conducted. The revised version of Let’s Talk is already undergoing extensive scale-up to benefit an estimated 150,000 adolescent-caregiver dyads in South Africa as part of the DREAMS program. This extensive reach coupled with the significant investment required to initiate family-based versus individual-level interventions warrants additional study as to the program’s impact. Recommendation 4: Adolescent-only models are necessary. In response to stakeholders’ requests for a streamlined version of Let’s Talk that could be implemented in schools and other settings without requiring caregiver co-engagement at in-person sessions, Tulane developed Let’s Talk Teens in 2018. While caregiver engagement offers notable advantages, Let’s Talk Teens is designed to reach adolescents whose circumstances make caregiver co-participation difficult or impossible. 6 2. Project Background South Africa is home to the largest HIV epidemic in the world and accounts for 25% of all new infections among adolescents (UNICEF., 2016b). An estimated 3.3 million children and adolescents in South Africa have lost one or both parents, many to AIDS (UNICEF., 2016a). Orphanhood increases the risk of HIV infection and poor mental health (Operario et al., 2011; Sharp et al., 2015). The mental status of care￾givers also plays a role in the wellbeing of children (Jardin et al., 2017). Programs that effectively mitigate HIV risk among adolescents – especially orphans and others whose vulnerability is greatest – are therefore vital to epidemic control. Interventions that address adolescents’ psychological health and behavioral HIV risk through improving child-caregiver communication, parenting and problem-solving skills hold significant promise for mitigating the epidemic in this priority population. Programs similarly need to address the mental health of adolescents and build their communication skills and self-efficacy. While recent decades have seen significant investment in a range of programming to support orphans and other vulnerable children (OVC), urgent need exists for evidence identifying the most effective interventions and services, with a focus on improving their health and psychosocial outcomes. In an effort to fill this need, USAID South Africa supported the development and evaluation of promising initiatives addressing priority problems among OVC. To that end, Tulane University and the University of Pretoria developed the USAID-funded adolescent and caregiver support intervention—called Let’s Talk—in 2015. This reports details program processes and evaluation over the course of the program development. Let’s Talk is a structured, family-centered adolescent HIV prevention program designed to address individual HIV transmission risk factors common among orphaned and vulnerable adolescents, including elevated risk for poor psychological health and sexual risk behavior. It is a structured, manualized HIV prevention intervention offered in a support group format to adolescents aged 13 years or older and their primary caregivers. The program aims to build core HIV knowledge and behavioral skills in tandem with support for caregiver and adolescent mental health, stronger relationships, and improved parenting practices. The intervention includes cognitive behavioural therapy-based components for adolescents and caregivers – emphasizing goal-setting, challenging negative thoughts, and problem-solving skills, as well as adolescent-caregiver communication and condom and sexual refusal negotiation. These social and self-regulatory strategies are coupled with knowledge and technical skills for condom use and pregnancy and HIV prevention. 7 3. Methodology 3.1 Evaluation purpose and questions The primary purpose of the evaluation study was to assess the potential effectiveness of an evidence￾based family-centered intervention on mental health, parenting and adolescent risk behavior outcomes. The pilot study also examined secondary outcomes including participants’ HIV knowledge and attitudes, caregiver psychological health, and the adolescent-caregiver relationship. Evaluation results have been used to guide programmers and policymakers in setting priorities for OVC programs, including allocating resources to maximize benefit to adolescent OVC and their families. The underlying study hypothesis is that the intervention will positively impact the primary study outcomes of adolescent psychological health and risk behaviors. Secondary outcomes include participant’s HIV knowledge and attitudes, caregiver psychological health, adolescent-caregiver relationship, their social support, and HIV positive caregivers’ illness management and prevention practices. In addition, focus groups discussions were conducted with the participants of Let’s Talk program in order to ascertain understanding of program impacts and areas for improvement. The qualitative component of the evaluation explored if, and in what ways, the Let’s Talk program succeeded in stimulating effective communication about sex between caregivers and the orphaned and vulnerable adolescents in their care, alongside psychological and behavioral outcomes. Specifically, the pilot study addressed the following evaluation questions and objectives:  Estimate the impact of Let’s Talk intervention on participating caregivers and the adolescents age 13-17 years under their care in the following areas: a) Adolescent’s psychological health, risk behavior, sexual intentions, HIV prevention knowledge and attitudes, social support and coping abilities. b) Caregiver’s ability to effectively cope with parenting as measured through their psychological health, general health, social support, and substance use. c) Caregiver and adolescent relationship as measured through reduced family conflict and harsh discipline practices, and improved communication and attachment. d) HIV+ positive caregivers’ illness management, HIV knowledge, and protective behaviors.  As a supplement to the quantitative evaluation, a series of focus group discussions (FGDs) with adolescents and caregivers who took part in the 2016 Let’s Talk sessions offered by Children in Distress Network’s (CINDI) community-based partner organizations were conducted. FGDs constituted a qualitative research technique that allows for in-depth exploration of issues with selected participants in a small group setting. The specific questions and objectives of the qualitative research were to: a) Understand how participants perceive the intervention’s effects, especially across the domains included in the pilot survey. b) Probe the language used by adolescents and caregivers to describe these outcomes, as a means of identifying potential ways to strengthen future survey measures.  Mixed methods, including structural equation modeling and qualitative data, were used to explore the role of various intermediate outcomes in achieving improved parent-adolescent 8 communication about sex among female intervention participants. The specific questions and objectives of this modeling were: a) How does parental knowledge affect the quality of the parent-adolescent relationship? b) What is the effect of the caregiver’s mental health and the adolescent’s mental health on the caregiver-adolescent relationship and sexual health communication? c) How does the caregiver-adolescent relationship affect the level of parental sexual communication?  Caregiver participation was identified as a significant implementation challenge to the Let’s Talk program. To inform future intervention offerings, the impact of varying incentives on caregivers’ attendance was investigated. The specific questions were: a) What is the effect of varying incentives on caregiver attendance to the Let’s Talk program?  The Let’s Talk intervention showed some improvements on adolescent and caregiver outcomes measured in the pre-post survey. Tulane University (Tulane) engaged with KPMG Services (Pty) Ltd (KPMG) to investigate the cost of implementing the Let’s Talk program for adolescents and caregivers. The objective of the study was to generate estimates and conclusions regarding the cost per participant as well as cost implications for expansion, replication, and modifiable implementation characteristics. Specific questions were: a) What lessons could be learned from the rollout of the program by three implementing partners and one of their community-based organisations respectively, in order to inform future funding requirements of the program? b) Based on lessons learned from current implementers, what key elements need to be accounted for when funding the program in order to achieve the desired level of quality? c) What is the cost of providing the program, based on a theoretical model which accounts for the minimum funding requirements? d) What factors will contribute to a change in this base theoretical cost when replicating the program in different contexts? 3.2 Program Design 3.2.1 Program Description Let’s Talk was developed by the Highly Vulnerable Children Research Center (HVC-RC) at Tulane University in collaboration with the University of Pretoria, with funding from the President’s Emergency Plan for AIDS Relief via the United States Agency for International Development Southern Africa. It is a structured, manualized HIV prevention intervention offered in a support group format to adolescents aged 13 to 19 and their primary caregivers. Let’s Talk was developed to address key issues facing adolescents affected by HIV and AIDS, including elevated risk for poor psychological health, sexual risk behavior and HIV infection. These efforts are accentuated by parallel support for caregivers, addressing their personal challenges and working to build skills for effective emotional coping and parenting. The program is designed to bolster psychosocial well-being, risk awareness and communication skills among adolescents and their caregivers, ultimately leading to better decision making for improved health. Let’s Talk sessions build caregivers’ competencies in parenting, psychological self-care and adolescent risk- 9 reduction, while helping adolescents develop and practice strategies for integrated wellness and healthy emotional and sexual development. Program development was based on three theoretical frameworks. Eco-developmental theory recognizes family dynamics as pivotal to adolescent outcomes (Szapocznik & Coatsworth, 1999) and is increasingly used to guide adolescent HIV prevention and care (Ortega et al., 2012; Perrino et al., 2000; Prado et al., 2010). Cognitive behavioral therapy (CBT) posits that thoughts, emotions and behaviors are linked and that modifying one can affect the others in predictable ways. The evidence base for CBT is robust, and several recent reviews support its efficacy for treating psychological problems, including depression and anxiety, in adults and children generally, as well as in HIV-affected subgroups (Butler et al., 2006; Hofmann et al., 2012; Sherr et al., 2011). Lastly, Bandura’s social learning theory upholds that learning occurs in a social context –such as a support group setting – through direct experience, observation, modelling and imitation (Bandura, 1977). A multi-level theory of change for the program was established (Figure 1), suggesting that better mental health among both caregivers and adolescents would improve their relationship, contributing to more positive parenting and adolescent self-efficacy. Thus, the program was intended to help participants build emotional coping, communication, and problem solving skills with a focus on resolving issues that commonly arise in family life. The group also provides a forum for participants to develop social and emotional support and expand their peer network. The development of key competencies specific to HIV and sexual health is also fundamental to the program, including ensuring all participants know how to prevent HIV and STI transmission and unintended pregnancy. To build self-efficacy, adolescents also require opportunities to consider real-life situations that may put them at risk, and to practice protective skills such as sexual refusal, condom use, and condom use negotiation. Meanwhile, caregivers’ parenting skills can be expanded to include strategies for mitigating adolescent sexual risk behavior and promoting safe intimate relationships, including open communication about sexual health issues, HIV testing, and dating violence. Several existing family-centered, evidence-based interventions addressing these factors were examined in order to identify candidate interventions for adaptation within the South African context. One intervention exhibiting particularly robust evidence was the Teens and Adults Learning to Communicate (TALC) program, developed by Rotheram-Borus and colleagues at the University of California’s Los Angeles Center for HIV Identification, Prevention and Treatment Services (M. J. Rotheram-Borus et al., 2001). TALC aims to support the mental health of caregivers and adolescents, improve family relationships and increase participants’ knowledge about HIV risk. While originally conceived as an intervention for parents living with HIV and their adolescents, the program builds emotional coping and problem solving skills that are broadly applicable. TALC was selected as the foundational model for the new program on the basis that it addressed many risk factors identified in the literature, was family￾based, had been applied in several low income country contexts such as Thailand and Haiti, demonstrated numerous long-term positive outcomes in follow-up studies, and incorporated CBT (Li et al., 2012; M. Rotheram-Borus et al., 2004; M. J. Rotheram-Borus et al., 2001; Smith Fawzi et al., 2012). 10 Figure 114. Let’s Talk Theory of Change Participants meet in small closed group sessions led by a trained facilitator and cover predetermined topics in sequence. Some sessions included caregivers and adolescents together; others involve separately the caregiver and adolescent groups to deliver content that is specific to each. Sessions are designed to help caregivers communicate more effectively with adolescents, build family strengths, set boundaries, solve problems and manage difficult situations at home. Participants increase their understanding about risk behavior among adolescents, discuss barriers to communication about sex in the home, gain HIV, STI and pregnancy prevention information; and practice communicating with adolescents about sensitive issues such as sexuality. Additionally, the program exposes adolescent participants to information and skills for talking about sensitive issues with their caregivers, saying no to sex, negotiating condom use and protecting themselves from risky situations, pregnancy and HIV. A comprehensive manual, covering all activities and materials needed for each session and including sample scripts and facilitation guidelines for each activity, is fundamental to the program. 3.2.2 Program structure and materials Let’s Talk was initially designed in 2015 as a three-phase program with 19 caregiver sessions and 14 adolescent sessions, of which six were joint caregiver-adolescent sessions (Table 1). Phase 1, or the first five sessions, allowed caregivers to build personal emotional coping and problem solving skills before shifting the focus to their adolescents. Phase 2 included four sessions in which caregivers focused on improving their parenting skills, which included developing a deeper understanding of adolescent behavior, learning to communicate more effectively with adolescents, and practicing setting boundaries. Simultaneously, adolescents focused on their own mental health and wellbeing during four individual 11 sessions – learning to cope with difficult emotions, set personal goals, and reduce family conflict. Through four joint sessions in Phase 2, caregivers and adolescents worked together to build mutual understanding, explore family strengths, and manage difficult situations at home. Finally, in Phase 3 during four individual sessions, caregivers learned about HIV and adolescent risk behavior, discussed barriers to communicating about sex, and learned ways to respond to a crisis. Adolescents gained sexual health knowledge, discussed the characteristics of healthy intimate relationships, and developed condom use negotiation skills and sexual self-efficacy. In the final two (joint) sessions, caregivers and adolescents reaffirmed their sexual risk knowledge, discussed a case study involving an unplanned pregnancy, and developed a shared vision for the future. The results of the formative and per-post evaluations led to a revised 14-week program which was s introduced in 2017; a 10-week version will be released in 2021. Table 1. Pilot program session outline Pilot program model Caregiver sessions Adolescent sessions Phase 1: Caregivers Matter 1. Building a healthy family No adolescent sessions 2. Emotional awareness 3. How to cope with sadness and fear 4. How to cope with anger 5. Family problem-solving skills Phase 2: Adolescents Matter 6. Joint - Introduction and getting to know one another 7. Raising an adolescent 7. My strengths and goal setting 8. Joint – Developing positive family relationships 9. Effective communication about emotions 9. Creating a positive atmosphere at home 10. Joint – Problem solving 11. Helping adolescents cope with difficult emotions 11. Coping with sadness 12. Behaviour management with adolescents 12. Coping with anger 13. Joint – Conflict management Phase 3: Protecting the Future 14. Adolescent risk taking 14. Sexual relationships 15. Communicating with adolescents about relationships and sexual health 15. Communicating about sex 16. Understanding HIV 16. HIV and STI’s – Fact and fiction 176. Preventing and responding to crises 17. Condom use 18. Joint – Future planning 19. Joint – Graduation and looking ahead 12 The program features a structured session approach, with a consistent pattern of activities delivered in each two-hour session, including an opening ritual, a discussion of the home practice from the previous session, and a series of 3 to 5 core exercises. Sessions close with a reflective discussion on lessons learnt; a home practice assignment, a closing ritual, and a lottery draw as a fun participation incentive. Recognizing the dynamics inherent in caregiving arrangements for children and adolescents in heavily HIV-affected communities, Let’s Talk engages the primary caregiver of the enrolled adolescent, regardless of the relationship between the two (Bray & Brandt, 2007). Interactive, culturally appropriate scenarios and stories were incorporated into the curriculum to foster experiential learning and promote relevance to participants’ daily lives through attention to issues including foster care, chronic illness and bereavement. A comprehensive manual detailing session activities and the materials required for each, together with sample facilitation scripts and guidance for facilitators, were also developed. Separate manuals were drafted for each program phase and provided to facilitators during training. Facilitators were also provided with an implementation guide detailing the program’s theoretical framework, the session outline, and facilitation tips. Worksheets, developed for program participants, were included as appendices in the curriculum manuals. Selected content, such as stories, scenarios, and worksheets, were professionally translated into two local languages (isiZulu and Sesotho) and provided to facilitators separately. 3.3 Program evaluation A mixed methods pilot study design was used to assess the initial implementation of the Let’s Talk program between 2015 and 2016. The study employed a mixed-methods approach with coordinated components: (1) Outcome evaluation using a pretest-posttest survey among intervention-exposed adolescents and their caregivers; and (2) qualitative research to examine the program’s perceived effects and areas for improvement. 3.3.1 Description of the setting and population The study was piloted within HIVSA and CINDI catchment sites in Gauteng and Kwa-Zulu Natal Provinces. HIVSA focuses on the following three areas in two districts of Gauteng Province: Soweto and Orange Farm of the City of Johannesburg district and Midvaal in the Sedibeng district. Soweto and Orange Farm are considered urban informal townships, while Midvaal is predominantly rural. CINDI operates within both peri-urban and rural communities in the Umgungundlovu District of KwaZulu-Natal (KZN) Province. Both of these districts are highly affected by HIV. According to the 2017 South African National HIV Prevalence, Incidence and Behaviour Survey, KZN had the highest HIV prevalence at 27.0% among adults aged 15-49 years, while Gauteng’s rate was slightly lower than the national average at 17.6% (Simbayi et al., 2009). The vast majority of residents are Black African, most of whom speak either isiZulu or Sesotho. 13 All study participants were beneficiaries of a HIVSA or CINDI CBO partner that targets services to orphans and vulnerable children. During the participant recruitment period, program staff approached potential caregiver beneficiaries to elicit their eligibility and interest in participating in the intervention through use of the enrollment form (see Appendix A). The returned forms served as the basis for the study population. Only partner CBO sites with at least 10 eligible caregivers and who were determined by HIVSA and CINDI to have the space and capacity to implement the intervention with fidelity were eligible to participate. Adults were eligible for the study if they were the primary female caregiver1 of at least one adolescent between the ages of 13 and 17 years residing in their home and have consented to intervention participation. All caregivers and adolescents in their care who met the inclusion criteria outlined in Table 2, were invited to participate in the study. If a caregiver had more than one adolescent in their care meeting eligibility criteria, multiple adolescents were allowed to participate in the study and intervention. Participants who were not able to give informed consent due to known or recognizable cognitive or psychiatric impairment were excluded from the participant recruitment stage. Table 2. Eligibility criteria for quantitative pilot study Inclusion criteria for HIVSA and CINDI partner sites • At least 10 women who act as the primary caregiver of an adolescent age 13-17 and who express potential willingness to participate in a support group • Private space to offer the intervention • Interest and capacity to offer the intervention Inclusion criteria for caregivers • Enrolled as a beneficiary of a selected HIVSA or CINDI site; • Primary caregiver of an adolescent aged 13-17; and • Willing and able to consent to study participation. Inclusion criteria for adolescents • Aged 13-17 years old as of May 1, 2015; • Enrolled in a selected HIVSA or CINDI site; • Willing and able to assent to study participation; and • Consent given by the primary caregiver for the adolescent to participate in the study. Focus group discussions were held with a separate set of 78 caregivers and 92 adolescents who completed at least 50% of the intervention sessions. Additional insights were acquired through five focus groups with program stakeholders, including facilitators, supervisors, and program managers at participating CBOs. The eligibility criteria for focus group discussions is displayed in Table 3. 1 For the purposes of this study, a primary caregiver is defined as the parent or guardian (legal or informal) who is responsible for the pre-selected adolescent’s welfare; specifically, the person that prepares meals, seeks medical attention when the adolescent is ill and otherwise cares for him/her. 14 Table 3. Eligibility criteria for qualitative study Inclusion criteria for partner sites • Run by a CINDI partner organization (Youth for Christ, Lifeline, or Community Care Project) • Private space to conduct FGDs • Consent to allow the research to take place on site Inclusion criteria for caregivers • Enrolled as a beneficiary of the above CINDI-affiliated program sites; • Participated in at least 50% of Let’s Talk joint or caregiver-only sessions; and • Willing and able to consent to study participation. Inclusion criteria for adolescents • Aged 13 years or above at the time of study enrolment; • Enrolled as a beneficiary of an above CINDI-affiliated program site; • Participated in at least 50% of Let’s Talk joint or adolescent-only sessions; • Willing and able to assent to study participation; and • Consent given by the primary caregiver for the adolescent under age 18 to participate in the study. 3.3.2 Implementing partners Let’s Talk was piloted by HIVSA and CINDI, two South African non-profit organizations focused on capacity building for community-based organizations (CBOs) serving orphans and vulnerable children and youth (OVCY) and their families. HIVSA and CINDI are based in Gauteng and KZN provinces, respectively, where HIV prevalence is particularly high (Simbayi et al., 2019). Intervention roll-out was supported by funding from the USAID Southern Africa/ U.S. President’s Emergency Plan for AIDS Relief (USAID/PEPFAR). The two organizations each offered six Let’s Talk groups during the pilot serving approximately 10 families in each group. Further background on the implementing partners is offered below:  HIVSA was established in March of 2002 with the purpose of strengthening community linkages to HIV prevention, treatment, care and support through community, health and social development systems strengthening initiatives. Their OVCY program, funded by USAID/PEPFAR, aims to support the Department of Social Development’s community based model of care by strengthening the institutional capacity and OVCY programming of Community Based Organisations. They hire, train and fund social auxiliary workers assigned to offer adolescent care and support interventions and other services at partnering CBO sites. HIVSA targets specific areas with high HIV prevalence, high maternal mortality rate, and a high number of orphans and vulnerable children. Currently, they provide support to approximately 50 CBOs working with OVCY in the areas of Soweto, Sedibeng and Orange Farm, within Gauteng province.  CINDI was established in 1996 and is a network of over 300 organizations in KZN who work for the interests of OVC. Similar to HIVSA, their emphasis is on capacity-building for organizations 15 serving OVC; primarily through the development of effective networks, linkages and referral systems to support OVC, and through community mobilization, participation and capacity enhancement through training, support and mentoring. Under their PEPFAR funded program, CINDI’s primary objective is to increase life skills and improve the well-being of OVC through small group HIV prevention education, promoting behavior change, improving access to counselling and testing, and provision or successful linkages to services. A second objective is to increase the knowledge, skills and competencies of primary caregivers and community members on parenting and basic OVC care and support through training in order to enhance the quality of care provided to the OVC population. In partnership with selected CBOs, CINDI has established relationships with 42 schools within uMgungundlovu District in KZN as a platform for identifying and serving OVCY and their caregivers. 3.3.3 A pre-test post-test pilot study design A pre-test post-test pilot study design was utilized to assess program effectiveness on key behavioral and mental health outcomes among caregivers and their adolescents, aged 13-19 years, who participated in the initial implementation of the program. Face-to-face interviews were conducted with caregiver and adolescent participants by interviewers not affiliated with the program. The analysis was restricted to the 95 female caregivers and 105 adolescents who participated in Let’s Talk and completed both surveys. The study was conducted in two provinces: KZN, where implementing partner CINDI offered it among 3 of their CBOs, and in Gauteng where HIVSA offered it among 4 of their CBOs. Study design and data collection The evaluation used a one-group pretest-posttest design with baseline surveys administered to everyone recruited for the program in June and July 2015, prior to intervention, and post-test surveys conducted three months after the final session (nine months after baseline assessment) in March 2016. Recruitment into Let’s Talk did not guarantee program participation, but among those who participated, high rates of study response were achieved. At baseline, 205 out of 216 adolescents recruited for the pilot program were interviewed (94.5%). Of these, 114 ultimately participated in the program (defined as attending at least one group session). At post-test, 112 of the 114 adolescent intervention participants with baseline surveys completed a second survey, reflecting a 98.2% response rate. Among caregivers, 175 out of 178 recruited for the Let’s Talk program pilot were interviewed at baseline (98.3%). At post-test, 131 of these caregivers participated in the program and 128 of them were re￾interviewed, reflecting a 97.7% response rate. Two survey instruments were developed, one for the adolescent and another for his/her primary caregiver. Both instruments were designed to gather three types of information: 1) Behavioral risk factors and related psychological outcomes hypothesized to be affected by the interventions, 2) Data on a series of secondary outcomes (e.g. relationships, knowledge, etc.) in order to better elucidate the pathways by which the interventions may be working, and 16 3) Demographics and background information that may serve as important explanatory and control variables. Many of these domains were captured applying standardized indicators and scales that have been previously validated with populations in sub-Saharan Africa (see Key Outcome Measures section below). At follow-up, the surveys also include a module to obtain information on participants’ perceptions of the interventions under study. Data collection was led by trained interviewers unaffiliated with the implementing organizations. Survey instruments were translated into isiZulu and Sesotho by professional translators familiar with the vernacular in study communities. Face-to-face interviews were administered to participants in their homes using portable electronic data collection devices, with adolescents using audio computer-assisted self-interviewing (ACASI) technology to respond to portions of the interview addressing sexuality topics (Morrison-Beedy et al., 2006). All interviews were delivered in the preferred language of the participant, namely English, isiZulu or Sesotho. The analytic sample presented in this report consists of the pre- and post-test data of 105 adolescents and their 95 caregivers who participated in Let’s Talk and completed both surveys. Ten caregivers had two adolescents enrolled in the program and provided separate answers for any adolescent specific questions during the interviews. Key outcome measures The surveys used in this study incorporated a range of child, caregiver and household well-being indicators that have been recommended by the United States Agency for International Development as important and expected outcomes of OVC programs (PEPFAR, 2012). The survey asked participants to report on age-appropriate well-being indicators and behavioral measures including psychosocial outcomes, HIV prevention and care measures, household economic stability indicators, and other relevant demographic information. Primary and secondary outcomes are detailed below: • Adolescent and Caregiver HIV Knowledge: Drawing from the South African Demographic and Health Survey, seven yes/no questions were used to develop an index variable reflecting respondents’ level of knowledge about HIV transmission and prevention, including rejection of common misconceptions (South African National Department of Health et al., 2007). • Adolescent Condom Use Knowledge. Five questions were used to develop an index measure reflecting awareness of how to use a condom correctly and how to put one on oneself or a male partner and its protective value. • Adolescent Condom Negotiation & Sexual Refusal Self-Efficacy: Condom negotiation self-efficacy was measured using an index adapted from the Condom Use Self-Efficacy Scale (Brafford & Beck, 1991). Three questions asked whether the adolescent believed that he/she could convince a new partner to use a condom, ask a partner to start using condoms, and convince an intoxicated partner to use a condom (Cronbach’s alpha=0.8236 at baseline). Sexual refusal self-efficacy was measured similarly with four questions asking how certain the respondent was that he/she could refuse sex under specific circumstances (Cronbach’s alpha=0.8689 at baseline). • Caregiver-Adolescent Sexual Risk Communication: Sexual risk communication was assessed using the sum of adolescents’ coded response scores reflecting the frequency (on a four-point scale) of 17 related conversational topics using five questions derived from a family communication scale used in previous evaluations of adolescent HIV prevention interventions in South Africa and Uganda (Bhana et al., 2004; Ismayilova et al., 2012). • Adolescent-Caregiver Connectedness: Connectedness was assessed using adolescent’s completion of the Inventory of Parental and Peer Attachment (IPPA) (Armsden & Greenberg, 1987). The IPPA includes 25 items measuring communication, trust, and alienation in parental attachment using a four-point likelihood scale with a total sum score (Cronbach's α=0.7580 at baseline). • Adolescent and Caregiver Mental Health: The 21-item Depression Anxiety Stress Scale (DASS 21) (Lovibond & Lovibond, 1995) was used with both adolescents and caregivers. Respondents recorded the frequency of symptoms during the previous seven days using a four-point scale, with results averaged (Cronbach’s alpha=0.8701 for adolescents and alpha=0.8971 for caregivers at baseline). Data analysis Prior to data analysis, data from the adolescent surveys were merged with data collected from their caregivers to create an analytical dataset containing a broad array of outcomes and potential confounders. Descriptive analyses of baseline data were conducted to generate frequencies of categorical outcomes, and means and/or medians of continuous outcomes, and to assess the extent of psychosocial problems and HIV risk factors among adolescents and their caregivers. To assess the impact of the study intervention on the key outcomes, univariate analyses comparing pre- and post-test assessments of key outcomes were performed using the paired t-test for continuous variables and McNamara’s test for categorical variables. Where a trend existed within bivariate results, fixed effects models applying linear or logistic regression models as appropriate were conducted to account for the correlation between siblings within families and control for important confounders. Analyses examined the influence of key factors that may modify intervention impact, such as adolescent age, gender and relationship to caregiver. Statistical analyses were performed using Stata and SAS. Significant changes in group means from baseline to follow-up were identified using generalized estimating equations (GEE) in Stata/IC 14 because of GEE’s flexibility in specifying outcome distributions and ability to correct for dependent observations. All models employed autoregressive correlation matrices to account for repeated measures, and an identity link function to express the coefficients of each outcome in the original scale units. Models of adolescent outcomes used bootstrapped standard errors to correct for clustering by caregiver. HIV transmission and condom use knowledge measures were expressed as a count of correctly answered questions; accordingly, both were modeled with a Poisson distribution. Condom negotiation self-efficacy, sexual refusal self-efficacy, adolescent/caregiver sexual communication and DASS-21 outcomes were modeled with a Gamma distribution due to their skewed distributions. Adolescent/caregiver connection was approximately normally distributed and was modeled with a Gaussian distribution. All models included an age term and the adolescent models also included a gender term. Coefficients from a dummy variable indicating data from the post-test survey round is the measure of change reported. Adjusted marginal means by survey round were generated post-estimation. 18 Summary of findings Among the 105 adolescents and their 95 caregivers who participated in Let’s Talk and completed both surveys, statistically significant improvements were found for adolescents’ HIV and condom use knowledge as well as condom negotiation self-efficacy, but not sexual refusal self-efficacy. Both caregivers and adolescents demonstrated significantly better mental health at post-test. Adolescent/caregiver connection and communication about healthy sexuality also improved. These preliminary results highlight the potential of HIV prevention interventions that engage caregivers alongside the vulnerable adolescents in their care to mitigate adolescent HIV risk factors. 3.3.4 Pathways to better intervention outcomes: structural equation models The initial evaluation of Let's Talk above described found an increase in caregiver-adolescent sexual communication. This analysis explored the role of parental knowledge, the quality of the parent￾adolescent relationship, and the mental health of both parties on caregiver-adolescent sexual health communication. Data collection and analysis This analysis was limited to data from the 64 female adolescents ages 13–17 and their caregivers who attended at least one Let's Talk session and completed both the pre- and post-intervention surveys. Using mixed data collected in 2015 and 2016, structural equation modeling of differenced pre-and post￾intervention survey data was used to explore the pathway to increased frequency of caregiver￾adolescent sexual health communication. Five caregivers had two female adolescents enrolled in Let's Talk and provided separate answers for any adolescent-specific survey questions. Post hoc path analysis was performed using structural equation models to test whether changes in key outcomes were associated with improvements in caregiver-adolescent communication about sexual health. Values of the key indicators from the baseline survey were subtracted from their values at follow-up to create measures of change. The change scores were approximately normally distributed. Data were analysed in Stata/IC 14 (College Station, TX) to estimate path coefficients. Robust standard errors were calculated to correct for the clustering of adolescents within caregivers. The error terms from the model were tested for correlation and found to be uncorrelated. The model fit was good with a standardized root mean squared residual (SRMR) of 0.03 and an overall coefficient of determination of 0.26. Summary of findings of the pathways analysis Improvements in the caregivers’ mental health was associated with improvements in the adolescents’ mental health. This in turn was associated with better connection with the caregiver and better sexual health communication. The results suggest that the level of sexual health communication within an adolescent-caregiver dyad depends in part on the mental health of both parties. 3.3.5 Focus Group Discussions (FGD) The content of the focus groups was divided into two themes among both caregivers and adolescents – some groups focused on mental health outcomes, while the others focused on sexual health, behavior and communication. Participants for FGD were recruited from the list of beneficiaries that had attended at least 50% of the program sessions. The primary goal of qualitative research was to explore the extent to which participants perceived results from program participation such as increased caregiver￾adolescent communication about sexuality, higher self-efficacy for sexual decision-making, improved 19 psychological wellbeing, lower propensity for risky behavior, and greater intentions to delay sexual debut and/or use effective methods of HIV and pregnancy prevention. FGD guides were developed collaboratively by members of the Tulane and UP research team members. Input was also sought from other stakeholders, including CINDI and its program partners, in order to ensure that key program-specific issues and priorities are addressed. Focus groups were limited to key questions based on forthcoming results of pilot survey data or divided across groups, thus ensuring sufficient exploration of particular themes or findings that warrant further exploration (e.g., aspects of the program perceived as less effective; intended and unintended results from the beneficiary perspective, barriers to full intervention participation). The discussion guides included general questions regarding participants’ perceptions of the quality and efficacy of the interventions and explore a range of effects from program engagement. Data collection methodology Focus group discussions were conducted in isiZulu by native speaking facilitators experienced in qualitative research and not affiliated with the program. Discussions took place at CINDI partner organization facilities where the Let’s Talk sessions were held, as these locations were both familiar and accessible to participants. However, no CINDI or partner program staff member were present during the groups. The discussions were facilitated only by trained, externally-affiliated researchers with prior experience conducting focus groups. Two were present, with one serving as a moderator and the other as an observer/note taker. Consent and assent were obtained by the moderator; forms were read out loud to all participants at the outset, followed by a short break to allow participants to approach the moderator with any questions, sign and submit consent forms if they so choose, or privately decline to participate without feeling pressure from the group. Caregivers of youth under 18 years of age were invited to attend the meeting so they can provide consent for the adolescent to participate; however, they were not permitted to be present in the room when focus group discussions with adolescents were in progress. Refreshments and a transport reimbursement of ZAR20 was made available to all participants at the outset, regardless of whether they decide to take part in the group discussion. Consent and Assent forms are found in Appendix B. Focus group facilitators and observers received at least one full day of training from a Senior researcher that authored the protocol prior to data collection in order to gain familiarity with the discussion guide, study objectives, protocol, and procedures for ethical research conduct. The field team used digital voice recorders to record all focus group discussions and the observer drafted field notes to be linked to each transcript by the Project Manager. At the end of each day of data collection, the research team conducted a field work debriefing meeting to gather information from the moderator and observer on the interview processes and any challenges that they encountered when facilitating the FGDs. Following the discussion, audio files were downloaded onto a laptop, quality-checked by the designated observer, and encrypted for security. The audio files were transcribed, translated into English by two independent translators, and the translations reconciled as a means of ensuring data quality. Transcripts and translated files were saved as password-encrypted MS Word documents and e-mailed to the Project Manager at Tulane at the end of each day of fieldwork. 20 Data analysis The research team compiled emerging topics, ideas, concepts, terms, phrases and keywords that were used to develop themes and codes. After the development of the themes and codes, data were analyzed using ATLAS.ti. New codes were identified on an on-going basis. A thematic analysis was resulting in qualitative synthesis of major themes. Common or cross-cutting themes emerging from the group discussions were identified with the aim to identify key findings for each and across respondent groups. Summary of findings Adolescents indicated that the Let’s Talk program taught them to make better decisions related to their sexual health, express their emotions appropriately, solve problems and communicate more effectively, which in turn contributed to improved relationships with their caregivers and others. Adolescents further reported positive changes in the caregivers’ behavior towards them, including less punitive responses and a willingness to listen to their opinions, even during disagreements. Caregivers indicated this helped them to be more effective parents and reported increases in positive communication with their children, resorting to punitive discipline less often. 3.3.6 Caregiver incentives Caregiver participation was identified as a significant implementation challenge to the Let’s Talk program. A study assessing the impact of varying incentive offerings on caregivers’ attendance was conducted to inform future intervention offerings. Data collection and analysis In 2017, four CBOs serving similar populations in KZN were randomly assigned to offer one of four incentive packages to ten Let’s Talk family groups, each consisting of 10-15 caregivers and their adolescents. Summary of findings More substantial incentive packages resulted in higher levels of caregiver attendance. Offering significant incentives may be crucial to sustaining caregiver participation in the program. 3.3.7 Costing of the program Tulane University (Tulane) engaged with KPMG Services (Pty) Ltd (KPMG) to investigate the cost of implementing the Let’s Talk program for adolescents and caregivers. The objective of the study was to generate estimates and conclusions regarding the cost per participant as well as cost implications for expansion, replication, and modifiable implementation characteristics. Three implementing partners, each with varying levels of experience in implementing the program, were engaged in order to understand the program and its funding requirements, and to develop recommendations for future funding. Data collection and analysis The study was structured into four phases: Inception; Fieldwork and Consultation; Analysis; and Reporting. The Inception Phase of the project involved the development of a research protocol in consultation with Tulane. The protocol clarified the project scope, expectations, roles, responsibilities, and timeframes for delivery. Activity and expenditure data were collected from implementing partners. Interviews were also conducted with one community-based implementing organization per 21 implementing partner. The data collected were consolidated in an Excel-based costing model which was built to estimate implementation costs per current implementer, and generate a theoretical model based on lessons learned from current implementation. Summary of findings The total cost of implementing Let’s Talk was found to be significantly higher with a greater number of program participants. Due to economies of scale, however, the average cost per participant was lower the more participants there were. The resulting costing tool was a useful resource for partners to use in planning for implementation. 3.4 Stakeholder engagement This evaluation was conducted with input from local implementing partners; CINDI and HIVSA. The engagement with CINDI and HIVSA was critical to defining the scope of the evaluation study and designing data collection methods that are contextually appropriate and responsive to the needs of the funders, program partners and participants. The evaluation team operated independently from implementers and other local key stakeholders. From the outset of the study, the evaluation team maintained frequent communication with the local stakeholders aiming to provide a well-developed and informed understanding of the program and its strengths and limitations among all stakeholders involved. Continued engagement of local stakeholders was maintained throughout the research process to ensure local relevance and the broader utilization of evaluation findings. Post-evaluation workshops and other forums were organized to help implementing partners and other local stakeholders understand and use study results. 3.5 Ethical considerations and assurances The study team was acutely aware of the importance of ethical standards and a number of best practice procedures were in place to ensure the rights and protection of participants. Caregivers provided written informed consent for their own and the adolescent’s participation, and adolescents provided written informed assent. Ethical approvals were granted by the Tulane University Institutional Review Board, USA and the Faculty of Health Sciences Research Ethics Committee at the University of Pretoria, South Africa. The research team is sensitive to special ethical considerations when working with persons under the age of 18 and has consulted a variety of experts and resources pertaining to research with children and youth, especially those who are HIV-affected or any other group considered especially vulnerable (Human Sciences Research Council, 2010; Petersen & Leffert, 1995b; K. Schenk & J. Williamson, 2005; South African National Department of Health, 2006). A number of mechanisms were instituted to protect adolescents and to ensure that their participation in this research is voluntary, that potential benefit outweighs potential risk, and that both adolescents’ assent and caregivers’ consent for adolescent participation is obtained. Special provisions include assent documents produced at the estimated language level of the subject and adolescent-specific reassurances to ensure voluntary participation. Additionally, training for facilitators will include techniques specific to facilitating conversations with adolescents. 22 It is notable that this population may also be vulnerable to coercion or undue influence due to their relationship with program partner organizations, and related service hopes or expectations. To address this risk, the respondents were made aware that: 1) the focus group discussion facilitator and observer are not affiliated with the program partner, 2) participants’ personal information will not be shared with anyone, and 3) the facilitator is only there to ask questions and not to provide humanitarian assistance. It was further made clear that participants’ responses will in no way affect the services they are eligible to receive—neither resulting in more or less services. Respondents were also made aware that they have the right to refuse to participate, skip questions or stop the discussion at any time, with no consequences. No compensation was provided to respondents for participation (other than refreshments and transport reimbursement) and respondents were made aware of this fact along with all study procedures as part of the consent procedure prior to being asked to participate. The following procedures were strictly adhered to ensure ethical implementation of the study: Informed consent: Caregivers provided written informed consent for their own and the adolescent’s participation, and adolescents provided written informed assent prior to beginning the interviews from field workers trained in strict ethical procedures. Voluntary participation: As part of the consent and assent procedures, the interviewer explained clearly before the questioning began that the respondent’s participation is entirely voluntary and that the respondent has the right to refuse to answer any question or any part of any question that he or she did not wish to respond to. In addition, all potential participants were reminded that they had the right to terminate the interview at any time. The research was considered to be of minimal risk as it posed risks no greater than that ordinarily encountered in daily life or during the performance of routine physical or psychological examination. Participants were made aware that they will not directly benefit from the study as part of the consent and assent procedures; however, the use of the study information for the formulation and betterment of programs that serve this population may indeed ultimately result in benefits to adolescents and their caregivers through improved intervention approaches. Finally, they were made aware at the outset that their decision to participate or not in the study did not affect their eligibility to receive services from the programs then or in the future. Privacy and confidentiality: Data collection was conducted by interviewers from a research team not affiliated with Let’s Talk or implementing partners. All participants were informed of provisions to ensure confidentiality, namely that that their answers would be held in strict confidence and stored in the different place as their contact information. Protection of data: All beneficiary data files that provide the information linking individuals to their unique identifiers were password-protected and kept in locked cabinets accesible only to the Tulane Univerity senior researchers. Similarly, the data for tracking sheets, which included contact and identifying information, was captured on a different database than the actual survey data. This data including audio-recordings of FGDs was also password protected with access limited to lead research team members. Conflict of interest: The investigators declare that they have no financial or conflicting interests to the study and the funders. 23 4. Findings and Limitations Results from the pilot survey study and focus group discussions are presented and discussed in the following two sections. 4.1 Pilot survey The pilot study evaluation used a one-group pretest-posttest design. Baseline surveys were administered to study participants recruited for the program in June-July 2015, prior to intervention, with post-test surveys completed three months after the final session (nine months after baseline assessment) in March 2016. Table 4 below presents descriptive statistics for adolescents and caregivers at baseline. The mean age of adolescents was 14.7 years and 61% were female. Almost half of adolescents (49%) had experienced the death of at least one parent and half (50%) were being cared for by their biological mothers, with another 30% cared for by a grandmother. Majority of adolescent participants (67%) attended more than 11 sessions, and 99% attended joint sessions with caregivers. The average age of caregivers was 48.1 years; all were female and almost half (47%) were married. Twenty-three percent of caregivers reported that they were living with HIV and 11% reported being chronically ill. Table 4. Baseline characteristics of adolescents and their caregivers Adolescents (N=105) Caregivers (N=95) N % N % Female 64 60.95 Female 95 100.00 Male 41 39.05 Married 45 47.37 Sexually active 17 16.19 Chronically ill 10 10.53 Orphan (single or double) 49 46.67 HIV positive 22 23.16 Caregiver Relationship Mother 53 50.48 Age Grandmother 32 30.48 19 to 29 9 9.47 Sister 9 8.57 30 to 39 23 24.21 Aunt 10 9.52 40 to 49 26 27.37 Non-relative 1 0.95 50 to 59 9 9.48 Age 60 to 69 16 16.84 12 to 13 21 20.00 70 to 82 12 12.63 14 to 15 55 52.38 16 to 17 29 27.62 Sessions Attended Sessions Attended 1 to 5 14 13.33 1 to 5 8 8.42 5 to 10 21 20.00 6 to 10 12 12.63 11 to 14 70 66.67 11 to 15 18 18.95 16 to 19 57 60.00 Attended any joint sessions w/caregiver 104 99.05 Attended any joint sessions w/ adolescent 92 96.84 24 Figure 2 – 10 present the GEE estimated baseline and follow-up adjusted group means and pretest￾posttest change for each outcome. Statistically significant improvements were found for all outcomes except sexual refusal self-efficacy. As seen in Figure 2, while the adjusted mean scores for adolescent sexual refusal self-efficacy increased over time by 1.69%, this change was not statistically significant (Beta=0.09, p=0.85). Figure 215. Adolescent Sex Refusal Self-Efficacy (N=105) Statistically significant improvements were found for all remaining outcomes. As seen in Figure 3 and Figure 4 below, adolescents showed a 7% increase in HIV transmission knowledge (Beta=0.42, p=0.008) and 17% increase in condom use knowledge (Beta=0.59, p=0.000). Figure 3. Adolescent HIV Transmission Knowledge (N=105) Figure 4. Adolescent Condom Knowledge (N=105) Figures 5 show 12% improvement in adolescents’ self-efficacy regarding condom negotiation (Beta=0.50, p=0.005) and decrease in adolescent mental health issues by 26% (Beta=-0.17, p=0.004). 5,14 5,22 5 5,1 5,2 5,3 5,4 Baseline Post-test 3,38 3,96 3 3,2 3,4 3,6 3,8 4 4,2 Baseline Post-test 5,69 6,11 5,4 5,5 5,6 5,7 5,8 5,9 6 6,1 6,2 Baseline Post-test 25 Figure 16. Adolescent Condom Negotiations (N=105) Figure 6. Adolescent DASS 21 (N=105) In addition, as seen in Figure 7 and Figure 8, there was a 21% increase in caregiver adolescent communication about sex (Beta=1.75, p=0.000) and 11% improvement in relationships connectedness (Beta=5.35, p=0.000). Figure 7. Caregiver-Adolescent Sexual Communication (N=105) Figure 8. Caregiver-Adolescent Connection (N=105) Results also showed that caregivers’ mental health issues decreased by 22% (Beta=-0.22, p=0.007) and caregivers’ HIV transmission knowledge increased by 3% (Beta=0.197, p=0.040); see Figures 9-10. Figure 9. Caregiver DASS 21 (N=95) Figure 10 Caregiver HIV Knowledge (N=95) 4,28 4,78 4 4,2 4,4 4,6 4,8 5 Baseline Post-test 0,64 0,47 0,3 0,4 0,5 0,6 0,7 0,8 Baseline Post-test 8,34 10,08 7 8 9 10 11 12 Baseline Post-test 50,63 55,99 48 50 52 54 56 58 Baseline Post-test 0,99 0,77 0,6 0,7 0,8 0,9 1 1,1 Baseline Post-test 6,28 6,48 6 6,1 6,2 6,3 6,4 6,5 6,6 Baseline Post-test 26 4.2 Process and qualitative data Attendance was one of the primary challenges encountered during pilot implementation, particularly among caregivers. As seen in Figure 11 and Figure 12, only 17% of caregivers and 21% of adolescents attended all of the sessions in the program. In total, 63% of caregivers and 69% of adolescents attended enough sessions to be considered as having completed the program (defined as attendance at 70% of sessions). Overall, caregivers attended an average of 12.4 sessions (out of 19 possible sessions) and adolescents an average of 10.1 sessions (out of 14 possible sessions). Attendance at the joint sessions was poor overall, with only 21% of caregiver-adolescent dyads attending all six joint sessions together. In response to these findings, a 14-week version was introduced in 2017 to help boost participant attendance and completion rates. Figure 11. Pilot program session attendance among caregivers (N=128) Figure 12. Pilot program session attendance among adolescents (N=112) Figure 13. Attendance of joint sessions by caregiver-adolescent dyads (N=97) 9% 70% 63% 10% 17% 0% 10% 20% 30% 40% 50% 60% 70% Attended one session only Attended 50% of sessions Completed programme, attended 70% of sessions Attended all but one session Attended all sessions 7% 82% 69% 12% 21% 0% 10% 20% 30% 40% 50% 60% 70% 80% Attended one session only Attended 50% of sessions Completed programme, attended 70% of sessions Attended all but one session Attended all sessions 73% 21% 0% 10% 20% 30% 40% 50% 60% 70% 80% Attended 50% of sessions Attended all sessions 27 In addition, a total of 30 facilitators provided feedback on the program training. As seen in Figure 14 below, facilitators rated the training highly overall; 95% reported that they understood the content and 91% felt they had learned skills they could use in their personal lives. Ninety-five percent indicated that they could effectively help participants manage and better express their anger, while 94% reported satisfaction with their ability to plan and facilitate sessions overall. Self-efficacy on some of the therapeutic components was somewhat less universal; 82% of facilitators felt that they could explain CBT effectively. As one trainee noted, “I need more practice on challenging negative thoughts.” Figure 14. Self-rated facilitator preparedness and competencies Adolescents reported that they learnt a lot about sexual health during the Let’s Talk program, and indicated that it taught them to take control of and make better decisions related to their sexual health. While for some participants, discussing sexual matters with their caregivers remained taboo due to cultural barriers and fear of punishment, the program helped many to establish channels of communication. Adolescents further indicated they had learnt to use a condom properly, some for the first time, and now better understood the risks associated with sex. Adolescents also reported a number of psychological benefits, such as learning to cope with and express their emotions appropriately. They also noted learning to solve problems and communicate more effectively, which in turn contributed to improved relationships with their caregivers and others. Adolescents further reported positive changes in the caregivers’ behavior towards them, including less punitive responses and a willingness to listen to their opinions, even during disagreements. Caregivers echoed these sentiments, noting benefits such as learning to solve problems and cope with anger, depression, and anxiety. They indicated this helped them 84% 93% 94% 95% 82% 98% 91% 95% 60 65 70 75 80 85 90 95 100 The curriculum manual is easy to use overall The training was able to retain trainee interest Satisfied with ability to plan and facilitate sessions overall Feel confident in ability to help participants manage and express their anger Feel confident in ability to explain CBT Very likely to recommend the training to others Learned skills in the training that could be applied in personal life Found the training content well-organised and easy to follow 28 to be more effective parents and reported increases in positive communication with their children, resorting to punitive discipline less often. 4.3 Pathways analysis Changes in the caregivers' and the adolescents’ depression, anxiety and stress scores showed that an improvement in caregiver mental health was associated with an improvement in adolescent mental health. A reduction in adolescent depression, anxiety, and stress was associated with adolescents’ improved sense of connection with their caregiver. Adolescent-caregiver connection was positively correlated with improvements in caregiver-adolescent sexual health communication. No statistically significant direct effects were found for change in caregiver mental health in predicting either changes in adolescent/caregiver connection or sexual health communication, nor was there a significant direct effect of change in adolescent mental health on change in sexual health communication. The statistically significant indirect effects suggest that the level of sexual health communication within an adolescent-caregiver dyad depends in part on the mental health of both parties. Quotes from Focus Group Participants “What helped me the most is that I was taught to say no if I don’t want to have sex and how to protect myself if I want to have sex.” FEMALE ADOLESCENT, AGE 14 “I would say the Let’s Talk group influenced me because I felt confident when the nurse asked if I wanted to test [for HIV]. I wanted to know my status.” FEMALE ADOLESCENT, AGE 18 “I would like to thank God first for bringing this program to us. I was depressed and hurting…and being here helped me a lot as my life is back to normal again.” CAREGIVER, AGE 54 “I used to be short tempered, but since attending the group I am not like that anymore. When I am angry I can control my anger.” FEMALE ADOLESCENT, AGE 17 “I now know that if l have a problem l can solve it with my family, not to keep it to myself and have suicidal thoughts.” FEMALE ADOLESCENT, AGE 19 “My mother used to beat us, me and my sister, but now we have a good relationship and she no longer even shouts at us.” FEMALE ADOLESCENT, AGE 16 “The group helped me a lot because we were not used to talking about such sensitive topics with our kids, thinking that they are still young for such talk…I am now able to talk to her about sex.” CAREGIVER, AGE 67 29 4.4 Caregiver incentives A study assessing the impact of varying incentive offerings on caregivers’ attendance was conducted to inform future intervention offerings. More substantial incentive packages resulted in higher levels of caregiver attendance. The provision of meals, transport reimbursements and add-on economic strengthening activities resulted in the highest level of attendance. Offering significant incentives may be crucial to sustaining caregiver participation in the program. 4.5 Costing analysis Tulane collaborated with KPMG Services Ltd. to investigate the cost of implementing the Let’s Talk program for adolescents and their caregivers. Activity and expenditure data were collected from three program implementing partners: the CINDI, HIVSA and the Networking HIV and AIDS Community of Southern Africa (NACOSA). An analysis of the results from this data was then conducted to identify key cost drivers and program requirements. A costing estimation tool was developed to help implementers estimate start-up and ongoing costs for their own programs independently. 4.6 Overall study limitations The study had several limitations, described below: Design of the pilot study: We used a one-group design with a small sample size. The lack of a comparison made it difficult to determine whether observed changes were due to the intervention or other factors. Small sample size: Few intervention participants were sexually active and the pre- and post-intervention assessment occurred over only a relatively short period. Path analyses are similarly typically performed with large datasets and the small sample size in this study prevented the examination of other pathways, including those leading to sexual risk behavior. Sample selection: The study population was limited to beneficiaries enrolled by community-based organizations into their programs and may not be generalizable to other groups. Participants also self￾selected into the program and the intervention dose varied among them. There is the potential for selection bias as a large portion of those who expressed interest in the program chose not to participate. Recruitment efforts targeted caregivers, and adolescents’ participation was likely contingent on their caregiver’s commitment to the program. All focus group participants had attended at least half of the intervention sessions whereas the survey sample did not have such a strict threshold. As such, participants in the qualitative analysis may have been more motivated to participate in the intervention than those included in the quantitative analysis, which could result in more positive affirmations about the program's potential among the former group. However, the inclusion of focus group participants with adequate program exposure was necessary and intentional to ensure an informed perspective on the program's processes and potential results. Attrition: The intervention was offered to 75% of the caregivers and 56% of the adolescents enrolled with program implementing partners. There is a need for an array of strategies to support participant 30 attendance and identify attrition as early as possible. Caregivers available to engage in the program likely differ in important ways from those with competing priorities that prohibited participation. Correspondingly, attendance was also treated uniformly in the pre post analysis despite variations, as dose response effects would require a larger sample size to discern. 31 5 Conclusions and Recommendations 5.1 Conclusion Results from this study suggest that Let’s Talk holds significant potential and helps to fill the gap in RSA for family centered programs. The uniqueness of the Let’s Talk program is that while HIV prevention is the aim, it goes beyond standard behavioural knowledge and skills programming, to address family dynamics and mental health--- strengthening these protective individual and family-level factors have the potential to heighten HIV preventive behaviour. Ultimately, programmers have a short stint in the lives of children – thus efforts that focus on building stronger support within their homes hold more promise for lasting effects. 5.2 Recommendations and future studies Based on the findings of Let’s Talk program evaluation, a number of recommendations have been compiled for further research and program scale-up: Recommendation 1: Family-centered program models hold promise to mitigate adolescent OVC risk. Findings from the pilot study highlight the significant potential of HIV prevention interventions that engage adolescents alongside their caregivers and go beyond standard behavioral knowledge and skills to address family dynamics and mental health. Importantly, targeted efforts to improve caregiver mental health and parenting skills can positively affect a range of intermediate outcomes associated with reduced adolescent risk behavior. Recommendation 2: A streamlined shortened programme model and quality control is necessary. Participant attendance challenges identified led leading to a shortened and simplified session set. Findings were used to finalize materials and guidance for a revised 14-week program released in 2017 consisting of individual and joint sessions for adolescents and their caregivers. A 10-week version is also being developed and will be released in 2021. The initial development and piloting of the Let’s Talk intervention highlighted the need to enhance training content related to cognitive behavioural theory and group management techniques, as well as increase the cultural relevance of activities in the curriculum. Tulane continues to work with Let’s Talk implementers to support ongoing program quality control, including training enhancements and efforts to maximize participant attendance at sessions. Recommendation 3: A randomized controlled trial aiming to provide more robust evidence of the program’s effectiveness should be conducted. This, to our knowledge, is the first study to investigate the impact of an HIV prevention intervention for adolescent girls in sub-Saharan Africa that includes caregiver co-participation, relative to an adolescent-only standard of care model. The revised version of Let’s Talk is already undergoing extensive scale-up to benefit an estimated 150,000 adolescent-caregiver dyads in South Africa as part of the DREAMS program. This extensive reach coupled with the significant investment required to initiate family-based versus individual-level interventions warrants additional study. 32 Recommendation 4: Adolescent-only models are necessary. In response to stakeholders’ requests for a streamlined version of Let’s Talk that could be implemented in schools and other settings without requiring caregiver co-engagement at in-person sessions, Tulane developed Let’s Talk Teens in 2018. The intervention, focused on adolescent participants, retains critical aspects of the original curriculum including emphasis on mental health; culturally-relevant HIV knowledge; structured engagement with peers; dynamic elements like role play, home practice and skills reinforcement. While caregiver engagement offers notable advantages, Let’s Talk Teens is designed to reach adolescents whose circumstances make caregiver co-participation difficult or impossible. 33 6. Dissemination To facilitate evidence-based decision-making, Tulane provided local and international stakeholders with current and cutting-edge information on an ongoing basis. This was achieved through delivery of results and research summaries at interactive workshops, policy-briefs, presentations at national and international forums, and peer reviewed publications. Such publications enhance the credibility of research findings and expand their influence. The following list details different ways through which the impact evaluation results and other findings from Let’s Talk pilot study have been disseminated among local and international stakeholders: International and national conference presentations: o Poster presentation at the 8th South African AIDS Conference: Let’s Talk about sex: Results from a pilot study of a family-centred HIV prevention program in South Africa (Durban, June 2017). o Oral Presentation at Best Practices and Innovations in Reaching and Linking Adolescent Girls and Young Women (AGYW) Meeting: Pilot study shows increased communication with caregivers and preparedness for healthy sexual decision-making among adolescent girls in the Let’s Talk program (Dr. Thurman, Pretoria, October 2016). o Oral Presentation at the International Conference of the Association of Psychology and Psychiatry for Adults and Children: Let’s Talk: A family centered intervention for HIV-affected caregivers and adolescents in South Africa (Dr. Visser, Athens Greece, May 2016). Presentations to study partners: o Interactive Workshop with Let’s Talk Implementers: Let’s Talk: Intervention Overview and Pilot Test Results (Dr. Thurman, Johannesburg, October 2016); Let’s Talk Implementation: Lessons & Considerations (Ms. Spyrelis, Johannesburg, October 2016). o Presentation to NACCA: Let’s Talk Caregiver-adolescent intervention to improve mental health and reduce HIV risk among orphans and vulnerable youth in South Africa (Dr. Visser, Pretoria, July 2016) Presentations at the PEPFAR OVCY technical meeting: o Presentation on Strengthening Relationships and Resilience: Lets’ Talk overview (Dr. Thurman, Pretoria, November 2015) o Western Cape Annual Partners Meeting: Strengthening Relationships and Resilience: Lets’ Talk overview (Dr. Thurman, Cape Town Oct 2015) Peer-reviewed publications: o Visser, M., Thurman, T.R., Spyrelis, A., Taylor, T.M., Nice, J.K., & Finestone, M. Development and formative evaluation of a family-centered adolescent HIV prevention program in South Africa. Eval Program Plann. 2018 Jun; 68:124-134. 34 o Thurman TR, Nice J, Luckett B, Visser M. Can family-centered programing mitigate HIV risk factors among orphaned and vulnerable adolescents? Results from a pilot study in South Africa. AIDS Care. 2018 Sep;30(9):1135-1143 o Thurman TR, Nice J, Visser M, Luckett BG. Pathways to sexual health communication between adolescent girls and their female caregivers participating in a structured HIV prevention intervention in South Africa. Soc Sci Med. 2020 Sep; 260:113168. o Spyrelis A., Taylor T & Thurman, T.R. (Under Review; available upon request) Incentivizing participant engagement in structured psychosocial interventions: Lessons learned from a family￾centered adolescent HIV prevention program in South Africa. Program and Research brief geared to programmers and policy makers: o The Let’s Talk Program: Origins, Development and Future Directions (2018). https://hvc￾tulane.org/the-lets-talk-program-origins-development-and-future-directions/ 35 7. References Armsden, G. C., & Greenberg, M. T. (1987). The inventory of parent and peer attachment: Individual differences and their relationship to psychological well-being in adolescence. J Youth Adolesc, 16(5), 427-454. doi:10.1007/bf02202939 Bandura, A. (1977). Social learning theory. Englewood Cliffs, New Jersey: Prentice Hall. Bhana, A., Petersen, I., Mason, A., Mahintsho, Z., Bell, C., & McKay, M. (2004). Children and youth at risk: adaptation and pilot study of the CHAMP (Amaqhawe) programme in South Africa. Afr J AIDS Res, 3(1), 33-41. doi:10.2989/16085900409490316 Brafford, L. J., & Beck, K. H. (1991). Development and validation of a condom self-efficacy scale for college students. J Am Coll Health, 39(5), 219-225. doi:10.1080/07448481.1991.9936238 Bray, R., & Brandt, R. (2007). Child care and poverty in South Africa. Journal of Children and Poverty, 13(1), 1-19. Butler, A., Chapman, J., Forman, E., & Beck, A. (2006). The empirical status of cognitive-behavioral therapy: A review of meta-analyses. Clinical psychology review, 26, 17-31. Hofmann, S. G., Asnaani, A., Vonk, I. J. J., Sawyer, A. T., & Fang, A. (2012). The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses. Cognitive therapy and research, 36(5), 427-440. doi:10.1007/s10608-012-9476-1 Human Sciences Research Council. (2010). Informed Consent Guidlines re minors (Including orphans and vulnerable children (OVC) and paternal substitutes). Retrieved from Cape Town, South Africa: http://www.hsrc.ac.za/Document-3934.phtm Ismayilova, L., Ssewamala, F. M., & Karimli, L. (2012). Family support as a mediator of change in sexual risk-taking attitudes among orphaned adolescents in rural Uganda. J Adolesc Health, 50(3), 228-235. doi:10.1016/j.jadohealth.2011.06.008 Jardin, C., Marais, L., Bakhshaie, J., Skinner, D., Neighbors, C., Zvolensky, M., & Sharp, C. (2017). Caregiver alcohol use and mental health among children orphaned by HIV/AIDS in South Africa. AIDS Care, 29(3), 399-407. doi:10.1080/09540121.2016.1220477 Li, L., Liang, L., Lee, S., Iamsirithaworn, S., Wan, D., & Rotheram-Borus, M. J. (2012). Efficacy of an intervention for families living with HIV in Thailand: A randomized controlled trial. AIDS and behavior, 16(5), 1276-1285. Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav Res Ther, 33(3), 335-343. doi:10.1016/0005-7967(94)00075-u Morrison-Beedy, D., Carey, M. P., & Tu, X. (2006). Accuracy of audio computer-assisted self-interviewing (ACASI) and self-administered questionnaires for the assessment of sexual behavior. AIDS Behav, 10(5), 541-552. doi:10.1007/s10461-006-9081-y Operario, D., Underhill, K., Chuong, C., & Cluver, L. (2011). HIV infection and sexual risk behaviour among youth who have experienced orphanhood: systematic review and meta-analysis. J Int AIDS Soc, 14, 25. doi:10.1186/1758-2652-14-25 Ortega, J., Huang, S., & Prado, G. (2012). Applying Ecodevelopmental Theory and the Theory of Reasoned Action to Understand HIV Risk Behaviors Among Hispanic Adolescents. Hispanic Health Care International, 10(1), 42-52. PEPFAR. (2012). Guidance for Orphans and Vulnerable Children Programming. President's Emergency Plan for AIDS Relief:. Perrino, T., Gonzalez-Soldevilla, A., Pantin, H., & Szapocznik, J. (2000). The role of families in adolescent HIV prevention: a review. Clin Child Fam Psychol Rev, 3(2), 81-96. Petersen, A. C., & Leffert, N. (1995a). Developmental issues influencing guidelines for adolescent health research: A review. Journal of Adolescent Health, 17(5), 298-305. Petersen, A. C., & Leffert, N. (1995b). Developmental issues influencing guidelines for adolescent health research: a review. J Adolesc Health, 17(5), 298-305. doi:10.1016/1054-139x(95)00184-t Prado, G., Huang, S., Maldonado-Molina, M., Bandiera, F., Schwartz, S. J., de la Vega, P., . . . Pantin, H. (2010). An empirical test of ecodevelopmental theory in predicting HIV risk 36 behaviors among Hispanic youth. Health Education & Behavior, 37(1), 97-114. Rotheram-Borus, M., Lee, M., Lin, Y., & Lester, P. (2004). Six-year intervention outcomes for adolescent children of parents with the human immunodeficiency virus. Archives of Pediatrics & Adolescent Medicine, 158(8), 742-748. doi:10.1001/archpedi.158.8.742 Rotheram-Borus, M. J., Lee, M. B., Gwadz, M., & Draiman, B. (2001). An intervention for parents with AIDS and their adolescent children. . American Journal of Public Health, 91(8), 1294-1302. Schenk, K., & Williamson, J. (2005). Ethical Approaches to Gathering Information from Children and Adolescents in International Settings: Guidelines and Resources. Retrieved from Washington DC: https://www.childwatch.uio.no/research/research-methodology/childrenethics.pdf Schenk, K. D., & Williamson, J. (2005). Ethical approaches to gathering information from children and adolescents in international settings: Guidelines and resources. Washington, D.C.: Population Council. Sharp, C., Jardin, C., Marais, L., & Boivin, M. (2015). Orphanhood by AIDS-Related Causes and Child Mental Health: A Developmental Psychopathology Approach. J HIV AIDS, 1(3). doi:10.16966/2380-5536.114 Sherr, L., Clucas, C., Harding, R., Sibley, E., & Catalan, J. (2011). HIV and Depression - a systematic review of interventions. Psychology, Health & Medicine, 16(5), 493-527. doi:10.1080/13548506.2011.579990 Simbayi, L. C., Zuma, K., Zungu, N., Moyo, S., Marinda, E., Jooste, S., . . . the SABSSM V Team. (2009). South African National HIV Prevalence, Incidence, Behaviour and Communication Survey, 2017. Retrieved from Cape Town: Smith Fawzi, M. C., Eustache, E., Oswald, C., Louis, E., Surkan, P. J., Scanlan, F., . . . Mukherjee, J. S. (2012). Psychosocial support intervention for HIV-affected families in Haiti: Implications for programs and policies for orphans and vulnerable children. Social science & medicine, 74(10), 1494-1503. doi:http://dx.doi.org/10.1016/j.socscimed.2012.01.022 South African National Department of Health. (2006). Guidlines for Good Practice in the Conduct of Clinical Trials with Human Participants in South Africa. Pretoria, South Africa Retrieved from http://www.kznhealth.gov.za/research/guideline2.pdf. South African National Department of Health, South African Medical Medical Research Council, & ORC Macro. (2007). South Africa Demographic and Health Survey 2003. Retrieved from Pretoria, South Africa: http://dhsprogram.com/pubs/pdf/FR206/FR206.pdf Szapocznik, J., & Coatsworth, J. D. (1999). An ecodevelopmental framework for organizing the influences on drug abuse: A developmental model of risk and protection. In M. Glantz & C. Hartel (Eds.), Drug abuse: Origins & interventions (pp. 331-366). Washington, DC: American Psychological Association. Thurman, T. R., Nice, J., Luckett, B., & Visser, M. (2018). Can family-centered programing mitigate HIV risk factors among orphaned and vulnerable adolescents? Results from a pilot study in South Africa. AIDS Care, 30(9), 1135-1143. doi:10.1080/09540121.2018.1455957 UNICEF. (2016a). For every child, end AIDS - Seventh Stocktaking Report. In. New York. UNICEF. (2016b). Key global charts and figures. Retrieved from https://data.unicef.org/topic/hivaids/adolescents￾young-people/#