Coping with Loss: Abangane Grief Groups for Adolescent Girls in the Free State, South Africa Final Evaluation Report October 2017 Support for this project is provided by the United States Agency for International Development (USAID) Southern Africa under the President’s Emergency Plan for AIDS Relief (PEPFAR) through Cooperative Agreement No. AID-674-A-12-0002 awarded to Tulane University. The views expressed in this document do not necessarily reflect those of USAID or the United States government. ii Project Title: “Coping with Loss: A Bereavement Initiative for Adolescent Girls in the Free State, South Africa” Contractor: Tulane University (Highly Vulnerable Children’s Research Center (HVC-RC), School of Social Work) Award No: AID-674-A-12-00002 Project Start Date: January 2015 Project End Date: April 2017 Persons responsible for compiling the report: Tonya Renee Thurman Chief of Party East Block, Tannery Park, 23 Belmont road, Rondebosch, Cape Town 7700 South Africa Tel: (+272) 1 685-1696 Fax: (+272) 1 685-1696 Email: tthurma@tulane.edu Tory Taylor M&E specialist 1440 Canal Street, New Orleans, LA 70112 U.S.A. Tel: (504) 988-4027 Email: ttaylor@tulane.edu Johanna Nice Senior Program Manager 127 Elk Place New Orleans, LA 70112 U.S.A. Tel: (504) 862-3476 Email: jnice@tulane.edu Dovile Vildaite Program Manager 127 Elk Place New Orleans, LA 70112 U.S.A. Tel: (503) 247-1941 Email: dvildaite@tulane.edu ii Table of Contents Acknowledgments ............................................................................................................................................... iv List of Acronyms.................................................................................................................................................... v List of Tables and Figures..................................................................................................................................... vi Executive Summary............................................................................................................................................... 1 Introduction...................................................................................................................................................... 1 Methodology .................................................................................................................................................... 1 Results............................................................................................................................................................... 2 Limitations ........................................................................................................................................................ 3 Conclusions and Recommendations................................................................................................................. 3 Project Background............................................................................................................................................... 6 Evaluation Purpose and Questions....................................................................................................................... 6 Purpose............................................................................................................................................................. 6 Evaluation Questions........................................................................................................................................ 6 Evaluation Theoretical Framework................................................................................................................... 6 Evaluation Design and Methodology.................................................................................................................... 9 Study setting and population............................................................................................................................ 9 Program description ......................................................................................................................................... 9 Stakeholder engagement................................................................................................................................10 Outcome evaluation sampling........................................................................................................................11 Data collection timeline and procedures........................................................................................................12 Key outcome measures...................................................................................................................................13 Data analysis...................................................................................................................................................16 Ethical considerations and assurances...........................................................................................................17 Deviations and adjustments...........................................................................................................................18 Procedures to ensure high quality data..........................................................................................................19 Findings and Limitations.....................................................................................................................................20 Program Effects...............................................................................................................................................21 Primary Outcome: Grief Intensity...............................................................................................................22 Primary Outcome: Mental Health...............................................................................................................23 Secondary Outcome: Caregiver-Adolescent Relationships.........................................................................24 Secondary Outcome: Social Support ..........................................................................................................25 iii Secondary Outcome: Academic Competence ............................................................................................27 Secondary Outcome: Resilience .................................................................................................................28 Program evaluation: Participants’ perspectives.............................................................................................29 Limitations ......................................................................................................................................................31 Conclusions and Recommendations...................................................................................................................31 Summary of findings.......................................................................................................................................32 Recommendations..........................................................................................................................................33 Dissemination .....................................................................................................................................................34 References..........................................................................................................................................................36 iv Acknowledgments Many people were instrumental to realizing the evaluation of Coping with Loss: A Bereavement Initiative for Adolescent Girls in the Free State, South Africa project. The study team is particularly grateful for the support provided from personnel at USAID Southern Africa, particularly Anita Sampson for her support and insights that contributed to the study’s development. The hard work and dedication of social workers and social auxiliary workers from our partner organization, Child Welfare Bloemfontein and Childline Free State, should be credited for the success of the program. The leadership and guidance of the organization’s Social Work Manager, Emma Francis, during intervention implementation and the research process was particularly noteworthy. We also thank the Director of Khululeka Grief Support, Deborah Diedericks, for her substantial technical support towards the intervention’s development and refinement. In addition, we are sincerely grateful to Epicentre for their rigorous fieldwork efforts, which resulted in the collection of high-quality data. Most importantly, we thank the caregivers and adolescents who participated in the study. It is only through their willingness to share their time and sensitive personal feelings and experiences that we can understand the potential of interventions to mitigate the challenges facing bereaved individuals. We sincerely hope that they and others like them benefit from this research effort. v List of Acronyms AIDS Acquired Immune Deficiency Syndrome CBO Community Based Organization CLFS Childline Free State CYCW Child and Youth Care Worker CWBFN Child Welfare Bloemfontein HCT HIV Counseling and Testing HIV Human Immunodeficiency Virus OVC Orphans and Vulnerable Children PEPFAR The United States President‘s Emergency Plan for AIDS Relief RCT Randomized Control Trial RSA Republic of South Africa USAID United States Agency for International Development vi List of Tables and Figures Tables Table 1. Content of the Abangane Grief support group.....................................................................................10 Table 2. Baseline characteristics of participants in the intervention and waitlist groups..................................21 Figures Figure 1. Theoretical framework linking survey content to expected program outcomes. ................................. 8 Figure 2. Study Design and Data Collection Timeline .........................................................................................13 Figure 3. Impact of Abangane program on normative grief...............................................................................22 Figure 4. Impact of Abangane program on intrusive grief thoughts..................................................................22 Figure 5. Impact of Abangane program on complicated grief............................................................................23 Figure 6. Impact of Abangane program on depression ......................................................................................24 Figure 7. Impact of Abangane program on problem behavior...........................................................................24 Figure 8. Impact of Abangane program on parental attachment.......................................................................25 Figure 9. Impact of Abangane program on supportive parenting ......................................................................25 Figure 10. Impact of Abangane program on emotional social support..............................................................26 Figure 11. Impact of Abangane program on support received in coping with grief...........................................26 Figure 12. Impact of Abangane program on academic self-efficacy...................................................................27 Figure 13. Impact of Abangane program on academic motivation ....................................................................27 Figure 14. Impact of Abangane program on post-traumatic growth ................................................................. 28 Figure 15. Impact of Abangane program on emotional self-efficacy ................................................................. 28 Figure 16. Participants’ perspectives on the program facilitators......................................................................29 Figure 17. Participants’ perspectives on the program groups............................................................................29 Figure 18. Perceived effects from program participation...................................................................................30 1 Executive Summary Introduction More than 46 million children in sub-Saharan Africa have experienced the death of one or both parents, nearly 11 million of whom lost a parent to AIDS (UNICEF, 2013). Early bereavement is common in South Africa, where almost a fifth of children have lost a parent and many more have experienced the death of another family member or friend (Brits et al., 2014; Meintjes, Hall, & Sambu, 2015). Early bereavement increases children’s risk for psychological disorders, highlighting the need for effective interventions, especially in areas where orphanhood is common. In addition, greater understanding of African children’s psychological responses to loss can help guide programs designed to provide social and emotional support following bereavement. This report presents findings of the impact evaluation of Abangane (“friends” in isiZulu).This was a structured and theory-based bereavement intervention for female adolescents in South Africa who have experienced the death of someone important in their lives. Adolescents aged 12 or older in the Free State province of South Africa were invited to participate. This evaluation, which commenced in January 2015 and was completed in April 2017, sought to assess the impact of Abangane program on psychological and behavioral problems among bereaved adolescents. Methodology The purpose of this randomized controlled trial (RCT) was to evaluate the impact of Abangane program on the psychological health of female adolescents in three peri-urban towns in Free State, South Africa. The intervention was conceived by Khululeka Grief Support and was adapted and implemented by Social Workers from Child Welfare Bloemfontein and Childline Free State (CWBFN & CLFS) in the Free State Province. The evaluation question was: what is the effectiveness of the Abangane program in reducing grief intensity and improving the mental health of the adolescents, caregiver-adolescent relationship, social support, academic competence and resilience? The program under study, Abangane (“friends” in isiZulu), is a locally derived, curriculum-based support group focused on coping with loss incorporating indigenous stories and cognitive behavioral therapy components. Eight weekly group session were facilitated by trained social workers or social auxiliary workers from a local non-profit organizations CWBFN and CLFS. The study population was identified through the CWBFN & CLFS intake process and consists of bereaved female adolescents from 11 participating schools. Within these schools, 453 bereaved ninth-grade students aged 13–17 years who had expressed interest in taking part in the group were randomly assigned to receive the intervention in 2015 or to be waitlisted for program enrollment after the study period and serve as the control group. 2 Data collection for the Coping with Loss trial began in FY2015 and was completed in FY2016. Approximately 400 adolescents and their caregivers completed a baseline survey prior to commencement of the support groups, and a follow-up survey three months after the groups’ completion. Utilizing standardized psychological scales, problematic grief and depression among participants were assessed through adolescents’ self‐report, while caregivers were asked to report on the adolescents’ overall emotional and behavioral functioning. Data collection occurred in waves consistent with the school term in which adolescents received the intervention. Baseline data collection was completed by March 2015 with 423 adolescents and their caregivers. Follow-up data collection was divided into three waves with the final wave of data collection completed in November 2015. Data analyses were completed with statistical programs SAS 9.3 and STATA 14 and focused on the assessment of pre- and post-test differences between the intervention and wait-listed group on the following primary and secondary outcomes:  Grief intensity  Mental health  Caregiver-adolescent relationship  Social support  Academic competence  Resilience Participants’ feedback on the program as well as perceived impact on other aspects of their lives were also collected during the evaluation process and presented in this report. Results Key findings of the program evaluation include the following: • Adolescent participants in the intervention group demonstrated statistically significant improvements in their psychological health compared to the wait‐listed group, including decreases in problematic grief and depression symptoms, as well as reductions in internalizing and attention problems. • Caregivers of intervention participants reported significantly lower levels of behavioral problems among adolescents, relative to those unexposed to the intervention. • Positive effects were also evident for secondary outcomes; for example, the program appeared to mitigate the worsening of caregiver-adolescent relationships and improved social support available to bereaved adolescents. • No significant program effects were found for adolescents’ academic performance and resilience outcomes. 3 Limitations Several limitations should be considered when interpreting evaluation results: • The study was conducted with adolescent girls and thus the findings cannot be generalized to male adolescents. The decision to restrict program groups to girls-only was based on the concerns about non-conducive behavior in mixed-gender adolescent groups. In addition, the program implementers felt that the intervention’s emphasis on emotional disclosure stood in contrast with local behavioral norms for adolescent males. • The intervention was initially designed to include a parallel support program for the primary caregivers of adolescent participants, aimed at helping to resolve caregivers’ grief so they could better support the children in their care. The caregiver component was eventually removed from the trial due to very low early session attendance among caregivers. • The median time between the focal death reported by participants and baseline assessment was relatively long, at more than 4 years. • Among adolescents who had lost one or both parents, an undetermined number were orphaned by AIDS; longitudinal research has shown greater psychological challenges in this population relative to adolescents orphaned by other means and non-orphans. Reliance on adolescents’ reports probably exacerbates misclassification. Future analyses could help to clarify the factors—such as prolonged illness, adolescent’s relationship to the deceased, baseline outcome values, and time since the loss—that affect treatment effects, with implications for intervention targeting and other aspects of implementation. Reliance on adolescents’ reports probably exacerbates misclassification. Future analyses could help to clarify the factors—such as prolonged illness, adolescent’s relationship to the deceased, baseline outcome values, and time since the loss—that affect treatment effects, with implications for intervention targeting and other aspects of implementation. • This evaluation included only one short-term follow-up assessment (i.e. follow-up surveys completed approximately 3 months post intervention), restricting an exploration of benefits that emerge over a longer timeframe. A longitudinal evaluations design with several long-term follow-up assessments follow-up need to be developed to examine the endurance of program effects over time. Despite these limitations, a short-term, structured, theory-based Abangane support group with contextually relevant content has shown promise in mitigating psychological and behavioral problems among bereaved adolescents in South Africa. Most importantly, Abangane program is replicable in resource-limited settings, using freely available curriculum materials, existing program structures, and appropriately trained personnel to implement it. Conclusions and Recommendations This evaluation study represents the first randomized controlled trial of a structured bereavement support program for adolescents in sub-Saharan Africa. The study offers new information about the 4 potential for the intervention under investigation, Abangane, and similar interventions to improve psychological health in this highly vulnerable population. In addition, this study offers evidence for the sizeable population of orphans and other bereaved children and adolescents in countries with generalized HIV epidemics and represents an unusual effort to ground evidence for psychological health programming in local cultural norms, by applying a locally derived intervention first, and strengthening its basis in theory second. Finally, this study includes a direct assessment of maladaptive grief using multiple measures cognitively adapted among young South Africans. Recommendations Recommendation 1: To scale-up cognitive behavioral therapy (CBT)-informed, culturally sensitive and logistically feasible interventions to enhance psychosocial wellbeing among children and young people in sub-Saharan Africa. The need for bereavement support among children in generalized HIV epidemic settings is high. Short-term, structured, theory-based support groups like Abangane show promise in mitigating psychological and behavioral problems among bereaved female adolescents. While the program effects reported in this study are modest overall, this evaluation offers important evidence of a culturally tailored and logistically feasible intervention addressing priority psychological needs among adolescent girls affected by loss in sub-Saharan Africa. Abangane is replicable using freely available curriculum materials and appropriately trained personnel. Recommendation 2: To address the need for family-focused/centered bereavement programs to mitigate grief and depression among children and caregivers. Many caregivers in South Africa experience high levels of bereavement-related grief themselves, which was found to be strongly associated with children’s risk for depression in previous studies. Engaging caregivers might be particularly crucial in South Africa, where previous research suggests that strong cultural norms prevent adults from discussing death with children. Recommendation 3: To address the need towards developing and evaluating sex-specific programming in the SSA. While separating program participants is common practice for youth interventions in sub-Saharan Africa, there is a need for greater attention towards developing, implementing, and evaluating programming directed at male adolescents or mixed-sex groups. Recommendation 4: To develop strategies to enhance program participation, particularly among caregivers. Strategies are required to support attendance and retention in programs, particularly those that enroll caregivers as co-participants. Examples may include reimbursement for transport costs, hiring a child-minder to watch younger siblings during sessions, and offering refreshments or other low￾cost participation incentives. Program implementers should further help caregivers to understand the goals of the program, and make them aware of the benefits of participation for both themselves and children / adolescents in their care. Recommendation 5: Maintain regular communication with local stakeholders about evaluation processes and results. Regular dialogue mechanisms with implementing organizations and other local key stakeholders are critical to addressing impediments to program success. Although it is important for 5 researchers to operate independently from implementers, it is essential to have open communication about roles, the purpose of the evaluation and its protocols and potential benefits. In the evaluation team’s experience, when implementers and other stakeholders have a well- developed understanding of how learning about the program supports quality improvement, they are often eager to support the research. Continued engagement of local stakeholders maintained throughout the research process contribute to ensuring local relevance and the broader utilization of findings. Recommendation 6: Future research should aim to provide additional evidence for the intervention models that best support integrated health and wellbeing among vulnerable youth. Little evidence exists to inform the design of community-based interventions targeted to the large and growing population of children in countries with generalized HIV epidemics who have experienced the loss of someone close to them. A greater understanding about the effectiveness of various approaches to psychological health among bereaved children in sub-Saharan Africa is needed. Programs serving orphans and vulnerable children should prioritize psychological support alongside other interventions, and future studies should focus on producing evidence for the intervention models that best support integrated health and wellbeing in this population. Overall, the group of adolescent girls assigned to the intervention reported lower scores for intrusive grief, complicated grief, and depression relative to the waitlist group, and caregivers reported significantly lower levels of behavioral problems among the adolescents in the intervention group. These findings build on existing research and offer new evidence in support of the use of time-limited group therapies for adolescent grief support incorporating cognitive behavioral techniques and gender￾sensitive, culturally aware approaches. 6 Project Background The HIV/AIDS epidemic, accidental injuries and violence have gradually increased the mortality rate among prime age adults in South Africa since 1997 (Statistics South Africa, 2006) leading to an increase in the number of children experiencing bereavement. Of the approximately four million orphans in South Africa in 2012, over half (62.5%) are due to parental death from AIDS (UNICEF, 2013). Studies have found that parental loss in childhood and adolescence is associated with a range of immediate and later life outcomes including problems at school, depression, criminality, early sexual activity, and unemployment (Aynsley-Green, Penny, & Richardson, 2012). However, support counseling for bereaved children and adolescents is virtually nonexistent in South Africa (Collingwood, Hough, & Jacobs, 2014) and children are often not encouraged to express grief or even attend memorial services for the deceased. A randomized controlled trial of AIDS orphans in Uganda found that school-based peer support groups reduced the symptoms of anxiety, depression and anger compared to a control group that did not participate in the peer support groups (Kumakech, Cantor-Graae, Maling, & Bajunirwe, 2009). The addition of a caregiver component that addresses the caregiver’s grief as well as support strategies for the grieving youth is in keeping with PEPFAR best practices for orphans and vulnerable children (OVC) programs as evidenced by multiple studies (Nyberg et al., 2012). This base of evidence demonstrates both the unmet need and potential impact of family bereavement initiatives that was offered by CWBFN & CLFS in Free State. This research aims to contribute to the evidence base for effective programming for bereaved adolescents in South Africa as well as other high HIV prevalence populations. Evaluation Purpose and Questions Purpose The purpose of this study was to assess the effectiveness of an eight session support group intervention on the psychosocial wellbeing of bereaved female adolescents in South Africa. Evaluation Questions What is the effectiveness of the ‘Abangane Grief Groups for Adolescent Girls’ program in reducing grief intensity and improving the mental health of the adolescents, caregiver-adolescent relationship, social support academic competence and resilience? Evaluation Theoretical Framework Evaluation framework was theoretically informed by the revised Stroebe and Schut’s Dual Process Model of Coping with Bereavement (2010) presented on the following page (Figure 1). The Dual Process Model identifies a range of stressors underlying bereavement, namely loss- and restoration-oriented stressors. Loss-orientation reflects aspects of the loss experience itself, such as rumination, emotional response and yearning, as well as the relocation/breaking of bonds with the deceased. Restoration￾orientation reflects the secondary sources of stress, that is, adjustments to changes in one’s life as a consequence of the loss. This may include changes to roles and responsibilities, family reorganization, and financial implications. Also central to the dual process model is the idea that bereavement is not a 7 linear progression; rather, it is a dynamic process where the grieving individual oscillates between confronting and avoiding the stressors and tasks of grieving as part of appraisal coping. Stroebe and Schut’s framework was revised by the evaluation team to elaborate upon loss- and restoration-oriented stressors, including unique stressors associated with HIV-related loss for children and adolescents. Our framework also integrates interpersonal and intrapersonal factors contributing to parental bereavement as identified by Haine et al.’s descriptive study of evidence-based bereavement interventions (2008) and Sherr and Mueller’s (2009) systematic review of HIV and mental health. These contributing factors interact with the bereavement stressors to influence the manner of appraisal coping, that is, how the bereaved individual assesses and copes with the stress of bereavement. Appraisal coping may be positive, resulting in personal growth, or negative, such as intrusive thoughts and self-blame. In turn, appraisal coping influences the bereaved child’s outcomes of grief intensity, depression, behavioral problems and academic performance. Implicit in the framework is that if a child experiences intrusive thoughts and self-blame, he/she is more likely to suffer from depression and behavioral disturbances; likewise, if he/she is able to find meaning and personal growth from the loss, the child is less likely to be depressed and may experience a lower intensity of grief. The relationship between outcomes and coping appraisal is bi-directional because the degree of grief intensity, depression, and behavioral problems can affect the individual’s appraisal of self-coping. Likewise, an individual dealing with depression and grief is less likely to sustain a sense of self-efficacy and more likely to have negative cognitions towards the loss. As the framework indicates, interpersonal and intrapersonal factors may also directly influence child outcomes. For example, high levels of social support and ample opportunity to talk about loss may serve as protective factors against grief, depression, and behavioral problems. Stigma surrounding a parental death due to AIDS, lack of familial and social support, and poverty may exacerbate levels of depression and grief. As with appraisal coping, interpersonal and intrapersonal factors have a bi-directional relationship with child outcomes, meaning that child outcomes can influence mutable factors such as social support and caregiver’s grief and distress. In interpreting the framework, it should be noted that factors in bold are items hypothesized to be influenced by the intervention under study. In this report the following predictions are tested: • Participants enrolled in the program/intervention group are hypothesized to have reduced grief, depression symptoms and problem behavior outcomes than participants in waitlist group. • Participants enrolled in the program/intervention group are hypothesized to have better social support, academic competence, resilience, and family relationships outcomes than participants in waitlist group. 8 Figure 1. Theoretical framework linking survey content to expected program outcomes. (Factors in bold color hypothesized to be influenced by intervention) 9 Evaluation Design and Methodology Study setting and population The study was conducted in the Free State Province of South Africa, which is located in south of Johannesburg on Lesotho’s western border. The capital of Free State Province is Bloemfontein. The prevalence of HIV in the 14-49 age group of Free State is among the highest of all of South Africa’s provinces at 19.6% (Human Sciences Research Council, 2013). However, the prevalence of HIV among women attending antenatal clinics is even higher at 32.9% (Department of Health, 2012). Twenty-seven percent of children in Free State are orphans (Meintjies & Hall, 2011). The program under evaluation is gender-specific, namely that it has been offered to female adolescents due to the greater enrollment of females to the program up to date, and concerns about non-conducive behavior in mixed-gender adolescent groups. As a result, study population consisted of female adolescents with the following characteristics: • 13-17 years old as of January 1, 2015 • Enrolled in one of the 11 program implementing schools • Enrolled in 8th or 9th grade in 2015, depending on the school intake process • Experienced the loss of someone close to them • Expressed interest in participating in the support group • Has not previously participated in the support group (which was offered to some 8th graders in participating schools). Program description The program under study, Abangane (“friends” in isiZulu), is a locally derived, curriculum-based support group focused on coping with loss incorporating indigenous stories and cognitive behavioral therapy components. The program originated from activities described in the Khu Kit, a guide developed by South African non-profit organization Khululeka Grief Support1 in response to the psychological needs of the country’s many children and adolescents who had lost a parent or someone else important in their lives. Child Welfare Bloemfontein and Childline Free State (CWBFN & CLFS) adapted activities from the Khu Kit into an eight-session curriculum (see Table 1) for female adolescents under the guidance of Khululeka Grief Support and Tulane University. Abangane support groups include activities guided by cognitive behavioral therapy principles and indigenous games and songs, contextually relevant stories and scenarios, as well as discussions about cultural rituals and traditions surrounding death. The intervention 1 Khululeka Grief Support was started in 2005 by a group of social workers, psychologists and counsellors in response to the large number of children orphaned by AIDS. From these efforts, the group developed the Khu Kit, a guide for training and mentoring grief group facilitators and creating peer support groups for bereaved children and youth. The Khu Kit includes a resource book that provides training and reference materials (including referral agencies) for grief group facilitators. The Khu Kit also includes exercises and creative activities designed to stimulate group discussions that will help children and adolescents to deal with loss and associated grief. More information about Khululeka Grief Support can be found on their website: http://www.khululeka.org/ 10 was pretested for acceptability in the target population. This is the first quantitative evaluation of Khululeka Grief Support’s services. Program partners CWBFN & CLFS is a non-governmental organization in South Africa offering an array of child protection services2 . Supported by PEPFAR funding from USAID Southern Africa, their services include the placement of qualified social workers within schools to identify and provide specialized care and support initiatives to orphans and vulnerable children (OVC). Prior to implementing the Abangane support group program, CWBFN & CLFS Social Workers received training and support from Khululeka and have since the year 2013 facilitated groups employing various activities among bereaved adolescents. Based on their implementation experience and with guidance from Khululeka, CWBFN & CLFS established a core standardized 8 session curricula for Abangane to be implemented across the study sites. Table 1. Content of the Abangane Grief support group Session Theme Session 1 Introduction and relationships building Session 2 Naming, identifying, understanding, and normalizing feelings Session 3 My personal experience of loss Session 4 Changes in my life resulting from loss Session 5 Rituals and traditions surrounding loss, and saying goodbye Session 6 Coping skills Session 7 Looking to the future Session 8 Closure of the group Stakeholder engagement The Abangane program is implemented with funding from the US President’s Emergency Plan for AIDS Relief (PEPFAR) through USAID Southern Africa. Training and resources were received from the Khululeka Grief Support Program in Cape Town, South Africa. Additionally, Tulane conducted an in￾depth workshop with CWBFN & CLFS staff in September 2014 to explain the study, gather feedback and answer questions. This helped ensure that program staff were well-informed prior to the evaluation beginning. Engagement of local stakeholders (e.g. implementing organizations, program partners) was critical to defining the scope of the evaluation study and designing survey methods that are contextually appropriate and responsive to the needs of the funders, program partners and participants. While it was important that the evaluation team operated independently from implementers and other local key 2 For more detailed information on CWBFN & CLFS, please visit their website: http://www.childwelfarebfn.org.za/index.php. 11 stakeholders, it was essential to have open communication about roles, the purpose of the evaluation and its protocol and potential benefits. As a result, from the outset of the study, the evaluation team maintained frequent communication with the local stakeholders aiming to provide a well-developed and informed understanding of the program and its strengths and limitations among all stakeholders involved. For example, implementation partners were invited to assist with identifying key services and expected outcomes to ensure that the evaluations study accurately captured the program efforts. Furthermore, continued engagement of local stakeholders was maintained throughout the research process to ensure local relevance and the broader utilization of evaluation findings. Post-evaluation workshops and other forums were organized to help implementing partners and other local stakeholders understand and use study results (see, for more detail, ‘Dissemination’ section of this report). Outcome evaluation sampling This study is an impact evaluation that used a randomized controlled trial to assess the effectiveness of a structured eight-session bereavement support group for female adolescents who have experienced the death of someone important in their lives. The program was implemented in 11 peri-urban schools in three towns (Bloemfontein, Ficksburg and Parys) by CWBFN & CLFS, with all 11 schools included in the evaluation. The schools were chosen by the provincial Department of Education in consultation with CWBFN with emphasis on serving those most in need considering vulnerability indicators of the area and school population. The sample for the evaluation study was identified through CWBFN & CLFS’s intake data which program partners routinely collect to identify OVC for their reporting and service planning purposes. Eighth grade students (ages 12–17 years) from participating schools completed an intake form collecting basic demographic and contact data from September 2014, through to February 2015. Students were asked to indicate whether they had experienced the loss of someone close to them and if they would be interested in participating in a bereavement support group. As outlined in previous sections, eligibility was limited to 13–17-year-old Sesotho-speaking girls enrolled in the ninth grade at a participating school in 2015 who reported experiencing a loss and expressed interest in taking part in a support group. CWBFN opted to limit the intervention to female participants due to concerns regarding participants’ ease of interaction in mixed-sex groups, and because girls had previously shown greater commitment than boys to session attendance. The number of eligible adolescents (N= 1,049) assessed for eligibility exceeded the organization’s service delivery capacity and therefore adolescents who had experienced the death of one or both parents were given priority as orphanhood is a well-documented risk factor for psychological problems among children. During the study recruitment, the social workers visited both the potential participants and their caregivers at home to explain the study and whether they were being invited to participate in the program or wait-listed for later enrollment. The caregivers were encouraged to consent to their adolescent’s involvement in the intervention. All selected adolescents and caregivers were also provided with a study information letter that explained in detail the nature of the study, including the completion of survey, voluntary aspect of their participation, and that their eligibility to receive services from CWBFN & CLFS regardless of their decision to participate or not in the study. Consent/assent for study 12 participation were then conducted separately by trained research personnel not affiliated with CWBFN & CLFS as described in subsequent sections. Data collection timeline and procedures A smaller sample of 704 eligible female adolescents at each school were randomly assigned to receive the intervention in 2015 or wait-listed for program enrollment until the end of the study. The sample was stratified on school and orphan status to ensure equal numbers of students in the intervention and wait groups as well as equal numbers of orphans in each group. All participants had access to the standard of care consisting of a school-based CWBFN counselor available to serve students based on self-referral or referral by a teacher. Participants in the intervention groups met for weekly, interactive 90 min sessions that included an average of three structured activities focused on experiences of loss and grief, coping skills, and the links between feelings, thoughts and behavior. All sessions included an opening and closing ritual and time for reflection. Homework was assigned at the end of each session and discussed at the start of the next one, including identifying sources of support, defining goals, and recognizing and challenging negative thoughts. Each participant was provided with a journal to use for recording their progress and feelings. CWBFN social workers or social auxiliary workers facilitated the groups in private meeting spaces during school hours with permission from the Provincial Department of Education. Facilitators completed a 4- day training from Khululeka Grief Support and began delivering the program to adolescents (not including the study population) about 1 year in advance of the study. They also participated in a 3-day refresher training immediately preceding the study. Facilitators attended weekly supervision meetings with the CWBFN program manager and provided a written account of each session for supervision and quality assurance purposes. An overview of the study design and data collection waves is depicted in Figure 2. Adolescents and their caregivers were asked to complete a baseline survey before beginning the Abangane sessions and a follow-up survey 3 months after the intervention ended in 2015. The intervention was offered to different groups of participants at three time periods (or waves as depicted in Figure 2), corresponding with the school year’s first three terms. Within each school, baseline and follow-up surveys were completed simultaneously with equal numbers of intervention and waitlist group participants to ensure equivalent time between survey rounds (an average of 184 days for the intervention group and 189 for the waitlist group). 13 Figure 2. Study Design and Data Collection Timeline Baseline and follow-up surveys were translated into Sesotho, back-translated into English, and pre-tested to help to improve clarity and accuracy. All surveys were conducted by trained interviewers from a research company unaffiliated with program partners. Face-to-face interviews were done with participants in their local languages and in their homes using portable electronic devices to read questions and record responses. Informed consent from caregivers and assent from adolescents were obtained at the outset of interviews in private and without the presence of program affiliates. Following the baseline survey, interviewers provided all adolescents, including those who declined to take part in the study, with a book of stories related to African children’s experiences with loss. Key outcome measures Two surveys were developed for this study: one for adolescents and one for their caregivers (see Appendix C). The adolescent instrument was designed to gather information on the primary and secondary outcomes hypothesized to be affected by the interventions as well as demographics and background information. In addition, the adolescent survey asked bereaved adolescents to identify a focal loss (i.e., the death that had affected them the most), and all questions about grief were oriented to this focal loss. The caregiver survey was designed to collect information relevant to the household and caregiver factors (e.g., poverty, chronic illness, caregiver mental health). Both the caregiver and adolescent surveys included scales to measure discipline practices and caregiver-child communication. At follow-up, the adolescent and caregiver surveys included a module to obtain information on their perceptions of the interventions of interest. Most of the evaluations outcome measures were captured applying standardized indicators and scales that have been previously validated with populations in sub￾Saharan Africa. Instruments were delivered in the preferred language of the participant, Sesotho. An overview and brief description of primary and secondary outcome measures are presented below. 14 Primary Outcome Measures Abangane grief support group was designed to reduce distress related to bereavement, thus adolescent grief intensity and mental health, including depression and problem behaviors, were the key psychological outcomes in this study. Grief intensity of the adolescent was measured using items from three standardized instruments: Core Bereavement Items (CBI), Intrusive Grief Thoughts Scale (IGTS) and the Inventory of Complicated Grief – Revised for Children (ICG-RC). • Core Bereavement Items (CBI) is a 17-item scale composed of three subscales: Images and thoughts; acute separation and grief(Burnett, Middleton, Raphael, & Martinek, 1997). The CBI is widely used to assess levels of normative grief and the adolescent survey included the five-item grief subscale of the CBI to assess how often the adolescents experienced common grief￾associated feelings, such as longing, loneliness, tearfulness, and sadness. Participants rated their frequency of experiencing these feelings (“never”, “sometimes”, “most of the time,” or “all of the time”) with corresponding values of 0 to 3 assigned. • Intrusive Grief Thoughts Scale (IGTS) is a nine-item scale that measures the extent to which bereaved individuals experience intrusive, undesired thoughts about a deceased loved one (ASU Program for Prevention Research, 1999). Participants were asked to report how often, in the past four weeks, they experienced interferences with daily activities such as difficulty falling asleep, bad dreams, and concentrating at school due to thinking about the loved one’s passing. A five-item response scale from “almost never or not at all” to “several times a day” was used, coded 1 to 5, respectively. • The Inventory of Complicated Grief–Revised for Children (ICG-RC) measures complicated grief and is a six-item subscale of the adult Inventory of Complicated Grief validated for use with children (Melhem, Porta, Walker Payne, & Brent, 2013). Items consist of longing and yearning for the deceased, inability to accept the death, shock, disbelieve, loneliness and a changed world view. The scale reflects lack of acceptance, shock, changed world perspective, loneliness, and longing for the deceased. Respondents’ ICG–RC scores were the sum of the item values, and use the same reference period, response options, and coding as the IGTS. Depressive symptomology was measured by the 20-item Center for Epidemiologic Studies Depression Scale for Children (CES-DC) (Faulstich, Carey, Ruggiero, Enyart, & Gresham, 1986; Weissman, Orvaschel, & Padian, 1980) previously used in research with South African youth (Pretorius, 1991; Smit et al., 2006). Questions assessed the past week frequency of depression symptoms, including unhappiness, loss of appetite, low self-esteem, low energy levels, and difficulty sleeping. Responses were totaled across a four-item scale from “not at all” to “a lot”, coded 0 to 3, respectively. Adolescent behavioral problems were measured with the Brief Problem Monitor-Parent Form (BPM-P) (Achenbach, McConaugh, Ivanova, & Rescorla, 2011) completed by caregivers. BPM-P is a 19-item subset of the Child Behavior Checklist (CBCL) that captures problem behavior, including internalizing behavior, attention problems, externalizing behavior in children aged 6-18. Caregivers used three 15 response options (“not true”, “somewhat true”, or “very true) to endorse or reject statements characterizing their child’s behavior during the past 4 weeks. Secondary Outcome Measures By reducing distress and promoting coping skills, Abangane Grief support group was expected to improve a set of secondary outcomes, including caregiver-adolescent relationship, social support available to program participants, academic competence and adolescent resilience conceptualized as the ability to assess and cope with the stress of bereavement. Caregiver- adolescent relationships were assessed through adolescent report of connectedness to caregiver and caregiver report of positive parenting practices and supportiveness: • Connectedness to caregiver was measured the 25-item parental subscale of the Inventory of Parent and Peer Attachment (IPPA) (Armsden & Greenberg, 1987). Adolescents were asked to report on the extent of respect, acceptance, attention, understanding, and communication they receive from their caregiver. Responses were summed across a four-item scale from “not at all true” to “completely true” and coded 0 to 3, respectively. • A ten-item supportive parenting subscale from the Parent Behavior Inventory (PBI-S) (Lovejoy, Weis, O'Hare, & Rubin, 1999) was used to assess caregiver affect towards the adolescent in their care. Caregivers were asked to indicate how often in the past four weeks they engaged in supportive behavior such as listening to the adolescent’s feelings and comforting her during a difficult time. Responses were summed across a five-item response scale from “never” to “always”, coded 1 to 5, respectively. Social support was measured through items focused on social support received in dealing with the bereavement itself and overall emotional social support available to adolescents. • Social support for coping with bereavement was measured using four self-generated and the Grief Evaluations Measure (GEM)-based items (Jordan, Baker, Matteis, Rosenthal, & Ware, 2005). For example, participants were asked if they had received the help they needed to cope with their loss, or whether they can discuss their feelings about the loss with their families. Response options ranged from “not at all true” to “completely true”, coded 0 to 3, respectively. • Adolescents’ perceptions of available emotional social support were measured using the receiving emotional support subscale of the 2-Way Social Support Scale (SSS-R)(Shakespeare￾Finch & Obst, 2011). Participants were asked about seven types of emotional support, including having someone in their life they can trust, who makes them feel worthwhile, and who they can turn to if they are feeling sad. Responses were summed across a four-point scale from “not at all true” to “completely true”, and coded 0 to 3, respectively. Academic competence was measured through academic self-efficacy, motivation and attendance: • The academic self-efficacy subscale of the Self-Efficacy Questionnaire for Children (SEQ-C) (Muris, 2001) was used to assess academic self-efficacy or perceived performance in school including success in paying attention, passing tests, and completing homework. Responses were 16 totaled across a seven-item scale from “not at all true” to “completely true”, coded 0 to 3, respectively. • Academic motivation assessed the perceived importance of education and efforts to do well in school. Six items were included in the survey with response options ranging from “not at all true” to “completely true”, coded 0 to 3, respectively. • School attendance was measured with one self-generated item which asked participants about absenteeism in the seven days prior to the survey, not including school holidays. Resilience was measured through positive appraisal coping abilities, including perceived coping, personal growth, and emotional self-efficacy: • The personal growth subscale of the Hogan Grief Reaction Checklist (Hogan, Greenfield, & Schmidt, 2001) was used to assess the adolescent’s perceived growth as a result of the loss. Eleven-item scale included questions on empathy, understanding and caring towards others, and becoming a better person. Response options ranged from “not at all true” to “completely true”, coded 0 to 3, respectively. • The emotional self-efficacy subscale of the Self-Efficacy Questionnaire for Children (SEQ-C) (Muris, 2001) was used to assess adolescent’s emotional self-efficacy, or perceived ability to regulate feelings and emotions such as anxiety, sadness, disappointment and fear. Responses were totaled across a seven-item scale from “not at all true” to “completely true”, coded 0 to 3, respectively. Data analysis The analysis plan included a preliminary analysis of baseline data to assess the prevalence and levels of bereavement among study participants and an impact analysis conducted using both pre- and post-test data to determine the effect of the intervention. Data from the adolescent surveys was merged with data collected from their caregivers to create an analytical dataset containing a broad array of risk factors and potential confounders. Frequencies, ranges, means, and standard deviations were generated in SAS version 9.3. Differences between the intervention and waitlisted groups were examined at baseline and between those lost to follow-up or not with t tests for continuous variables and χ² tests for dichotomous variables. Mann￾Whitney U tests were used to test for differences in the outcomes by treatment group at baseline. Program effects were estimated using generalized estimating equations (GEE) in STATA version 14. The GEE method was chosen for its flexibility in handling skewed data and its ability to yield population￾averaged estimates in intention-to-treat analyses such as this one (i.e., analysis by treatment assignment rather than recorded attendance). All models used autoregressive covariance structures at the first level to account for correlations within individuals, and bootstrapped standard errors to adjust for correlations within treatment groups. All models included a dummy variable for group assignment and a dummy variable for survey round, with an interaction term for group assignment by survey round. Program impact was assessed using the group 17 assignment by survey round interaction effect. Two control variables were included in all models: the participant’s baseline value for the outcome and the years (or fraction of years) elapsed since the adolescent experienced the focal loss. Ethical considerations and assurances This evaluation study was subject to the approval of the Institutional Review Board at Tulane University, to which the evaluation team is affiliated. Data collection commenced following review and approval of the study protocol, data collection instruments and related materials by the Faculty of Health Sciences Ethics Committee at the University of Free State and the Tulane University Human Research Protection Program in the USA in January 2015. The following procedures were strictly adhered to, to ensure ethical implementation of the study: • Informed consent/assent: Both caregiver consent and adolescent assent were required for study inclusion. Individuals were not eligible if they were not able to give informed assent due to known or recognizable cognitive or psychiatric impairment. Informed consent and assents were obtained from every participant prior to beginning data collection, and caregivers were asked to provide consent for the youth in their care to participate. All adolescents who participated in the study were provided informed assent using culturally and age appropriate information. • Voluntary participation: As part of the consent and assent procedures, the interviewer explained clearly before the questioning began that the respondent’s participation is entirely voluntary and that the respondent has the right to refuse to answer any question or any part of any question that he or she did not wish to respond to. In addition, all potential participants were reminded that they had the right to terminate the interview at any time. Finally, they were made aware at the outset that their decision to participate or not in the study did not affect their eligibility to receive services from the programs then or in the future. • Confidentiality: All participants were informed of provisions to ensure confidentiality (e.g. that their answers would be held in strict confidence and stored in the different place as their contact information. In addition, a number of mechanisms were in place to protect the privacy and confidentiality of participants during the data collection procedures (e.g. interviews conducted in a private settings with no observers allowed; no interviewer conducted an interview in their own community). In addition, Tulane researchers adhered to strict confidentiality guidelines in handling the survey data (e.g. the program partners were not be made aware of individual survey results of any child or caregiver; all participants were assigned unique identification numbers, and their actual identification information (i.e., names and addresses) were kept separately from the questionnaires). • Protection of the data: All beneficiary data files that provide the information linking individuals to their unique identifiers were password-protected and kept in locked cabinets accesible only to the Tulane Univerity senior researchers. Similarly, the data for tracking sheets, which included contact and identifying information, was captured on a different database than the actual 18 survey data. This data is also password protected with access limited to lead research team members. • Referrals: Given the sensitive nature of discussions of loss and grief for participants, especially adolescents, the research team ensured that the participants were aware of the survey included questions on highly personal and sensitive topics. During interviews, questions concerning loss were approached in a standardized, careful and sensitive manner by trained interviewers. Further, as a precautionary and support measure, a counseling referral mechanism was organized in collaboration with local program partners. In advanced of fieldwork, Tulane research staff arranged with the CWBFN & CLFS for qualified community-based counselors to be available during the interview period should a respondent became upset during the interview, thus requiring a counseling referral. Interviewers were also trained to recognize this need and to handle the situation in a sensitive, non-judgmental manner. In addition, participants were provided with a social service sheet detailing local resources available to them. • Wait groups service delivery: The research team was cognizant of the potential concerns associated with randomized trials and the delay of services for wait group participants. At the time of the study, there were no grief counseling or grief support groups available to children or adolescents outside of the Abangane support program at these schools and so the wait group was not deprived of standard services or treatment. Further, the study was occurring within the context of CWBFN & CLFS expected service delivery target numbers, and there was thus no intentional denial of services – rather a method to allow for equal opportunity for immediate enrollment. Lastly, all adolescents received useful resources and opportunities, including the book and social service sheets as well as an eventual opportunity to participate in the intervention. The decision to prioritize service delivery to the wait group adolescents was encouraged by the study team and approved by CWBFN & CLFS, and was a provision that may not have otherwise been available outside of the study context given resource constraints. Deviations and adjustments This evaluation had a one major adjustment from the protocol originally submitted and approved by the IRB (see Appendix A), namely omitting caregiver program from the trial. The evaluation trial was originally designed to include a parallel support program for the primary caregivers of adolescent participants, aimed at helping to resolve caregivers’ grief so they could better support the children in their care. The caregiver component was dropped from the trial due to very low early session attendance among caregivers. Possibly, caregiver participation could have resulted in different outcomes, for example, more pronounced improvements in adolescents’ psychological health and social support outcomes. Furthermore, engaging caregivers is particularly crucial in South Africa, where previous research suggests that strong cultural norms prevent adults from discussing death with children (Van der Heijden & Swartz, 2010). 19 Procedures to ensure high quality data Several procedures were followed throughout the implementation of the program to ensure the highest quality of the data collected: • Survey Preparation: The survey was developed in English and reviewed by the implementing organization to ensure the appropriateness of the content and terms used. The questions were translated from English into Sesotho by a professional translation company not affiliated with the research team or CWBFN & CLFS. Two different translators fluent in both English and Sesotho independently translated each instrument. A third party then compared translations and resolved any discrepancies. During training, the field team will also extensively review all translations to ensure proper translation of the meaning and understanding of the contents. The translated survey were then pilot tested among students who have experienced the loss of a loved one in a school that were not included to participate in the study, but was to receive the grief groups from CWBFN & CLFS in the future. Results from the pilot testing were used to further refine the survey questions and to adjust the length of the survey so that it does not take more than one hour to administer. • Survey Modality: Surveys were administered face-to-face using computer-assisted personal interviewing (CAPI) technology. Questions and responses were displayed on the screen of a netbook held by the interviewer. The interviewer read the questions and responses and recorded the respondent’s answer by using the touch screen. The data were then automatically stored in the database and encrypted, thereby eliminating the need for double capture of surveys and other time-consuming data entry error control processes. System checks and automatic skip patterns were built into the system to avoid interviewer error in question ordering. • Data collection & Field Team: The data collection was led by Tulane researchers and implemented by a qualified research partner selected through a competitive bid process. Required qualifications for the research partner and field team included: extensive evaluation experience in South Africa and undertaking similar evaluations; experience conducting quantitative interviews in English and Sesotho; and research experience working with vulnerable populations, including adolescents. In conjunction with Tulane research staff, the research partner developed interviewer and supervisor training guides and facilitated an extensive training session. All fieldworkers were briefed on the purpose of the research and trained on how to apply the research tools and methodologies. In addition all fieldworkers weree specifically trained in ethical procedures, including maintaining respondent privacy, ensuring voluntary participation and confidentiality, carrying out participant referrals, and dealing with sensitive topics. 20 Findings and Limitations Findings presented in this section entail descriptive characteristics of the overall sample used in the study, program effects for intervention and waitlist groups, and participants’ feedback on the program. During the participant recruitment period (September 2014 to February 2015), 704 eligible participants were identified, of whom 226 were assigned to Abangane grief support group and 227 were assigned to waitlist. Four-hundred and one adolescents completed the baseline survey (200 assigned to intervention and 201 assigned to waitlist), with 96·5% retention at follow-up (387participants in total; 196 assigned to intervention and 191 assigned to waitlist). No statistical differences were found between participants retained and those lost to follow-up on the outcomes under investigation and the sample characteristics at baseline. Evaluation results on the primary and secondary outcomes (i.e. adolescent-reported grief, depression, social support, and supportive parenting) are restricted to the 382 adolescents (193 assigned to intervention and 189 assigned to waitlist) who completed both survey rounds and were able to estimate the time elapsed since their focal loss. Analysis of caregiver-reported indicators of adolescent behavioral problems and supportive parenting was restricted to the 376 adolescents whose caregivers completed both survey rounds (189 assigned to intervention and 187 assigned to waitlist). Baseline characteristics for the intervention and waitlist groups are presented in Table 2. Mean age of adolescents in the study at baseline was 14.5 (SD=1.2) and an average of 4.5 years (SD=3.7) had passed since participants’ focal loss. On average, the focal loss occurred more than 4 years prior to the intervention, and the average age at which adolescents experienced loss was about 10 years of age at both the intervention and waitlist groups. About half of all adolescents (N=173 or 45%) reported the loss of a biological parent as the one that had affected them the most. Nearly three-quarters of adolescents reported that the loss was due to an illness, among whom only 17 (4%) cited AIDS as the cause of death (although 146 or 38% indicated it was due to a prolonged illness that lasted more than 3 months). Overall, the intervention and waitlist groups were similar with respect to their histories of loss (see Table X). 21 Table 2. Baseline characteristics of participants in the intervention and waitlist groups Waitlisted (N=189) Intervention (N=193) Mean (SD) Mean (SD) Age (years) 14.5 (1.2) 14.4 (1.2) Years since focal loss 4.5 (3.5) 4.6 (3.9) Loss that most affected them N (%) N (%) Biological parent 81 (43%) 92 (48%) Grandparent 54 (29%) 53 (28%) Other 54 (29%) 48 (25%) Cause of death Accident 9 (5%) 11 (6%) Violence (including suicide) 5 (3%) 6 (3%) Illness 142 (75%) 142 (74%) Unknown 33 (18%) 34 (18%) Experienced multiple losses 83 (44%) 98 (51%) Primary caregiver Mother 90 (48%) 87 (45%) Father 9 (5%) 5 (3%) Grandparents 34 (18%) 49 (25%) Aunt or uncle 19 (10%) 19 (10%) Other 37 (20%) 33 (17%) Orphan (single or double) 96 (51%) 106 (55%) Program Effects The effects of the program and mean scores for each outcome by treatment group are presented visually in Figures 3 – 15. Mann-Whitney U tests showed that baseline scores did not differ between the two groups (data not shown), with the exception of Brief Problem Monitor (BPM-P) which was significantly higher for the intervention group than for the waitlist group (p=0.008). Scores for primary outcomes declined between baseline and follow-up survey rounds in both groups, suggesting increasing psychological health, although greater improvements were noted in the intervention group. 22 Primary Outcome: Grief Intensity Three standardized instruments were used to measure grief intensity, namely Core Bereavement Items (CBI), Intrusive Grief Thoughts Scale (IGTS) and the Inventory of Complicated Grief –Revised for Children (ICG-RC). As seen in Figure 3, the mean scores of normative grief (measured with the first instrument – CBI-G) decreased between survey rounds in both groups. However, the reduction in CBI-G scores for the intervention group was not statistically significant (p=0.44). Figure 3. Impact of Abangane program on normative grief Figure 4 indicates the change in intrusive grief thoughts (measured with IGTS) among the intervention and wait groups at baseline and follow-up. While both groups reported very similar levels of intrusive grief thoughts at baseline, at follow up the intervention groups showed a significant reduction in IGTS scores (p=0.00) relative to the waitlisted group. These findings indicate that while both groups had fewer intrusive grief thoughts over time, there was a significantly higher reduction in intrusive grief thoughts among program participants. Figure 4. Impact of Abangane program on intrusive grief thoughts 7.3 5.8 6.6 4.4 3 4 5 6 7 8 Baseline Follow-up Mean score Changes in normative grief scale (CBI-G) Wait group Intervention 23 In addition, the intervention group showed statistically significant reduction in the complicated grief index (ICG-RC) indicating that while the symptoms of complicated grief decreased between survey rounds in both groups (see Figure 5), the decrease was significantly more pronounced in the intervention group as compared to the waitlist group (p=0.015). Figure 5. Impact of Abangane program on complicated grief Primary Outcome: Mental Health Depression among adolescents was measured using survey items drawn from the Depression Scale for Children (CES-DC), specifically designed to measure depression among children and adolescents. These items assess the frequency of symptoms of depression experienced in the preceding 7 days, including feelings of happiness, appetite, self-esteem, energy levels, and sleep patterns. Figure 6 shows the change in depression levels among the treatment and wait group at baseline and follow-up suggesting that the symptoms of depression decreased over time in both groups. However, the regression analysis indicated that decrease in depression levels was more pronounced for the intervention group as compared to the waitlist group (p=0.009). 21.9 18.7 21.7 16.6 15 17 19 21 23 Baseline Follow-up Mean score Changes in intrusive grief thoughts (IGTS) Wait group Intervention 16.6 13.4 16.5 12.3 11 12 13 14 15 16 17 Baseline Follow-up Mean score Changes in complicated grief scale (ICG-RC) Wait group Intervention 24 Figure 6. Impact of Abangane program on depression The Brief Problem Monitor (BPM-P) scale designed for youth aged 6-18 years and adapted from a broader Child Behavior Checklist was used to assess adolescents’ behavior problems. The survey questions on this measure were completed by caregivers and captured adolescent’s internalizing, and externalizing behavior, as well as attention problems in the preceding 4 weeks. Figure 7 indicates the change in behavior problems among the treatment and wait group at baseline and follow-up. The mean scores in BPM-P measure show that adolescents in the intervention group had higher levels of behavioral problems at baseline than those in the wait group. However, while behavioral problems decreased among both groups over time, the intervention groups showed significantly higher reduction in problem behaviors than the waitlist group (p=0.017). Figure 7. Impact of Abangane program on problem behavior Secondary Outcome: Caregiver-Adolescent Relationships Caregiver-adolescent relationship, social support, academic competence and resilience comprised secondary outcomes assessed in this study. Caregiver-adolescent relationship was conceptualized as adolescents’ sense of attachment and connectedness to their caregivers and was measured using the Inventory of Parental and Peer Attachment (IPA). The IPA items included in the adolescent survey assessed the communication, trust, and alienation aspects of caregiver-child attachment in the 16.6 13.4 16.5 12.3 11 13 15 17 Baseline Follow-up Mean score Changes in depression scores (CES-DC) Wait group Intervention 8.2 6.6 10.2 5 6 7 8 9 10 11 Baseline Follow-up Mean score Changes in problem behavior (BPM-P) Wait group Intervention 25 preceding 4 weeks. Figure 8 indicates the change in positive attachment levels among the intervention and wait group adolescents at baseline and follow-up. At baseline, both groups reported similar levels of positive caregiver attachment. While positive caregiver attachment remained stable among adolescents in the intervention group, it decreased significantly among the wait group over time. Thus, the program effect was statistically insignificant on the parental attachment index (p=0.10). Figure 8. Impact of Abangane program on parental attachment The Parent Behaviour Inventory (PBI) was used to assess supportive parenting behavior. Survey questions on this measure were completed by caregivers who indicated how often in the past 4 weeks they engaged in supportive behavior such as listening to the adolescent’s feelings and comforting her during a difficult time. As seen in Figure 9, there was a slight increase in supportive parenting behavior among intervention group whereas the mean scores in PBI measure decreased in the wait group over time. Overall, no statistically significant improvement in supportive parenting was found in the intervention group (p=0.067). Figure 9. Impact of Abangane program on supportive parenting Secondary Outcome: Social Support Social support available to adolescents was measured through items focused on emotional social support and social support received in dealing with the bereavement. Emotional social support was measured with the emotional support subscale of the 2-Way Social Support Scale (SSS-R), and Figure 10 47.4 49.4 49.5 46 47 48 49 50 Baseline Follow-up Mean score Changes in Inventory of Parental Attachment (IPA) Wait group Intervention 42.3 40.9 41.8 42.2 40 41 42 43 Baseline Follow-up Mean score Changes in supportive parenting (PBI-S) Wait group Intervention 26 shows the change in emotional social support received from others at baseline and follow up. While the waitlist group reported a slightly higher level of support received at baseline compared to the intervention group, the wait group reported a decrease in the emotional social support they received at the follow up. The intervention group, on the contrary, reported significantly higher levels of emotional social support available to them at the follow up. While small in effect size, there was a statistically significant program effect in emotional social support available to the intervention group (p=0.005). Figure 10. Impact of Abangane program on emotional social support Several self-generated and the Grief Evaluations Measure (GEM)-based items were used to assess the support adolescents received in coping with their grief. In their surveys, adolescents indicated the levels of support they had received or available to them in coping with loss of their loved one. Figure 11 shows the change in grief support levels among the intervention and wait groups at baseline and follow-up. While groups reported relatively similar levels of grief support at baseline, adolescents in the intervention group reported a significant increase in grief support over time, while there was almost no change among adolescents in the wait group. The difference in grief support mean scores was statistically significant (p=0.000) indicating significant program effect on the grief support available to adolescents in the intervention group. Figure 11. Impact of Abangane program on support received in coping with grief 15.3 15 14.6 15.5 13 14 15 16 Baseline Follow-up Mean score Changes in emotional social support (SSS-R) Wait group Intervention 6.9 6.8 7.3 8.5 5 6 7 8 9 10 Baseline Follow-up Mean score Changes in grief support (GEM) Wait group Intervention 27 Secondary Outcome: Academic Competence Academic confidence was measured with academic self-efficacy assessing academic motivation. The academic self-efficacy subscale of the Self-Efficacy Questionnaire for Children (SEQ-C) was used to assess perceived performance in school including success in paying attention, passing tests, and completing homework. As seen in Figure 12 below, at both baseline and follow-up rounds, both groups reported similar levels of academic self-efficacy which slightly decreased over time. No significant intervention effect was found for academic self-efficacy (p=0.35). Figure 12. Impact of Abangane program on academic self-efficacy Adolescents’ academic motivation assessed the perceived importance of education and efforts to do well in school. As Figure 13 shows, there was no the change in academic motivation reported by the study participants in the intervention group, while a slight decrease in academic motivation was observed among the wait list group. In summary, there was no intervention effect found for academic motivation (p=0.75). Figure 13. Impact of Abangane program on academic motivation School attendance was measured with one item asking adolescents if any days of school had been missed in the preceding week. No significant intervention effect was found for this outcome either (data not shown). 16.95 15.96 17.04 16.27 15 16 17 18 Baseline Follow-up Mean score Changes in academic self-efficacy (SEQ-C) Wait group Intervention 12.7 12.5 12.6 12.6 12 12.2 12.4 12.6 12.8 13 Baseline Follow-up Mean score Changes in academic motivation Wait group Intervention 28 Secondary Outcome: Resilience Resilience or the adolescents’ post-traumatic growth as a result of their loss was assessed using the personal growth subscale of the Hogan Grief Reaction Checklist that measured a sense of growth in compassion, tolerance, caring and hopefulness following the loss of a loved one. As seen in Figure 14 below, there was a slight increase in reported post-traumatic growth in both study groups over time. No significant program effect was found for this resilience measure (p=0.22). Figure 14. Impact of Abangane program on post-traumatic growth Adolescent’s emotional self-efficacy was measured using a subscale of the Self-Efficacy Questionnaire for Children (SEQ-C) and assessed adolescent’s perceived ability to cope with negative emotions and feelings, such as fear, sadness and anxiety. Figure 15 below shows very minimal changes in the emotional self-efficacy scores for both groups over time. In addition, no significant intervention effect was found for this outcome (p=0.85). Figure 15. Impact of Abangane program on emotional self-efficacy 20.8 21.4 21.6 22.2 20 21 22 23 Baseline Follow-up Mean score Changes in post-traumatic growth (HGRC subscale) Wait group Intervention 11.1 11.1 11.2 11.5 10 10.5 11 11.5 12 Baseline Follow-up Mean score Changes in emotional self-efficacy (SEQ-C subscale) Wait group Intervention 29 Program evaluation: Participants’ perspectives The overall attendance of the Abangane groups was high, with 178 (92%) of adolescents assigned to take part in the intervention attended at least one session, 148 (77%) attended seven or more sessions, and 126 (65%) completed the full eight-session program. After the program was completed, adolescents assigned to the intervention group were asked to share their experiences about the program and lessons they had learned. Participants were asked to rate a series of statements about the program, including their experiences with the facilitators and other group members, using a 4-point rating scale (ranging from “not true at all” to “completely true”). The findings presented in Figure 16 and Figure 17 show the percentages of positive experiences (i.e. those who answered “mostly true” or “completely true”) reported by participants3 . Overall, the vast majority of program participants expressed high levels of satisfaction with the facilitators and the group experience. In terms of their experiences with the facilitators (Figure 16), 95% of participants indicated that the facilitators understood their problems and feelings, while 92% indicated that facilitators explained things in a clear and understandable manner. With regard to their experiences of the groups (Figure 17), 81% felt understood by other group members, 92% felt comfortable sharing their personal stories in the group, and 82% felt supported by other group members. Figure 16. Participants’ perspectives on the program facilitators Figure 17. Participants’ perspectives on the program groups 3 Percentages presented in this section are based on the sample of 196 adolescents enrolled in the program. 92% 95% 80% 85% 90% 95% 100% Facilitators explained things in an understandable manner Felt that facilitators understood problems and feelings Experiences with facilitators 30 To capture participants’ perspectives of the program impact on their lives, adolescents were also asked to indicate some of the program benefits or lessons learned while participating at the Abangane grief support group. This helped to capture their own perspectives and thoughts on the impact and unique changes that program participation brought to their lives, otherwise difficult to capture with standardized scales. As seen from adolescents’ responses presented in Figure 18, the vast majority reported several positive changes that occurred following program participation. More specifically, 97% reported having learned new ways of solving problems, 88% made at least one new friend as a result of the attending the program, and 93% felt better able to deal with their difficult emotions. A further 95% felt the program helped them to cope with the loss of a loved one. Most importantly, all participants (or 100%) who attended the program said that they would recommend the Abangane grief support group to a friend who experienced the loss of a loved one. Figure 18. Perceived effects from program participation 82% 92% 81% 0% 20% 40% 60% 80% 100% Felt supported by other group members Felt comfortable sharing personal stories in the group Felt understood by other group members Group-specific experiences 95% 93% 88% 97% 80% 85% 90% 95% 100% Helped them to cope with their loss Better able to deal with difficult emotions, such as anger or sadness Made at least one new friend Learned new ways to solve problems Participants' perspectives on the program benefits 31 Limitations There are several limitations that need to be taken into consideration with respect to the study findings: • The study population was restricted to girls, and thus the findings cannot be generalized to male adolescents. The decision to restrict program groups to girls-only was intentional one based on the greater enrollment of females in the pilot, and concerns about non-conducive behavior in mixed-gender adolescent groups. In addition, the program implementers felt that the intervention’s emphasis on emotional disclosure stood in contrast with local behavioral norms for adolescent males. • The median time between the focal death reported by participants and baseline assessment was relatively long, at more than 4 years. Eighteen percent of the adolescent didn’t know the cause of death, whether it was natural or unnatural causes. Reliance on adolescents’ reports probably exacerbates misclassification. Three quarters had died from Illness, however, the study did not collect information on the exact death-causing illness and therefore did not determine the number of children who had lost one or both parents to an AIDS-related illness. Longitudinal research has shown greater psychological challenges in this population relative to adolescents orphaned by other means and non-orphans. • This evaluation included only one short-term follow-up assessment (i.e. follow-up surveys completed approximately 3 months post intervention), restricting an exploration of benefits that emerge over a longer timeframe. A longitudinal evaluations design with several long-term follow-up assessments follow-up need to be developed to examine the endurance of program effects over time. Despite the limitations, this evaluation offers important new evidence of a culturally tailored and logistically feasible intervention addressing priority psychological needs among adolescent girls affected by loss in sub-Saharan Africa. Conclusions and Recommendations Little evidence exists to inform the design of community-based interventions targeted to the large and growing population of children in countries with generalized HIV epidemics who have experienced the loss of someone close to them. This study offers new information about the potential for the Abangane program and others like it to mitigate the harmful psychological consequences of loss among adolescents in places where loss is common. The program was associated with measurable decreases in adolescent-reported depression and maladaptive grief, as well as caregiver-reported behavioral problems among adolescents. In this section, a summary of key research findings is presented followed by a set of recommendations. 32 Summary of findings Significant changes were observed in the outcomes presented in this report. Most importantly, statistically significant program effects have been found in all primary outcomes (i.e. adolescent grief intensity and mental health) and several secondary outcomes. Grief Intensity • The evaluation results indicate that the program helped adolescents to address some of their intrusive grief thoughts and complicated grief symptoms but did not reduce the actual prevalence of complicated grief. In addition, while the program successfully reduced intrusive grief thoughts among participants, no change in adolescents’ resilience or their ability to function as a result of grief were found in this evaluation. This lack of change in functioning could explain why the prevalence of complicated grief did not decline. Alternatively, the prevalence of complicated grief may not have changed as a result of that fact that caregivers were still functioning poorly. Thus, while the program shows some positive results for grief, given the complex and clinical nature of complicated grief, it likely requires more intensive intervention that what is offered by the program. Mental Health • Similarly, while the program participants demonstrated significant improvements in their depression symptoms, the overall prevalence of depression did not change. As with grief, depression is a complex clinical illness that requires more intensive intervention, and would likely benefit further from an intervention for caregivers. • The caregivers of adolescents in the treatment group also reported a significant reduction in adolescents’ behavior problems, specifically with regard to their internalizing (anxiety) and attention problems. This may be attributed to the reduction in young participants’ experiences of intrusive thoughts and depressive symptoms. Social Support • The program positively impacted the sense of social support adolescent participants experienced as they reported significantly improved levels of social and grief support available to them. However, no change was noted for their provision of support to others. The support groups likely provided an opportunity for adolescents to talk about their loss, which they may not be able to do otherwise, and to interact and develop bonds with peers with similar experiences. Caregiver/Adolescent Relationship • Caregivers of adolescents in the treatment group reported significantly higher levels of supportive parenting behavior. Further, adolescents in the treatment group reported stable positive attachment levels to their caregivers while those in the waitlist experienced a significant decline in this area. 33 No significant changes were seen in the academic competence of adolescent participants, nor in terms of caregiver wellbeing. Overall, the Abangane grief support program shows a lot of promise in improving mental health and grief resolution among adolescent females. However, the program could be more powerful if it incorporated components that promote daily functioning, including resilience, emotional self-efficacy, and even academic performance. Possibly, this could be achieved by including life skills that adolescents can use to cope and regulate their emotions. The program could be also improved by adding a component that focuses on encouraging school attendance and performance. Recommendations Based on the findings of Abangane program evaluation, a number of recommendations have been compiled for further research and program scale-up: Recommendation 1: To scale-up CBT-informed, culturally sensitive and logistically feasible interventions to enhance psychosocial wellbeing among children and young people in sub-Saharan Africa. The need for bereavement support among children in generalized HIV epidemic settings is high. Short-term, structured, theory-based support groups like Abangane show promise in mitigating psychological and behavioral problems among bereaved female adolescents. While the program effects reported in this study are modest overall, this evaluation offers important evidence of a culturally tailored and logistically feasible intervention addressing priority psychological needs among adolescent girls affected by loss in sub-Saharan Africa. Abangane is replicable using freely available curriculum materials and appropriately trained personnel. Recommendation 2: To address the need for family-focused bereavement programs to mitigate grief and depression among children and caregivers. This trial was initially designed to include a parallel support program for the primary caregivers of adolescent participants, aimed at helping to resolve caregivers’ grief so they could better support the children in their care. The caregiver component was eventually removed from the trial due to very low early session attendance among caregivers. Thus, although social support outcomes were found to be improved (probably as a result of the intervention’s group dynamic), no significant improvements in supportive parenting were noted. Possibly, caregiver participation could have resulted in more pronounced improvements in adolescents’ psychological health outcomes. Many caregivers in South Africa experience high levels of bereavement-related grief themselves, which was found to be strongly associated with children’s risk for depression in previous studies. Engaging caregivers might be particularly crucial in South Africa, where previous research suggests that strong cultural norms prevent adults from discussing death with children. Recommendation 3: To address the need towards developing and evaluating sex-specific programming in the SSA. While separating program participants is common practice for youth interventions in sub-Saharan Africa, there is a need for greater attention towards developing, implementing, and evaluating programming directed at male adolescents or mixed-sex groups. Recommendation 4: To develop strategies to enhance program participation, particularly among caregivers. Strategies are required to support attendance and retention in programs, particularly those 34 that enroll caregivers as co-participants. Examples may include reimbursement for transport costs, hiring a child-minder to watch younger siblings during sessions, and offering refreshments or other low￾cost participation incentives. Program implementers should further help caregivers to understand the goals of the program, and make them aware of the benefits of participation for both themselves and children / adolescents in their care. Recommendation 5: Maintain regular communication with local stakeholders about evaluation processes and results. Regular dialogue mechanisms with implementing organizations and other local key stakeholders are critical to addressing impediments to program success. Although it is important for researchers to operate independently from implementers, it is essential to have open communication about roles, the purpose of the evaluation and its protocols and potential benefits. In the evaluation team’s experience, when implementers and other stakeholders have a well- developed understanding of how learning about the program supports quality improvement, they are often eager to support the research. Continued engagement of local stakeholders maintained throughout the research process contribute to ensuring local relevance and the broader utilization of findings. Recommendation 6: Future research should aim to provide additional evidence for the intervention models that best support integrated health and wellbeing among vulnerable youth and the factors that affect intervention effects. Little evidence exists to inform the design of community-based interventions targeted to the large and growing population of children in countries with generalized HIV epidemics who have experienced the loss of someone close to them. A greater understanding about the effectiveness of various approaches to psychological health among bereaved children in sub-Saharan Africa is needed. Programs serving orphans and vulnerable children should prioritize psychological support alongside other interventions. Future studies should focus on producing evidence for the intervention models that best support integrated health and wellbeing in this population and the factors—such as prolonged illness, adolescent’s relationship to the deceased, baseline outcome values, and time since the loss—that affect treatment effects. Dissemination This study was designed to produce knowledge to inform existing practices and guide future investment in programming for bereaved adolescents in South Africa. Findings have been made available to partners (CWBFN & CLFS and Khululeka Grief Support) as soon as they were available. Dissemination occurred through workshops and presentations, including at the National Action Committee for Children Affected by HIV/AIDS (NACCA) in which the Principal Investigator is a member. To encourage the scale￾up of proven interventions and improvement of existing programming for orphans in Sub-Saharan Africa, findings were submitted to peer-reviewed journals for publication. Such publications enhance the credibility of research findings and expand their influence. The following list details different ways through which the impact evaluation results and other findings from Coping with Loss study have been disseminated among local and international stakeholders: International and national conference presentations: 35 o Presentation at the Children and HIV: Equity Now meeting that proceeded the AIDS Conference: Coping with loss: A randomized controlled trial of a structured peer support group for bereaved female adolescents in South Africa (Dr. Thurman, Durban, July 2016) o Presentation at 1st South African National Conference on Violence: Caregivers’ physical abuse of orphans and vulnerable children in the Free State: Correlates & programmatic recommendations (Ms. Alex Spyrelis, Johannesburg, August 2016) o Presentation at the REPSSI Psychosocial Forum: Grief Among Parentally Bereaved Adolescent Girls in South Africa: Adversity and Resilience (Zimbabwe, September 2015) o Presentation at the Conference of the International Society for Child Indicators: Every time that month comes, I remember: Using Cognitive Interviewing to Adapt Grief Measures for Use with Bereaved Adolescents in South Africa (Cape Town, September 2015) Presentations to study partners: o Presentation to the Khululeka Board Meeting: Coping with Loss: A randomized controlled trial of bereavement support groups for adolescent girls in the Free State, South Africa (Dr. Thurman, Cape Town, July 2016). Evaluation results presented to Khululeka director, staff and board members. o Presentation to Child Welfare Bloemfontein: Coping with Loss: A randomized controlled trial of bereavement support groups for adolescent girls in the Free State, South Africa (Ms. Spyrelis, Bloemfontein, July 2016). Evaluation results presented to CWBFN & CLFS director, program manager, facilitators and other staff during their staff meeting. o Interactive workshop with CWBFN & CLFS: Coping with Loss: A Family Bereavement Initiative for Adolescent Girls in the Free State, South Africa (Bloemfontein, January 2015) o Three presentations delivered as part of a workshop pertaining to the Coping with Loss study held with program staff from Child Welfare Bloemfontein and Childline Free State that addressed the psychological health of OVC, related programming principles, and lessons learned from the qualitative research for program improvement (Bloemfontein, September 2014) Presentations at the PEPFAR OVCY technical meeting: o Presentation on Caregiver Burden and Risk of Child Abuse (Ms. Spyrelis, September 2016, Pretoria) Peer-reviewed publications: o Thurman, T. R., Taylor, T. M., Luckett, B., Spyrelis, A. & Nice, J. (2018). Complicated grief and caregiving correlates among bereaved adolescent girls in South Africa. Journal of Adolescence, 62, 82-86. 36 o Thurman, T. R., Luckett, B. G., Nice, J., Spyrelis, A., & Taylor, T. M. (2017). Effect of a bereavement support group on female adolescents' psychological health: a randomised controlled trial in South Africa. The Lancet Global Health, 5(6), e604-e614. o Thurman, T. R., Luckett, B. G., Taylor, T. M., Nice, J. K., Carnay, M. R., & Spyrelis, A. (2017). Best practices for evaluating care and support programs for HIV-affected families: Lessons learned from South Africa. In M. Henrickson, D. Chipanta, V. Lynch, H. Muñoz Sanchez, V. Nadkarni, & T. Semigina, & Sewpaul, V. (Eds.), Getting to zero: Global social work responds to HIV (pp. 15-38). Geneva: UNAIDS and IASSW. o Taylor, T. M., Thurman, T. R., & Nogela, L. (2016). “Every time that month comes, I remember”: using cognitive interviews to adapt grief measures for use with bereaved adolescents in South Africa. Journal of Child & Adolescent Mental Health, 28(2), 163-174. Research brief geared to programmers and policy makers: o Structured support groups improve the psychological health of bereaved female adolescents in South Africa: Results from a randomized controlled trial in South Africa. Summary of Oral Conference Presentation at Children and HIV: Equity Now, 2016. o Structured support groups improve the psychological health of bereaved female adolescents in South Africa: Results from a randomized controlled trial in South Africa. https://hvc￾tulane.org/downloads/Policy_Brief_Grief_Groups_RCT_May_2017_FINAL_DoE.pdf o Complicated grief among bereaved adolescent girls in South Africa: Correlates and programmatic implications. https://hvc￾tulane.org/downloads/Policy_Brief_Complicated_Grief_Jan2018_final.pdf Conflict of interest The researchers declare that they have no financial interests associated with the program being evaluated and have no conflicting interests. References Achenbach, T. M., McConaugh, S. H., Ivanova, M. Y., & Rescorla, L. A. (2011). Manual for the ASEBA Brief Problem Monitor (BPM). Burlington, VT: University of Vermont, Research Center for Children, Youth, and Families. Armsden, G. C., & Greenberg, M. T. (1987). The inventory of parent and peer attachment: Individual differences and their relationship to psychological well-being in adolescence. Journal of Youth and Adolescence, 16(5), 427-454. doi:10.1007/bf02202939 ASU Program for Prevention Research. (1999). Family bereavement program manual: Program for Prevention Research. Tempe, AZ: Arizona State University. 37 Aynsley-Green, A., Penny, A., & Richardson, S. (2012). Bereavement in childhood: risks, consequences and responses. BMJ Support Palliat Care, 2(1), 2-4. doi:10.1136/bmjspcare-2011-000029 Brits, H., Branders, L., Claassen, M., Saaiman, D., Strauss, R., van Staden, A., & Joubert, G. (2014). The prevalence of multiple losses experienced by children from birth to 18 years in the National District Hospital in Bloemfontein. South African Family Practice, 56(5), 286-290. doi:10.1080/20786190.2014.975478 Burnett, P., Middleton, W., Raphael, B., & Martinek, N. (1997). Measuring core bereavement phenomena. Psychological Medicine, 27(1), 49-57. Collingwood, C., Hough, A., & Jacobs, V. (2014). Khululeka Grief Support: Report on an evaluation of the work of Khululeka. Inkanyezi Social Evaluation PracticeGrief Support. Cape Town, South Africa: Khululeka Department of Health. (2012). The National Antenatal Sentinel HIV and Syphilis Prevalence Survey in South Africa, 2011. Pretoria: Department of Health, South Africa. Faulstich, M. E., Carey, M. P., Ruggiero, L., Enyart, P., & Gresham, F. (1986). Assessment of depression in childhood and adolescence: an evaluation of the Center for Epidemiological Studies Depression Scale for Children (CES-DC). American Journal of Psychiatry, 143(8), 1024-1027. doi:10.1176/ajp.143.8.1024 Haine, R. A., Ayers, T. S., Sandler, I. N., & Wolchik, S. A. (2008). Evidence-Based Practices for Parentally Bereaved Children and Their Families. Professional Psychology, Research and Practice, 39(2), 113-121. doi:10.1037/0735-7028.39.2.113 Hogan, N. S., Greenfield, D. B., & Schmidt, L. A. (2001). Development and validation of the Hogan Grief Reaction Checklist. Death Studies, 25(1), 1-32. doi:10.1080/07481180125831 Human Sciences Research Council. (2013). South African National HIV Behavior & Health Survey: Preliminary Results. Pretoria: HSRC. Jordan, J. R., Baker, J., Matteis, M., Rosenthal, S., & Ware, E. S. (2005). The grief evaluation measure (GEM): an initial validation study. Death Studies, 29(4), 301-332. doi:10.1080/07481180590923706 Kumakech, E., Cantor-Graae, E., Maling, S., & Bajunirwe, F. (2009). Peer-group support intervention improves the psychosocial well-being of AIDS orphans: cluster randomized trial. Social Science & Medicine 68(6), 1038-1043. doi:10.1016/j.socscimed.2008.10.033 Lovejoy, M. C., Weis, R., O'Hare, E., & Rubin, E. C. (1999). Development and initial validation of the Parent Behavior Inventory. Psychological Assessment, 11(4), 534-545. doi:http://dx.doi.org/10.1037/1040-3590.11.4.534 Meintjes, H., Hall, K., & Sambu, W. (2015). Demography of South Africa’s children. In A. De Lannoy, S. Swartz, L. Lake, & C. Smith (Eds.), South African Child Gauge (pp. 102–106). Cape Town: Children’s Institute, University of Cape Town. Meintjies, H., & Hall, K. (2011). Demography of South Africa’s children. In L. Jamieson, R. Bray, L. Lake, S. Pendlebury, & C. Smith (Eds.), South African Child Gauge 2010/2011 (pp. 79-83). Cape Town: Children’s Institute, University of Cape Town. Melhem, N. M., Porta, G., Walker Payne, M., & Brent, D. A. (2013). Identifying prolonged grief reactions in children: dimensional and diagnostic approaches. Journal of the American Academy of Child & Adolescent Psychiatry, 52(6), 599-607.e597. doi:10.1016/j.jaac.2013.02.015 38 Muris, P. (2001). A Brief Questionnaire for Measuring Self-Efficacy in Youths. Journal of Psychopathology and Behavioral Assessment, 23(3), 145-149. doi:10.1023/A:1010961119608 Nyberg, B. J., Yates, D. D., Lovich, R., Coulibaly-Traore, D., Sherr, L., Thurman, T. R., . . . Howard, B. (2012). Saving lives for a lifetime: supporting orphans and vulnerable children impacted by HIV/AIDS. Journal of Acquired Immune Deficiency Syndromes, 60 Suppl 3, S127-135. doi:10.1097/QAI.0b013e31825da836 Pretorius, T. B. (1991). Cross-cultural application of the Center for Epidemiological Studies Depression Scale: a study of black South African students. Psychol Rep, 69(3 Pt 2), 1179-1185. doi:10.2466/pr0.1991.69.3f.1179 Shakespeare-Finch, J., & Obst, P. L. (2011). The development of the 2-Way Social Support Scale: a measure of giving and receiving emotional and instrumental support. Journal of Personality Assessment, 93(5), 483-490. doi:10.1080/00223891.2011.594124 Sherr, L., & Mueller, J. (2009). Where is the evidence base? Mental health issues surrounding bereavement and HIV in children. Journal of Public Mental Health, 7(4), 31-39. doi:doi:10.1108/17465729200800027 Smit, J., Myer, L., Middelkoop, K., Seedat, S., Wood, R., Bekker, L. G., & Stein, D. J. (2006). Mental health and sexual risk behaviours in a South African township: a community-based cross-sectional study. Public Health, 120(6), 534-542. doi:10.1016/j.puhe.2006.01.009 Statistics South Africa. (2006). Adult mortality (age 15-64) based on death notification data in South Africa: 1997-2004. Pretoria: Statistics South Africa. UNICEF. (2013). Statistics – South Africa. Pretoria: UNICEF. Van der Heijden, I., & Swartz, S. (2010). Bereavement, silence and culture within a peer-led HIV/AIDS￾prevention strategy for vulnerable children in South Africa. African Journal of AIDS Research, 9(1), 41-50. doi:10.2989/16085906.2010.484563 Weissman, M. M., Orvaschel, H., & Padian, N. (1980). Children's symptom and social functioning self￾report scales. Comparison of mothers' and children's reports. Journal of Nervous Mental Disorders, 168(12), 736-740. 39