External Outcomes Evaluation Report Children in Distress Network (CINDI) May’khethele OVC Programme Prepared by Impact Consulting Evaluation Team Michelle Stewart Tracey Konstant Anje Coetzer Lindy Dlamini Gethwana Mahlase Jessica Williams Jerushah Rangasami 17 July 2012 Children in Distress Network (CINDI) May’khethele OVC Programme External Outcomes Evaluation Report Commissioned by Pact South Africa under Associate Award No. 674-A-00-08-00001 Evaluation Team Director Jerushah Rangasami Associates Michelle Stewart Tracey Konstant Anje Coetzer Lindy Dlamini Gethwana Mahlase Jessica Williams Impact Consulting PO box 1113 Green Point 8051 South Africa Cell: +27827762286 Phone/Fax: +27214244166 Email Address: info@impactconsulting.co.za Disclaimer: This publication was made possible through support provided by the Office of HIV/AIDS, Bureau for Health, U.S. Agency for International Development, under the terms of USAID South Africa Associate Award No. 674-A-00-08-00001-00. The opinions expressed herein are those of the author(s) and do not necessarily reflect the views of Pact or the U.S. Agency for International Development. ACKNOWLEDGEMENTS Impact Consulting would like to take the opportunity to sincerely thank the following individuals: All participating intervention and control schools, in particular learners who completed the survey and participated in the focus group discussions, and the principals and teachers who participated in interviews Programme managers and other key staff at CINDI, CCP, YFC and Lifeline All of the key informants who participated in the evaluation Addis Berhanu from PACT South Africa. Without the willingness and assistance from these individuals, this report would not have been possible. TABLE OF CONTENTS 1 THE MAY’KHETHELE PROGRAMME................................................................. 2 1.1 The Children in Distress Network (CINDI)...................................................................................... 2 1.2 The May‟khethele Orphans and Vulnerable Children Programme................................................. 2 2 BACKGROUND.................................................................................................... 5 2.1 The impact of HIV on OVC ............................................................................................................. 5 3 EVALUATION PURPOSE AND GUIDING QUESTIONS ..................................... 7 4 METHODOLOGY.................................................................................................. 8 4.1 Evaluation design ........................................................................................................................... 8 4.2 Sampling strategy........................................................................................................................... 8 4.3 Sample demographics.................................................................................................................... 9 4.4 Limitations in the methdology ....................................................................................................... 10 4.5 Analysis ........................................................................................................................................ 10 5 FINDINGS, ANALYSIS AND EMERGING RECOMMENDATIONS ....................11 5.1 Evaluation question 1: Change in OVC wellbeing........................................................................ 11 5.2 Evaluation question 2: The HIV education programme................................................................ 14 5.3 Evaluation question 3: Community and household services........................................................ 29 5.4 Additional Evaluation question: Progamme management and institutional arrangements .......... 33 6 CONCLUSION.....................................................................................................37 7 SUMMARY OF RECOMMENDATIONS AND GOOD PRACTICE ......................38 LIST OF TABLES Table 1. Service entry points and referral web among the three programme partners operating in secondary schools............................................................................................................ 4 Table 2. Quantitative survey sample .............................................................................................. 9 Table 3. Qualitative sample............................................................................................................ 9 LIST OF FIGURES Figure 1. Comparison of OVC and non-OVC psychosocial support items.................................... 13 Figure 2. Percentage of learners who correctly answered each HIV knowledge item for intervention and control groups ...................................................................................... 19 Figure 3. Comparison of intervention and control groups on emotional wellbeing items.............. 20 Figure 4. Time since last HIV test.................................................................................................. 22 Figure 5. Time since last HIV test broken down by partner organisation...................................... 23 Figure 6. Attitudes to HIV prevention A ........................................................................................ 24 Figure 7. Attitudes to HIV prevention B ........................................................................................ 24 Figure 8. Scores on stigma related items for intervention and control groups .............................. 25 Figure 9. Responses to stigma items on supporting HIV positive peers and disclosing HIV positive status for intervention and control groups ...................................................................... 26 Figure 10. Child has a school uniform to wear to school................................................................. 32 ACRONYMS AIDS Acquired Immune Deficiency Syndrome ART Antiretroviral Treatment ARV Anti Retroviral CINDI Children in Distress Network CCP Community Care Project DHA Department of Home Affairs DOH Department of Health DSD Department of Social Development HCT HIV Counselling and Testing (an opt-out approach) HIV Human Immunodeficiency Virus ID Identity Document LL Lifeline LO Life Orientation NAP National Action Plan NGO Non-Governmental Organisation PEPFAR President‟s Emergency Plan For Aids Relief OVC Orphans and Vulnerable Children STI Sexually Transmitted Infections VCT Voluntary Counselling and Testing YFC Youth For Christ Executive Summary The May‟khethele program was launched in October 2007 in the uMgungundlovu district of KwaZulu-Natal and aims to improve the lives of orphans and vulnerable children (OVC) through the provision of a comprehensive range of services. The programme operates at two levels. It provides school-based HIV education and HIV Counselling and Testing (HCT); and personalised, household level attention for OVCs who are identified and enrolled in the programme. The programme uses a consortium approach. Under the CINDI umbrella, it is implemented by four CINDI members: the Community Care Project (CCP), Lifeline, Youth for Christ (YFC) and, up until 2010, Sinani. The evaluation is a retrospective outcomes evaluation of the May‟khethele programme. It was guided by the following evaluation questions: 1. To what extent has the wellbeing of OVCs changed during their participation in the May‟khethele programme? 2. To what extent did the school-based HIV prevention education programme improve attitudes and knowledge about HIV and AIDS, reduce stigma and influence change in sexual behaviours and HIV infection risk reduction among targeted adolescents? (Note that this includes all of the children who received the HIV prevention education programme). 3. How effective were the services provided to OVC households in terms of improving care to OVCs? 4. What management and institutional arrangements best support programme effectiveness? (This is an additional question emanating from the evaluation). A mixed methods evaluation design was applied. This included primary document review, programme records analysis, a quantitative learner survey and qualitative focus groups and interviews. Qualitative results were analysed against the evaluation questions, using the PEPFAR programme areas as a thematic framework. Quantitative results were analysed using Chi-square and Fisher tests for significant differences. Findings show that the wellbeing of OVCs enrolled in the programme has improved. Severely under-parented learners have been motivated to focus on their education, overcome substance abuse problems, and engage more positively in society. OVC have improved resilience and coping mechanisms, better access to HIV care and treatment, and have significantly better food security than their non-OVC counterparts in the same schools. The programme also assisted in improving child protection through removing OVC from unsafe living situations and finding homes for OVC living alone. p 1 The school-based HIV prevention education programme is widely considered to have been highly successful, and an essential on-going element in school services. The availability of lay counselling by an approachable, knowledgeable, independent adult is a key success factor. There was increased knowledge among learners at intervention schools on transmission and prevention of HIV, improvement in learners‟ confidence and self-esteem and following their May‟khethele experience, more learners are practicing safer sex or abstaining from sex. In terms of stigma, results were mixed with some schools experiencing a decrease, but stigma remaining a challenge in others. For HIV testing, uptake varied across the three May‟khethele partners – which possibly indicates the need to strengthen the referral approach to testing used in the programme. The comprehensive, integrated, child-centred, holistic, household level nature of OVC support has been effective and is recommended as good practice. The model demonstrates the value of the real needs of OVCs being addressed by a single caring adult and the value of support to physical needs being strongly complemented by support to emotional needs. The consortium model employed by the May‟khethele programme has been a highly successful approach. It enables both reach and depth in servicing OVCs; flexibility and adaptability; and capitalising on the strengths of each partner organisation providing for a range of skills and professional capacity. A key success of the model has been the commitment to strong project management and the establishment of crucial operating systems and processes. In addition to raising funds to roll out the model, the May‟khethele programme intends to share this model with the sector as an example of best practice. Overall the evaluation found that the programme is effectively designed to provide holistic education, care and support at school, home and community levels. This has provided several principles of good practice, having demonstrated how school-wide education, life￾skills facilitation and counselling can be effectively integrated with needs-based household support. Recommendations include sharing the examples of good practice used by the programme as well as sharing the programme model with the OVC sector as part of May‟khethele‟s sustainability plan. p 2 1 THE MAY’KHETHELE PROGRAMME 1.1 THE CHILDREN IN DISTRESS NETWORK (CINDI) The Children in Distress Network (CINDI) is a network of people and organisations that support children affected and infected by HIV and AIDS in the province of KwaZulu-Natal. The network consists of over 300 civil society and government organisations, including non￾governmental organisations, community-based organisations, faith-based organisations and local and regional government departments1 . 1.2 THE MAY’KHETHELE ORPHANS AND VULNERABLE CHILDREN PROGRAMME The May‟khethele programme was launched in October 2007 in the uMgungundlovu district of KwaZulu-Natal, and has been fully functional since January 2008. Services were provided to 5204 OVC in October 2007-September 2008; 11,722 in October 2008-September 2009; and 12,193 in October 2009-September 20102 . May‟khethele expanded from 16 schools in 20 wards at the outset, to 57 schools in 29 wards by the end of 2011. Aims and objectives The May‟khethele programme aims to assist and support orphans and vulnerable children (OVC) through schools-based interventions3 . The main objectives of the programme are: To increase life skills and improve the wellbeing of OVC under the age of 18 in 57 schools in uMgungundlovu district, through HIV/AIDS prevention education, promoting behaviour change, improving access to counselling and testing, and provision or linkage to other OVC services; and Identification of learners with particular vulnerability and providing them with responsive household level services, support and mentorship tailored to their needs, and addressing as far as possible the causes of their vulnerability. To increase knowledge and understanding of OVC care and support through provision of informal training to primary caregivers4 . Services 1 www.cindi.org.za 2 PACT, 2011 3 PACT, 2011 4 PACT, 2011 p 3 The programme aims to improve the lives of vulnerable children through the provision of a comprehensive range of services. The model includes: HIV prevention education for all learners in participating schools in grades 8-10, with an emphasis on holistic emotional well-being and personal development, integrated with sexuality and HIV education Psychological care through lay-counselling and referral for professional psychological support Voluntary counselling and testing for HIV Health care support and general healthcare referrals Educational support through provision of school uniforms and stationery packs for enrolled OVC Directly assisting OVC to access enabling documents and facilitating the provision of documents for all learners at participating schools Enabling and ensuring child protection, including removal from abusive situations Household Economic Strengthening, for OVC living in poverty, including facilitating access to social grants and establishment of household/community gardens. The programme operates at two main levels: School-based HIV education and HIV Counselling and Testing (HCT), and general counselling for all learners, including those enrolled as OVC in the programme. In the process, teachers develop skills and are exposed to rights-based learner support and HIV education, and schools develop systems which attempt to address vulnerability in their communities as a whole. OVCs are identified and enrolled in the programme to receive personalised, household level attention through visits by a May‟khethele carer. Their needs and those of their caregivers are assessed, and support or referral and follow-up services ensure that the child has greater access to rights. Implementing partners The May‟khethele Orphans and Vulnerable Children‟s Programme is implemented by four CINDI members: the Community Care Project (CCP), Lifeline, Youth for Christ (YFC) and, up until 2010, Sinani. Participating schools are divided among the programme partners as lead agencies, each calling on the expertise and cooperation of the other members of the May‟khethele team where needed (Table 1). p 4 Partner Primary services Collaborative services YFC Delivers HIV education at schools Household based, personalised services to enrolled OVC at YFC schools YFC refers OVC to Lifeline for facility based HCT CCP Delivers HIV education at schools Household based, personalised psychosocial and livelihood support services to enrolled OVC at CCP Provides HCT in Schools, homes and communities (since 2010) CCP refers learners to Lifeline for Rape Counselling HCT and professional counselling Lifeline Provides HCT at schools, and includes sexuality and HIV education in individual counselling sessions Provides professional psychological counselling where needed Refers OVC to CCP and YFC for follow ups Table 1. Service entry points and referral web among the three programme partners operating in secondary schools p 5 2 BACKGROUND The HIV and AIDS epidemic in South Africa has vastly increased the number of orphans and other vulnerable children (OVC) in need of care and protection. The United States President‟s Emergency Plan for AIDS Relief (PEPFAR), the funder of the May‟khethele programme, defines OVC as follows: “A child, 0-17 years old, who is either orphaned or made more vulnerable because of HIV/AIDS. Orphan: has lost one or both parents to HIV/AIDs. Vulnerable child: is more vulnerable because of any or all of the following factors that result from HIV/AIDS - is HIV-positive; lives without adequate adult support (eg in a household with chronically ill parents, a household that has experienced a recent death from chronic illness, a household headed by a grandparent, and/or a household headed by a child); lives outside of family care (eg in residential care or on the streets); or is marginalised, stigmatised or discriminated against5 ”. South Africa has one of the highest HIV prevalence rates, and the world‟s largest epidemic, with 5.7 million people living with HIV in the country in 20086 . Due to under-delivery of treatment to the necessary scale, around 400,000 AIDS deaths per year were recorded at a peak in HIV mortality rates7 . Approximately 1.4 million children in South Africa had been orphaned by HIV/AIDS. Just over 1 out of every 4 children in the province of KwaZulu-Natal has lost one or both biological parents8 , with the uMgungundlovu district affected by one of the highest HIV prevalence rates in the country. In addition, many children are HIV-positive themselves, through mother-to-child transmission of the virus. 2.1 THE IMPACT OF HIV ON OVC Compared to other children, OVC are more likely to: have poor health and nutrition become HIV infected suffer sexual abuse, including child prostitution and trafficking lack emotional support to deal with grief and trauma experience long-term psychological problems 5 PEPFAR, 2006 6 Yezingane Network, 2010 7 Tremendous Hearts, 2010 8 Yezingane Network, 2010 p 6 lack love, care and attention take drugs and other substances do badly in school and/or drop out of school have poor educational and vocational opportunities begin working early become involved in crime experience stigma and discrimination experience exploitation and abuse lose their rights to land and property9 . In an attempt to address the OVC crisis, the Department of Social Development (DSD) issued a National Action Plan (NAP) for OVCs in 2006. The current 2009 - 2012 NAP aims, inter alia, to strengthen family capacity; mobilise community-based responses for protection of OVC; ensure that legal and policy frameworks are enabling; and provide essential services for OVC10 . Aligned to the NAP‟s minimum package of services for OVCs, PEPFAR has identified a set of core programme areas11: Clinical Nutritional Support Child Protection Interventions General Healthcare Referrals Healthcare Support for Access to Anti-Retroviral Treatment (ART) HIV Prevention Education Psychological Care Educational Support Household Economic Strengthening. The May‟khethele programme is closely aligned to the NAP objectives, and corresponds to the PEPFAR thematic programme areas to varying degrees. 9 International HIV/AIDS Alliance, 2003 10 Khulisa Management Services, 2008 11 PEPFAR, 2012 p 7 3 EVALUATION PURPOSE AND GUIDING QUESTIONS This report is structured according to the following evaluation questions: 1. To what extent has the wellbeing of OVCs changed during their participation in the May‟khethele programme? 2. To what extent did the school-based HIV prevention education programme improve attitudes and knowledge about HIV and AIDS, reduce stigma and influence change in sexual behaviours and HIV infection risk reduction among targeted adolescents? (Note that this includes all of the children who received the HIV prevention education programme). 3. How effective were the services provided to OVC households in terms of improving care to OVCs? 4. What management and institutional arrangements best support programme effectiveness? (This is an additional question emanating from the evaluation). The evaluation is a retrospective outcomes evaluation of the May‟khethele programme using both quantitative and qualitative research methods. It is important to note that although the programme operated in primary and high schools this focus shifted towards high schools in the fourth year. As a result, primary schools were not included in this evaluation. In addition, one of the four partner organisations, Sinani, has left the progamme and their work was not included in the evaluation. p 8 4 METHODOLOGY 4.1 EVALUATION DESIGN A mixed methods evaluation design was applied. This included primary document review, programme records analysis, a quantitative survey and qualitative focus groups and interviews. Respondents included OVC and other learners, teachers, programme facilitators and staff, NGOs and government stakeholders working in closely related fields. Qualitative data sources included May'khethele staff interviews, OVC focus groups at schools, a Lifeline HIV support group for OVC who are HIV-positive, OVC caregiver interviews and focus groups, and key informant interviews. Quantitative data were collected through a learner survey conducted in participating May‟khethele schools (intervention schools) and control schools. The comparison with control schools was provided in an attempt to detect the relative outcomes of the programme at intervention schools. 4.2 SAMPLING STRATEGY The sampling strategy for the learner survey used randomised, multi-stage sampling with probability proportionate to the size of beneficiary groups per school. Ten intervention schools were selected randomly from the 35 participating high schools, with an even distribution of schools among the three partner organisations (Table 2). A total of 849 learners were randomly selected from 10 intervention schools and 4 control schools. School Number of learners Intervention versus control Count Percentage Classification Percentage Gobindlovu 81 9.5% Intervention YFC 64% Edendale HS 77 9.1% Umthoqotho HS 66 7.8% Sukuma Comprehensive 69 8.1% Intervention CCP Zamazulu HS 68 8.0% Bongudunga 62 7.3% Georgetown 50 5.9% Skhululiwe SS 28 3.3% Intervention Life Line Imvunulo SS 24 2.8% Ikusaselihle HS 20 2.4% p 9 Willowfountain 92 10.8% Control 36% Bheximba HS 91 10.7% Mcomjwana HS 81 9.5% ML Sultan 40 4.7% Total 849 100% Table 2. Quantitative survey sample Enrolled OVC (63% of the sample) and learners not enrolled in the OVC programme (37% of the sample) were selected in proportion with their numbers at the schools. 4.3 SAMPLE DEMOGRAPHICS Although equal numbers of boys and girls were included from each stratified sample, more females (57%) than males (43%) were available to participate in the survey, for both control and intervention schools. Learners from the control schools were older than those from the intervention schools, with 73% being aged 16 to 18, where only 48% of intervention school participants were in this age range. This is likely to have affected the results, particularly those related to sexual maturity. Qualitative sample A total of 181 participants made up the qualitative sample (Table 3): Qualitative data source Organisation / School Number of participants No. of females No. of males Staff interviews Lifeline, CCP, YFC, CINDI 7 3 4 OVC focus groups 9 intervention schools An average of 10 per group 46 43 HIV support group Lifeline 12 10 2 3 x caregiver focus groups 42 41 1 5 x caregiver home visits 5 5 0 School key informant interviews 5 intervention schools 17 4 13 Key informant interviews Dept of Health Dept of Home Affairs 5 NGOs 9 5 4 Totals 181 114 67 Table 3. Qualitative sample p 10 4.4 LIMITATIONS IN THE METHDOLOGY 4.4.1 Use of control schools Although a few particularly dominant trends emerged, the comparison with control schools has been less effective than hoped. The quantitative results may be under-reporting positive outcomes of the programme. Where significant differences are detected between control and intervention schools, however, these would suggest a substantial level of outcomes, sufficient to overcome the sampling bias. Three critical factors have contributed to the limited value of the intervention: control design: All schools in the area have had exposure to HIV education and OVC interventions. The age distribution of the learners in the control schools is not equivalent to the intervention schools. One of the control schools, ML Sultan, differed substantially from the intervention group. It is located in town, better resourced and has a high matric pass rate. 4.5 ANALYSIS Qualitative results were analysed against the evaluation questions, using the PEPFAR programme areas as a thematic framework. Wherever appropriate the voices of respondents are provided in the text as an authentic representation of the perspectives that were communicated. Trends, examples and particular cases were identified, and the perspectives of informants used to explain, enrich and sometimes to contradict with quantitative findings. Quantitative results were analysed using Chi-square and Fisher tests for significant differences. The survey responses from learners were analysed for comparison according to: Analysis for differences between intervention schools and control schools. o In order to attempt to dilute bias, the outlying control school, ML Sultan, was excluded from the analysis of sexuality and HIV education. Within intervention schools data, a further analysis was conducted on differences between respondents who were enrolled as OVC in the May‟khethele programme and those who were not. p 11 5 FINDINGS, ANALYSIS AND EMERGING RECOMMENDATIONS 5.1 EVALUATION QUESTION 1: CHANGE IN OVC WELLBEING To what extent has the wellbeing of OVCs changed during their participation in the May‟khethele programme? Major conclusions The wellbeing of OVCs enrolled in the programme has improved. Severely under-parented learners have been motivated to focus on their education, overcome substance abuse problems, and engage more positively in society. Situations have been resolved where OVC are reported or found to live in abusive or unsafe home situations. OVC in need of medical treatment, and particularly learners living with HIV, have been referred and accompanied to ensure that treatment is provided. Several OVC have recovered from severe illness under the care of the programme. Nutritional needs have been met through referral and facilitation of welfare grants or food parcels, and through establishment of food gardens. Although OVC were enthusiastic about the improvements in their confidence, self-esteem and emotional well-being, they continue to require regular personal support. Having been abandoned in some way in their past, programme continuity and trustworthiness is particularly critical if interference is not to have additional negative impacts in their lives. 5.1.1 Improved resilience and coping mechanisms Many OVCs stressed that the counselling services are an important element of the programme, enabling them to cope better with problems and become more resilient. Counselling greatly assists OVC in coping with traumas such as the death of a parent/s or being HIV-positive themselves. “May‟khethele has changed my life because I used to smoke dagga and drugs. Since I got involved with May‟khethele and learnt the dangers of drugs, I have stopped. I‟m more focused on my school work and go to church. I do my chores and homework. I have also distanced myself from bad friends and associate myself with friends that motivate me.” 12 “My life has changed a 100% with May‟khethele. I used to do drugs because I was always stressed. May‟khethele has shown that there are people I can talk to when I‟m feeling down.” 13 12 OVC focus group 2 13 OVC focus group 2 p 12 Improved emotional well-being was especially visible among teenagers who participate in the programme‟s support group for learners living with HIV. The group had been facilitated through acceptance of their positive HIV status. They explained how the support group had restored their hope and has helped them cope with issues of anger, fear and isolation. “Before May‟khethele I used to blame my mother for my status. I hated her, because amongst all my siblings, I‟m the only one who is HIV positive. After joining the support group I made peace with myself. I forgave my mother.” 14 “When I found out that I was HIV positive, I got confused. I did not understand anything. I felt stupid and dirty. I was a bad girl. They made me become my old self and I realise that this is not my fault.” 15 “...I was angry because I was still a virgin. I even thought of sleeping around with everybody because I had saved myself and yet I got infected. They [May‟Khethele] saved my life.” 16 Many OVCs experience stigma and discrimination and they often feel isolated17. The programme helps OVCs feel more included in society. “When they first visited my home, my family was happy that there were people who cared enough about me to want to see where and how I lived.” 18 The programme strategy of going into OVCs‟ homes shows these learners that their school life isn‟t separate from the community. They feel included, better connected and more valued19 . “They [OVCs] feel they can be normal.” 20 5.1.2 Child protection and psychological support Interviews and focus groups indicate that May‟khethele is, to a certain extent, effective at identifying and removing OVCs from unsafe living situations. A children‟s welfare organisation noted that, “Their support makes it easier to do our work - even though we may be working on a case, because they are in the community if they noticed a problem they will visit the child‟s home and check up for us”21 . People trust the programme enough to feel comfortable to report concerns they have about children22 . 14 HIV positive support group focus group 15 HIV positive support group focus group 16 HIV positive support group focus group 17 PEPFAR, 2006 18 OVC focus group 2 19 Principal interview 4 20 Teacher interview 2 21 NGO interview 3 22 NGO interview 3 p 13 The programme has also helped learners who do not have parents or who are living alone to find homes: “I know one set of twins. They were staying at the orphanage. May‟khethele organized foster parents for them.” 23 “There were four siblings going around asking for food, May‟khethele took those learners to a foster home. They organised birth certificates for them and maybe IDs.” 24 Enrolled OVC, understandably, have more psychosocial challenges than their non-OVC peers. Significantly25 fewer OVC felt “able to do things as well as most other people” and “as happy as other learners my age”, than non-OVC learners at intervention schools (Figure 1). Figure 1. Comparison of OVC and non-OVC psychosocial support items 5.1.3 Improved access to HIV care and treatment By enabling access to HIV testing the programme has identified adults and learners who are eligible for ART, who may otherwise have been missed. It has assisted them with accessing care and treatment, enabling the recovery of some of whom had reached Stage 4 AIDS. “We have given learners another opportunity to live. Some learners‟ CD4 count was 10. They didn‟t know they had HIV and are 13 or 14 years old.” 26 “If it wasn‟t for May‟khethele my grandchild wouldn‟t be alive today.” 27 23 OVC focus group 4 24 OVC focus group 4 25 Chi-square = 8.665 p<0.05 and Chi-square = 6.688 p<0.05 26 Staff interview 1 27 Caregiver home visit 1 0% 10% 20% 30% 40% 50% 60% 70% 80% OVC Not OVC OVC Not OVC OVC Not OVC I have people I can talk to when I have a problem I am able to do things as well as most other people I am as happy as other children my age All the time Some of the time None of the time Significant difference p 14 May‟khethele facilitates immediate follow-up of positive HIV tests in consultation with the child and with his or her consent. A partner organisation conducts a CD4 count, enabling rapid access to treatment if necessary. May‟khethele ensures continuity from testing, through treatment and care, to psychosocial support. 5.1.4 Differences in food security among enrolled OVC Learners enrolled as OVC in the May‟khethele programme have significantly better food security than non-OVC counterparts in the same schools: 31% always having 2 meals a day, compared to 20% of non-OVC learners. The majority of caregivers felt that the nutritional support provided by the programme was extremely valuable. “The most important thing to us is providing food and clothes for our children. May‟khethele really comes through for us.” 28 5.2 EVALUATION QUESTION 2: THE HIV EDUCATION PROGRAMME To what extent did the school-based HIV prevention education programme improve attitudes and knowledge about HIV and AIDS, reduce stigma and influence change in sexual behaviours and HIV infection risk reduction among targeted adolescents? Note that this includes all of the children who received the HIV prevention education programme. Major conclusions The school-based HIV prevention education programme is widely considered to have been highly successful, and an essential on-going element in school services. Programme beneficiaries recommended that May‟khethele increase the amount of time it spends at schools and recommended that all grades in high schools be included in the programme, rather than only Grades 8 to 10. The facilitation model provides an example of good practice, particularly with regard to building self￾esteem, confidence, social responsibility and emotional maturity. A combination of personal upliftment with technical information has been particularly successful. Facilitation could be further enhanced with clear norms and standards for school facilitators. A range of participatory teaching practices are used in the model, including dialogues, edutainment, and drama. Respondents noted the value of these practices and suggested including additional techniques such as learner debates on various salient topics and inviting more HIV positive champions to give talks to the learners. 28 OVC caregiver focus group 2 p 15 The availability of lay counselling by an approachable, knowledgeable, independent adult is a key success factor, offering support, access to problem solving, and respect, thereby enhancing self-value and self-esteem. While professional psychologists are clearly essential in cases of severe trauma or abuse, readily available facilitators fulfill a valuable role for a greater number of learners. Uptake of HIV testing varied considerably between the three May‟khethele partners. HIV testing rates were highest at schools where the relevant partner organisation in the school provided testing directly. Testing rates were lower at schools where the partner organisation referred learners to another partner organisation or facility for testing. It is recommended that lessons learnt from the direct approach to testing are used to strengthen the referral approach. Although condom distribution in schools is against DoE school policy, given the importance of condoms in HIV prevention, availability of condoms at convenient and youth-friendly collection sites is essential. Realistically, many learners are likely to be sexually active before they leave school at the age of 18. Access to condoms is facilitated by the May‟khethele programme, although various social pressures around abstinence messaging, privacy and HIV may make it difficult for learners to request or obtain condoms openly. Carefully thought out strategies are needed which do not casualise sex among young learners, but which give young adults access to HIV prevention. Although stigma has diluted substantially across schools in the sample, it is recommended that the programme continues to place emphasis on this topic. Deeper issues of stigma and fear of disclosure, and the need for peer support and supportive oversight for learners who become infected with HIV, all require continuous diligence around stigma and discrimination. OVC have higher risk attitudes for certain HIV questions than non-OVC. This suggests that, while May‟khethele has helped OVCs to develop safer and healthier attitudes to sexuality, these learners continue to be vulnerable and are the group that requires information and attention. Individual conversations on sexuality and HIV seemed to have been missed by the community outreach element of the programme. Home visits and frank, personal, relaxed conversation with a trusted, knowledgeable adult, remain a valuable resource for OVC. Continuing to discuss HIV and sexuality at home, being available to answer questions and, if appropriate, supplying condoms, is a valuable form of support. School-based programmes are necessarily long-term, as generations of learners continue to need the same level and nature of educational and emotional support. Schools have gained enhanced capacity, but teachers feel that May‟khethele fills a role that they themselves lack the time, skills or relationships with learners to fulfill. p 16 5.2.1 School partnership The presence of the programme in schools has enabled teachers to gain skills; HIV knowledge and understanding; access to Voluntary Counselling and Testing (VCT) for themselves and better relationships and communication with learners. Learners benefit from more supportive teachers and greater access to helpful adults. Schools now facilitate more parent participation and have increased outreach to their communities. May‟khethele schools are gaining reputations for being schools of choice in their areas: “More Grade 8s came to the school because of May‟khethele. They heard what is going on here from their siblings.” 29 The majority of the schools stressed the importance of a good relationship between the school and the programme: “we are working hand in hand with the programme.” 30 Principals and teachers agreed that it had taken some time to build a good relationship between the school and the programme but this had been achieved at most schools. “May‟khethele has become part of the school‟s family.” 31 Some respondents suggested that academic achievement has improved at their schools because of the programme. Others have noticed that learners are more focused in class and are learning better. A number of teachers attributed this improvement to the fact that May‟khethele has mitigated many of the social, interpersonal and confidence issues which negatively affect learners‟ academic ability. “We had a 91% pass rate in 2011. This has never happened before. The learners are more serious about their school work because May‟khethele has helped with their home situations.” 32 5.2.2 School-based facilitation The May‟khethele facilitators and counsellors at schools are young and engage with learners in a light and accessible manner. Learners relate to facilitators more openly than to their teachers, and do not regard them as authority figures. This enables a trusting relationship to develop and learners become comfortable discussing sensitive issues, disclosing abuse and requesting counselling. “They do not discriminate against the positives and they are able to keep your secrets.” 33 “They do not shout at us, they talk to us like they are of our age.” 34 29 Teacher group interview 4 30 Principal interviews 1, 2, 4 31 Principal interview 2 32 Principal interview 5 33 OVC focus group 1 34 OVC focus group 1 p 17 “The way they teach LO is different from how the teachers do it. They clarify things.” 35 “They have great sense of humour you do not get bored with them.” 36 Teachers felt that it is easier for the facilitator to discuss sensitive topics such as sexuality and condom use. Facilitators also have more specialised knowledge and skills around social issues than many teachers, and are better equipped to share accurate information appropriately and effectively with learners37 . Although the majority of school facilitators are effective and much appreciated, there have been problems with a few facilitators. For example, at one school a facilitator was replaced because the school felt that she was unable to relate to the learners and was straining the relationship between the school and the programme. May‟khethele responded well and resolved the issue. 5.2.3 HIV prevention education in schools Knowledge CCP offers a school-based HIV prevention education programme for Grades 8 and 9, which takes place during Life Orientation class (LO) slots within the school timetable. YFC provides a similar programme for Grades 8-10. Lifeline offers HCT information sessions at schools. Focus groups and interviews all reported substantially increased knowledge among learners on transmission and prevention of HIV. “I‟m teaching Life Orientation and you can see they [learners] have an understanding from May‟khethele. They already know before you teach them.” 38 The survey confirms that learners at the intervention schools have significantly39 more HIV knowledge than learners at the control schools (Figure 2). The higher scores from intervention schools on items such as “a person with HIV can look healthy”, and that “sharing eating utensils does not carry a risk of HIV” illustrate significant positive shifts in learners‟ attitudes towards stigma, stereotypes and assumptions. Learners now subscribe to fewer HIV myths, and teachers commented that learners can readily identify and explain typical HIV misconceptions. Learners are more aware of the consequences of unsafe sex and they understand the benefits of abstaining and using 35 OVC focus group 4 36 OVC focus group 1 37 Teacher interview 38 Teacher group interview 6 39 Independent t-test with t = 2.140 and p<0. 05 p 18 condoms. Learners now have a detailed understanding of the different means of HIV transmission. “The information we get from them [May‟Khetele] helps us to know the importance of using a condom so as to prevent the chances of getting HIV.” 40 “We have learnt how to protect ourselves from STIs and HIV.” 41 Of concern is that between 36% of the intervention group and 46% of the control group are under the impression that mosquitoes can transmit HIV. Although the intervention group are significantly better informed on this item, it is something that should be addressed, since it would drastically aggravate both anxiety and stigma. An understanding of HIV as a chronic, treatable and manageable condition has been gained, and with it, important potential impacts on stigma and denial. “If you are positive, you cannot become negative again. But if you listen to doctors, you can live for a long time”42 “They teach you how you must behave if you are HIV positive.”43 40 OVC focus group 1 41 OVC focus group 3 42 OVC focus group 4 43 OVC focus group 2 p 19 Figure 2. Percentage of learners who correctly answered each HIV knowledge item for intervention and control groups Increased confidence and self-esteem One of the major outcomes of the HIV education programme was an improvement in learners‟ confidence and self-esteem. Principals and teachers agreed that many learners are more outspoken and are able to ask questions more boldly. They are less withdrawn and have confidence to speak in front of the class and express themselves more freely. One teacher noted that “there is boosted morale” at the school. OVC focus groups also recognised an improvement in self-esteem. One boy who had lost confidence when his father died shared that May‟khethele had helped him to believe in himself again: “When he died, I could not believe in myself and there was nobody to praise me when 0% 10%20%30%40%50%60%70%80%90%100% A person can’t get HIV from mosquito … Antiretroviral drugs (ARVs) cure HIV (False) Kissing transmits HIV/AIDS (False) Sex with a married person is safe (False) Only people with multiple partners … If a man is circumcised he will definitely … If a couple have been faithful for a long … A person can’t get HIV by drinking from … HIV causes AIDS (True) HIV/AIDS can be transmitted from a … Taking a shower or a bath immediately … A person with HIV can look healthy (True) You can get HIV by sitting on the same … You can only get HIV/AIDS through sex … Even if both partners have HIV they … Having sex with a virgin can cure HIV (False) Sangomas know how to cure HIV/AIDS … Intervention Control Significant difference p 20 I have done well. May‟khethele told me that I will still be a man one day even if my dad is not around.”44 “I once had low self-esteem, when I pass a group of people and I hear them laughing I would think that they were laughing at me. Now I do not care what the next person says or think about me.” 45 Focus group respondents felt that a strength of the programme is that it teaches self￾awareness and that they are better able to recognise their emotions: “It is important to know who you are and what you want in life.” 46 Significantly47 more learners in the intervention schools described themselves as “…as happy as other learners my age” than in control schools (Figure 3), reflecting an increase in self-esteem, and echoing the qualitative findings. Figure 3. Comparison of intervention and control groups on emotional wellbeing items Enhancing positive social behaviours Another programme outcome was an increase in learners‟ positive social behaviour. Focus groups commented that the programme had taught them concepts such as sharing, forgiveness and empathy and that they are applying these in their lives. Teachers at some schools added that since the programme learners have exhibited improved discipline and there is less fighting at school. 44 OVC focus group 1 45 OVC focus group 1 46 OVC focus group 1 47 Chi-square = 7.700 p<0.05 0% 10% 20% 30% 40% 50% 60% 70% Intervention Control Intervention Control Intervention Control I have people I can talk to when I have a problem I am able to do things as well as most other people I am as happy as other children my age All of the time Some of the time None of the time Significant difference p 21 “I have learnt to share with those that have less than me.” 48 “They taught me the importance of forgiving. Before May‟khethele, I used to keep grudges. There were many people that I did not talk to. I talk to all of them now...” 49 “They taught me to respect other people and to say things that will make the other person happy.” 50 “I was short-tempered. Always fighting with people and get suspended from school. I now can be patient with other people.” 51 Schools and learners have gained greater awareness of the needs of others and have initiated social support projects for those in need. “We see what is happening in the community and now we are more involved.” 52 Learners themselves have become more open to sharing and helping others. “Because of what I learnt from May‟khethele I was able to help a friend. His family was struggling and didn‟t have any decent clothes. I told my family and since then we help them with whatever we can clothes, food, etc.” 53 “I had a friend who also boards at the school. In January she couldn‟t come to school because she didn‟t have uniform. I spoke to my mother who gave me permission to give her my other uniform because I have two sets. If I hadn‟t learnt about sharing at May‟khethele I don‟t think I would have even cared about my friend.” 54 5.2.4 HIV testing as a prevention intervention The May‟khethele programme provides school-based or highly accessible non-medical sites for HIV testing for learners, as well as CCP offering home testing where appropriate. Lifeline also offers an immediate CD4 cell count should a learner test positive. Qualifying learners are referred to the nearest clinic for ART treatment, and learners are followed up both in terms of their health and their psychological wellbeing. Learners who test HIV+ are included as vulnerable, and enrolled for intensive support under the OVC programme. 48 OVC focus group 1 49 OVC focus group 4 50 OVC focus group 5 51 OVC focus group 5 52 Principal interview 4 53 OVC focus group 4 54 OVC focus group 3 p 22 May‟khethele staff and OVC focus groups considered an increase in testing to be a major outcome of the programme at most of the intervention schools. “If they had not spoken at length about the importance of HIV testing, I will not know my status. Before they came, I used to hear other people talking about testing. Even on radio they talk about it but I was just not interested.” 55 Respondents regarded the many learners who have taken an HIV test as a direct result of the programme. “They advised and encouraged us to go for HIV tests and I went to get tested in 2010.” 56 “My family and I now get tested every 3 months.” 57 The majority of learners in both the control and intervention groups reported that they had been tested within the past year (Figure 4), without significant difference between the control group and the May‟khethele schools. Figure 4. Time since last HIV test Testing rates varied considerably between the three May‟khethele partners (Figure 5). Testing rates were highest at Lifeline schools, where testing is a core service provided by the organisation, with 96% of learners having had an HIV test. Testing rates were lower at CCP schools (64% of learners had been tested) where learners were referred to Lifeline for testing, until 2010 when CCP began to provide HCT in schools and in OVC homes. Testing was lowest at YFC schools (58% of learners had been tested) where learners were refered to Lifeline, despite provision of transport, and Lifeline and YFC collaborating on testing campaigns designed specifically for learners at these schools. 55 OVC focus group 1 56 OVC focus group 3 57 OVC focus group 2 6% 16% 44% 27% 7% 3% 12% 52% 27% 6% 0% 10% 20% 30% 40% 50% 60% Between 2-4 years Between 1-2 years Within the past year Never Don’t know Intervention group ( n = 542) Control group ( n = 302) p 23 Figure 5. Time since last HIV test, broken down by partner organisation 5.2.5 Prevention and attitudes to risk behaviour Following their May‟khethele experience, more learners are practicing safer sex or abstaining from sex. Principals and teachers at 3 of the 5 interviewed intervention schools reported a decrease in the pregnancy rate, especially for Grades 8 to 10. “Before May‟khethele I was always giving in to peer pressure, doing drugs and going around with boys. I realise that if I‟d continued the way I was going, I‟d have ended up pregnant or even worse – HIV positive.” 58 “Because of the May‟khethele teachings, I stopped being a player, fooling around with girls because I learnt that this behaviour will lead me to getting HIV.” 59 “AIDS is dangerous and there is no cure for it, you must know ways to protect yourself from it.” 60 The great majority of learners give correct and appropriate answers to survey question on HIV prevention. Equal percentages (78%) of learners at the intervention and control schools believe that abstinence is the best method to prevent HIV (Figure 6 and 7). Significantly61 more learners at the intervention schools stated their intention to abstain up until marriage than at the control schools. Whilst OVC focus groups recognised the importance of abstinence, there was debate about whether this is realistic. Most participants felt that abstaining until the age of 18 years was more realistic than until marriage. “I prefer a condom even though it is not 100%. It is unlikely that one will abstain, you can abstain if you are younger, for example 13 years, but once you are 18 years it will be difficult to abstain.” 62 58 OVC focus group 2 59 OVC focus group 2 60 OVC focus group 1 61 Chi-square = 6.291 p<0.05; comparison of 3 control schools 62 OVC focus group 1 0% 20% 40% 60% 80% Sukuma Comprehensive Zamazulu HS Bongudunga Georgetown Skhululiwe SS Imvunulo SS Ikusaselihle HS Gobindlovu Edendale HS Umthoqotho HS CCP Lifeline YFC More than 1 year ago Within the past year Never Don’t know p 24 Figure 6. Attitudes to HIV prevention A 63 The vast majority of learners believe that it is not acceptable for males or females to have multiple concurrent partners (be a player). OVC focus groups reported that since the programme they have greatly reduced the risk in their own sexual behaviour. They are using condoms and are no longer „sleeping around‟. 78% and 75% of learners from the intervention and control groups respectively indicated that they would use a condom every time they have sex. Peer leaders at the schools are less convinced that other learners are now using condoms, but observed that “some are trying to be faithful to their partners.” 64 Figure 7. Attitudes to HIV prevention B 65 63 Note that OVC in all graphs refers to learners enrolled in the May‟khethele household level support programme, and non-OVC refers to all other learners at intervention schools 64 Peer leader group interview 65 Note that OVC in all graphs refers to learners enrolled in the May‟khethele household level support programme, and non-OVC refers to all other learners at intervention schools 78% 15% 6% 3% 78% 18% 4% 3% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Abstinence (not having sex) is the best method of preventing HIV/AIDS There‟s no point in abstaining or practicing safe sex because in the end we will all die anyway It is fine for men or boys to be a „player‟ It is fine for women or girls to be a „player. Percentage of learners who agreed with the statment Intervention Control 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% OVC Not OVC Intervention Control Intervention Control I will abstain (not have sex) up until marriage I will abstain (not have sex) up until marriage I will use condoms every time when I have sex Definitely Maybe No Significan p 25 5.2.6 Stigma associated with HIV and AIDS Focus groups and interviews suggested a noticeable decrease in stigma at some schools, although it remains a challenge in others. While some teachers felt that there is still a long way to go to combat HIV stigma, teachers at other schools noticed that learners talk about HIV more and find it easier to open up. “I know now that I should not discriminate against HIV positive people because they are still the same people as they were before they found out that they are HIV positive.” 66 “There is not as much denialism. They acknowledge HIV exists.” 67 “The learners we get from the programme are open-minded. When we talk to them about HIV they do not see it as a rude subject. They have the knowledge.” 68 Predictably, stigma attitudes depend on the level of social closeness (Figure 8). 89% of learners from intervention schools, significantly69 more that those in control schools, stated that they would support their HIV-positive peers. This would seem to reflect a clear outcome of May‟khethele‟s focus on self-esteem and interpersonal relationships. “I learnt not discriminate but to take care of my loved ones if they are HIV positive.”70 Figure 8. Scores on stigma related items for intervention and control groups 66 OVC focus group 4 67 Principal interview 4 68 NGO interview 69 Chi-square = 8.003 p<0.05 70 OVC focus group 3 62% 63% 75% 83% 92% 61% 68% 79% 81% 92% I will never fall in love with an HIV positive person (% of learners who disagreed with … Only promiscuous (sleeping around) people get infected with HIV / AIDS (% of learners who disagreed with … If a person has HIV they must keep it a secret (% of learners who disagreed with statement) I would share things (toilet seats, utensils, bath, etc.) with an HIV positive person (% of learners who agreed with statement) A teacher who is HIV positive but is not sick should be allowed to continue teaching in school … Intervention Control Significant difference p 26 Figure 9. Responses to stigma items on supporting HIV positive peers and disclosing HIV positive status for intervention and control groups Impersonal relationships and sharing of objects are held with comfort by over 80% of respondents. The thought of communicating and disclosure are accepted by 79% of the intervention group. A personal relationship with a person living with HIV, however, is considered too close by 39% of learners in the intervention group, demonstrating an undercurrent of fear and difference associated with HIV. The link between „social proximity‟ and stigma was echoed in the qualitative data. Learners are comfortable to talk about HIV. They know not to discriminate against people who are HIV positive and have become more accepting of people with HIV71. However, when it comes to openly disclosing their own status, teachers and caregivers noted that the majority of learners are not willing to do so for fear of discrimination (Figure 9). 5.2.7 Improved communication about HIV Previously a taboo subject in families, learners feel more informed and confident to speak about HIV in their families. “I‟m now able to talk to my parents about HIV and STIs.” 72 “My mother used to say I‟m too young to be talking about sex. Because of May‟khethele, I‟m now comfortable to go talk to her about everything and go to her for advice.” 73 71 Teacher group interview 6 72 OVC focus group 2 73 OVC focus group 2 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Intervention Control Intervention Control I will disclose my HIV/AIDS status if I am HIV positive I will support fellows/peers who are infected and affected by HIV/AIDS Definitely Maybe No Significan p 27 “Parents are allowing learners to go for testing, because the learners understand about HIV and transfer this knowledge to parents. Because of the programme there are no misconceptions and they know exactly what HIV is about and the consequences.” 74 “Talking about sex and HIV was always taboo but my children say „Gogo, we learn about these things at the L.O. class and the May‟khethele programme.” 75 “In the past, we couldn‟t talk about HIV. Now I‟m comfortable to talk to my children about HIV. This removes the stigma of HIV. It is now like any other illness, like diabetes.” 76 5.2.8 Identifying and resolving child abuse A major outcome observed was an increase in disclosure of domestic violence and sexual abuse. This is attributed to learners being more aware of their rights and also having the opportunity, often for the first time in their lives, to disclose to a trusted adult. “Learners know that if a person touches you in an improper way they must report it. They are aware of their rights because before they weren‟t aware that this was wrong.” 77 Learners are not only reporting abuse to the May‟khethele facilitator but have also started to disclose to teachers. This may reflect a deepening in the relationship between teachers and learners, and greater trust of adults in positions of support. Teachers also spoke about how they themselves have learned to be more aware of signs of abuse and appropriate action to help learners who are affected by abuse. “I even went to court as a witness for child abuse, I was her teacher and she disclosed to me.” 78 “There was domestic violence and this learner became reserved and wouldn‟t participate. She was able to speak out and through the programme we relocated her... The father came to the school to threaten us but the school stood up for her. We are willing to help the learners but May‟khethele gives us the advice and skills.” 79 5.2.9 HIV attitudes and HIV education outcomes among enrolled OVC A positive attitude change has been seen across learners in the intervention schools, and particularly among enrolled OVC. Peer leaders at the schools explained that many OVC are depressed and even suicidal, but that the programme, “gives confidence so that you know what 74 Teacher interview 1 75 OVC caregiver focus group 1 76 OVC caregiver focus group 1 77 Principal interview 4 78 Teacher group interview 4 79 Teacher interview 5 p 28 you want in life and so that you don‟t lose hope.” 80 Other NGO‟s which work with the programme also observed less hopelessness amongst OVC and attributed this to the psychosocial support provided by the programme. “The programme provides counselling and it helps the learners to value themselves and to accept their situation – that they [learners who are HIV positive] can live after HIV. It helps them think of the future and that they can live a positive life.” 81 “They have hope because they have someone to talk to; they see someone is interested in their life and willing to help.” 82 The programme has enabled OVCs to speak out about their problems and to ask for help. OVC focus groups describe how before the programme they had been unable to talk about their problems through fear, a lack of opportunity, or not seeing the value in talking to someone. “I couldn‟t talk about the things that were troubling me, but now I can easily talk to the May‟khethele people.” 83 “They make it easy for you to talk about things that are worrying you. I like them because they are loving and they help us forget our problems.” 84 “Now I am no longer afraid to talk to them about anything.” 85 While enrolled OVC and non-OVC answered similarly for the majority of questions around HIV and sexuality in the survey, there were a few notable exceptions. Enrolled OVC have less cautious, and perhaps more worldly-wise, expectations of abstinence (Figure 7). Significantly more OVCs (37%), than non-OVCs (27%), in the intervention group believe that only promiscuous people become infected with HIV. Although low, significantly more enrolled OVCs (6%) than non-OVCs (2%) believe it is acceptable for males to be promiscuous. 80 Peer leader interview 81 NGO interview 5 82 Teacher interview 5 83 OVC focus group 5 84 OVC focus group 1 85 OVC focus group 5 p 29 5.3 EVALUATION QUESTION 3: COMMUNITY AND HOUSEHOLD SERVICES How effective were the services provided to OVC households in terms of improving care to OVCs? Major conclusions The comprehensive, integrated, child-centred, holistic, household level nature of OVC support has been effective and is recommended as good practice. The model demonstrates the value of the real needs of OVCs being addressed by a single caring adult, whether these relate to food, housing, protection, and psychosocial support and information, and when support to physical needs are strongly complemented by support to emotional needs. This is an essential design element for programmes hoping to enhance the rights and wellbeing of children and youth. Given the lack of hope and general family depression among people living in poverty, even greater availability of support groups, home visits, and peer support would be appropriate. Innovative approaches to sustainable livelihood support have been a valued part of the programme, particularly in the form of food gardens. Training and projects in local economic development are beyond the scope of the CINDI network‟s role, but active partnerships with organisations that focus on economic self-reliance and sustainable financial security would benefit OVC. Continuity and follow-up of OVC needs and services are a key success factor. Encouraging community buy-in and participation is an ongoing area of effort as OVCs continue to be excluded and isolated. May‟khethele has focused on building community spirit, and should continue to work to achieve a sense of community responsibility for OVCs. Involvement of community leaders and partnerships with churches and religious leaders as powerful sources of influence is recommended by respondents. 5.3.1 Psychological care Research shows that there is a dire need for psychosocial support for OVCs86 . Young people who are orphaned or made vulnerable by illness, poverty, homelessness or abuse invariably struggle with the trauma of these experiences. In addition, they are likely to encounter stigmatisation and rejection, aggravating their emotional distress. They are less likely to have sufficient adult supervision, support or containment. They may respond with inappropriate and unsafe behaviour, less maturity of judgement and low self-esteem, and are therefore particularly vulnerable to HIV infection. The May‟khethele programme provides psychosocial support to OVCs in the school setting, with additional support in the home environment, as well as referral for professional counselling where needed. 86 PEPFAR, 2006 p 30 “The programme is doing a good job because it doesn‟t concentrate only on HIV. It also looks at all of the child‟s needs and then helps them.” 87 Caregivers also receive support that helps them to feel cared for and less isolated. Some caregivers explained that the programme helped to restore their dignity and others felt more confident to care for their children. By empowering caregivers and providing support the programme contributes to helping them better care for OVCs. 5.3.2 Enabling documents Birth certificates, death certificates and identity documents are essential for access to virtually any service, and to normal engagement in society: “Without these documents you can‟t have a normal life.” 88 One of the most compelling motivations for families to obtain documents is that access to social grants requires children‟s birth certificates and, if relevant, parents‟ death certificates. May‟khethele has offered support, advice and information to caregivers, improving access to birth certificates for learners. 95% of learners in the intervention group have a birth certificate. This is significantly89 more than the 90% of learners in the comparable control group. May‟khethele„s partnership with the Department of Home Affairs (DHA) has helped to improve OVCs‟ access to enabling documents, helping to identify learners and schools where the need is greatest. May‟khethele provides the department with a link into communities by generating a list of schools which need DHA services, and arranging DHA visits to schools. 5.3.3 Home- based HIV testing The benefits of testing are recognised by OVCs: “Knowing your status helps you to take ARVs early; even here at school there are learners who take ARVs. I know because my uncle was taking the same pills.” 90 “If you discover that you are HIV positive you will be able to protect another person from being infected by you.” 91 Although HIV testing is offered at some of the intervention schools and at partner organisations‟ premises, often the most vulnerable families find it difficult to go out to access testing services. It is much easier to accept a caregiver into their homes. May‟khethele 87 NGO interview 88 DHA official interview 89 Fisher‟s test = 6.188 p<0.05 90 OVC focus group 1 91 OVC focus group 1 p 31 therefore includes an invitation for home-based testing as part of home visits which has been led by CCP. “We enjoy their home visit. I wish they would visit again. They came to do VCT after my mother had passed away. They explained nicely why this was necessary.” 92 “They helped with the testing of my late aunt‟s children that I‟m taking care of. Knowing their status put my mind at ease.” 93 May‟khethele‟s testing service is provided in close partnership with the Department of Health (DOH). The department provides testing supplies and the programme increases learner access to HCT. “DOH can‟t infiltrate the whole community. DOH don‟t have the resources to do what they [May‟Khethele] are doing.” 94 5.3.4 HIV management support Learners diagnosed as HIV-positive are enrolled into the May‟khethele OVC programme. Caregivers also learn how to better care for HIV-positive learners, as well as themselves if they are HIV positive. “The programme goes beyond the child – it goes to the family and teaches caregivers about HIV. They learn that it is manageable.” 95 “They taught me how to care for HIV-positive person. One is able to ask questions and be assisted.” 96 “Because of the training I‟m now able to see if a child is sick and know what to do.” 97 “I‟ve done home-based care training and I‟m now able to share my knowledge with my children.” 98 5.3.5 Educational support Lack of school fees and school uniforms are major obstacles to school attendance. The May‟khethele programme has improved OVCs‟ access to both. May‟khethele arranges direct assistance to some OVCs for school fees and negotiates with schools on their behalf if they cannot pay school fees. May‟khethele also ensures that OVC learners have uniforms, with 92 OVC focus group 3 93 Caregiver focus group 2 94 DOH officials‟ group interview 95 Staff interview 1 96 OVC caregiver home visit 1 97 OVC caregiver focus group 2 98 OVC caregiver focus group 1 p 32 significantly99 more learners in the intervention group having uniforms than in the control group (Figure 10). Although significantly fewer OVCs have a uniform than non-OVCs, the 95% who do have uniforms remain higher than the 89% of learners in control schools. Figure 10. Child has a school uniform to wear to school 5.3.6 Household economic strengthening The household economic strengthening element of the May‟khethele programme concentrates on establishing food gardens in the households of enrolled OVC as well as helping caregivers to access social grants. The programme conducts OVC nutritional assessments. Learners in need of immediate food support are referred to appropriate programmes. For longer term food security, the programme encourages and supports the establishment of food gardens in OVC households, providing both enhanced nutrition and a source of income. Although food gardens have taken a long time to take hold in people‟s minds, successes have started to be seen. “The food gardens were not working because there was no hope. They have started to work and this indicates the emotional health of the family.” 100 For the majority of caregivers participating in focus groups, the food gardens were a key programme element. “Selling the vegetables gives us income to provide other necessities for our families without depending solely on the grants.” 101 “We never have to spend money on vegetables, having the gardens helps keep the hunger at bay.” 102 “We are able to sell our vegetables and buy bread.” 103 99 Chi-square = 19.402; p<0.05 100 Staff interview 2 101 OVC caregiver focus group 1 102 OVC caregiver home visit 4 95% 99% 89% 5% 1% 11% 0% 20% 40% 60% 80% 100% OVC (n = 314) Not OVC (n = 164) Control (n = 294) Yes No p 33 5.3.7 Continuity and follow up May‟khethele‟s systems of household centred, holistic support, and careful follow-up on OVCs‟ access to services was identified as a key factor for programme success. “May‟khethele follow up on the child, they don‟t just leave them.” 104 “May‟khethele visit and check us regularly.” 105 5.3.8 Community-based presence The programme is well-positioned to access the community. It can target areas where there is the most need. By “taking services to the area”106 , May‟khethele reaches the most vulnerable. “They are a useful resource because they are community based so they know what is out there.” 107 May‟khethele has experienced a lack of community buy-in and has had to work at involving communities in the programme and bringing back a community spirit. Although there have been challenges to community participation in the programme, May‟khethele has assisted with community development in a number of ways. Through working with local churches it has encouraged church members to be more community-minded and challenged them to uplift the community108. By sourcing field staff from local communities, mutual understanding and respect between the programme and its client base is enhanced. 5.4 ADDITIONAL EVALUATION QUESTION: PROGAMME MANAGEMENT AND INSTITUTIONAL ARRANGEMENTS What are the lessons and good practices learned from the management and organisational approach taken by the May‟khethele programme? Major conclusions The consortium approach under the CINDI umbrella has offered several key advantages. The programme has been able to provide a range of services in a responsive, flexible and yet 103 OVC caregiver focus group 1 104 NGO interview 1 105 OVC caregiver home visit 3 106 NGO interview 1 107 NGO interview 3 108 NGO interview 5 p 34 comprehensive manner. A key success of the programme has been strong project management and the establishment of effective systems and processes, particularly processes for regular reflection and action learning. With facilitation by CINDI leadership, the team has been able to design approaches and adjust them with experience, providing a continuously improving model approach. While the consortium approach offers the advantages of a range of capabilities from different partners and opportunities for exchange and synergy, variation between the approaches taken by partners sometimes requires compromises and careful management. As this phase of the programme draws to a close, managers intend to share the May‟khethele joint partnership model with the OVC and education sectors as an example of good practice. Funds are also being raised for CINDI itself to continue to work within the same paradigm. It is recommended that the key success factors be carefully taken into account when working towards taking the model to scale. These include firstly, the enthusiastic investment of young, well-trained, well-mentored adults, who provide the entire school with lay counseling alongside holistic sexuality, lifeskills and HIV education. Secondly, the model‟s success depends on being able to identify OVC and provide family￾centred community care, which is regular, personalised and prepared to take action to resolve the range of constraints to OVC realising their rights. 5.4.1 Reflective, learning-oriented joint venture No single organisation or government department is able to provide services to OVCs that cover all aspects of their needs. Coordination within and between community organisations, government departments and different levels of government, is paramount in achieving comprehensive and holistic service provision. The consortium model employed by the May‟khethele programme has been highly successful. It enables both reach and depth in servicing OVCs; flexibility and adaptability; and capitalises on the strengths of each partner organisation providing for a range of skills and professional capacity109 . A key success of the model has been the commitment to strong project management and the establishment of effective operating systems and processes, some of which included: effective communication and feedback mechanisms (in part through a live database) streamlined coordination genuine critical reflection and a culture of learning which was incorporated back into the programme monitoring and evaluation systems. 109 Staff interviews p 35 Another essential element contributing to programme success has been ongoing capacity building and support at different levels. CINDI and partner organisations received valuable technical training from PACT SA, and partner organisations received further significant training from CINDI. When second level support was needed by partner organisations, such as financial management and monitoring and evaluation, this was also provided by CINDI. The joint partnership model, with its key systems and management practices, provides a good practice model for the sector. The May‟khethele programme intends to share this model with the sector. such that the valuable lessons learnt can contribute towards sustained, effective OVCs support. 5.4.2 Partnerships and networking Beyond its core group of partners, May‟khethele links with other relevant services to enable comprehensive service provision. Shared standards and professional culture are established through networking and partnership building, demonstration, benchmarking and communication. Government Many of the May‟khethele programme staff commented on the challenges in building relationships with government departments, describing it as a slow process which requires much time and effort. May‟khethele has been relatively successful at strengthening partnerships with government. The programme works predominantly with the four government Departments of Education; Health; Social Development; and Home Affairs. As a result, May‟khethele‟s beneficiaries receive services more comprehensively and expediently. Government departments gain access to hard-to-reach households and receive information on prioritising the most vulnerable. “DHA is happy to be in partnership with them. They assist us a lot and the mobile unit is there to work closely with NGOs.” 110 May‟khethele‟s involvement has greatly increased access to HCT for learners, and has included the critical role of follow-up and support needed to safely offer HCT services to school youth. “They [May‟khethele] contribute a significant number to the HCT campaign. Whenever we have events they assist us.” 111 110 DHA official interview 111 DOH officials group interview p 36 Linking with NGOs Effective referral system: May‟khethele networks, refers clients and follows up with NGOs to enable access by their beneficiaries to specific services and professional skills. Working together to utilise each other‟s strengths: May‟khethele partners with other NGOs to offer more effective services and to minimise duplication. For example, if legal intervention is needed to remove the child from an unsafe home, May‟khethele asks Pietermaritzburg Child Welfare to assist. May‟khethele reciprocates by identifying places of safety for children and helping them to access enabling documents. Capacitating other NGOs: May‟khethele has helped to capacitate other NGOs that work with learners in the community. For example, through May‟khethele a shelter NGO receives resources, training, and assistance with funding applications112. In turn, the May‟khethele programme brings learners in need to this place of safety so that they are cared for until they can be permanently placed. 112 NGO interview 2 p 37 6 CONCLUSION The programme is effectively designed to provide holistic education, care and support at school, home and community levels. This has provided several principles of good practice, having demonstrated how school-wide education, life-skills facilitation and counselling can be effectively integrated with needs-based household support. The streamlined model demonstrates the flexibility possible to provide services as varied as HCT, documentation and life-skills around sexuality and HIV prevention, while also ensuring that a consistent, rigorously designed curriculum can be developed and rolled out across a large number of participating schools. While fine programme refinements might benefit the model, the May‟khethele programme has largely been a success. Although the programme is coming to an end at the conclusion of the grant period of the PEPFAR funding, the relevance of the programme in terms of lasting solutions to the OVC crisis is assured in the programme‟s sustainability plans. The model will be shared with the OVC and education sectors as an example of best practice in order to assist with developing a scaled, sustainable programme model for the sector. The design of systems that effectively address what amounts to a 20-year emergency is a major achievement. An entire generation of traumatised, damaged and under-parented children require society as a whole to care for them. Holding this generation through to adulthood as self-reliant, effectively participating and well-adjusted members of society is a daunting challenge facing all those working in the children‟s sector, and indeed, all members of a responsible and ethical society. p 38 7 SUMMARY OF RECOMMENDATIONS AND GOOD PRACTICE 1. The comprehensive, integrated, child-centred, holistic, household level approach of the OVC element of the programme is a key success factor. Despite care, OVCs remain more vulnerable than non-OVC. The model is effective when the real needs of OVCs are addressed by a single caring adult, whether these relate to food, housing, protection and psychosocial support and information, and when physical needs are strongly complemented by support to emotional needs. 2. An especially powerful programme element, which is recommended across school-based OVC programmes, is the provision of lay counsellor–facilitators who are approachable, knowledgeable, independent young adults. This person offers support, access to problem solving and focused attention, thereby enhancing self-value and self-esteem among learners. While professional psychologists are clearly essential in cases of severe trauma or abuse, readily available facilitators fulfil a valuable role for a larger number of learners. Both professional and non-professional levels of psychological support are recommended. 3. The school-based HIV education model used, which combines personal upliftment with technical information, has been particularly successful. It builds self-esteem, confidence, social responsibility and emotional maturity, while enhancing HIV knowledge. Facilitation could be further enhanced with facilitator support, supervision, codes of conduct and on￾going mentorship and debriefing. A greater range of creative educational approaches could be included, particularly those that exercise learners‟ expanding confidence in self￾expression, participation and critical thinking. 4. Innovative approaches to sustainable household, caregiver and welfare support have been a strong part of the programme, particularly in the form of food gardens, home visits and support groups. With the lack of hope and general family depression among people living in poverty, even greater investment and strategies for confidence-building and emotional support for households would be appropriate. Training and local economic development to households are beyond the scope of CINDI‟s niche. Active networking with partners who build economic self-reliance and sustainable financial security would benefit OVC. 5. An area where the survey results might have been unexpected is that of HIV testing in CCP and YFC schools, where learners were referred to Lifeline for testing (until 2010 for CCP schools). The quantitative data do not support respondents‟ impressions that the great majority of learners have tested for HIV. A closer investigation of the obstacles to access and testing uptake, especially when using the referral approach, is advised. p 39 6. Community buy-in and participation is an on-going area of effort as OVCs continue to be excluded and isolated. May‟khethele should continue its valuable work stream towards achieving a sense of community responsibility for OVCs. Continued involvement of community and religious leaders are powerful sources of influence. Without a child￾friendly, child-embracing community, organisations cannot single-handedly create a safe and nurturing environment for all children, including those who are vulnerable. 7. The consortium approach under the CINDI umbrella has offered several key advantages. The programme has been able to provide a range of services in a responsive, flexible and yet comprehensive manner. The model does, however, require a clear awareness and management of the variance between the approaches taken by partners. 8. The May‟khethele programme intends to share this model with the sector as an example of best practice, to encourage its ultimate uptake in schools across the country. It is recommended that the key success factors be carefully taken into account when working towards taking the model to scale. These include firstly, the enthusiastic investment of young, well-trained, well-mentored adults, who provide the entire school with lay counselling alongside holistic sexuality, lifeskills and HIV education. Secondly, the model‟s success depends on being able to identify OVC and provide family-centred community care, which is regular, personalised and prepared to take action to resolve the range of constraints to OVC realizing their rights. 1 Appendices to the Evaluation Report Children in Distress Network (CINDI) May’khethele OVC Programme Prepared by Impact Consulting Evaluation Team Michelle Stewart Tracey Konstant Anje Coetzer Lindy Dlamini Gethwana Mahlase Jessica Williams Jerushah Rangasami 17 July 2012 2 Children in Distress Network (CINDI) May’khethele OVC Programme Appendices to the Evaluation Report Commissioned by Pact South Africa under Associate Award No. 674-A-00-08-00001 Evaluation Team Director Jerushah Rangasami Associates Michelle Stewart Tracey Konstant Anje Coetzer Lindy Dlamini Gethwana Mahlase Jessica Williams Impact Consulting P O Box 1113 Green Point 8051 South Africa Phone: +27 (0) 21 424 4166 Email Address: Jerushah@impactconsulting.co.za www.impactconsulting.co.za Disclaimer: This publication was made possible through support provided by the Office of HIV/AIDS, Bureau for Health, U.S. Agency for International Development, under the terms of USAID South Africa Associate Award No. 674-A-00-08-00001-00. The opinions expressed herein are those of the author(s) and do not necessarily reflect the views of Pact or the U.S. Agency for International Development. 3 1 APPENDICES APPENDIX 1- ADDITIONAL EVALUATION QUESTIONS  What have been the benefits of accessing HIV counseling and testing services and knowing one‟s status?  Has there been change in HIV/AIDS related knowledge?  Has there been a change in sexual practices among target beneficiaries?  How has the educational support received from the program improved school attendance and performance?  What, if any change did access to care and support in schools have on educational outcomes such as attendance and performance?  Has the OVC emotional wellbeing improved? Has the program in any way helped to improve self-esteem?  Has the program helped in accessing legal protection in case of need?  How well did the program address the need for acquiring legal document s; like birth registration or ID?  How well has the program facilitated access to services to children which were denied legal status? Has the program helped access to HIV related health care services including ART?  Was the training effective in improving skills and knowledge among care gives  How does the approach to and model of training compare with others in terms of delivering the intended results  Did the training enhance good family functioning (relationship between the OVC and their primary caregiver)?  Have the parents/primary care givers improved their parenting skills? Have care provider attitudes‟ improved? 4 APPENDIX 2 – QUANTITATIVE ANALYSIS RESULTS Breakdown of quantitative survey sample per school School Number of learners Intervention versus control Count Percentage Classification Percentage Gobindlovu 81 9.5% Intervention 64% Edendale HS 77 9.1% Intervention Sukuma Comprehensive 69 8.1% Intervention Zamazulu HS 68 8.0% Intervention Umthoqotho HS 66 7.8% Intervention Bongudunga 62 7.3% Intervention Georgetown 50 5.9% Intervention Skhululiwe SS 28 3.3% Intervention Imvunulo SS 24 2.8% Intervention Ikusaselihle HS 20 2.4% Intervention Willowfountain 92 10.8% Control 36% Bheximba HS 91 10.7% Control Mcomjwana HS 81 9.5% Control ML Sultan 40 4.7% Control Total 849 100% 5 Gender of learners in intervention and control groups Gender Intervention group (n=545) Control group (n=304) Female 57% 62% Male 43% 38% Percentage of learners older than 16 at Intervention and Control Schools Percentage of learners older than 16 Intervention group (n=545) 48% Control group (n=304) 73% 6 Percentage of learners who CORRECTLY ANSWERED each of 17 HIV knowledge items, by three sets of comparisons Item Comparison 1 Comparison 2 (#) Comparison 3 Intervention (10 schools) Control (4 schools) Statistically significant difference? Intervention (10 schools) Control (3 schools) Statistically significant difference? Intervention OVC Intervention Not OVC Statistically significant difference? HIV causes AIDS (True) 86% 89% No 86% 89% No 85% 88% No A person with HIV can look healthy (True) 92% 87% Yes 92% 85% Yes 91% 93% No You can only get HIV/AIDS through sex (False) 95% 89% Yes 95% 89% Yes 96% 92% No Kissing transmits HIV/AIDS (False) 74% 77% No 74% 74% No 75% 74% No If a man is circumcised he will definitely not be infected by HIV (False) 82% 82% No 82% 82% No 82% 82% No HIV/AIDS can be transmitted from a mother to her unborn child (True) 86% 84% No 86% 82% No 84% 90% No Antiretroviral drugs (ARVs) cure HIV (False) 72% 65% Yes 72% 63% Yes 71% 72% No You can get HIV by sitting on the same toilet seat used by someone who has HIV (False) 92% 95% No 92% 95% No 92% 90% No Taking a shower or a bath immediately after sex prevents HIV infection (False) 86% 87% No 86% 88% No 83% 86% No Even if both partners have HIV they should always use condoms when having sex (True) 95% 95% No 95% 95% No 94% 96% No Only people with multiple partners contract HIV/AIDS (False) 80% 72% Yes 80% 73% Yes 77% 80% No 7 Sangomas know how to cure HIV/AIDS (False) 99% 98% No 99% 98% No 99% 98% No If a couple have been faithful for a long time they don‟t have to use a condom (False) 83% 84% No 83% 83% No 82% 83% No A person can‟t get HIV from mosquito bites (True) 64% 54% Yes 64% 52% Yes 64% 63% No Sex with a married person is safe (False) 79% 84% No 79% 84% No 77% 81% No Having sex with a virgin can cure HIV (False) 96% 95% No 96% 96% No 96% 96% No A person can‟t get HIV by drinking from the same cup as someone who is infected (True) 83% 75% Yes 83% 73% Yes 85% 79% No Note: Statistical significance was determined by means of Chi-square tests (#) Comparison 2: One control school excluded 8 Percentage of learners who AGREED with each of 11 statements concerning HIV/AIDS, by three sets of comparisons Item Comparison 1 Comparison 2 (#) Comparison 3 Intervention (10 schools) Control (4 schools) Statistically significant difference? Intervention (10 schools) Control (3 schools) Statistically significant difference? Intervention OVC Intervention Not OVC Statistically significant difference? I am tired of hearing about HIV/AIDS 24% 29% No 24% 25% No 24% 26% No Everybody knows about HIV/AIDS 62% 59% No 62% 62% No 63% 62% No There‟s no point in abstaining or practicing safe sex because in the end we will all die anyway 18% 17% No 18% 15% No 19% 21% No If a person has HIV they must keep it a secret 21% 25% No 21% 25% No 24% 19% No Only promiscuous (sleeping around) people get infected with HIV / AIDS 32% 37% No 32% 39% No 37% 27% Yes Abstinence (not having sex) is the best method of preventing HIV/AIDS 78% 78% No 78% 78% No 76% 81% No I will never fall in love with an HIV positive person 39% 38% No 39% 37% No 42% 42% No It is fine for women or girls to be a „player. 3% 4% No 3% 3% No 4% 1% No It is fine for men or boys to be a „player‟ 4% 8% Yes 4% 6% No 6% 2% Yes I would share things (toilet seats, utensils, bath, etc.) with an HIV positive person 81% 83% No 81% 84% No 79% 84% No A teacher who is HIV positive but is not sick should be allowed to continue teaching in school 92% 92% No 92% 92% No 92% 92% No Note: Statistical significance was determined by means of Chi-square tests(#) Comparison 2: One control school excluded Responses with regard to five HIV/AIDS related undertakings: Comparison 1 Comparison 1 No Maybe Definitely Number of respondents Statistically significant difference? I will disclose my HIV/AIDS status if I am HIV positive Intervention (10 schools) 21% 44% 35% 542 No Chi-square = 0.074 p = 0.964 Control (4 schools) 20% 44% 36% 301 Total 21% 44% 35% 843 I will support fellows/peers who are infected and affected by HIV/AIDS Intervention (10 schools) 1% 10% 89% 545 Yes Chi-square = 8.003 p<0.05 Control (4 schools) 2% 15% 83% 304 Total 1% 12% 87% 849 Responses with regard to five HIV/AIDS related undertakings: Comparison 2 Comparison 2 (#) No Maybe Definitely Number of respondents Statistically significant difference? I will abstain (not have sex) up until marriage Intervention (10 schools) 6% 42% 52% 542 Yes Chi-square = 6.291 p<0.05 Control (3 schools) 9% 47% 44% 264 Total 7% 43% 50% 806 I will use condoms every time when I have sex Intervention (10 schools) 2% 21% 78% 545 No Chi-square = 4.608 p = 1.00 Control (3 schools) 4% 22% 75% 263 Total 2% 21% 77% 808 I am interested in testing for HIV/AIDS and knowing my status Intervention (10 schools) 2% 8% 90% 545 Yes Chi-square = 6.213 p<0.05 Control (3 schools) 4% 3% 93% 263 Total 3% 6% 91% 808 (#) Comparison 2: One control school excluded Responses with regard to five HIV/AIDS related undertakings: Comparison 3 Comparison 3 No Maybe Definitely Number of respondents Statistically significant difference? I will abstain (not have sex) up until marriage Intervention: OVC 7% 44% 49% 317 No Chi-square = 4.232 p = 0.121 Intervention: Not OVC 4% 37% 59% 162 Total 6% 42% 52% 479 I will use condoms every time when I have sex Intervention: OVC 1% 20% 79% 317 No Fisher‟s test = 0.246 p = 1.000 Intervention: Not OVC 1% 20% 79% 165 Total 1% 20% 79% 482 I will disclose my HIV/AIDS status if I am HIV positive Intervention: OVC 24% 41% 35% 316 No Chi-square = 1.592 p = 0.451 Intervention: Not OVC 19% 44% 37% 163 Total 22% 42% 36% 479 I am interested in testing for HIV/AIDS and knowing my status Intervention: OVC 2% 6% 92% 317 No Chi-square = 2.590 p = 0.274 Intervention: Not OVC 3% 10% 87% 165 Total 3% 7% 90% 482 I will support fellows/peers who are infected and affected by HIV/AIDS Intervention: OVC <1% 10% 89% 317 No Fisher‟s test = 2.923 p = 0.204 Intervention: Not OVC 2% 9% 89% 165 Total 1% 10% 89% 482 “Learners have access to HIV testing in my community”, by three sets of comparisons Comparison Yes No Don’t know Number of respondents Statistically significant difference? Comparison 1 Intervention (10 schools) 59% 15% 26% 543 No Chi-square = 0.096 p = 0.953 Control (4 schools) 59% 15% 26% 304 Total 59% 15% 26% 847 Comparison 2 (#) Intervention (10 schools) 59% 15% 26% 543 No Chi-square = 2.585 p = 0.275 Control (3 schools) 63% 17% 20% 264 Total 60% 16% 24% 807 Comparison 3 Intervention: OVC 59% 14% 27% 316 No Chi-square = 2.161 p = 0.340 Intervention: Not OVC 59% 18% 23% 164 Total 59% 15% 26% 480 (#) Comparison 2: One control school excluded Time since most recent HIV test, by three sets of comparisons Time since test Comparisons Comparison 1 Intervention group (10 schools, n=542) Control group (4 schools, n=302) Statistically significant difference? Between 2-4 years 6% 3% No Chi-square = 9.309 p = 0.054 Between 1-2 years 16% 12% Within the past year 44% 52% Never 27% 27% Don‟t know 7% 6% Total 100% 100% Comparison 2 (#) Intervention group (10 schools, n=542) Control group (3 schools, n=263) Statistically significant difference? Between 2-4 years 6% 3% Yes Chi-square = 15.970 p<0.05 Between 1-2 years 16% 11% Within the past year 44% 57% Never 27% 24% Don‟t know 7% 5% Total 100% 100% Comparison 3 Intervention: OVC (n=316) Intervention: Not OVC (n=163) Statistically significant difference? Between 2-4 years 5% 9% No Chi-square = 5.831 p = 0.212 Between 1-2 years 18% 13% Within the past year 45% 41% Never 24% 31% Don‟t know 7% 6% Total 100% 100% (#) Comparison 2: One control school excluded Time since most recent HIV test, by school School Classification Time since test Number of More than 1 learners year ago Within the past year Never Don’t know Gobindlovu Intervention 33% 52% 10% 5% 79 Edendale HS Intervention 8% 35% 48% 9% 77 Sukuma Comprehensive Intervention 27% 28% 33% 12% 69 Zamazulu HS Intervention 13% 38% 40% 9% 68 Umthoqotho HS Intervention 22% 23% 46% 9% 65 Bongudunga Intervention 16% 60% 18% 7% 62 Georgetown Intervention 24% 48% 22% 6% 50 Skhululiwe SS Intervention 25% 75% 0% 0% 28 Imvunulo SS Intervention 34% 58% 0% 8% 24 Ikusaselihle HS Intervention 35% 60% 5% 0% 20 Willowfountain Control 17% 60% 22% 1% 92 Bheximba HS Control 12% 58% 24% 6% 91 Mcomjwana HS Control 11% 53% 28% 9% 80 ML Sultan Control 21% 15% 49% 15% 39 Total 7% 27% 46% 19% 844 Responses with regard to nutrition statements: Comparison 2 Comparison 2 (#) None of the time Some of the time All of the time Number of respondents Statistically significant difference? I eat at least two meals a day Intervention (10 schools) 8% 66% 26% 544 No Chi-square = 3.885 p = 0.143 Control (3 schools) 6% 62% 32% 262 Total 7% 65% 28% 806 I have enough food to eat Intervention (10 schools) 11% 49% 40% 543 No Chi-square = 0.186 p = 0.911 Control (3 schools) 11% 50% 39% 262 Total 11% 49% 40% 805 I go to bed hungry Intervention (10 schools) 59% 35% 6% 541 Yes Chi-square = 9.014 p<0.05 Control (3 schools) 51% 45% 4% 263 Total 56% 38% 6% 804 (#) Comparison 2: One control school excluded Responses with regard to nutrition statements: Comparison 3 Comparison 3 None of the time Some of the time All of the time Number of respondents Statistically significant difference? I eat at least two meals a day Intervention: OVC 4% 65% 31% 317 Yes Chi-square = 16.119 p<0.05 Intervention: Not OVC 13% 67% 20% 164 Total 7% 66% 27% 481 I have enough food to eat Intervention: OVC 12% 48% 40% 315 No Intervention: Not OVC 8% 51% 41% 165 Chi-square = 1.767 Total 10% 50% 40% 480 p = 0.413 I go to bed hungry Intervention: OVC 7% 36% 57% 313 No Chi-square = 1.453 p = 0.484 Intervention: Not OVC 7% 31% 62% 165 Total 7% 34% 59% 478 Average school attendance, by three sets of comparisons Comparison Never Once a month Once a week 5 times a week Number of respondents Statistically significant difference? Comparison 1 Intervention (10 schools) 0.6% 1.5% 0.4% 97.6% 543 No Fisher‟s test = 0.789 p = 0.915 Control (4 schools) 0.7% 1.3% 0.7% 97.3% 301 Total 0.6% 1.4% 0.5% 97.5% 844 Comparison 2 Intervention (10 schools) 0.6% 1.5% 0.4% 98% 543 No Fisher‟s test = 0.625 p = 0.963 Control (3 schools) 0.8% 1.1% 0.4% 97.7% 261 Total 0.6% 1.4% 0.4% 97.6% 804 Comparison 3 Intervention: OVC 0.6% 1.9% 0.0% 97.5% 315 No Fisher‟s test = 3.640 p = 0.220 Intervention: Not OVC 0.6% 1.2% 1.2% 97.0% 165 Total 0.6% 1.7% 0.4% 97.3% 480 (#) Comparison 2: One control school excluded Learner has a school uniform to wear to school, by three sets of comparisons Comparison Yes No Number of respondents Statistically significant difference? Comparison 1 Intervention (10 schools) 97% 3% 541 Yes Chi-square = 19.402 p<0.05 Control (4 schools) 89% 11% 294 Total 94% 6% 835 Comparison 2 Intervention (10 schools) 97% 3% 541 Yes Chi-square = 25.236 p<0.05 Control (3 schools) 87% 13% 255 Total 93% 7% 796 Comparison 3 Intervention: OVC 95% 5% 314 Yes Chi-square = 6.320 p<0.05 Intervention: Not OVC 99% 1% 164 Total 96% 4% 478 Personal identity: Comparison 1 Comparison 1 Yes No Don’t know Number of respondents Statistically significant difference? I have a birth certificate Intervention (10 schools) 95% 5% <1% 545 No Fisher‟s test = 3.882 p = 0.137 Control (4 schools) 91% 8% 1% 303 Total 1% 6% 93% 848 I have a green identity document book (*) Intervention (10 schools) 27% 71% 2% 261 Yes Fisher‟s test = 13.057 p<0.05 Control (3 schools) 42% 57% 1% 221 Total 34% 64% 2% 482 (*) Only calculated for learners who are 16 years and older; only three control schools are used as all of the learners in one of the control schools are younger than 16 years Personal identity: Comparison 2 Comparison 2 Yes No Don’t know Number of respondents Statistically significant difference? I have a birth certificate Intervention (10 schools) 95% 5% <1% 545 Yes Fisher‟s test = 6.188 p<0.05 Control (3 schools) 90% 9% 1% 263 Total 93% 6% 1% 808 I have a green identity document book (*) Intervention (10 schools) 27% 71% 2% 261 Yes Fisher‟s test = 13.057 p<0.05 Control (3 schools) 42% 57% 1% 221 Total 34% 64% 2% 482 (#) Comparison 2: One control school excluded (*) Only calculated for learners who are 16 years and older Personal identity: Comparison 3 Comparison 3 Yes No Don’t know Number of respondents Statistically significant difference? I have a birth certificate Intervention : OVC 95% 4% 1% 317 No Fisher‟s test = 1.792 p = 0.371 Intervention : Not OVC 93% 7% <1% 165 Total <1% 5% 94% 482 I have a green identity document book (*) Intervention : OVC 25% 72% 2% 166 No Fisher‟s test = 0.716 p = 0.730 Intervention : Not OVC 30% 68% 3% 71 Total 27% 71% 2% 237 (*) Only calculated for learners who are 16 years and older Learners older than 16 years who have a green ID book, by school School Classification Age of learners % of learners older than 16 who have a green ID book Number of learners who provided their age Number of learners older than 16 % of learners older than 16 Gobindlovu Intervention 80 42 53% 19% Edendale HS Intervention 77 18 23% 28% Sukuma Comprehensive Intervention 69 28 41% 7% Zamazulu HS Intervention 68 18 26% 14% Umthoqotho HS Intervention 66 39 59% 31% Bongudunga Intervention 62 35 56% 29% Georgetown Intervention 50 32 64% 19% Skhululiwe SS Intervention 28 18 64% 39% Imvunulo SS Intervention 24 18 75% 61% Ikusaselihle HS Intervention 20 14 70% 43% Willowfountain Control 92 90 98% 40% Bheximba HS Control 91 66 73% 39% Mcomjwana HS Control 81 66 81% 48% ML Sultan Control 40 0 0% -- Having a house where to sleep at night, by three sets of comparisons Comparison None of the time Some of the time All of the time Number of respondents Statistically significant difference? Comparison 1 Intervention (10 schools) 1% 1% 98% 541 No Chi-square = 1.717 p = 0.424 Control (4 schools) 2% 2% 96% 302 Total 1% 2% 97% 843 Comparison 2 Intervention (10 schools) 1% 1% 98% 541 No Fisher‟s test = 2.959 p = 0.220 Control (3 schools) 2% 3% 95% 263 Total 1% 2% 97% 804 Comparison 3 Intervention: OVC 1% 1% 98% 316 No Fisher‟s test = 2.025 p = 0.444 Intervention: Not OVC 1% 3% 96% 162 Total 1% 1% 98% 478 (#) Comparison 2: One control school excluded Responses with regard to statements about contentment/happiness of learner: Comparison 1 Comparison 1 None of the time Some of the time All of the time Number of respondents Statistically significant difference? I have people I can talk to when I have a problem Intervention (10 schools) 6% 35% 59% 544 No Chi-square = 4.14 p = 0.813 Control (4 schools) 7% 34% 59% 304 Total 6% 35% 59% 848 I am able to do things as well as most other people Intervention (10 schools) 2% 41% 58% 545 No Chi-square = 0.038 p = 0.981 Control (4 schools) 2% 41% 57% 303 Total 2% 41% 57% 848 I am as happy as other children my age Intervention (10 schools) 2% 39% 59% 544 Yes Chi-square = 7.700 p<0.05 Control (4 schools) 3% 48% 49% 304 Total 3% 42% 56% 848 Responses with regard to statements about contentment/happiness of learner: Comparison 2 Comparison 2 None of the time Some of the time All of the time Number of respondents Statistically significant difference? I have people I can talk to when I have a problem Intervention (10 schools) 6% 35% 59% 544 No Chi-square = 1.404 p = 0.496 Control (3 schools) 8% 33% 59% 264 Total 7% 34% 59% 808 I am able to do things as well as most other people Intervention (10 schools) 2% 41% 58% 545 No Control (3 schools) 2% 40% 58% 264 Chi-square = 0.248 Total 2% 40% 58% 809 p = 0.884 I am as happy as other children my age Intervention (10 schools) 2% 39% 59% 544 Yes Chi-square = 9.663 p<0.05 Control (3 schools) 3% 49% 48% 264 Total 3% 42% 55% 808 (#) Comparison 2: One control school excluded Responses with regard to statements about contentment/happiness of learner: Comparison 3 Comparison 3 None of the time Some of the time All of the time Number of respondents Statistically significant difference? I have people I can talk to when I have a problem Intervention : OVC 5% 39% 55% 316 No Chi-square = 4.545 p = 0.103 Intervention : Not OVC 4% 30% 66% 165 Total 5% 36% 59% 481 I am able to do things as well as most other people Intervention : OVC 2% 44% 54% 317 Yes Chi-square = 8.665 p<0.05 Intervention : Not OVC <1% 33% 67% 165 Total 2% 40% 58% 482 I am as happy as other children my age Intervention : OVC 3% 41% 56% 316 Yes Chi-square = 6.688 p<0.05 Intervention : Not OVC <1% 33% 67% 165 Total 2% 38% 60% 481 Scores on measure of level of contentment/happiness, by three sets of comparisons Comparison Poor (Score: 0-1 out of 6) Average (Score: 2-4 out of 6) Good (Score: 5-6 out of 6) Number of respondents Statistically significant difference? Comparison 1 Intervention (10 schools) 2% 37% 61% 543 No Chi-square = 3.336 p = 0.189 Control (4 schools) 1% 44% 55% 303 Total 2% 39% 59% 846 Comparison 2 Intervention (10 schools) 2% 37% 61% 543 No Chi-square = 3.767 p = 0.152 Control (3 schools) 2% 44% 54% 264 Total 2% 39% 59% 807 Comparison 3 Intervention: OVC 2% 41% 57% 315 Yes Fisher‟s test = 6.829 p<0.05 Intervention: Not OVC 1% 30% 69% 165 Total 2% 37% 61% 480 (#) Comparison 2: One control school excluded Newly created measure of vulnerability that consists of five categories There is an adult over the age of 24 living in my home The adult is … Count Percentage Label All of the time Mother or father 321 38% Category 1 All of the time Other relative or foster parent 304 36% Category 2 Some of the time Mother or father or other relative or foster parent 143 17% Category 3 None of the time Mother or father or other relative or foster parent 41 5% Category 4 All of the time / some of the time No adult 6 1% None of the time No adult 21 3% Category 5 Total 836 100% Category 4 represents inconsistent responses and is another indicator of vulnerability Newly created measure of vulnerability, broken down in terms of OVC status New measure OVC status Total OVC Not OVC Unknown Category 1 33% 33% 34% 321 Category 2 45% 8% 47% 304 Category 3 31% 19% 50% 143 Category 4 38% 9% 53% 47 Category 5 38% 0% 62% 21 Total 37% 19% 43% 836 A statistically significant relationship exists between the newly created variable of vulnerability and OVC status (Chi-square = 75.609, p<0.05). APPENDIX 3 - LEARNER SURVEY SCHEDULE Today’s date:_____________________ Name of school:____________________________ Your gender: Male Female Your age:_____________________ INSTRUCTIONS: Please indicate your chosen answer with a tick in the appropriate column. HIV STATEMENT 1. HIV causes AIDS. Tru e False 2. A person with HIV can look healthy. Tru e False 3. You can only get HIV/AIDS through sex. Tru e False 4. Kissing transmits HIV/AIDS. Tru e False 5. If a man is circumcised he will definitely not be infected by HIV. Tru e False 6. HIV/AIDS can be transmitted from a mother to her unborn child. Tru e False 7. Antiretroviral drugs (ARVs) cure HIV. Tru e False 8. You can get HIV by sitting on the same toilet seat used by someone who has HIV. Tru e False STATEMENT 9. Taking a shower or a bath immediately after sex prevents HIV infection. True False 10. Even if both partners have HIV they should always use condoms when having sex. True False 11. Only people with multiple partners contract HIV/AIDS. True False 12. Sangomas know how to cure HIV/AIDS. True False 13. If a couple have been faithful for a long time they don‟t have to use a condom. True False 14. A person can‟t get HIV from mosquito bites. True False 15. Sex with a married person is safe. True False 16. Having sex with a virgin can cure HIV. True False 17. A person can‟t get HIV by drinking from the same cup as someone who is infected. True False STATEMENT 18. I am tired of hearing about HIV/AIDS. Agree Disagree 19. Everybody knows about HIV/AIDS. Agree Disagree 20. There‟s no point in abstaining or practicing safe sex because in the end we will all die anyway. Agree Disagree 21. If a person has HIV they must keep it a secret Agree Disagree 22. Only promiscuous (sleeping around) people get infected with HIV / AIDS. Agree Disagree 23. Abstinence (not having sex) is the best method of preventing HIV/AIDS. Agree Disagree 24. I will never fall in love with an HIV positive person. Agree Disagree 25. It is fine for women or girls to be a „player‟. Agree Disagree 26. It is fine for men or boys to be a „player‟. Agree Disagree 27. I would share things (toilet seats, utensils, bath, etc) with an HIV positive person. Agree Disagree 28. A teacher who is HIV positive but is not sick should be allowed to continue teaching in school Agree Disagree STATEMENT 29. I will abstain (not have sex) up until marriage. Definitely Maybe No 30. I will use condoms every time when I have sex. Definitely Maybe No 31. I will disclose my HIV / AIDS status if I am HIV positive. Definitely Maybe No 32. I am interested in testing for HIV / AIDS and knowing my status. Definitely Maybe No 33. I will support fellows/peers who are infected and affected by HIV/AIDS. Definitely Maybe No 34. Learners have access to HIV testing in my community Yes No Don’t know 35. My most recent HIV test was… Between 2-4 years Between 1-2 years Within the past year In the past six months Never Don’t know 36. The nurse gave me the result of my test Yes No Don’t know NS STATEMENT 37. I eat at least 2 meals a day None of the time Some of the time All of the time 38. I have enough food to eat None of the time Some of the time All of the time 39. I go to bed hungry None of the time Some of the time All of the time ED STATEMENT 40. On average, I go to school… Never Once a month Once a week 5 times a week 41. I have a school uniform to wear to school Yes No CP STATEMENT 42. I have a birth certificate Yes No I don’t know 43. I have a green identity document book Yes No I don’t know STATEMENT 44. There is an adult over the age of 24 living in my home None of the time Some of the time All of the time 45. The adult (a person over 24 years of age) in my home is: My mother or father A relative (e.g. my brother or sister, my grandmother, the sister or brother of my parents, other relative) Someone not in my family takes care of me (e.g. foster parent) Not applicable, there is no adult at my home. 46. I have a house where I can sleep at night None of the time Some of the time All of the time PS STATEMENT 47. I have people I can talk to when I have a problem None of the time Some of the time All of the time 48. I am able to do things as well as most other people None of the time Some of the time All of the time 49. I am as happy as other children my age None of the time Some of the time All of the time HES STATEMENT 50. My school attendance is affected because I need to work for money None of the time Some of the time All of the time APPENDIX 4 – LIST OF PEOPLE INTERVIEWED May’khethele Staff interviews Date Organisation Number of participants Venue 10.02.2012 Lifeline 2 CINDI offices 10.02.2012 Community Care Project 2 10.02.2012 Youth For Christ 1 10.02.2012 CINDI 2 Total 7 OVC focus groups Date School Male participants Female participants Total 13.02.12 Georgetown High School 6 5 11 14.02.12 Edendale High 4 7 11 14.02.12 Sukuma Comprehensive 5 5 10 15.02.12 Zamazulu Secondary 6 3 9 16.02.12 Bongudunga Secondary 6 4 10 16.02.12 Imvunulo Senior Secondary 4 6 10 17.02.12 Ikusaselihle High 4 5 9 17.02.12 Umthoqotho High 4 5 9 24.02.12 Gobindlovu Secondary 4 6 10 Total 43 46 89 Lifeline support group focus group Date Male participants Female participants Total 24.02.2012 2 10 12 Caregiver focus groups and interviews Date Activity Area Number of females Number of males Total number of participants 12.02.2012 Focus group Copeville 11 0 11 12.02.2012 Focus group Imbali 10 1 11 18.02.2012 5 x household visits Imbali, Dambura, Ashdown 5 0 5 19.02.2012 Focus group Gobindlovu High School 20 0 20 Total 46 1 47 School key informant interviews Date School Position of participant Number of participants Gender Comment 13.02.12 Edendale High Principal 1 Male 2 LO teachers were unavailable for the interviews LO teacher 1 Male Peer leaders 2 2 x Females 14.02.12 Sukuma Comprehensi ve LO teacher / Deputy Principal 1 Male The school is currently finalising the timetable and therefore there is 1 LO teacher Principal 1 Male 15.02.12 Zamazulu Secondary Principal and Deputy Principal 2 1 x Male; 1 x Female All interviewees were available and eager to participate in the research LO teachers 2 2 x Males LO teacher 1 Male LO teacher 1 Male 16.02.12 Imvunulo Senior Secondary Principal 1 Male The school has 1 LO teacher LO teacher 1 Female 17.02.12 Umthoqotho High Principal 1 Male 1 LO teacher was absent; 1 LO teacher had a family death LO teachers 2 2 x Males Total 17 Government interviews Date Department Position of participant Number of participants Gender 13.02.12 Department of Home Affairs MD 1 Male 21.02.12 Department of Health Senior technical advisor 1 Male Mentor coordinator 1 Female Department of Education Total 3 NGO interviews Date Organisation Position of participant Number of participants Gender 20.02.12 Medical Male Circumcision Clinic MMC acting senior counsellor 1 Female Zamimpilo drop in centre Project Director 1 Female 21.02.12 Pmb Child Welfare Intake Manager 1 Female Esther House 2 x Project Directors 2 1 x Male; 1x Female African Enterprise Pastor 1 Male APPENDIX 5 – QUALITATIVE DATA COLLECTION TOOLS STAFF INTERVIEWS CINDI / Partner Organisations Interview Schedule 1) Please can you describe the May‟khethele programme from your perspective? 2) In what way does working in a consortium help? a. Are there any challenges? b. Has being part of the programme helped your organisation to build capacity? 3) What have the major achievements of the programme been in terms of outcomes for OVCs? Probe:  HIV knowledge, attitudes, stigma among learners – how do you know?  Learners‟ sexual behaviour – how do you know?  OVC wellbeing – examples, are you trying to reach all children in schools? Is this a programme goal?  Lifeline – HIV support to PLHIV 4) What are the major constraints to achievement of outcomes, particularly with reference to the main programme objectives:  to increase life skills and improve the wellbeing of OVC under the age of 18 in 55 schools within uMgungundlovu district KZN, through HIV/AIDS prevention education, promoting behaviour change, improving access to counselling and testing and provision or linkage to other OVC services; and  to increase knowledge and understanding of OVC care and support through provision of informal training to primary caregivers. 5) Have there been any unexpected negative outcomes for OVCs and the HIV situation in schools and communities? (eg, abstinence messages are unrealistic, less access to condoms, HCT testing in schools) 6) What have the major benefits been for your organisation from participation in the programme in terms of capacity, programming, partnerships and relationships? 7) Have there been any unexpected negative outcomes for any of the organisations in this programme? (eg, vision shift, unsustainable increases in HR commitments, donor specific reporting or management requirements, ??) 8) Can you see points at which the programme‟s influence or advocacy has brought about changes in the way services are delivered? Probe: By DSD, DoE, DoH, social welfare 9) What is the long term solution to challenges you are addressing? Please comment on if and how the programme intends to sustain change, influence national agendas, or contribute to the situations of OVC beyond those in your partner schools? KEY INFORMANT INTERVIEWS IN SCHOOLS Principal / Life Orientation Teacher / HIV Committee Member Interview Schedule 1. What do you see as the role of the May‟khethele programme? 2. What are your impressions about the May‟khethele programme? 3. What value does the programme bring to your school? 4. Would you like the programme to be at the school in 2013? Probe: If YES Why? and if NO Why Not? 5. Do you think the programme has increased learners knowledge about HIV? Probe: how do you know; eg, what makes you feel that information is being absorbed and knowledge is increasing? What makes you think that it might not be changing …. Do the learners continue to believe any HIV myths? Which myths? 6. Do you think the programme has influenced learners to practise safer sex or abstain? Probe: how do you know; eg, has there been a change in pregnancy rates? 7. Has fear and stigma around HIV changed in your school because of the programme? Probe: What changes do you see? What difference has that made (explore the theme in a discussion – High stigma people don’t want to test or suggest using a condom. If stigma has changed people would be more willing to do this and talk about sex) 8. How could the HIV education programme be improved? 9. Are you aware of the OVC support element in the programme? Probe: If yes: What value does this element bring to households and OVCs? Is the programme able to improve the wellbeing of OVCs at your school? In what ways; probe for in terms of increasing access to core services – food support, health, child protection, psychosocial, education support, economic support 10. How could the OVC support programme be improved? 11. In closing, do you have any advice or final comments you would like to share with CINDI and partner organisations? Peer Educator Interview Schedule 1. What do you see as the role of the May‟khethele programme? 2. What are your impressions about the May‟khethele programme? 3. What value does the programme bring to your school? 4. Would you like the programme to be at the school in 2013? Probe: If YES Why? and if NO Why Not? 5. Do you think the programme has increased learners knowledge about HIV? Probe: how do you know; eg, what makes you feel that information is being absorbed and knowledge is increasing? What makes you think that it might not be changing …. Do the learners continue to believe any HIV myths? Which myths? 6. Do you think the programme has influenced learners to practise safer sex or abstain? Probe: how do you know? 7. Has fear and stigma around HIV changed in your school because of the programme? Probe: What changes do you see? What difference has that made (explore the theme in a discussion – High stigma people don’t want to test or suggest using a condom. If stigma has changed people would be more willing to do this and talk about sex) 8. How could the HIV education programme be improved? 9. Are you aware of the OVC support element in the programme? Probe: If yes: What value does this element bring to households and OVCs? Is the programme able to improve the wellbeing of OVCs at your school? In what ways; probe for in terms of increasing access to core services – food support, health, child protection, psychosocial, education support, economic support 10. How could the OVC support programme be improved? 11. What extra support have you had to become a peer educator? How has this changed your life? GOVERNMENT OFFICIALS Interview Schedule (DoE; DSD; municipality HIV or/and vulnerable groups worker 1. What do you see as the role of the May‟khethele programme? 2. In what way has there been collaboration between your department and the programme? How has this helped you? 3. What value does the programme bring to the community? Probe: in terms of: Access to services, Improving knowledge on HIV and AIDS, Promoting safer sexual practice, Overall wellbeing of children and youth 4. How could the programme be improved? 5. In closing, do you have any advice or final comments you would like to share with CINDI and partner organisations? Other NGOs in the area / Social Worker Interview Schedule 1. What do you see as the role of the May‟khethele programme? 2. In what way has there been collaboration between your NGO/work and the programme? How has this helped you? 3. What value does the programme bring to the community? Probe: in terms of: Access to services, Improving knowledge on HIV and AIDS, Promoting safer sexual practice, Overall wellbeing of children and youth 4. How could the programme be improved? 5. In closing, do you have any advice or final comments you would like to share with CINDI and partner organisations? OVC focus group schedule 1. What does the May‟khethele programme do? 2. Of these things you have spoken about, please explain which of these is the most important thing that the May‟khethele programme helps you with or does for you? 3. Think back to before you were a part of the programme – a. What was different in your life? b. Are there any things that have improved or got better since then? c. Are there any things that have got worse? 4. What is the thing you like most about the May‟khethele programme? 5. Is there anything you don‟t like about the May‟khethele programme? 6. What other things should the programme do? Caregiver Focus Group / OVC Household Interview Schedule 1. What do you see as the role of the May‟khethele program (CINDI partner organization XXXXX) in your community? 2. What kinds of services and support have you and your household received from the organisation? Probe: What do you value the most? 3. How satisfied are you with the services and support you and your household receive from the organisation? a) How does the xxxx organization help you? b) Are there any problems with the xxx organization ? Please explain. 4. What has changed in your life and the life of your child/the child you take care of since you started receiving services from the organisation? a. What has changed b. Are there any things that have improved or got better since then? c. Are there any things that have gotten worse? 5. Are there any needs that you and your household have that are not being met? 6. What, if anything, can the organisation do to make the services it provides to you and your household more effective? Lifeline Support Group Focus Group Schedule 1. In what way does being a member of the Lifeline support group help you? Probe:  Dealing with your HIV status  Issues of anger and depression  Issues of confidence and self esteem  Supportive people to talk to within the group  Disclosing to your family  Issues of disclosure at school and stigma from peers? 2. Is there any other support, eg individual counselling, support at home, or other support through referrals, that Lifeline has helped to arrange? 3. If you are on ART treatment, does this support group help with starting treatment and adherence? What happens at this support group that helps with adherence? Is this enough for people on treatment to maintain full treatment compliance? What other support is helpful? 4. How has Lifeline helped in creating this support group and keeping it going? 5. What about the group helps it to keep going? 6. Is there anything you don‟t like about the support group? 7. What other things should the support group do? APPENDIX 6 – PACT TOR: CINDI MAY’KHETHELE OVC PROGRAMME EXTERNAL EVALUATION Request for Proposals (RFP) External Evaluation Evaluating the Outcome of Children in Distress Network (CINDI) MAY‟KHETHELE OVC PROGRAM Children in Distress Network (CINDI) South Africa OVC Program 1- Background / Rationale The Children in Distress Network (CINDI) is a partnership of people and organisations that support children affected and infected by HIV and AIDS in KwaZulu-Natal province. The May‟khethele orphans and vulnerable children‟s program is part of a larger initiative implemented by four CINDI members namely: Community Care Project (CCP), Lifeline (LL), Sinani and Youth for Christ (YFC-KZN). The program provides support for orphaned and vulnerable children through primarily school based interventions and had been funded by the United States President‟s Emergency Plan for AIDS Relief (PEPFAR) since October 2007. The May‟khethele OVC program aims to improve the lives of orphans and other children made more vulnerable by HIV and AIDS through provision of a comprehensive range of services. These services include provision of HIV prevention education, psychological care, voluntary counselling and testing (VCT), health care support specifically for antiretroviral treatment (ART), educational support in the form of school uniforms and stationery packs, general healthcare referrals and helping qualifying children access enabling documents (birth certificates and identity documents) and social grants. May‟khethele OVC program operated in sixteen schools in its first year. These schools were spread across 20 wards of uMgungundlovu district of KwaZulu-Natal. This number has increased dramatically over the last three years and the program now operates in fifty-five schools located in 29 wards. The program has been running for three years and provided services to 5204 OVC in its first year, 11 722 in the second year and 12 193 in its third year. The impact of the program on the wellbeing of children however has not been assessed so far. This evaluation therefore seeks to measure the effect that the program has brought about on its beneficiaries. 1.2- Program objectives and key priority areas May‟khethele OVC program‟s goal is to improve the health and psycho-social wellbeing of orphaned and vulnerable children of the greater uMgungundlovu district of KwaZulu-Natal Province through improved access to services. CINDI, through May‟khethele OVC program put in place a set of interventions to be implemented in schools to improve OVC‟s lives. One of the key interventions implemented was the HIV prevention education designed to support improved attitudes about HIV and AIDS, reduced stigma, increased knowledge of the disease and improved prevention behaviour amongst the youth. The main objectives of the program are: - to increase life skills and improve the wellbeing of OVC under the age of 18 in 55 schools within uMgungundlovu district KZN, through HIV/AIDS prevention education, promoting behaviour change, improving access to counselling and testing and provision or linkage to other OVC services; and - to increase knowledge and understanding of OVC care and support through provision of informal training to primary caregivers. 2. Purpose of the Evaluation Over the past three years, a substantial proportion of the May‟khethele orphans and vulnerable children‟s program resources have been invested in supporting the four implementing partners to enhance their capacity as well as on provision of services to address the needs of orphans and vulnerable children. A large amount of data has been generated from the program mainly on inputs and key activities implemented as well as on immediate results such as children served per different service types, age and gender. However, a considerable gap in available data is the documentation of outcome level results that reflect the value of the program in changing the lives of its beneficiaries. The overall purpose of this evaluation is therefore to assess the outcome of the program on the wellbeing of children. Although there may be other relevant questions and knowledge about the program, the limited resources available for the external evaluation call for a more focused assessment that will generate essential information around the prime focus of the program. The new knowledge generated by the evaluation is expected to enrich learning on what worked and didn‟t work, and to inform future program design and implementation Key Evaluation Questions The key evaluation questions include the following  To what extent did the school based HIV prevention education intervention improve attitudes and knowledge about HIV and AIDS, reduce stigma and influence change in sexual behaviours and HIV infection risk reduction among targeted adolescents?  How effective was the training of primary care givers in improving their abilities and coping skills in caring for children? In addition to these, additional questions relevant to the evaluation are included in the table under Annex A. 3. Key Stakeholders (users of the evaluation findings) The Key stakeholders for this evaluation include government managers in various departments, the program beneficiaries, schools (teachers and learners), primary caregivers/parents, program staff, program partners, donor agencies, CINDI board and CINDI network. Stakeholders Reasons why the stakeholder should be involved in the evaluation How the Stakeholder might use or be affected by the evaluation’s results Stakeholders role in the evaluation Government Stakeholders ; Provincial Departments of Social development (DSD) and department of Education (DoE) The Departments particularly Social Development and Education are key stakeholders for the May‟khethele program given their mandates in policy and implementation of programs for OVC. Perspectives of the DSD and DoE are therefore essential in this evaluation. The DSD and DOE will use the evaluation results to inform potential improvements in OVC programs in schools. The results may be used to inform future funding decisions and policy related to programs that are run in schools. Respondents in key informant interview Children and their families Primary beneficiaries of the program and their views on what has worked and what hasn‟t is essential in assessing the value of the program Participate in providing feedback on the program and informing decision making processes on how best to respond to the needs of vulnerable children and families Key respondents in focus groups and survey Schools (Teachers and Principals) Principals and teachers work closely with the program and children and they are essential in the provision of Learn from the evaluation – what worked well, what didn‟t and how to improve their involvement in the program to enhance the value Respondents in Individual In-depth services to children in need Interviews and/or Focus Group Discussions Program Staff and Partners Program staff and partners are central to the implementation of the program Learn from the evaluation – what worked well, what didn‟t and how to improve on the program to enhance its value to the targeted beneficiaries Key respondents in focus group discussions and key informant interview CINDI Network and Board The CINDI board provides overall guidance on program implementation The board will use the results for future planning on whether to allow similar programs to be carried out by CINDI, what should be done differently in future. Lessons learnt from the evaluation process will be shared with the broader CINDI network and potentially contribute to influencing programming by other organisations None USAID and Pact SA USAID provided funding for the program implementation. Pact has worked with CINDI as a Umbrella Grants Manager (UGM) partner over the duration of the grant and provided substantial technical support to the program Learn the value of the program and whether intended overall goals were met. USAID is commissioning the Evaluation while Pact is providing evaluation management support External Evaluation RFP _CINDI-May’khethele OVC program_2011 47 | P a g e 4. Evaluation Design The focus of the evaluation is to assess key program outcomes related to strengthening response to the needs of OVC, as such, the evaluation design should enable the determination of the cause-effect relationship between potential improvements that may be found and the program interventions. Quasi-experimental designs are likely to be most appropriate however budgetary constraints may limit options available. This design will enable the comparison of intervention and non-intervention sites with regards to effectiveness of response to needs of OVC within school settings, self-reported behaviors among adolescents as well as abilities and coping skills among caregivers. However the final design to be employed will be determined after the external evaluators have had a chance to undertake a frontend analysis and are therefore able to select the best design option that specifies the kind of comparison that should be made. 5. Key Data Sources and Methods The data collection methods will be mixed aiming to collect both qualitative and quantitative data. Data sources will include target OVCs and their caregivers, adolescents and school staff (teachers and principals) in selected May‟khethele and comparison schools, representatives from the department of education and possibly other relevant government officials as well as May‟khethele program staff. Data collection methods will include a survey in schools, focus groups of program beneficiaries, key informant interviews and a review of the May‟khethele program database. 6. Sampling Quantitative Data The evaluation will be based on primary survey data collected from randomly selected children in May‟khethele as well as from comparison schools. The sample will be drawn using a two -stage cluster sampling with probability proportionate to size (PPS). External Evaluation RFP _CINDI-May’khethele OVC program_2011 48 | P a g e In calculating the sample size several points will be taken into consideration; the anticipated magnitude of change (related to the key program interventions as reflected in the evaluation questions), the desired degree of confidence (the level of statistical significance), and the statistical power. Qualitative Data Purposeful sampling will be used to identify respondents to participate in key informant interviews and focus group discussions. 7. Key Data Analysis Procedure Analysis methods will depend largely on the type and quantities of data collected. However the data analysis will basically focus around comparison of differences in response to the survey by children targeted by the program compared to those in comparison sites. Comparisons will include the different key variables such as length of contact with the program, age, gender, vulnerability (OVC vs non OVC) etc. Furthermore, analysis of qualitative data obtained from focus groups and key informant interviews will demonstrate program beneficiaries‟ feedback on the extent to which the program facilitated improved response to the needs of children as well as extent to which the program improved abilities and coping skills of caregivers in caring for children. Analysis will be undertaken using various tools available for qualitative and quantitative data as deemed appropriate. 8. Evaluation Process; activities and deliverables Key Aspects of the evaluation scope of work (SOW) 8.1- Undertaking a comprehensive front end analysis; including the following  Understanding the relationship between program stages and the proposed broad evaluation question External Evaluation RFP _CINDI-May’khethele OVC program_2011 49 | P a g e  Understanding the context for program delivery and key factors that influence program implementation  Understanding the existing theoretical and empirical knowledge about the program and examining program theory  A comprehensive stakeholder analysis and determination of roles of key stakeholders in the evaluation  Balancing costs and benefits of the evaluation and advising on the most strategic questions to include in the evaluation  Developing the detailed evaluation protocol The Key deliverable is a detailed evaluation protocol including  Key evaluation questions and linkages to program theory  Stakeholder analysis including their roles in the evaluation  Evaluation approach, design and sampling methods  Key measures and data collection tools to be used  Data analysis strategy including dummy table/graphs for presenting data  Evaluation work-plan including key activities and timeframes  Detailed budget 8.2- Following submission and approval of the detailed evaluation protocol, the consultants will implement the evaluation process including the following key steps.  Pre-test instruments  Train data collectors  Undertake the evaluation data gathering process  Prepare data for analysis  Clean data  Enter data into electronic data analysis systems  Undertake comprehensive data analysis  Formulate the findings Key deliverables include  Submission of a final tested data collection instruments to be used  Report on the data gathering process after it is completed 8.3- Consultants will be required to prepare a range of reports on the findings of the evaluation and to participate in the provision of feedback and dissemination of key findings  Identify major findings: what works, what does not, key lessons  Develop clear and specific recommendations to address key findings and proposals for action  Prepare reports using various communication tools directed at different stakeholders as appropriate  Participate in provision of feedback to selected stakeholders External Evaluation RFP _CINDI-May’khethele OVC program_2011 50 | P a g e Key deliverables  Detailed written report including an executive summary with highlights of the evaluation and key findings  Power Point Presentation providing summary of evaluation process and results  Brief paper targeting community audiences on the key findings from the evaluation  participation in dissemination of evaluation findings (various events will be organized by Save the Children for the different stakeholder groups 9. Evaluation Team- Required expertise and experience The evaluation team should comprise of individuals with the following expertise  Extensive evaluation experience particularly in the South Africa; demonstrated experience in undertaking similar evaluations  Programmatic experience in orphaned and vulnerable children‟s programs as well as HIV and AIDS including experience with School-based programs  Familiarity with the South African government systems, particularly in relation to working with school-based programs  Capacity development expertise  Extensive experience in employing both qualitative and quantitative data collection methods including participatory evaluation techniques 10. Roles and Responsibilities: undertaking and managing the evaluation Who will be involved Main Role External Evaluators Lead evaluator Evaluation/research officers Data collectors  Develop the evaluation design and key measures for each evaluation question.  Develop the data collection strategy; sampling and instruments.  Developing data analysis strategy.  Pre-test instruments and train data collectors.  Undertake the evaluation data collection process.  Prepare data and undertake comprehensive data analysis.  Formulate the key findings and recommendation.  Prepare reports; identify major findings, develop recommendations. External Evaluation RFP _CINDI-May’khethele OVC program_2011 51 | P a g e CINDI and Partners Staff Program Managers Program staff, M&E team, Field staff Administrative staff  Work with the External Evaluator in facilitating access to required information and resources.  Provide input in finalizing the evaluation design, sampling, data collection tools and processes by the External Evaluator.  Assist with coordinating and providing logistical support for field visits and meetings with key stakeholders during data collection.  Plan for and undertake dissemination of findings. Pact SA MERL department Programs department Contracts management team  Management of the solicitation process for identifying suitable External Evaluator.  Provide input in finalizing the evaluation design, sampling, data collection tools and processes.  Management of the External Evaluators contract.  Monitoring the implementation and deliverables of the evaluation.  Preparation of evaluation management documents- RFP, SOW, Contract USAID Activity manager Overall guidance and approval of the following;  Evaluation Terms of Reference  Scope of work and contract for the External Evaluator  Evaluation budget  Final evaluation Report 11. Documentation and Data Use Plan Final Report: The final deliverable of the evaluation should be a transparent, credible and comprehensive report of all findings. This document will be primarily for internal use at CINDI, Partner organisations and USAID levels and will be freely available to external technical specialists through the CINDI Program. Suggested Evaluation Report Format: Cover page Table of Contents Acronyms used in the report External Evaluation RFP _CINDI-May’khethele OVC program_2011 52 | P a g e Executive Summary: includes the major findings of the evaluation and summarised conclusion and recommendations. Introduction : background to the program evaluated Evaluation Purpose and Methods  Literature review  Purpose and Guiding Questions  Methodology and data collection techniques  Limitations Findings: findings of the of the Evaluation Conclusions: should be clearly based on evaluation findings and include their implications for future interventions Recommendation: should be clearly related to conclusions, should be practical and if necessary divided up for various actors or program partners Appendices: schedule, list of people interviewed, questionnaires, TOR, bibliography and list of documents reviewed 12. Timeframes/ level of effort The evaluation activities are expected to be undertaken between October 2011 and February 2012. Estimated level of effort is 60-70 consultant days depending on the final agreed evaluation plan. This timeframe will cover the full range of evaluation processes. 13. The Evaluation Budget The total estimated cost for this evaluation is between $40,000 and $50,000. Consultants will be expected to submit detailed budgets as part of the evaluation proposals for consideration. The estimation includes Consultants time, costs of data collection, and the logistical support and travel costs during the evaluation process. 14. Submission of Proposals The outline of the technical proposals should include the following: 1. Introduction 2. Key Evaluation Questions 3. Proposed Evaluation Approach and Design 4. Sampling Strategy 5. Plan for data acquisition 6. Data analysis Plan 7. Evaluation Team (brief Resumes; provide detailed CVs in Appendix). The detailed CV should include the names and contact numbers of the External Evaluation RFP _CINDI-May’khethele OVC program_2011 53 | P a g e staff/consultants assigned to the project. A summary of the role and responsibility of each staff person/consultant and estimated time to be spent by each staff person/consultant; CVs must address all key elements in the evaluation matrix included below. 8. Team members time commitment and availability over the evaluation period 9. Evaluation work plan reflecting proposed time frames and outputs/deliverables (including Gantt chart) 10. Budget - detailed budget including daily fees for each staff person/consultant and breakdown of all other costs to be charged to the contract. The prospective service provider must submit an all-inclusive price for all activities proposed in the application. 15. Evaluation of Proposals The proposals received will undergo a technical evaluation by a selection committee; The selection committee reserves the right not to accept the lowest bid, as the elements listed in the evaluation matrix below will play a major role when evaluating proposals; In order to ensure meaningful participation and effective comparison prospective service providers are requested to furnish detailed information in substantiation of compliance to the technical evaluation criteria. 16. Proposal Scoring Criteria The review of proposal submitted by potential evaluators will be based on the following allocation of points. ELEMENT Range Evaluation Design (suitability & rationale) (0-20) Data Collection Strategy including sampling (methods, process & involvement of key stakeholders) (0-25) Evaluation team (range of skills and experience) (0-35) Availability and commitment of required level of effort (LOE) by key staff over the duration of evaluation (0-10) Cost Efficiency (budget versus proposed output) (0-10) Total (0-100) 17. Proposal Submission Details External Evaluation RFP _CINDI-May’khethele OVC program_2011 54 | P a g e All proposals should be submitted by email to rfp@pactsa.org.za by 24th August 2011, at 5pm South African time. Late submissions will not be considered. Please ensure the subject line states “Application – Evaluating the Outcome of Children in Distress Network (CINDI) May‟khethele OVC Program”. In accordance with US Government regulations on free and fair competition, all prospective service providers must have access to the same information. Therefore all enquiries regarding these terms of reference should be directed to rfp@pactsa.org.za. Pact will create a distribution list and periodically send answers to questions and updates to all prospective applicants. Please note, Pact cannot commit to providing answers to all questions asked. Pact will do its best to source answers, but can only commit to making the same information available to all prospective applicants via this question and answer forum. External Evaluation RFP _CINDI-May’khethele OVC program_2011 55 | P a g e Annex A: Additional Questions Relevant to the Evaluation Components of the program which we would like to learn more about Questions we have that we would like answered What data do we have to help us analyze this question? What further data do we need? Who should be involved? School based approaches to providing care and support to OVC HIV - prevention education, OVC educational support, Psychosocial and Child Protection Support  What have been the benefits of accessing HIV counseling and testing services and knowing one‟s status?  Has there been change in HIV/AIDS related knowledge?  Has there been a change in sexual practices among target beneficiaries?  How has the educational support received from the program improved school attendance and performance?  What, if any change did access to care and support in schools have on educational outcomes such as attendance and performance?  Has the OVC emotional wellbeing improved? Has the program in any way helped to improve self-esteem?  Has the program helped in accessing legal protection in case of need?  How well did the program address the need for acquiring legal document s; like birth registration or ID?  How well has the program facilitated access to services to children which were denied legal status?  Has the program helped access to HIV related health care services including ART?  Information from the program database on services provided to children  Program process evaluation reports  Program performance reports  Key respondents in survey  School attendance and data from class registers  Progression report cards/stats from school  Key respondents in individual in￾depth interview and Focus Group Discussions  Sampled beneficiaries  Program Staff  School principals, teachers, caregivers, parents, External Evaluation RFP _CINDI-May’khethele OVC program_2011 56 | P a g e Components of the program which we would like to learn more about Questions we have that we would like answered What data do we have to help us analyze this question? What further data do we need? Who should be involved? Training of OVC care givers  Was the training effective in improving skills and knowledge among care gives  How does the approach to and model of training compare with others in terms of delivering the intended results  Did the training enhance good family functioning (relationship between the OVC and their primary caregiver)?  Have the parents/primary care givers improved their parenting skills?  Have care provider attitudes‟ improved?  Program performance data  Key respondents in survey  Feedback from stakeholders  Document review of other training programs (formal/informal; accredited versus non-accredited) for care givers  Primary caregivers/parents  Stakeholders  Program staff 57 APPENDIX 7 - BIBLIOGRAPHY CINDI, 2008. PowerPoint presentation May‟khethele: My dreams - My future. Available at: http://www.cindi.org.za/files/MaykhetheleProgramme20080522.pdf International HIV/AIDS Alliance, 2003. Building Blocks: Africa-wide briefing notes - Resources for communities working with orphans and vulnerable children. Khulisa Management Services. 2008. OVC Programmes in South Africa Funded by the U.S. President‟s Emergency Plan for AIDS Relief. Available at: http://www.cpc.unc.edu/measure/our-work/program-areas/ovc/ovc-program-case￾studies/ovc-case-studies-sa/Hope%20WorldwideSA-%20SR-842-H3.pdf National Action Plan for Orphans and Other Children made Vulnerable by HIV/AIDS in South Africa 2009-2012. 2009. Available at: http://www.cindi.org.za/files/nap_2009- 2012_v8_final.pdf PACT, 2011. Request for Proposals (RFP) External evaluation – Evaluating the outcome of Children in Distress Network (CINDI) May‟khethele OVC Programme PEPFAR. 2006. Orphans and Other Vulnerable Children Programming Guidance. Available at: http://www.pepfar.gov/guidance/78217.htm Tremendous Hearts. 2010. South Africa‟s OVC Crisis. Available at: http://www.tremendoushearts.org/content/south-africas-ovc-crisis USAID. 2008. Orphans and Vulnerable Children in High HIV-prevalence Countries in Sub￾Saharan Africa. Available at: http://pdf.usaid.gov/pdf_docs/PNADM647.pdf Yezingane Network. 2010. Some Facts about the Situation of Children in Kwa-Zulu Natal. Available at: http://www.childrensrightscentre.co.za/site/files/6592/Prov_Prof_KwaZulu%20Natal.p df 58 APPENDIX 8 – LIST OF DOCUMENTS REVIEWED CINDI May‟khethele OVC Programme 2010 annual progress report to PACT SA CINDI May‟khethele OVC Programme 2009 annual progress report to PACT SA CINDI May‟khethele OVC Programme 2008 annual progress report to PACT SA May‟khethele process evaluation report 2009 May‟khethele process evaluation report 2008 CCP HIV prevention education manual YFC HIV prevention education manual Lifeline HIV/AIDS manual for participants Lifeline programme description and implementation plan Lifeline VCT process SASI manual, April 2011 May‟khethele indicator information sheet (PEPFAR year 4) Programme quality assessment tool – YFC, Lifeline CINDI stakeholder list Organograms – CINDI, CCP, YFC, Lifeline Evaluation planning workshop presentations – CINDI, CCP, YFC Database framework May‟khethele information form Child profile form Referral and monitoring form Revised data quality management procedures List of May‟khethele high schools