“Linkages to Care: Home Visiting in Urban Townships”
Outcome and Impact Evaluation of the Future Families 
program
Final Evaluation Report
November 2017
Support for this project is provided by the United States Agency for International Development (USAID) 
Southern Africa under the President’s Emergency Plan for AIDS Relief (PEPFAR) through Cooperative 
Agreement No. AID-674-A-12-0002 awarded to Tulane University. The views expressed in this document 
do not necessarily reflect those of USAID or the United States Government.
i 
Project Title: Linkages to Care: Home Visiting in Urban Townships (FF)
Contractor: Tulane University, School of Social Work’s Highly Vulnerable Children’s Research Center 
(HVC-RC)
Award No: AID-674-A-12-00002 
Project Start Date: June, 2014
Project End Date: April, 2017
Persons responsible for compiling the report:
Tonya Renee Thurman
Chief of Party
East Block, Tannery Park, 23 Belmont road, 
Rondebosch, Cape Town 7700
South Africa
Tel: (+272) 1 685-1696
Fax: (+272) 1 685-1696
Email: tthurma@tulane.edu
Tory Taylor
M&E Specialist
1440 Canal Street,
New Orleans, LA 70112
U.S.A.
Tel: (504) 988-4027
Email: ttaylor@tulane.edu 
Johanna Nice
Senior Program Manager
127 Elk Place
New Orleans, LA 70112
U.S.A.
Tel: (504) 862-3476
Email: jnice@tulane.edu
Dovile Vildaite
Program Manager
127 Elk Place
New Orleans, LA 70112
U.S.A.
Tel: (503) 247-1941
Email: dvildaite@tulane.edu 
ii
Table of Contents
Acknowledgments ............................................................................................................................................... iv
List of Acronyms.................................................................................................................................................... v
List of Tables and Figures..................................................................................................................................... vi
Executive Summary.............................................................................................................................................. 1
Introduction...................................................................................................................................................... 1
Methodology .................................................................................................................................................... 1
Findings............................................................................................................................................................. 3
Limitations ........................................................................................................................................................ 5
Conclusions and Recommendations................................................................................................................. 5
Project Background.............................................................................................................................................. 7
Evaluation Purpose and Questions...................................................................................................................... 8
Purpose............................................................................................................................................................. 8
Evaluation Questions........................................................................................................................................ 8
Evaluation Design and Methodology .................................................................................................................. 8
Study setting and population............................................................................................................................ 8
Stakeholder engagement.................................................................................................................................. 9
Quasi-experimental post-test study: sampling and analysis strategy .............................................................. 9
Impact evaluation sampling and analysis strategy (Randomized controlled trial).........................................10
Data collection methodology..........................................................................................................................11
Key outcome measures...................................................................................................................................11
Data analysis...................................................................................................................................................12
Ethical considerations and assurances...........................................................................................................13
Deviations and adjustments...........................................................................................................................14
Procedures to ensure high quality data..........................................................................................................15
Findings and Limitations....................................................................................................................................16
Key findings and limitations of the quasi-experimental study .......................................................................16
Outcome 1: HIV Testing ..............................................................................................................................17
Outcome 2: School Attendance ..................................................................................................................19
Outcome 3: Caregiver Psychosocial Wellbeing...........................................................................................19
Outcomes 4: Early Childhood Development...............................................................................................20
Outcome 5: Access to Social Services.........................................................................................................21
Limitations of the quasi-experimental design ............................................................................................22
iii
Key findings and limitations of the RCT ..........................................................................................................22
Outcome 1: HIV Testing ..............................................................................................................................24
Outcome 2: School Attendance ..................................................................................................................26
Outcome 3: Caregiver Psychosocial Wellbeing...........................................................................................27
Outcomes 4: Early Childhood Development...............................................................................................29
Outcome 5: Access to Social Services.........................................................................................................30
Limitations of the RCT................................................................................................................................. 32
Conclusions and Recommendations..................................................................................................................33
Summary of key findings................................................................................................................................. 33
Recommendations..........................................................................................................................................35
Dissemination.....................................................................................................................................................38
References..........................................................................................................................................................40
iv
Acknowledgments 
The study team wants to thank all the people who made this evaluation project possible. We are 
particularly grateful for the support provided from personnel at USAID Southern Africa, particularly 
Anita Sampson and Naletsana Masango. We further extend our appreciation to Future Families and its 
staff members for their insight and logistical support, to the local partners who facilitated our access to 
the field and to our local collaborators. A special thanks to the community care workers who devoted a 
considerable part of their valuable time to answer our questions and to arrange meetings with the 
caregivers of the children. We also thank Impact Research International under the leadership of Patrick 
Chiroro and Tronny Mawadzwa for their exemplary fieldwork. Importantly, we extend our gratitude to 
the participating caregivers who despite the endless challenges of their everyday life, generously shared 
intimate details of their lives with the research team in order to increase our understanding of their 
circumstances and the impact programs can have on their lives. We sincerely hope that they and others 
in similar circumstances will be the ultimate beneficiaries of these research efforts.
v 
List of Acronyms
AIDS Acquired Immune Deficiency Syndrome
CAPI Computer-Assisted Personal Interviewing
CBO Community Based Organization
CSG Child Support Grant
ECD Early Childhood Development
FF Future Families 
HCT HIV Counselling and Testing
HIV Human Immunodeficiency Virus
NGO Nongovernmental Organization
OAG Old Age Grant
OVC Orphans and Vulnerable Children
PEPFAR U.S. President‘s Emergency Plan for AIDS Relief
RCT Randomized Control Trial
RSA Republic of South Africa
UNAIDS Joint United Nations Programme on HIV/AIDS
UNICEF United Nations Children’s Fund
USAID United States Agency for International Development
vi
List of Tables and Figures
Table 1. Rounds of data collection and analysis................................................................................................. 11
Table 2. Sample description from the quasi-experimental component of evaluation.......................................16
Table 3. Sample baseline characteristics of caregiver and their children by group assignment........................23
Figure 1. Caregivers ever tested for HIV.............................................................................................................17
Figure 2. Caregivers tested for HIV in last year...................................................................................................17
Figure 3. Children ever tested for HIV ................................................................................................................18
Figure 4. School attendance in preceding week among all school-age children................................................19
Figure 5. School attendance in preceding week among children 15 years and older........................................19
Figure 6. Caregiver Psychosocial Wellbeing: Basic Support................................................................................20
Figure 7. Caregiver Psychosocial Wellbeing: Negative Feelings.........................................................................20
Figure 8. Early Childhood Development: Immunization.....................................................................................21
Figure 9. Early Childhood Development: Stimulating activities..........................................................................21
Figure 10. Access to Social Services: Child Support Grant by eligibility..............................................................22
Figure 11. Access to Social Services: Old Age Grant by eligibility .......................................................................22
Figure 12. Caregivers ever tested for HIV...........................................................................................................24
Figure 13. Caregivers tested for HIV in last year................................................................................................. 24
Figure 14. Caregivers 35 years and younger: tested in last year........................................................................25
Figure 15. Children ever tested for HIV ..............................................................................................................25
Figure 16. Children tested in last year................................................................................................................26
Figure 17. Impact of caregiver testing on children HIV testing ..........................................................................26
Figure 18. School attendance in preceding week among all school-age children..............................................27
Figure 19. School attendance in preceding week among children 15 years and older......................................27
Figure 20. Caregiver Psychosocial Wellbeing: Basic Support..............................................................................28
Figure 21. Caregiver Psychosocial Wellbeing: Negative Feelings.......................................................................28
Figure 22. Child Health Clinic Cards....................................................................................................................29
Figure 23. Early Childhood Development: Immunization...................................................................................29
Figure 24. Early Childhood Development: Stimulating activities........................................................................30
Figure 25.Access to Social Services: Verified Birth Certificate............................................................................30
Figure 26. Access to Social Services: Child Support Grant by eligibility..............................................................31
Figure 27. Access to Social Services: Old Age Grant by eligibility .......................................................................31
Figure 28. Participants’ perspectives on the program impact on their lives......................................................32
1 
Executive Summary 
Introduction
Evaluation purpose. The purpose of this study was to evaluate the impact of Future Families services, 
with an emphasis on their care worker home visiting program provided to families with orphans and 
vulnerable children (OVC) in Tshwane (aka Pretoria) townships, South Africa. Future Families, a non￾profit organization operating in South Africa, takes a family-based approach to addressing the needs of 
OVC through paraprofessional home visiting services delivered by mobilized and trained care workers. 
The study encompasses a quasi-experimental component designed to provide rapid feedback related to 
program implementation, and a randomized controlled trial (RCT) offering rigorous evidence of program 
impact. Findings are expected to strengthen the evidence base for effective family centered 
programming to address the needs of vulnerable children and their caregivers.
Evaluation questions. What is the impact of the Future Families’ home visiting program provided to 
families with orphans on child, guardian and household wellbeing indicators? These indicators are : 
• HIV testing uptake among caregivers and children
• School attendance among school-aged children
• Caregiver psychosocial wellbeing
• Early childhood development
• Access to social services
Methodology 
Study design: Using a quasi-experimental design, this programme evaluation compared outcomes of the 
recipients of the Future Families programme during its scale up phase, who were previously enrolled 
and received the intervention, with outcomes of newly enrolled recipients before the receipt of the 
intervention. The characteristics of recipients from both arms were matched using the demographic, 
health and socioeconomic status of the guardian. All households who had been previously enrolled for 
approximately 18 months and those who were newly enrolled but had not yet received any services
were invited to participate in the study. Alongside, a waiting-list randomized controlled trial was 
conducted to evaluate the effectiveness of this programme. Beneficiaries who were new to the 
program at the time of enrollment in the study were randomly assigned to receive the program’s full 
complement of services either immediately or approximately 15 months after the baseline survey.
Intervention description: Under the supervision of a qualified social worker, care workers with a
secondary education and recruited from the local community received ongoing training from Future 
Families. These care workers provided monthly home visits to vulnerable families (including households 
with orphans, chronically ill adults, HIV/AIDS-affected family members and/or extreme poverty) in 
Tshwane townships, South Africa. Care workers provide households with information, psychosocial 
support, and access to external services. 
2 
Data collection methods: Future Families expanded their staff and other program resources to 
accommodate planned program expansion, necessitating the use of staggered service delivery to new 
enrollees. This situation provided a natural opportunity to adopt an experimental design, the strongest 
kind of research evidence, while ultimately offering full program services to all enrollees. Beneficiaries 
who were new to the program at the time of enrollment in the study were randomly assigned to receive 
the program’s full complement of services either immediately or approximately 12 months after the 
baseline survey. Alongside this randomized controlled trial (RCT), data was also collected at baseline
from current long-term beneficiaries to facilitate an immediate quasi-experimental post-test 
assessment. 
Study participants were recruited through the Future Families’ community based organization (CBO) 
through trained care workers from the local communities. Data was collected from adult caregivers
(“guardians”) defined as the adult in the home who had primary responsibility for the care of residing 
children. The survey included age-appropriate wellbeing indicators for each child in the home, and 
characteristics and outcome indicators for the guardian and the household. Primary caregivers 
completed their survey through a face-to face interview with a trained interviewer in English or Sepedi.
The interviewer used computer-assisted personal interviewing (CAPI) technology to read and record 
responses. IRB approval for this study was obtained from the Tulane University Institutional Review 
Board in the United States and the University of Limpopo Medunsa Campus Ethics Committee in 
Pretoria. Standard procedures for informed consent, confidentiality, quality assurance and participant 
referral were followed. 
Of the total 822 households enrolled in the Future Families home visiting program, 763 guardians 
completed baseline surveys (93% response rate) in June-August 2014: including 282 guardians who had 
been previously enrolled in the Future Families program (89% response rate) and 481 newly enrolled 
guardians (95% response rate) who had not yet received any services. The quasi-experimental study
used propensity score matching to compare 231 (82%) previously enrolled households to 231 (48%) 
newly enrolled households for an analytical dataset of 462 households. In the RCT, caregivers newly 
enrolled into the FF program during the months of May-June 2014 were randomly assigned to 
participate in the program beginning in September 2014 or waitlisted for participation to occur after the 
follow-up survey, about one year later. The assignment to groups was accomplished using computer 
software designed for random allocation of subjects to groups, and interviewers were blind to 
individuals’ group assignments. Follow up data collection occurred in November 2015 after one year of 
service delivery. While 481 guardians participated at baseline, 431 caregivers completed the follow-up 
survey (providing information on 1123 children) yielding a 90% participant retention at follow‐up.
Data Analysis: In the quasi-experimental study, propensity scored matched data was used to compare 
caregiver and child outcomes between program-participating and non-participating households. 
Multivariate logistic regression models were estimated using PROC SURVEYLOGISTIC in the SAS 
statistical software to adjust the standard errors for clustering of children by household. The predictor of 
primary interest was previous program participation, using an intent-to-treat design. The logistic 
regression models included basic demographics that may influence outcomes and all categorical 
variables were entered into the model as mutually exclusive dichotomous dummies. In the RCT study, 
3 
differences between the treatment groups by sample characteristics were assessed at baseline using 
chi-square tests for categorical variables and t-tests for continuous variables in SAS version 9.3. Logistic 
mixed effects models tested group differences for evaluation outcomes using the MELOGIT command in 
Stata 14 and an intent-to-treat group assignment. All models included control variables for caregiver’s 
gender and the child’s age at baseline.
Findings
The summaries of key findings from both study components are presented separately below.
Key findings from the quasi-experimental analysis: 
 Caregiver HIV Testing: The percentage of caregivers who had ever been tested for HIV was 
significantly higher in households previously enrolled in the FF home visiting program than newly 
enrolled households (78% vs. 69%, Chi-square = 4.46, p= 0.035). There was no significant difference 
in HIV testing in the previous year between the two groups (46% vs. 42%, Chi-square = 0.71, p=0.40). 
 Child HIV testing: Previously enrolled households had a significantly higher percentage of their 
children tested for HIV compared to newly enrolled families (44% vs. 30%, Chi square =29.56, 
p=0.0001). Receiving FF services increased the odds of a child being tested by 97% (OR= 1.97, 95% CI 
= 1.34–2.92) compared to children from similar households that had not yet received services. In 
addition, the difference in testing was more pronounced for orphans, with 49% of orphans from 
previously enrolled households tested compared to 24% among children from newly enrolled 
households. Orphans living in program-participating households had double the odds of being 
tested compared to non-orphans living in similar non-participating households (OR = 2.12, 95% CI = 
1.00–4.47). 
 School Attendance: School age children from program-participating households attended school 
more regularly compared to children from newly enrolled households (59% vs. 52%, Chi￾square=4.20, p=0.04). In addition, a significantly higher percentage of adolescents aged 15 and older 
from program-receiving households (63%) attended school more regularly as compared to 
teenagers from newly enrolled households (50%) (Chi-square=5.81, p=0.02). 
 Caregiver Psychosocial Wellbeing: The difference in percentages of basic support available to 
caregivers in program-participating and non-participating households was statistically insignificant
(66% vs. 63%, Chi-square=0.34, p=0.56). In addition, no significant difference in caregivers’ 
experiences of negative feelings was found between two groups of households (47% vs. 52%, Chi￾square=1.05, p=0.31). 
 Early Childhood Development: The percentage of children aged 1-5 years with Road to Health clinic 
cards observed by the interviewer and indicating full age-appropriate immunization was significantly 
higher in newly enrolled households (37%) as compared to program-receiving households (24%) 
(Chi-square=7.43, p=0.01). There was no significant difference between groups regarding 
engagement in stimulating activities (77% vs. 83%, Chi-square=1.69, p=0.19).
4 
 Access to Social Services: Among households eligible to receive the Child Support Grant (CSG), there 
was no significant difference in family’s access to the grant as an equal numbers of caregivers 
(approx. 80%) in both enrollment groups reported receiving the CSG (Chi-square=0.07, p=0.79). The 
percentage of eligible households receiving Old Age Grants (OAG) was 11% higher in previously 
enrolled households, but this difference was not statistically significant (93% vs. 84%, Chi-square= 
1.91, p=0.17). 
Key findings from the randomized controlled trial: 
 Caregiver HIV Testing: Caregivers enrolled in the home visiting program were three times more 
likely to have been tested for HIV in the preceding year than caregivers in the wait group (OR = 3.3; 
p = 0.001). This program effect was particularly prominent among intervention caregivers younger 
than 35 years, as they were ten times more likely to have been tested in the previous year than their
wait-listed counterparts (OR = 10.3, p = 0.01). No significant differences were found among lifetime 
testing rates between groups.
 Child HIV testing: No significant program effect was found for lifetime rates of HIV testing among 
children. Child HIV testing in the previous year was only assessed at follow up and showed that a 
significantly higher percentage of children enrolled in the program were recently tested than wait￾listed children (OR = 1.37; p = 0.02). However, due to the lack of longitudinal data on child HIV 
testing, this effect cannot be attributed to the program. Children whose caregivers (1) participated 
in the home visiting program and (2) had been tested in the previous year, were almost five times 
more likely to have been tested at follow-up than other children (OR=4.7, 95% CI=1.2-18.3). 
 School Attendance: Positive trends were observed in regular school attendance among all school￾age children in both intervention and wait groups, yet no significant program effect was found for 
this outcome. However, children aged 15 and older in the intervention group were 6 times more 
likely to attend school regularly than those in the wait list group (OR = 6.5, p = 0.015).
 Caregiver Psychosocial Wellbeing: There was no significant difference between intervention and 
wait group caregivers in overall basic support received or overall negative feelings scores. However, 
there was a significant program effect on the frequency of negative feelings. That is, the frequency 
of negative feelings experienced by caregivers dropped by 21% in the intervention group, and 10% 
in the wait group (OR=0.05; p=0.038). 
 Early Childhood Development: No program effect was found for immunization rates or stimulating 
activities for children under the age of 5. 
 Access to Social Services: While there were positive trends observed in the percentage of children 
with verified identity documentation in both intervention and wait groups, no significant program 
effects was found for this indicator. There was no significant program effect found for the uptake in 
child and old age support grants among intervention households. 
5 
Limitations 
Study findings must be considered within the context of the evaluation study design, which has several 
important limitations. The quasi-experimental study establishes a strong correlation between program 
participation and outcome measures but cannot assign causality as with a truly experimental design. In 
addition, while propensity score matching is very effective in creating similar comparison groups, it does 
not necessarily address potential participation bias associated with needier households enlisting in the 
program earlier. Thus, the households in the intervention group may differ from those enrolled later 
with regard to factors not included in the analysis that could potentially affect outcomes. Further, the 
questionnaire asked only if the child had ever been tested for HIV and not if the child had been tested 
since enrollment in the Future Families home visiting program. If child testing is associated with early 
enrollment in the Future Families home visits, the program effect may be inflated. 
For both the quasi-experimental and RCT, social desirability bias may also affect reports of HIV testing in 
the intervention group, given the program’s emphasis on education and prevention. The lack of 
universally accepted outcome measures that capture the range of services provided by home visiting 
organizations such as Future Families is another key limiting factor in program evaluations. While the 
study used measures with a history of testing and successful application in similar populations and 
contexts, alternative measures may have produced different results, especially for the outcomes 
pertaining to early childhood development and access to social services.
Conclusions and Recommendations
The evaluation study presented in this report was designed to assess the effectiveness of Future 
Families’ holistic, family-based approach to addressing the needs of OVC through paraprofessional home 
visiting in urban township communities in Tshwane, South Africa. In so doing, the study contributes to 
strengthening the evidence base for effective family-centered programming by demonstrating that 
providing vulnerable families with one-on-one support and education from trained care workers has the 
potential to increase HIV testing uptake, improve caregivers’ psychosocial wellbeing, and increase 
school attendance among vulnerable youth. The following list of recommendations has been drawn 
from the evaluation results; rationale and further details for each recommendation is provided in the 
body of this report: 
Recommendation 1: Improve efforts to better reach the most vulnerable populations, including HIV 
positive caregivers & their children in order to bolster HCT promotion. 
Recommendation 2: Explore ways to improve school attendance among enrolled children and 
adolescents, such as raising awareness about the importance of attending school every day from an 
early age or non-monetary incentives to recognize children with regular attendance. 
Recommendation 3: Strengthen efforts to improve caregiver psychosocial health through structured, 
time-limited interventions and enhanced targeted care worker support to those living with HIV. 
Recommendation 4: The training course for the FF care workers should include methods and content on 
how to address HIV stigma in the community to avoid further stigmatization or discrimination of 
program enrollees.
6 
Recommendation 5: Establish referral procedures that are sensitive to the research context by working 
with the implementing partner and ensuring trained, sensitive interviewers. 
Recommendation 6: Maintain regular communication with local stakeholders about evaluation 
processes and results in order to address impediments to program success and to ensure the local 
relevance and the broader utilization of findings.
Recommendation 7: Adhere to a culturally sensitive research approach, by ensuring cultural and 
linguistic competence in the development of consent/assent language and procedures. 
Recommendation 8: To help tailor programming, future research should focus on identifying the aspects 
of home visiting intervention models that are most influential for HCT and other services uptake in 
different contexts. Future studies may help to identify the key attributes of home visiting that can most 
effectively link at risk and vulnerable families to HIV testing, grant access and other social services. 
7 
Project Background
With an estimated 5.6 million people living with HIV in South Africa, the AIDS epidemic has led to a large 
number of orphaned or otherwise vulnerable children (OVC) and families. Of South Africa’s 18 million 
children, nearly 21% have lost one or both parents (UNAIDS, 2012); and with an HIV rate of 28% among 
pregnant women, many more likely reside with HIV positive parents (Department of Health, 2010). Both 
psychosocial challenges and economic hardship are disproportionately common in homes where an 
orphan or HIV-infected adult resides (Deininger, Garcia, & Subbarao, 2003). Children affected by AIDS 
have an increased likelihood of psychological challenges, sexual risk behavior, and poorer school 
attendance (UNICEF, 2003). Effective responses to addressing the needs of OVC must include efforts to 
strengthen the capacity of families that care for them (Schenk, Michaelis, Sapiano, Brown, & Weiss, 
2010). OVC caregivers may suffer emotional distress, have limited social support and experience family 
discord. Research reports a heightened risk of mental health problems among OVC caregivers, including 
grandparents and HIV positive parents (Ice, Yogo, Heh, & Juma, 2009). Addressing the needs of OVC 
caregivers is integral to a comprehensive, family-based approach. OVC programming principles 
therefore emphasize the importance of interventions that endeavor to enhance the psychosocial well￾being and caregiving capacity of OVC guardians (UNICEF, 2003). 
The purpose of this study was to evaluate the impact of the Future Families care worker home visiting 
program offered to vulnerable families through trained and mobilized care workers. Supported by the 
President of the United States’ Emergency Plan for AIDS Relief (PEPFAR) through USAID Southern Africa, 
Future Families is a South African non-governmental organization providing a range of services to 
vulnerable children and their families, affected and infected by HIV/AIDS. Future Families was 
established in 2010 and now works in six townships located within the greater Tshwane area (also 
known as Pretoria) of South Africa. Specifically, they operate in Sunnyside, Eersterus, Mamelodi 
(Central), Mamelodi East, Nellmapius/Mamelodi South and Olievenhoutbosch. As of early 2016, Future 
Families was providing community care services to the families of over 8,900 children in the Greater 
Tshwane Area (Future Families, 2016). However, at the time of evaluation planning, Future Families was 
just expanding into the Nellmapius/Mamelodi South community and intended to enroll as many as 800 
new families during an estimated three-month period from May-July 2014 in addition to those already 
enrolled in the Mamelodi Central and Mamelodi East townships. 
To implement their services, Future Families trains and mobilizes care workers from the local 
community to provide monthly home visits to vulnerable families. Minimum qualifications require that 
care workers completed secondary education and are recruited from the community. They work under 
the supervision of a licensed social worker and receive ongoing training from Future Families. 
Additionally, they receive enhanced support related to HCT, including an HIV prevention training module
with refresher trainings, targeted encouragement from supervisory staff, and community resource 
information designed specifically to promote testing uptake. During the initial home visits, a customized 
action plan is developed to meet the needs of each family, including material support, counseling, and 
referral for a variety of health and social services. 
8 
Evaluation Purpose and Questions
Purpose
The evaluation of Future Families consisted of two components with distinct purposes:
(1) a quasi-experimental study that aimed to provide an immediate outcome evaluation to inform 
programming and;
(2) A one-year randomized controlled trial study designed to provide an impact evaluation of the 
home visiting program. 
Evaluation Questions
Both studies were designed to provide a range of child, guardian and household wellbeing indicators 
that have been endorsed by the United States Agency for International Development (USAID) as 
important and expected outcomes of OVC programs: 
• HIV testing uptake among caregivers and children
• School attendance among school-aged children
• Caregiver psychosocial wellbeing
• Early childhood development
• Access to social services
The quasi-experimental study investigated to what extent outcomes differed between newly enrolled 
beneficiaries and established beneficiaries, while the RCT investigated to what extent changes in these 
indicators among the intervention group were attributable to the Future Families program.
Evaluation Design and Methodology 
Study setting and population
The study was conducted in three contiguous peri-urban townships of the city of Pretoria/Tshwane, 
Mamelodi, Mamelodi East and Nellmapius, located in the Gauteng Province of South Africa. Future 
Families program services targeted households containing children who have lost one or both parents, 
chronically ill adults, and/or households in extreme poverty. Prior to the study in May – June 2014, 
potential beneficiaries were visited in their homes by a care worker who collected detailed information 
about the family for program (not study) eligibility determination. All households who had been 
previously enrolled in Future Families programs for approximately 18 months (receiving an average of 
11 home visits over that time based on Future Families monitoring data), and those who were newly 
enrolled but had not yet received any services were invited to participate in the study. All interested 
caregivers were selected/recruited based on the following participant inclusion criteria:
• Aged 18 or over at the time of study enrollment;
9 
• Enrolled in a selected Future Families site;
• Willing and able to consent to study participation
Beneficiaries of Future Families services who were not able to give informed consent due to known or 
recognizable cognitive or psychiatric impairment were excluded from the participant recruitment stage.
Stakeholder engagement
Engagement of local stakeholders, including implementing organizations and program partners, was 
critical to defining the scope of the evaluation and designing survey methods that are contextually 
appropriate and responsive to the needs of the funders, program partners and participants. While it was 
important that the evaluation team operated independently from implementers and other local key 
stakeholders, it was essential to have open communication about roles, the purpose of the evaluation 
and its protocol and potential benefits. As a result, from the outset of the study, the evaluation team 
maintained frequent communication with the local stakeholders ensuring a well-developed and 
informed understanding of the program and its strengths and limitations among all stakeholders 
involved. For example, implementation partners were invited to assist with identifying key expected 
outcomes to ensure that the evaluations study accurately captured the program efforts. In addition, 
Tulane conducted an in-depth workshop with FF staff in 2014 (see ‘Dissemination’ section in this report)
to explain the study, gather feedback and answer questions. Based on input from stakeholders, new 
indicators were added to the evaluation instruments to more fully reflect the program priority 
outcomes. Engaging the program implementers from the outset of the evaluation project helped ensure 
that program staff were well-informed prior to the evaluation beginning.
Continued engagement of local stakeholders was maintained throughout the research process to ensure 
local relevance and the broader utilization of evaluation findings. For example, the results of the quasi￾experimental analysis were shared with the FF to inform program implementation over the following 
year. Baseline results were used by partners to prioritize focal areas for optimal program impact, 
including HIV testing of children, immunizations, promotion of early childhood development program 
enrolment and school attendance, and birth certificate uptake and verification. Post-evaluation 
workshops and other forums were organized to help implementing partners and other local 
stakeholders understand and use study results (see ‘Dissemination’ section in this report). 
Quasi-experimental post-test study: sampling and analysis strategy
The quasi-experimental study was designed to indicate potential program effects by comparing the data 
from families’ already receiving Future Families services (i.e. established enrollees or beneficiaries) with 
the data from families newly enrolled but not yet receiving home visits. 
Data was collected from current long-term beneficiaries already enrolled in Future Families home 
visiting program for approximately 18 months and those who were newly enrolled but had not yet 
received any services. Prior to the data collection, potential beneficiaries were visited in their homes by 
a care worker who collected detailed information about the family for eligibility determination. 
Households containing the most vulnerable children were prioritized for program enrollment. Of the 
10
total 822 households enrolled in the Future Families home visiting program, 763 guardians from 763 
households (93% response rate) completed baseline surveys: including 282 guardians who had been 
previously enrolled in the Future Families program (89% response rate) and 481 newly enrolled 
guardians (95% response rate) who had not yet received any services. 
While the program eligibility criteria were the same for both groups, previously enrolled households 
were generally more disadvantaged than newly enrolled households. Propensity score matching was 
thus used to identify a subset of newly enrolled households most similar to those previously enrolled to 
create a quasi-experimental design. Propensity scores were calculated in SAS 9.3 software for each 
household using logistic regression predicting previous program participation. Matching was conducted 
using Coca-Perraillon’s (2007) greedy match macro without replacement and a caliper of 0.01. This
resulted in 231 (82%) previously enrolled households matched to 231 (48%) newly enrolled households 
for an analytical dataset containing 462 households. Within the 462 households included in the analysis, 
there were 1324 children: 644 in the intervention households and 680 in control households. 
Impact evaluation sampling and analysis strategy (Randomized controlled trial) 
The RCT was designed to assess the impact of Future Families home visiting services on beneficiaries 
over a one year period. RCT data was collected in two rounds with the same beneficiaries over a one 
year period. Beneficiaries who were new to the program at the time of enrollment in the study were 
randomly assigned to receive the program’s full complement of services either immediately or 
approximately 15 months after the baseline survey. School-aged children in both groups of newly￾enrolled beneficiaries were offered homework assistance classes at a Future Families facility, to ensure 
that even the control group received some benefit from enrollment from the start. The assignment to 
groups was accomplished using RANUNI function in the SAS statistical software designed for random 
allocation of subjects. Baseline data from these participants were collected within four weeks of their 
enrollment and prior to program commencement between July – August 2014. Follow-up surveys were 
completed in November of 2015 after one year of service delivery to the intervention group, but prior to 
commencement of service delivery to the wait-listed control group. 
A total of 484 newly enrolled caregivers were recruited to participate in the study with one refusal. At 
baseline, the 483 participating caregivers provided data on 1,245 children under the age of 18. At 
follow-up, 431 caregivers completed surveys (216 intervention subjects and 215 control subjects) with 
one refusal and 51 who had either relocated or were unavailable after three visits for a loss to follow-up 
of 10.8%. Among those caregivers completing follow-up surveys, 22 were not the same as the caregiver 
who completed the baseline survey, but cared for the same children. Evaluation findings in this report 
pertain to the final sample of 431 households.
Differences between the treatment groups by sample characteristics were assessed at baseline using 
chi-square tests for categorical variables and t-tests for continuous variables in SAS version 9.3. Logistic 
mixed effects models tested group differences for HIV testing using the MELOGIT command in Stata 14 
using intent-to-treat group assignment. All models included control variables for the age at baseline and 
gender of the caregiver. Terms for the three townships were tested but eliminated because they were 
non-significant and did not improve model fit.
11
Table 1. Rounds of data collection and analysis
Study Participant 
Recruitment
Data Collection Data Analysis
Quasi￾experimental
May – June 2014 Surveys completed 
in July 2014
Analysis of data collected in 2014 included: 
• Assessment of the baseline data of 
newly enrolled beneficiaries to assess 
their current situation with respect to 
outcomes of interest; 
• Comparison of outcomes between the 
new and longstanding beneficiaries to 
discern potential program effects.
Randomized 
Controlled 
Trial
May – June 2014 Baseline (Round 1): 
surveys completed 
in July – August 
2014
Follow-up (Round 
2): surveys 
completed in 
November 2015
Analysis of follow-up data collected in 
2015, included: 
• Change over a one year period in the 
sample enrolled in 2014, and 
comparisons between outcomes by 
study group (based on the timing of 
service delivery);
• Potential risk and protective factors 
predictive of outcomes of interest.
Data collection methodology
Survey data were collected from adult primary caregivers (“guardians”) of children residing in the 
household after obtaining informed consent in face-to-face interviews using computer-assisted personal 
interviewing (CAPI) technology to read and record responses. Participants were interviewed at home in 
English or Sepedi by a trained interviewer not affiliated with Future Families and blind to study 
allocation. All study instruments were translated into Sepedi and verified by three professional 
translators. The survey asked guardians to report on age-appropriate wellbeing indicators for each child 
in the home, including age, gender and the child’s HIV testing history. Caregiver demographic 
information, HIV testing and status, and health indicators were also collected. IRB approval for this study 
was obtained from the Tulane University Institutional Review Board in the United States and the 
University of Limpopo Medunsa Campus Ethics Committee in Pretoria. Standard procedures for 
informed consent, confidentiality, quality assurance and participant referral were followed and are 
discussed in more detail in ‘Ethical considerations and assurances’ section. 
Key outcome measures
The surveys used in this study incorporated a range of child, caregiver and household well-being 
indicators that have been recommended by the United States Agency for International Development as 
important and expected outcomes of OVC programs (PEPFAR, 2012). The survey asked participants to 
report on age-appropriate wellbeing indicators for each child in the home, and collected characteristics 
and outcome indicators for the guardian and the household. A set of primary outcomes reported in this 
12
article includes HIV testing, school attendance, caregiver’s psychosocial well-being, early childhood 
development, and access to social services. 
• HIV Testing was assessed by asking caregivers if they and the children in their household had ever 
been tested for HIV and when they received their most recent test results. Caregivers were also 
asked if they were tested for HIV in the last year. Response options were “Yes”, “No”, and “Don’t 
know”.
• School Attendance was measured by asking caregivers if the school-aged child (children) under 
their care were absent from school in preceding week (i.e. “During the last full school week, did 
[child’s name] miss any days (not including holidays)?”). Response options were “Yes”, “No”, and 
“Don’t know”. 
• Caregiver Psychosocial Wellbeing was measured by asking caregivers to indicate the basic social 
and emotional support available to them (e.g. “Do you have someone in your life to turn to for 
suggestions about how to deal with a personal problem?” or “Do you have someone in your life to 
help with daily chores if you were sick?”), with response options being “Yes” and “No”. In addition, 
participants were asked about the frequency of experiencing negative feelings (“How often do you 
have negative feelings such as sad mood, despair, or depression?”, with response options ranging 
from “Never” (0) to “Always” (4). 
• Early Childhood Development was assessed by measuring two components: child immunization 
and engagement in any stimulating activities. To measure immunization, caregivers were firstly 
asked if the child (children) in their care had a health clinic or Road to Health Card 
(unverified/reported card). Caregivers were then asked to produce the health care card as evidence 
(verified card). While the possession of a health care card was not an outcome itself, the 
immunization status of children was only recorded for those children (aged 5 years and younger) 
whose caregivers could produce a card (i.e. interviewers recorded the immunizations listed on the 
card by using response options “Yes” and “No”). In addition, caregivers were asked to indicate if 
the child (or children) in their care recently engaged in any stimulating activities (e.g., read books or 
looked at picture books, listened to stories, sang songs, played games, etc.) with any household 
member over 15 years of age. Response categories were “Yes” and “No”. 
• Access to Social Services was measured by asking caregivers to indicate if the child (or children) 
under their care had birth certificates or identity documentation, as well as a series of questions to 
determine if they were eligible for and receiving a Child Support Grant or a Foster Care Grant. The 
survey also asked whether anyone in the household was in receipt or had applied for an Old Age 
Grant. Response categories to all these questions were “Yes” and “No”. 
Data analysis
As presented in Table 1 above, the data analysis plan included a preliminary analysis of baseline data 
and more advanced analysis to establish the difference between the previously and newly enrolled 
households (a quasi-experimental component) as well as the intervention and waitlist groups (a RCT 
component). Descriptive analyses included frequencies of categorical outcomes, and means and/or 
13
medians of continuous outcomes to depict the characteristics of the OVC, caregivers, and households in 
the study. These basic analyses also illustrated areas of met and unmet need to guide Future Families 
programmatic efforts. 
In the quasi-experimental study, propensity scored matched data was used to compare caregiver and 
child outcomes between program-participating and non-participating households. Multivariate logistic 
regression models were estimated using PROC SURVEYLOGISTIC in the SAS statistical software to adjust 
the standard errors for clustering of children by household. The predictor of primary interest was 
previous program participation, using an intent-to-treat design. The logistic regression models included 
basic demographics that may influence testing and all categorical variables were entered into the model 
as mutually exclusive dichotomous dummies. In the RCT study, differences between the treatment 
groups by sample characteristics were assessed at baseline using chi-square tests for categorical 
variables and t-tests for continuous variables in SAS version 9.3. Logistic mixed effects models tested 
group differences for evaluation outcomes using the MELOGIT command in Stata 14 using intent-to￾treat group assignment. All models included control variables for caregiver’s gender and the child’s age 
at baseline. Terms for the three townships were tested in both evaluation components but eliminated 
because they were non-significant and did not improve model fit. In all analyses, significance probability 
values of .05 or less on the variables ‘program’ (for the post-test) and ‘time’ (for the pre and post) were 
used to capture program effects. 
Ethical considerations and assurances
The study team was acutely aware of the importance of ethical standards and a number of best practice 
procedures were in place to ensure the rights and protection of participants. Prior to the 
commencement of data collection activities in June 2014, the full research protocol and survey 
instruments were approved by the Tulane University Institutional Review Board in the United States and 
the University of Limpopo Medunsa Campus Ethics Committee in Pretoria (ethical approvals obtained in 
June 2014). Ethical guidelines for this study were developed according to the South African Department 
of Health Ethical Guidelines (Department of Health, 2004, 2006), Human Science Research Council’s 
guidelines on research with orphans and vulnerable children (HSRC, 2010) and other reputable 
resources for the ethical conduct of research with adolescents (Petersen & Leffert, 1995; Schenk & 
Williamson, 2005). The randomized control trial was registered with ClinicalTrials.gov [NCT02395328].
The following procedures were strictly adhered to in order to ensure ethical implementation of the 
study:
Informed consent: Written informed consent was obtained from all adult participants prior to beginning 
the interview from field workers trained in strict ethical procedures. Adult respondents were asked to 
provide consent to participate in the survey when they were first approached at their home residence 
by fieldworkers. In the RCT study, written informed consent was obtained from all adult participants 
prior to both baseline and follow-up interviews to ensure continued voluntary and informed 
participation.
14
Voluntary participation: As part of the consent and assent procedures, the interviewer explained clearly 
before the questioning began that the respondent’s participation is entirely voluntary and that the 
respondent has the right to refuse to answer any question or any part of any question that he or she did 
not wish to respond to. In addition, all potential participants were reminded that they had the right to 
terminate the interview at any time. Finally, they were made aware at the outset that their decision to 
participate or not in the study did not affect their eligibility to receive services from the programs then 
or in the future.
Privacy and confidentiality: All data collection rounds were completed by interviewers from a research 
team not affiliated with Future Families. Interviewers visited families in their home and conducted 
interviews in private and without the presence of program affiliates in caregiver’s local language using 
the structured survey instrument described above. Prior to interviews, participants were asked to select 
a private room, area in the yard, or other space where s/he was comfortable, free of observers, and 
could not be overheard. All participants were informed of provisions to ensure confidentiality, namely 
that that their answers would be held in strict confidence and stored in the different place as their 
contact information. 
Protection of data: All beneficiary data files that provide the information linking individuals to their 
unique identifiers were password-protected and kept in locked cabinets accesible only to the Tulane 
Univerity senior researchers. Similarly, the data for tracking sheets, which included contact and 
identifying information, was captured on a different database than the actual survey data. This data is 
also password protected with access limited to lead research team members. 
Deviations and adjustments
No major deviations from or adjustments to the protocol occurred. There were, however, several 
additions to the research instruments for the RCT study that were incorporated into the previously 
approved baseline questionnaire to make it suitable for follow up interview purposes. More specifically, 
changes to the caregiver survey included the addition of questions regarding exposure to and 
experiences with the intervention under study (care worker visiting program), as well as questions 
regarding services the respondent may have received from community organizations or programs. 
Other newly included questions asked about caregiver’s ability to pay for unexpected household 
expenses; caregiver's attitude towards corporal punishment in the home or at school; whether the 
caregiver has identified someone to care for the children if something should happen to him/her; 
whether the caregiver was forced to relocate due to activities known to have occurred in the community 
in the past year; and whether children who have ever been tested for HIV have had a test in the past 12 
months. Further, the updated survey ascertained whether a new caregiver wass caring for the children 
enumerated at baseline and the circumstances of such. It also determined which of the baseline children 
were still under care and whether there were any new children who have moved into the home since 
baseline. Finally, one survey item was deleted that explored receipt of material assistance and accessing 
referrals. These changes received IRB approval prior to follow up data collection.
15
Procedures to ensure high quality data
Several procedures were followed throughout the evaluation process to ensure high quality data: 
Survey Preparation: All survey instruments were translated in Sepedi through use of a local professional 
translation company. Two different translators independently translated the instruments, and a third 
translator then compared and rectified the translations to ensure accuracy. Translated surveys were 
reviewed by local program staff for clarity and pilot tested among a small sample of previously enrolled 
participants.
Survey Modality: Primary caregivers completed their survey through a face-to face interview with a 
trained interviewer. The interviewer used computer-assisted personal interviewing (CAPI) technology to 
read and record responses. All questions and responses were simultaneously displayed on the screen 
and the respondent's answers were recorded by the interviewer on the touch screen. The data were 
automatically stored in the database and encrypted, thereby eliminating the need for double capture of 
surveys and other time-consuming data entry error control processes. It was also programmed to allow 
for system checks and to do auto-skips for the various applicable questions.
Tracking Form: Procedures to reduce loss to follow-up were also be in place. After completion of the 
baseline survey, primary caregivers were asked to complete a Household Tracking Sheet (see Appendix 
C) which solicited information such as caregiver surnames, contact numbers, GPS coordinates of the
household location, and other identifying information. This information was collected to facilitate 
follow-up data collection. Respondents consented to this portion separately and were reassured that 
contact information would not to be stored with their questionnaires. Future Families program 
registers also served as a secondary source for tracking participants, as they contained contact 
information for all beneficiaries and Future Families staff assisted with keeping the registers up-to-date.
Training and Field Procedures: All fieldwork staff was culturally and linguistically representative of the 
targeted respondents and received a thorough 7-day training pertaining to the questionnaires and 
ethical conduct. Throughout the data collection process, the research team engaged in a range of quality 
control measures that met national and international social research standards. In addition to 
comprehensive training of field staff, these included weekly fieldwork review meetings and dedicated 
quality control teams that checked every paper and electronic questionnaire handed in from the field.
16
Findings and Limitations
The following sections present findings separately for the key outcomes of the quasi-experimental and 
RCT studies. The presentation of the findings of the surveys will be articulated on two levels: the 
caregiver (HIV testing, psychological wellbeing, access to social services) and the child (HIV testing, 
school attendance, early childhood development). 
Key findings and limitations of the quasi-experimental study
The purpose of the quasi-experimental study was to compare a sample of previously enrolled caregivers 
to a sample of newly enrolled caregivers in a post-test propensity score matched analysis that aimed to 
provide immediate information to inform program implementation. While the program eligibility criteria 
were the same for both groups, previously enrolled households were generally more disadvantaged 
than newly enrolled households. For example, previously enrolled households tended to have guardians 
with less education, more adults with a chronic illness (self-reported sickness for three months of the 
past year), be food insecure, have fewer assets and contain more orphans (see Thurman et al., 2016 for 
a more detailed discussion of the quasi-experimental study). Propensity score matching was thus used 
to identify a subset of newly enrolled households that could act as a similar comparison group for the 
established beneficiaries. Propensity score matching resulted in 231 (82%) previously enrolled 
households matched to 231 (48%) newly enrolled households for a final analytic sample of 462 
households.
Table 2 below lists the breakdown of the sample of caregivers by gender and group and the breakdown 
of the sample of children by gender, group and age. Of the total caregiver sample (n=462), the vast 
majority were female (89.83%). Of the total child sample (n=1,324 children), 680 (51.36%) were from 
previously enrolled families and 644 (48.64%) were from newly enrolled families. In terms of age, 349 
(26.36%) children were under the age of 5, including 178 (51%) in the families already receiving FF 
services and 171 (49%) in newly enrolled households. The majority of children were of school age (903 
or 68.20%), with similar numbers coming from previously enrolled and newly enrolled households (460 
or 50.94% and 443 or 49.06%, respectively). Gender distribution was similar across all age and 
enrollment groups.
Table 2. Sample description from the quasi-experimental component of evaluation
Characteristics Group Previous enrollees New enrollees Total sample
N % N % N %
Caregivers All 231 50.00 231 50.00 462 100
Men 21 9.09 26 11.26 47 10.17
Women 210 90.91 205 88.74 415 89.83
Children All 680 51.36 644 48.64 1324 100
17
Boys 328 48.24 318 49.38 646 48.79
Girls 352 51.76 326 50.62 678 51.21
Children under 
5 years of age
All 178 51.00 171 49.00 349 100
Boys 90 50.56 94 54.97 184 52.72
Girls 88 49.44 77 45.03 165 47.28
Children 1 to 5 
years of age
All 195 52.70 175 47.30 370 100
Boys 101 51.79 91 52.00 192 51.90
Girls 94 48.21 84 48.00 178 48.10
School age 
children (6 to 
17)
All 460 50.94 443 49.06 903 100
Boys 215 46.74 211 47.63 426 47.18
Girls 245 53.26 232 52.37 477 52.82
The following sections outline results on the key outcome indicators including visual presentation of 
percentages for different caregiver and child groups.
Outcome 1: HIV Testing
HIV testing among caregivers was assessed with two indicators: whether they had received an HIV test 
in the past year and whether they had received an HIV test in their lifetime. As seen in Figure 1, 78% of 
caregivers from previously enrolled households and 69% from newly enrolled households had ever been 
tested for HIV. This difference in lifetime HIV testing was found to be statistically significant (Chi-square 
= 4.46, p= 0.035) indicating that the percentage of caregivers who had ever been tested for HIV is 
significantly higher in the households already enrolled in FF home visiting program. With respect to the 
second HIV testing indicator – HIV testing in the previous year – there was no statistically significant 
difference between new and old program enrolees (Chi-square = 0.71, p=0.40, see Figure 2). 
Figure 1. Caregivers ever tested for HIV
78%
69%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Established enrollees New enrollees
Percentage of caregivers ever tested for 
HIV (N=462) 
46% 42%
0%
10%
20%
30%
40%
50%
60%
70%
Established enrollees New enrollees
Percentage of caregivers tested for HIV in 
the last year (N=462)
Figure 2. Caregivers tested for HIV in last year
18
In the sample of 1,324 children, 44% of children from previously enrolled households had been tested 
for HIV, while 30% of children from newly enrolled households had been tested (Figure 3). There was a 
statistically significant difference between the two groups – previously enrolled households had a higher 
percentage of their children tested for HIV compared to newly enrolled families (Chi square =29.56, 
p=0.0001). In addition, the difference in testing was more pronounced for orphans, with 49% of orphans 
from previously enrolled households tested compared to 24% among children from newly enrolled 
households. 
Figure 3. Children ever tested for HIV
A more in-depth examination of HIV testing among children presented in a recent publication (Thurman, 
Luckett, Taylor, & Carnay, 2016) found that younger children were more likely to have been ever tested, 
with the odds of being tested falling by 6% for each additional year of life until age 17 (OR = 0.94, 95% CI 
= 0.91–0.96). In terms of the impact of household and caregiver characteristics, children with a male 
guardian had 68% lower odds of being tested than children with a female guardian (OR = 0.32, 95% CI = 
0.14–0.72), while having a guardian younger than 25 years of age reduced a child’s odds of being tested 
by 82% (OR = 0.18, 95% CI = 0.04–0.69). Guardians’ education and marital status did not predict testing, 
and children cared for by someone who is not a biological parent had similar odds of being tested as 
children living with a biological parent. The odds that a child would be tested more than tripled for those 
whose guardian was a biological parent living with HIV (OR = 3.14, 95% CI = 1.77–5.56). Children living 
with a guardian who exhibited accurate knowledge of HIV transmission risks had 70% higher odds of 
being tested (OR = 1.70, 95% CI = 1.13–2.54). 
Receiving FF services increased the odds of a child being tested by 97% (OR= 1.97, 95% CI = 1.34–2.92) 
compared to children from similar households that had not yet received services. When including an 
interaction term for orphan by program participation, orphans living in program-participating 
households had double the odds of being tested compared to non-orphans living in similar non￾participating households (OR = 2.12, 95% CI = 1.00–4.47). Orphan status alone was not significant (OR = 
0.77, 95% CI =0.43–1.39), although the conditional effect for program participation remained significant 
(OR = 1.69, 95% CI = 1.09–2.62).
44%
30%
0%
10%
20%
30%
40%
50%
Established enrollees New enrollees
Percentage of children ever tested for HIV 
(N= 1324)
19
Outcome 2: School Attendance
School attendance among the two study groups was assessed by looking at school absenteeism in the 
preceding week among all school-age children (N=903). As shown in Figure 4, 59% of children aged 6-17 
years from previously enrolled households and 52% from newly enrolled households attended school 
regularly. This difference between the two groups was statistically significant (Chi-square=4.20, p=0.04) 
indicating that children from previously enrolled households attended school more regularly compared 
to the children from newly enrolled households. 
Figure 4. School attendance in preceding week 
among all school-age children
Figure 5. School attendance in preceding week 
among children 15 years and older
Further examination of school attendance rates among the two groups of enrollees showed no 
significance in school attendance rates among the children aged 6 to 12 years. However, there was a 
significant difference in school attendance in previous week among children age of 15 and older. As seen 
in Figure 5, a significantly higher percentage of teenagers from previously enrolled households (63%) 
attended school regularly as compared to teenagers from newly enrolled households (50%) (Chi￾square=5.81, p=0.02). 
Outcome 3: Caregiver Psychosocial Wellbeing
Caregiver psychosocial wellbeing among established and new program enrollees was assessed by 
measuring different types of basic social and emotional support available to caregivers. As shown in 
Figure 6, 66% of caregivers from previously enrolled households reported receiving basic support in 
their lives, while a slightly lower percentage of 63% of caregivers from newly enrolled households 
reported the same. The difference in percentages was statistically insignificant (Chi-square=0.34, 
p=0.56), indicating that a similar amount of basic support was available to newly enrolled and previously 
enrolled caregivers. 
59%
52%
0%
20%
40%
60%
80%
Established enrollees New enrollees
Percentage of school-age children 
attending school regularly (N=903)
63%
50%
0%
20%
40%
60%
80%
Established enrollees New enrollees
Percentage of children aged 15 years 
and older attending school regularly 
(N=342)
20
Figure 6. Caregiver Psychosocial Wellbeing:
Basic Support
Figure 7. Caregiver Psychosocial Wellbeing: 
Negative Feelings
Self-reported experiences of negative feelings was the second indicator measuring caregiver’s 
psychosocial wellbeing. As seen in Figure 7, approximately half of the samples of previously enrolled 
caregivers (47%) and newly enrolled caregivers (52%) reported experiencing negatives feelings 
frequently. The difference between the two groups of enrollees was statistically insignificant (Chi￾square=1.05, p=0.31) supporting similar levels of negative feelings found between the two groups of 
caregivers.
Outcomes 4: Early Childhood Development
Early childhood development outcomes were collected for children ages 1-5 (n=370). Indicators 
included whether all age-appropriate immunizations had been received among children and whether an 
adult had engaged in stimulating activities with the child in the past three days. The percentage of 
children aged 1-5 years with a Road to Health clinic card observed by the interviewer and indicating full 
age-appropriate immunization was significantly higher in newly enrolled households (Chi-square=7.43, 
p=0.01). As seen in Figure 8, the percentage of fully immunized children was 37% in newly enrolled 
households and 24% in previously enrolled households. 
66% 63%
0%
10%
20%
30%
40%
50%
60%
70%
Established enrollees New enrollees
Percentage of caregivers with basic 
support (N=462)
47% 52%
0%
10%
20%
30%
40%
50%
60%
70%
Established enrollees New enrollees
Percentage of caregivers experiencing 
negative feelings frequently (N=462)
21
Figure 8. Early Childhood Development: 
Immunization
Figure 9. Early Childhood Development: 
Stimulating activities
Figure 9 shows caregiver reported percentages of children under five years of age who recently engaged 
in stimulating activities with other household members (e.g. read books or looked at picture books, 
listened to stories, sang songs, played games). Similar numbers of children in both enrolment groups 
were reported to have recently engaged in stimulating activities, the percentages reached 77% of 
current child enrolees and 83% of children in newly enrolled households. The difference between the 
two groups was statistically insignificant (Chi-square=1.69, p=0.19). 
Outcome 5: Access to Social Services
The two groups of enrollees were also compared on two indicators measuring their access to social 
services, including documents and grants important for social protection. Caregivers were asked to 
verify whether the child or children in their household had any identity documentation (e.g. birth 
certificate or ID). Almost identical numbers of caregivers indicated and showed child’s identity 
documents in both enrollment groups (i.e. 77% in previous and 76% in newly enrolled groups). In 
addition, there was no statistically significant difference in families’ access to the Child Support Grant
(CSG) among households determined eligible to receive this grant (Chi-square=0.07, p=0.79); Equal 
numbers of caregivers (80%) in both enrollment groups reported being in receipt of a CSG (Figure 10). 
77% 83%
0%
20%
40%
60%
80%
Established enrollees New enrollees
Percentage of children under 5 years who 
recently engaged in stimulating activities 
(N=370)
24%
37%
0%
10%
20%
30%
40%
50%
Established enrollees New enrollees
Percentage of children 1-5 years with a 
verified health clinic card who were fully 
immunized (N=370)
22
Figure 10. Access to Social Services: Child 
Support Grant by eligibility
Figure 11. Access to Social Services: Old Age 
Grant by eligibility
As seen in Figure 11, the percentage of eligible households receiving Old Age Grants was 11% higher in 
previously enrolled households (93%) as compared to newly enrolled households (84%). However, this 
difference was not statistically significant (Chi-square= 1.91, p=0.17) indicating that both groups of 
enrollees had similar access to social services. 
Limitations of the quasi-experimental design
These findings must be considered within the context of the evaluation study design, which has several 
important limitations. This quasi-experimental study establishes a strong correlation between program 
participation and outcome measures, including HIV testing, but cannot assign causality as with a truly 
experimental design. Furthermore, the questionnaire asked only if the child had ever been tested for 
HIV and not if the child had been tested since enrollment in the Future Families home visiting program. 
If child testing is associated with early enrollment in the Future Families home visits, the program effect 
may be inflated. Social desirability bias may also affect reports of HIV testing in the intervention group, 
given the program’s emphasis on education and prevention. Incorporating follow-up questions about 
the recency of testing and the factors that prompted it would provide useful additional information for 
better understanding HCT practices and distinguishing the program’s effects.
In general, while propensity score matching is very effective in creating similar comparison groups, it 
does not necessarily address potential participation bias associated with needier households enlisting in 
the program earlier. Thus, the households in the intervention group may differ from those enrolled later 
with regard to factors not included in the analysis that could potentially affect outcomes.
Key findings and limitations of the RCT
A total of 484 newly enrolled caregivers were recruited to participate in the RCT study with one refusal. 
The 483 participating caregivers provided data on 1245 children under the age of 18 at baseline. At 
follow-up, 431 caregivers completed surveys (216 intervention subjects and 215 control subjects). One 
caregiver refused and 51 either relocated or were unavailable after three visits for a loss to follow-up of 
93%
84%
0%
20%
40%
60%
80%
100%
Established enrollees New enrollees
Percentage of eligible households 
receiving an Old Age Grant (N=87)
80% 80%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Established enrollees New enrollees
Percentage of children eligible for Child 
Support Grant (under 18 years) that 
applied for or received it (N=1 324)
23
10.8%. Loss to follow-up was similar in both arms, with 11.2% in the wait-listed group and 9.6% in the 
home visit group. In anticipation that some caregivers would pass away or relocate, efforts were made 
to identify changes in the primary caregiver of study children. Among those caregivers completing 
follow-up surveys, 22 were not the same as the caregiver who completed the baseline survey, but cared 
for the same children and thus were interviewed. 
Baseline characteristics of the sample by group assignment are presented below in Table 3. Of the total 
sample of 431 caregivers, 92.11% were female. Caregivers in the study provided information on 1,123 
children (556 in intervention group and 567 waitlist group). With regards to age, 349 (26.36%) children 
were under the age of five (178 or 51% in the families already receiving FF services and 171 or 49% in 
newly enrolled households). Of the 1,123 children in the study, the majority were of school age (903 or 
68.20%), with similar numbers coming from previously enrolled and newly enrolled households (460 or 
50.94% and 443 or 49.06%, respectively). Among the baseline sample of children, 15.7% were single 
orphans, 2.8% were double orphans, 13% were living with a chronic illness, 16.5% lived with a 
chronically ill caregiver, and 3.6% lived in a child or youth headed household. In the sample of all 
children, gender distribution was similar across all age and enrollment groups. No significant differences 
were found between the groups with regards to any of the characteristics presented.
Table 3. Sample baseline characteristics of caregiver and their children by group assignment
Characteristics Group Intervention Wait group Total sample
N % N % N %
Caregivers All 216 50.12 215 49.88 431 100
Men 15 6.94 19 8.84 34 7.89
Women 201 93.06 196 91.16 397 92.11
Children All 556 49.51 567 50.49 1123 100
Boys 298 53.60 288 50.79 586 52.18
Girls 258 46.40 279 49.21 537 47.82
Children under 
5 years of age
All 166 53.55 144 46.45 310 100
Boys 104 62.65 73 50.69 177 57.10
Girls 62 37.35 71 49.31 133 42.90
Children 1 to 5 
years of age
All 158 50.80 153 49.20 311 100
Boys 90 56.96 77 50.33 167 53.70
Girls 68 43.04 76 49.67 144 46.30
School age 
children (6 to 
17)
All 365 48.09 394 51.91 759 100
Boys 184 50.41 199 50.51 383 50.46
Girls 181 49.59 195 49.49 376 49.54
The following sections outline results for the key outcome indicators including visual presentation of 
percentages for caregivers and children from the intervention and control groups.
24
Outcome 1: HIV Testing 
HIV testing uptake among caregivers and children was assessed by asking caregivers at both survey 
rounds if they and children in their care had ever been tested for HIV. Caregivers were also asked if they 
were tested for HIV in the previous year at both rounds. Figures 12 – 17 present results for HIV testing 
uptake among study participants. In the caregiver sample, while lifetime rates were not significantly 
affected by the program enrolment (see Figure 12), recent testing among caregivers was positively 
impacted by the program (see Figure 13). That is, at follow-up, caregivers enrolled in the home visiting 
program were three times more likely to have been tested for HIV in the preceding year than caregivers 
in the wait group (OR = 3.3; p = 0.001).
Figure 12. Caregivers ever tested for HIV
Figure 13. Caregivers tested for HIV in last year
In addition, the program effect was particularly prominent among the intervention group caregivers 
younger than 35 years, as they were ten times more likely (OR=10.3, 95% CI=2.3-45.5) to have been 
tested than the wait-listed caregivers (see Figure 14). The effect of the program did not vary by caregiver 
80% 74%
85% 85%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Intervention Wait group
Percentage of caregivers ever tested for HIV 
(N=431) 
Baseline Follow-up
48% 51%
64%
48%
0%
10%
20%
30%
40%
50%
60%
70%
Intervention Wait group
Percentage of caregivers tested for HIV in the last 
year (N=431) 
Baseline Follow-up
25
gender. This finding is important given that the HIV prevalence rates in South Africa are highest among 
the 25-49 age females (Shisana et al., 2014). 
Figure 14. Caregivers 35 years and younger: tested in last year
In the child sample, while positive trends can be seen in child HIV testing percentages between both the 
intervention and waitlist groups (Figure 15), no significant program effect was found for lifetime rates of 
HIV testing among children. The effect did not vary by child’s age nor gender.
Figure 15. Children ever tested for HIV
This evaluation has focused on the lifetime rates of child HIV testing which was assessed at both the 
baseline and follow up. However, child HIV testing in the previous year was only assessed at follow up. 
As seen in Figure 16, the percentage of children tested for HIV in the previous year was significantly 
higher among children enrolled in the program than the wait-list children (OR = 1.37; p = 0.02). 
However, due to the lack of longitudinal data on child HIV testing, this effect cannot be attributed to the 
program. 
60% 64%
81%
56%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Intervention Wait group
Percentage of caregivers under 35 years of age 
tested for HIV in the last year (N=155)
Baseline Follow-up
37% 35%
44% 45%
0%
10%
20%
30%
40%
50%
Intervention Wait group
Percentage of children ever tested for HIV
(N= 1 123)
Baseline Follow-up
26
Figure 16. Children tested in last year
Figure 17 shows the impact of caregiver testing for HIV on child’s HIV testing uptake indicating that 
children whose caregivers (1) participated in the home visiting programme and (2) had been tested in 
the previous year, were almost five times more likely to have been tested at follow-up than other 
children (OR=4.7, 95% CI=1.2-18.3). These findings highlight the importance of a family-based approach 
to testing as caregiver testing was found to be an important precursor to child testing. 
Figure 17. Impact of caregiver testing on children HIV testing
Outcome 2: School Attendance
As in the quasi-experimental study, school attendance was measured by asking caregivers how often the 
school-age children in their care had missed school in the preceding school week (excluding public or 
school holidays). Although Figure 18 shows positive trends in regular school attendance among all 
30%
24%
0%
10%
20%
30%
40%
Intervention Wait group
Percentage of children tested for HIV in the last 
year (N=1 123)
Follow-up
17% 19%
39%
29%
0%
10%
20%
30%
40%
50%
Intervention Group Wait Group
Impact of program on child HIV testing rates in past 
year by treatment arm and caregiver testing history
Caregiver not tested in past year Caregiver tested in past year
27
school-age children in this study, with increases in both intervention and wait groups of 22% and 20% 
respectively, no significant program effect was found for this indicator. 
Figure 18. School attendance in preceding week among all school-age children
However, a significant program effect in school attendance was found among children aged 15 and 
older. As seen in Figure 19 below, there was a sharp decline in school attendance among the children in 
the waitlist group, and a slight increase among the intervention group. In addition, children aged 15 and 
older in the intervention group were 6 times more likely to attend school more regularly than those in 
the wait list group (OR = 6.5, p = 0.015).
Figure 19. School attendance in preceding week among children 15 years and older
Outcome 3: Caregiver Psychosocial Wellbeing
In both survey rounds, caregiver psychosocial well-being was measured with two indicators, self￾reported basic support they receive and negative feelings they experience. Basic support was assessed 
by asking caregivers whether they had someone in their lives to turn to for help with a personal 
49%
62%
56%
37%
0%
20%
40%
60%
80%
Intervention Wait group
Percentage of children aged 15 years and older 
attending school regularly (N=158)
Baseline Follow-up
52% 54%
76% 74%
0%
20%
40%
60%
80%
Intervention Wait group
Percentage of school-age children attending school 
regularly (N=759)
Baseline Follow-up
28
problem, with daily chores if they are sick, to show them love and affection, and with whom to do
enjoyable things. Figure 20 below shows the percentage of caregivers who reported receiving basic
support at the baseline and follow-up. As seen in the graph, while there was a slight increase in basic 
support reported among intervention group caregivers between baseline and follow-up (i.e. 60% to 
67%), there was no significant difference found between intervention and wait group caregivers in the 
overall basic support they reported.
Figure 20. Caregiver Psychosocial Wellbeing: Basic Support
Self-reported experiences of negative feelings such as sadness, depression and anxiety, was the second 
indicator of caregiver psychosocial wellbeing. Although there was no significant program effect found in 
overall negative feelings reported by caregivers in the intervention and waitlist groups, there was a 
significant program effect on the frequency of negative feelings. That is, caregivers in the intervention 
group had significantly lower odds (cut in half) of having negative feelings (OR=0.05; p=0.038) than those 
in the wait group. As seen in Figure 21, the frequency of negative feelings experienced by caregivers 
dropped by 21% in intervention group, and 10% in the wait group.
Figure 21. Caregiver Psychosocial Wellbeing: Negative Feelings
55%
45%
34% 35%
0%
10%
20%
30%
40%
50%
60%
70%
Intervention Wait group
Percentage of caregivers experiencing negative 
feelings frequently (N=431)
Baseline Follow-up
60% 60%
67%
60%
0%
10%
20%
30%
40%
50%
60%
70%
Intervention Wait group
Percentage of caregivers with basic support 
(N=431)
Baseline Follow-up
29
Outcomes 4: Early Childhood Development
As in the quasi-experimental study, two indicators were used to capture changes in Early Childhood 
Development among children ages 1-5 (N=310), namely immunization and stimulating activities. At both 
rounds, the vast majority of caregivers (96-100%) reported having Road to Health clinic cards for their 
children. However, considerably smaller percentage of caregivers were able to produce the health card 
as evidence (i.e. verified card) upon request (see Figure 22). For children whose caregivers produced 
verified health cards, immunization was assessed by recoding child’s immunization status indicated on 
the health card. As seen in Figure 22 and Figure 23, no program effects were found on the number of 
verified health cards and immunization rates of children under the age of 5.
Figure 22. Child Health Clinic Cards
Figure 23. Early Childhood Development: Immunization
Stimulating activities was the second measure of Early Childhood Development and it was assessed by 
asking caregivers whether, in the preceding 3 days, children under 5 years of age engaged in stimulating 
66% 65%
57% 57%
0%
20%
40%
60%
80%
Intervention Wait group
Percentage of children under 5 years with verified 
health clinic cards (N=310)
Baseline Follow-up
36% 38%
23% 22%
0%
10%
20%
30%
40%
50%
Intervention Wait group
Percentage of children 1-5 years with a verified 
health clinic card who were fully immunized 
(N=310)
Baseline Follow-up
30
activities (e.g. reading books, telling stories, singing songs, playing and taking out of the yard) with other 
household members. Figure 24 below shows steady rates of stimulating activities reported by caregivers 
in the intervention and waitlist groups, at both baseline and follow-up stages of data collection, with no 
program effect found. 
Figure 24. Early Childhood Development: Stimulating activities
Outcome 5: Access to Social Services
To assess the final outcome ‘Access to Social Services’, caregivers were asked whether the children in 
their care had a birth certificate or ID document. At baseline, 95% of caregivers reported that the 
children in their household had identity documentation. However, as reported in Figure 25, a lower 
percentage were actually able to produce the ID document for children in their care upon interviewer 
request (i.e. verified identity document). Although the percentage of children with verified identity 
documentation has increased from the baseline to follow-up surveys in both intervention and wait 
groups, no significant program effects was found for this indicator. 
Figure 25.Access to Social Services: Verified Birth Certificate
80% 80% 83%
76%
0%
20%
40%
60%
80%
Intervention Wait group
Percentage of children under 5 years who 
recently engaged in stimulating activities (N=299)
Baseline Follow-up
75% 77%
85% 85%
0%
20%
40%
60%
80%
Intervention Wait group
Percentage of children with a birth 
certificate/identity document verified by 
interviewer (N=1 123)
Baseline Follow-up
31
The remaining indicators of access to social services included questions of whether caregivers, at both 
rounds of data collection, were in receipt of, or had applied for, the Child Support Grant for any of the 
children in their care, and whether any members in their household were in receipt of, or had applied 
for, the Old Age Grant. Analysis of these indicators was limited to children and household member 
eligible for these grants. With respect to Child Support Grants, Figure 26 below shows a slight decrease 
in the percentage of eligible children who were receiving or had applied for the Child Support Grant. 
There was no significant program effect found for this indicator.
Figure 26. Access to Social Services: Child Support Grant by eligibility
With respect to Old Age Grants, Figure 27 shows slight increases in the percentage of eligible adults who 
either received or had applied for the grant. However, there was no significant program effect found, 
possibly due to a small sample size (N=70).
Figure 27. Access to Social Services: Old Age Grant by eligibility
Finally, at the follow up data collection round caregivers were asked to share their opinions about the 
program effects on their lives. More specifically, caregivers were asked about the home visits they received 
82% 84% 80% 78%
0%
20%
40%
60%
80%
Intervention Wait group
Percentage of children eligible for Child Support 
Grant (under 18 years) that applied for or received 
it (N=1 220)
Baseline Follow-up
83% 87% 92% 88%
0%
20%
40%
60%
80%
100%
Intervention Wait group
Percentage of eligible households receiving an Old 
Age Grant (N=70)
Baseline Follow-up
32
and whether (and how) their lives may have changed as a result of program participation. Responses to these 
questions were obtained from 167 of 216 program receiving caregivers. Eighty-four percent of respondents 
said they had at least a monthly care worker visit and 43% reported bi-monthly care worker visits. The
average duration of the visits was one hour. Figure 28 below reports participants’ perceptions on the overall 
changes in their lives resulting from the program participation. That is, 75% felt that their ability to care for 
their children had improved. Seventy-five percent thought that their overall circumstances had improved, 
and 90% said the overall circumstances of children in the household had improved as a result of the FF home 
visiting program. In addition, approximately 70% of caregivers reported regular discussions of their own HIV 
testing and 76% of children’s HIV testing with care workers during these visits. 
Figure 28. Participants’ perspectives on the program impact on their lives
Limitations of the RCT
Several limitations should be taken into account when considering the generalizability of findings from 
the RCT study. For example, while the estimates of program effect for caregiver HIV testing was based 
on change over the study period, the program effect for child HIV testing in the previous year was 
estimated in cross-sectional models controlling for ever having been tested at baseline which limits 
causal inference for this indicator. The lack of universally accepted outcome measures that capture the 
range of services provided by home visiting organizations such as Future Families is another key limiting 
factor in the program evaluation. While the study used measures with a history of testing and successful 
application in similar populations and contexts, alternative measures may have produced different 
results, especially for the outcomes pertaining to early childhood development and access to social 
services. Finally, this evaluation included only one follow-up assessment, restricting an exploration of 
program effects that emerge over a longer timeframe. A longitudinal evaluations design with several 
long-term follow-up assessments would need to be developed to examine the endurance of program 
effects over time. 
90%
75%
75%
0% 20% 40% 60% 80% 100%
Thought that the overall circumstances of
children in the household had improved
Thought that their overall circumstances had
improved
Thought that their ability to care for their
children had improved
33
Conclusions and Recommendations
The evaluation study presented in this report was designed to address the effectiveness of Future 
Families’ holistic, family-based approach to addressing the needs of OVC by providing caregivers with 
information, psychosocial support, and access to external services through paraprofessional home 
visiting in urban township communities in Pretoria/Tshwane area in South Africa. It included a quasi￾experimental design component and a randomized control trial to provide both initial findings to inform 
immediate programming and provide rigorous evidence of program impact. Overall, this study 
contributes to strengthening the evidence base for effective family-centered programming by 
demonstrating that providing vulnerable families with one-on-one support and education from trained 
care workers have the potential of increasing HIV testing uptake, contributing to improved caregivers’ 
psychosocial wellbeing, and increasing school attendance among vulnerable youth. This adds to the 
growing body of recent research that have emphasized the value of trained and supported home visitors
to OVC families in poorly resourced settings by increasing social grant uptake and improving children’s 
wellbeing (Thurman, Kidman, & Taylor, 2015). 
Summary of key findings
Key findings from the quasi-experimental analysis: 
 Caregiver HIV Testing: The difference in lifetime HIV testing between caregivers already in receipt of 
FF home visiting program and caregivers from newly enrolled households was statistically significant 
indicating that the percentage of caregivers who had ever been tested for HIV is significantly higher 
in the households already enrolled in FF home visiting program. There was no significant difference 
between the two groups of households in HIV testing in the previous year.
 Child HIV Testing: There was a statistically significant difference between the two groups of 
households in child HIV testing suggesting that program-receiving households had a higher 
percentage of their children tested for HIV compared to newly enrolled families. In addition, the 
difference in testing was more pronounced for orphans, with 49% of orphans from previously 
enrolled households tested compared to 24% among children from newly enrolled households. 
Receiving FF services increased the odds of a child being tested by 97% to children from similar 
households that had not yet received services. Orphans living in program-participating households 
had double the odds of being tested compared to non-orphans living in similar non-participating 
households.
 School Attendance: The difference between the two groups of school-age children on school 
attendance indicator was statistically significant indicating that children from program-participating 
households attended school more regularly compared to the children from newly enrolled 
households. In addition, a significantly higher percentage of adolescents aged 15 and older from 
program-receiving households (63%) attended school more regularly as compared to teenagers 
from newly enrolled households (50%).
34
 Caregiver Psychosocial Wellbeing: The difference in percentages of basic support available to 
caregivers in both program-participating and non-participating households was statistically 
insignificant indicating that a similar amount of basic support was available to newly enrolled and 
previously enrolled caregivers.
 Early Childhood Development: The percentage of children aged 1-5 years with Road to Health clinic 
cards observed by the interviewer and indicating full age-appropriate immunization was significantly 
higher in newly enrolled households (37%) as compared to program-receiving households (24%). In 
terms of stimulating activities indicator, similar numbers of children in both enrolment groups were 
reported to have recently engaged in stimulating activities, and the difference between the two 
groups was statistically insignificant.
 Access to Social Services: There was no statistically significant difference in family’s access to Child 
Support Grant (CSG) among the two groups of households determined eligible to receive this grant 
as equal numbers of caregivers (approx. 80%) in both enrollment groups reported being in receipt of 
a CSG. The percentage of eligible households receiving Old Age Grants (OAG) as 11% higher in 
previously enrolled households as compared to newly enrolled households. However, this difference 
was not statistically significant indicating that both groups of enrollees had similar access to OAG. 
Key findings from the randomized control trial: 
 Caregiver HIV Testing: In the caregiver sample, while lifetime HIV testing rates were not significantly 
affected by the program enrolment, recent HIV testing (in the past 12 months) was positively 
impacted by the program: at follow-up, caregivers enrolled in the home visiting program were three 
times more likely to have been tested for HIV in the preceding year than caregivers in the wait 
group. In addition, the program effect was particularly prominent among the intervention group 
caregivers younger than 35 years, as they were ten times more likely to have been tested in the 
previous year than the wait-listed caregivers.
 Child HIV Testing: While positive trends were observed in child HIV testing between both the 
intervention and waitlist groups, no significant program effect was found for lifetime rates of HIV 
testing among children. In terms of HIV testing more recently, in this evaluation child HIV testing in 
the previous year was only assessed at follow up. The results showed that the percentage of 
children tested for HIV in the 12 months was significantly higher among children enrolled in the 
program than the wait-list children. However, due to the lack of longitudinal data on child HIV 
testing, this effect cannot be attributed to the program. Most importantly, the was a significant 
impact of caregiver testing for HIV on child’s HIV testing uptake indicating that children whose 
caregivers (1) participated in the home visiting program and (2) had been tested in the previous 
year, were almost five times more likely to have been tested at follow-up than other children. 
 School Attendance: While positive trends were observed in regular school attendance among all 
school-age children in both intervention and wait groups, no significant program effect was found 
for this outcome. However, a significant program effect in school attendance was found among 
35
children aged 15 and older as children of this age in the intervention group were 6 times more likely 
to attend school more regularly than those in the wait list group.
 Caregiver Psychosocial Wellbeing: There was no significant difference between intervention and 
wait group caregivers in overall basic support they reported available to them. In addition, while 
there was no significant program effect found in overall negative feelings reported by caregivers in 
the intervention and waitlist groups, there was a significant program effect on the frequency of 
negative feelings. That is, caregivers in the intervention group had significantly lower odds (cut in 
half) of having negative feelings than those in the wait group (the frequency of negative feelings 
experienced by caregivers dropped by 21% in intervention group, and 10% in wait group).
 Early Childhood Development: No program effect was found on the number of verified health cards 
and immunization rates of children under the age of 5 whose caregivers produced verified health 
cards. In addition, there was no program effect found on the stimulating activities indicator. That is, 
there was no difference between the intervention and waitlist group and across time within groups 
suggesting steady rates of stimulating activities reported by caregivers in the intervention and 
waitlist groups throughout the study. 
 Access to Social Services: While there were positive trends observed in the percentage of children 
with verified identity documentation in both intervention and wait groups, no significant program 
effects was found for this indicator. There was a slight decrease in the percentage of eligible 
children who were receiving or had applied for the Child Support Grant, however there was no 
significant program effect found for the uptake in CSG among intervention households. In addition, 
while there was a slight increase in the percentage of eligible adults who either received or had 
applied for the grant, there was no significant program effect found for this indicator.
Recommendations 
The study reports significant improvements in several primary outcomes among program participants 
and serves as a window into the factors associated with vulnerable children’s and at-risk families access 
to HCT in peri-urban settings in South Africa, as well as aspects of programming that may contribute to 
gains in testing prevalence. Nevertheless, there is a clear need for further testing and scale-up of the 
Future Families and other interventions to supplement home visiting. The evaluation team hopes that 
the results of this study can be used to guide resource allocation and program design to provide support 
to HIV-affected families. Priorities should include identifying sources of funding, providing specialized 
training and compensation to this cadre of workers, and incorporating monitoring and evaluation in 
every program implementation plan. Drawing on the findings of this evaluation, a number of 
recommendations for programming and future research have been identified:
Recommendation 1: Explore ways and improve efforts to better reach most vulnerable populations.
This evaluation has found a significant program effect on caregivers HIV testing uptake which, in turn, 
was an important precursor to child’s testing. The program, however, had no direct effect on child 
testing. In addition, ever tested rates among children were still low. Only a few caregivers surveyed in 
this study were HIV+ which may be due to low levels of disclosure. These findings point to the need to 
36
find ways to reach the most vulnerable populations, particularly OVC, in order to link them to services 
and bolster HCT promotion.
Recommendation 2: Explore ways to improve school attendance among children and adolescents in 
the FF serviced families. Although the results of the RCT study indicate positive program effect on 
school attendance among adolescents aged 15 and older, absenteeism among children from different 
age groups did not improve. While some reasons behind children’s absenteeism (e.g. bad health and 
child labor) cannot be addressed by the program, there are ways to improve school attendance. For 
example, efforts should be directed to raising awareness about the importance of attending school 
every day even in the early grades. Non-monetary incentives could be used to recognize children and 
families that have started the year on time, or show high attendance. 
Recommendation 3: Strengthen efforts to improve caregiver psychosocial health. While no program 
effect was found in overall negative feelings and social support reported by caregivers in the study, 
there was a significant program effect on the frequency of negative feelings and a positive trend in basic 
support among caregivers. Regardless, there remains a clear need for alternative evidence-based 
interventions that can more effectively address high levels of psychological distress which is common in
HIV-affected families and particularly caregivers. The potential for structured time-limited interventions 
such as interpersonal group therapy, cognitive behavioral therapy, and other curriculum-based support 
groups has been highlighted by research evidence from sub-Saharan Africa to effectively alleviate 
psychological distress in caregivers, including those affected by HIV and AIDS. As such, targeted 
education and psychosocial support that home visitors provide may also help caregivers to understand 
the benefits of HIV testing, cope with the implications of a positive diagnosis and reduce stigma 
associated with HIV and AIDS. 
Recommendation 4: The training course for the FF care workers should include methods and content 
on how to address stigma in the community. HIV-affected households are particularly vulnerable and 
thus the care worker training course should include special modules and techniques on how to work and 
communicate sensitive information in a non-judgmental way to avoid stigmatization or discrimination of 
program enrollees.
Recommendation 5: Establish referral procedures that are sensitive to the research context. In this 
study, social service referral sheets were generated in cooperation with program partners for 
distribution to all potential study participants. This information was intended to enhance participants’ 
knowledge about services available in their community. However, referral sheets are not a substitute for 
comprehensive active referral protocols. Interviewers must be provided with explicit guidelines for 
referring participants who are in overt distress or at risk of harm. In addition, interviewers/fieldworkers 
are generally not social workers and need to be trained to recognize and recommendations produced 
from program evaluation do not find their way into practice. Evaluators can promote both local 
ownership and improved program practice by facilitating workshops with implementing partners that 
are designed to help enumerate and expand the application of minimum standards for service delivery. 
37
Recommendation 6: Maintain regular communication with local stakeholders about evaluation 
processes and results. Regular dialogue mechanisms with implementing organizations and other local 
key stakeholders are critical to addressing impediments to program success. Although it is important for 
researchers to operate independently from implementers, it is essential to have open communication 
about roles, the purpose of the evaluation and its protocols and potential benefits. In the evaluation 
team’s experience, when implementers and other stakeholders have a well- developed understanding of 
how learning about the program supports quality improvement, they are often eager to support the 
research. Continued engagement of local stakeholders maintained throughout the research process 
contributes to ensuring local relevance and the broader utilization of findings.
Recommendation 7: Adhere to culturally sensitive research approach. Sensitivity in the wording of 
research instruments and consent/assent forms is necessary to avoid stigmatization or discrimination of 
participants throughout the research process. All research documents must be written at the language 
level of participants (e.g. with youth- specific reassurances) designed to ensure well-informed and 
voluntary participation. In addition, research in international settings also requires cultural and linguistic 
competence in order to develop appropriate consent/assent language and procedures. Evaluation 
research also demands sensitivity to service expectations and researchers need to verify that 
participants understand the nature and intent of the study, that it will not affect service delivery and is 
strictly voluntary. 
Recommendation 8: To help tailor programming, future research should focus on identifying the 
aspects of home visiting intervention models that are most influential for HCT and other services 
uptake in different contexts. Future studies may help to identify the key attributes of home visiting that 
can most effectively link at risk and vulnerable families to HIV testing, grant access and other social 
services. In addition, to strengthen evidence base for home visiting programs provided by trained care 
workers, an RCT design employing several long-term follow-up stages should be used, if possible. There 
is also scarce evidence on whether limited focus programming (i.e. targeting one or two outcomes) 
results in better outcomes compared broad programming (i.e. targeting multiple outcomes), and thus 
future research should seek to compare interventions that target HIV-affected families on multiple 
outcomes with those that target HIV-affected families on fewer and more specific outcomes.
38
Dissemination
This study was designed to produce knowledge to inform existing practices and guide future investment 
in programming for HIV-affected families in South Africa. Findings from the baseline and quasi￾experimental analysis were made available to Future Families as soon as they were available in order to 
identify service gaps and successes. Preliminary data from the RCT study was shared with Future 
Families and other stakeholders in 2016. In addition, to encourage the scale-up of proven intervention 
and improvement of existing programming for orphans in Sub-Saharan Africa, findings were submitted 
to peer-reviewed journals for publication. Such publications enhance the credibility of research findings 
and expand their influence. The following list details different channels through which the impact 
evaluation results and other findings from Linkages to Care have been disseminated among local and 
international audiences: 
International and national conference presentations:
o Poster presentation at the 21st International AIDS Conference: Home visiting increases HIV 
testing uptake among vulnerable children and their caregivers: Results from a randomized 
controlled trial in South Africa (Durban, July 2016).
o Poster Presentation at the 7th Annual SA AIDS Conference: Future Families Home Visiting 
Program in Tshwane Increases Child HIV Testing (Durban, June 2015).
Presentations to study partners: 
o Presentation at Future Families AGM: Future Families Home Visiting Program Evaluation in 
Tshwane, Gauteng (Ms. Spyrelis, Pretoria, August 2016). Evaluation results from the RCT study 
presented to Future Families director, staff and board members at their AGM. In addition, a 
workshop with Future Families director and key staff members was organized and followed by 
an in-depth discussion of the abovementioned presentation, resulting in an action plan based on 
the results. 
o Interactive Workshop with Future Families: Future Families Baseline Survey: Preliminary Findings 
to Inform the Way Forward (Pretoria, November 2014) 
Presentations at the PEPFAR OVCY technical meeting: 
o Presentation on Home Visiting: Let’s Get Focused (Dr. Thurman, Pretoria, September 2016)
o Presentation on Key Considerations – Program attendance and fidelity in structured programs 
(Dr. Thurman, Pretoria, November 2015). 
o Roundtable discussion on: Future Families Home Visiting Program Increases HIV Testing (Dr. 
Thurman, Pretoria, November 2015). 
39
Peer-reviewed publications:
o Thurman, T. R., Luckett, B., Taylor, T., & Carnay, M. (2016). Promoting uptake of child HIV 
testing: an evaluation of the role of a home visiting program for orphans and vulnerable children 
in South Africa. AIDS Care, 28(sup2), 7-13. 
Research brief geared to programmers and policy makers: 
o HIV testing uptake is greater among orphans and vulnerable children exposed to a home visiting 
program: Results from a quasi-experimental study in South Africa. Summary of paper published 
in AIDS Care, 2016.
o Home visiting increases HIV testing among caregivers of orphans and vulnerable children: 
Results from a randomized controlled trial in South Africa. Summary of Conference Poster at 
International AIDS Conference, 2016. 
40
References
Coca-Perraillon, M. (2007). Local and global optimal propensity score matching. Paper presented at SAS 
Global Forum 2007, Orlando, FL. Retrieved from 
http://www2.sas.com/proceedings/forum2007/185-2007.pdf 
Deininger, K., Garcia, M., & Subbarao, K. (2003). AIDS-Induced Orphanhood as a Systemic Shock: 
Magnitude, Impact, and Program Interventions in Africa. World Development, 31(7), 1201-1220. 
doi:https://doi.org/10.1016/S0305-750X(03)00061-5
Department of Health, S. A. (2004). Ethic in Health Research Principles, Structures and Processes. 
Pretoria, South Africa. 
Department of Health, S. A. (2006). Guidlines for Good Practice in the Conduct of Clinical Trials with 
Human Participants in South Africa. Pretoria, South Africa. 
Department of Health, S. A. (2010). The National HIV and Syphillis Prevalence Survey, South Africa 2010.
Pretoria, South Africa. 
Future Families. (2016). Future Families Annual report 2015/2016. Pretoria, South Africa. 
HSRC. (2010). Informed Consent Guidlines re minors (Including orphans and vulnerable children (OVC) 
and paternal substitutes). Retrieved from: http://www.hsrc.ac.za/Document-3934.phtml. 
Ice, G. H., Yogo, J., Heh, V., & Juma, E. (2009). The Impact of Caregiving on the Health and Well-being of 
Kenyan Luo Grandparents. Research on Aging, 32(1), 40-66. doi:10.1177/0164027509348128
PEPFAR. (2012). Guidance for Orphans and Vulnerable Children Programming. President's Emergency 
Plan for AIDS Relief:. 
Petersen, A. C., & Leffert, N. (1995). Developmental issues influencing guidelines for adolescent health 
research: A review. Journal of Adolescent Health, 17(5), 298-305. 
Schenk, K. D., Michaelis, A., Sapiano, T. N., Brown, L., & Weiss, E. (2010). Improving the lives of 
vulnerable children: implications of Horizons research among orphans and other children 
affected by AIDS. Public Health Rep, 125(2), 325-336. doi:10.1177/003335491012500223
Schenk, K. D., & Williamson, J. (2005). Ethical approaches to gathering information from children and 
adolescents in international settings: Guidelines and resources. Washington, D.C.: Population 
Council.
Shisana, O., Rehle, T., Simbayi, L. C., Zuma, K., Jooste, S., Zungu, N., . . . Onoya, D. (2014). South African 
National HIV Prevalence, Incidence and Behaviour Survey, 2012. Cape Town, HSRC Press.
Thurman, T. R., Kidman, R., & Taylor, T. M. (2015). Bridging the gap: The impact of home visiting 
programs for orphans and vulnerable children on social grant uptake in South Africa. Children 
and Youth Services Review, 48, 111-116. doi:http://dx.doi.org/10.1016/j.childyouth.2014.12.002
Thurman, T. R., Luckett, B., Taylor, T., & Carnay, M. (2016). Promoting uptake of child HIV testing: an 
evaluation of the role of a home visiting program for orphans and vulnerable children in South 
Africa. AIDS Care, 28 Suppl 2, 7-13. doi:10.1080/09540121.2016.1176679
UNAIDS. (2012). Report on the Global AIDS Epidemic. UN Joint Programme on HIV/AIDS (UNAIDS). 
UNICEF. (2003). Africa's Orphaned Generation. New York: Unicef.