Page 1 of 57 ZAMFAM Evaluation Report Zambia Family Activity in Southern and Central Provinces ZAMFAM SC Project Implemented by Development Aid from People to People in Zambia with funding from USAID and PEPFAR Prepared by Chainama Hills College Hospital, P.O Box, 30043, +260 974576962 0 or +260 97159388 primzambia2017@gmail.com Page 2 of 57 9th December, 2020 Acknowledgements We acknowledge the financial support from the United States Agency for International Development (USAID) Fund to the Zambia Family Activity South – Central (ZAMFAM SC) Project and technical support from Development Aid from People to People (DAPP) without which this project would not have been possible. We are grateful to the ZAMFAM Project teams (the Chief and Deputy Chief of Party and the M&E Specialist) as well as DAPP Management (The Executive Director, the Procurement and Finance) teams for the technical and logistical support before, during and after the process of designing the evaluation, collecting data and report writing. We would like to thank all Project Implementing Teams, partners and stakeholders in all the participating districts of (Chibombo, Kabwe, and Kapiri Mposhi) in central province and (Choma, Mazabuka, Monze and Livingstone) in Southern Province for the support and taking part in the evaluation and providing perspectives of the programs. We extend our sincere gratitude to the program beneficiaries who took their time to participate in the final evaluation, giving us valuable insight into the impact of the programs they participated in and the contributions made in their lives and/or day to day lives. Page 3 of 57 Contents Executive summary . ...............................................................................................................5 1.0. Introduction...................................................................................................................11 1.1 Background ...............................................................................................................11 1.2 The ZAMFAM Project...................................................................................................11 1.3. Project Purpose and Results..........................................................................................12 1.3.1 Main Project Purpose...............................................................................................12 1.3.2 ZAMFAM planned results areas .............................................................................12 1.4 Evaluation Purpose.........................................................................................................14 1.4.1 Main Objective ........................................................................................................14 1.4.2 Specific Objectives..................................................................................................14 1.4.3 Evaluation Focus and framework...........................................................................14 2.0. Methodology....................................................................................................................15 2.1 Evaluation Design ..........................................................................................................15 2.2 Data Collection Tools and techniques............................................................................15 2.3 Sample and Sampling Process .......................................................................................15 2.3.1 Quantitative sample .................................................................................................15 2.3.2 Qualitative sample ...................................................................................................16 2.3.3 Evaluation Data Collection Summary .....................................................................16 2.4 Data Analysis .................................................................................................................17 2.5 Quality Control Strategy ................................................................................................18 2.6 Response to COVID 19..................................................................................................18 2.7 Limitations .....................................................................................................................18 3.0 Evaluation Findings...........................................................................................................19 3.1 Introduction ....................................................................................................................19 3.2 Participant demographic data .........................................................................................19 3.3 The Relevance of the ZAMFAM Project.......................................................................22 3.4 The Project Effectiveness...............................................................................................26 3.5. Project Efficiency..........................................................................................................47 3.6. Project equity - cross-cutting issues..............................................................................49 3.7. Project impact................................................................................................................50 3.8. Lessons Learned...........................................................................................................52 3.8. Sustainability of project activities................................................................................53 4.0. Conclusion ........................................................................................................................54 4.1. Recommendations.........................................................................................................54 Page 4 of 57 Glossary/Acronyms and Abbreviations AIDS Acquired Immune Deficiency Syndrome CAG Community Action Group CCV Child Care Volunteer CHW Community Health Worker Creative Creative Associates International CWAC Community Welfare Assistance Committee DAC Development Assistance Committee DAPP Development Aid from People to People DMS Data Management System DWAC District Welfare Assistance Committee HIV Human Immune-Deficiency Virus HPP The Federation Humana People to People IEC Information, Education and Communication materials M&E Monitoring and Evaluation MCDSS Ministry of Community Development and Social Services MTE Mid-term Evaluation OVC Orphans and Vulnerable Children PCSC Parent Community School Committees PLWHA People Living with HIV and AIDS SILC Savings and Internal Lending Communities USAID United States Agency for International Development VAG Village Action Group WASH Water, Sanitation and Hygiene YWCA Young Women Christian Association ZAMFAM SCZambia Family South Central Page 5 of 57 Executive Summary Introduction With funding from the United States Agency for International Development (USAID), Development AID from People to People (DAPP) implemented the Zambia Family Activity South – Central (ZAMFAM SC) Project with the aim of improving care and resilience for Orphaned and Vulnerable Children (OVC) as well as building resilience in their families. ZAMFAM SC project was implemented in 14 districts of Zambia in partnership with Creative Associates International, Kabwe Adventist Family Health Institute (KAFHI), Network of Zambian People Living with HIV and AIDS (NZP+). The ZAMFAM SC Project also worked in cooperation with the Government of Zambia through line departments such as the District Health Office (DHO), District Education Boards Secretary (DEBS) Community Development, Social Welfare and Agriculture, livestock and Fisheries. This end of project evaluation aimed to document the overall project performance towards meeting the set project goals, objectives, and outcomes and draw lessons useful for future OVC programming for improving DAPP and partners’ work and organisational learning, program replication and wider stakeholder information dissemination. Evaluation Methodology This evaluation utilized both quantitative and qualitative methods to obtain in-depth understanding of how the ZAMFAM SC project interventions contributed to the wellbeing of children, families and communities. Based on the sampling targets provided by DAPP, a three￾stage stratified cluster design was used to select a sample of 700 households from the eight participating districts. Six districts were old districts i.e. (Kapiri Mposhi, Chibombo, Livingstone, Kabwe, Mazabuka, Choma) that started implementation of the project from nception while two districts i.e. (Mkushi and Kalomo) started implementing the activities in November 2019. The sample was represented by 325 households in Central province (representing 75 in Kapiri Mposhi, 125 in Kabwe and 125 in Chibombo). In addition, 325 households were sampled in southern province (representing 75 in Livingstone, 125 in Mazabuka and 125 in Choma). Qualitative data were collected using 14 FGDs (7 with children and 7 with caregivers). FGDs had a total of 112 participants while 47 participants were interviewed through KII/IDIs. In this evaluation, the new districts that had a limited implementation time in the project were not sampled. This ensured effective comparison of the results between the baseline, Mid-term Evaluation (MTE) and the endline evaluation. Evaluation Findings ZAMFAM Project Relevance - The ZAMFAM interventions were relevant to international commitments in building and strengthening governmental, and community capacities to provide a supportive environment for vulnerable children infected with and affected by HIV/AIDS, including achievement of the goals of Accelerating Children’s HIV/AIDS Treatment (ACT) in line with the core principle of SDGs 17 on HIV/AIDS response and USAID’s OVC child and family strengthening programme. In addition, the project was relevant to the WHO goal of 95:95:95 by working to enhance screening, testing and reduced loss to follow￾up among children living with HIV. Page 6 of 57 - The provision of Block Grants provided to schools by the ZAMFAM project were in line with the Ministry of General Education (MoGE) Education and Skills Sector Plan 2017-2021 and the 7NDP which aims at ensuring that no citizen is left behind, particularly with regard to accessing education services. - The project was also relevant to the National Health Strategy Plan and National HIV and AIDS Strategic Framework 2017 – 2021 as well as the Ministry of Community Development and Social Services programme on providing basic social protection services to vulnerable people in society and reducing the high poverty levels in the country targeting vulnerable households. - At the community levels, the project responded to the low child wellbeing status in relation to their nutritional, HIV, educational, care and support as well as poor capacity of child support workers and community structures, and poor WASH facilities. In addition, the project addressed the low integration and collaboration with government structures and community engagement in improving, health, education and the wellbeing of children affected by and infected with HIV. The Project Effectiveness Strategic Objective 1: Child wellbeing status measurably improved due to provision and accessing of quality care and support services Findings shows that the objective of improving child wellbeing status through the provision and accessing of quality care and support services was achieved. Secondary data demonstrated that the ZAMFAM SC facilitated the establishment of 3,021 Community Actions Groups (CAGs) against a target of 2,900 CAGs, whose aim was to support child wellbeing. In addition, through the action groups, more than 80,000 families carried out activities to improve WASH on 5 basic needs (latrine/ toilet, hand wash, drink clean water, kitchen hygiene, bathing facilities). Further children were screened for HIV to ensure that children/ adolescents that might have been at risk since the last screening were tested, and if found positive, linked to HIV services. In collaboration with other organizations, CHWs, CWACs and health facility staff encouraged children between the ages of 10-17 living with HIV to join support groups were they drew inspiration, energy and strength to cope up with challenges related to their HIV status. This contributed to 95% of the people screened knowing their HIV status. Strategic Objective 2: Capacity of government and community structures to care for and support children and adolescents living with, affected by and/or vulnerable to HIV increased The evaluation findings show that the project strengthened the capacity of government and community structures for them to be able to improve the quality of care for and support children and adolescents living with, affected by and/or vulnerable to HIV. Results showed that the project strengthened the coordination of activities as well as referrals through district stakeholders’ meetings, as well as by strengthening networking and coordination of OVC support and care. At district level, the project networked with districts and sub-district structures through the government line ministries namely Ministry of Health, Ministry of Community Development and Social Services, Ministry of Education as well as Ministry of Local Government and Housing. At community level the project strengthened the involvement of local leaders who played a critical role in the coordination of activities. Page 7 of 57 Strategic Objectives 3: Resilience of households to care for children and adolescents living with, affected by and/or vulnerable to HIV measurably increased Evaluation findings shows that this project objective was attained by strengthening the knowledge and skills of caregivers through Action Groups and supporting them with pass on gifts of small livestock and legumes, providing cash gifts as well as strengthening the Savings and Internal Lending Groups (SILC). In addition, Para-vets engaged and supported caregivers through continued monitoring of the health of the livestock, the process which helped in reducing the risk of losing animals from preventable diseases. Strategic Objective 4 - Reduced new infections among adolescents as a result of behavior change communication to the adolescents and their families Findings demonstrate that this objective was achieved as the project supported the adolescents with knowledge of and information on HIV prevention, treatment, care and support. This was done through the establishment of Support Groups for OVCs living with HIV and collaboration with different stakeholders such as NZP+, health facilities and Action Groups among others. Secondary data review showed that the ZAMFAM SC project successfully managed to establish 320 clubs. The action targeted OVC aged 10 to 17 years of age. The clubs were connected to the health facilities. The groups allowed members to be sharing information on life skills as well as sport and cultural activities. Support clubs were also using sport and cultural activities as a forum for them to act as champions in the fight against HIV by reaching other youth with information on behavior change communication. In this regard, open interactions with adolescents through life skills training and sports provided an opportunity for the adolescents living with HIV to be supported, practice prevention of HIV and adopt behavior change communication strategies to enhance their well-being. Strategic Objective 5 - Shared learning and evidence base to improve programming and inform policy and program investment strengthened. The review of documents in the districts visited established that this objective was attained with case plans for activities available at district level for the family and for each registered child to monitor progress in situations where the families did not meet graduation criteria. Case forms were updated after each contact with the beneficiaries for the case managers together with the case worker to keep track of activities in relation to the family until they graduate. Primary data sources were in place that helped in tracking project activities and reporting on the activities. Findings showed that the available data sources helped in generating results for sharing with stakeholders in district meetings, radio programs and dialogue meetings. The project had dedicated staff that managed project learning, and documentations for decision making. PEFPAR Essential Indicators (EI) under ZAMFAM for the evaluation Percentage of primary caregiver who know the child’s HIV status - Findings showed an increase in the % of primary caregivers who knew the HIV status of their children from 52.8% at baseline, 98.6% at MTE and 99% during the End line in all the targeted six (6) old districts. Page 8 of 57 Percentage of children between 6 -59 months who are undernourished - Evaluation findings showed a reduction in the percentage of children between 6 -59 months who are undernourished as measured by MUAC from 3.7% at baseline to 13.4% at MTE and 0% at end line. Percent of children too sick to participate in daily activities - Results from the valuation showed that the percentage of children who were reported to have been too sick to participate in daily activities two weeks preceding the evaluation reduced from 36.5% at baseline to 13% during the MTE and 12.8% during the end of project evaluation. Percent of children who have a birth certificate, observed or self-reported - Compared to ZAMFAM SC resultsfor the baseline, MTE and end-line showed that there was an increase in the number of children having birth certificates from 9.5% at baseline to 22.2% at MTE and 27.2% during the end-line. Percent of children aged 5–17 years regularly attending school - In relation to school attendance, i.e. a child was in school the last week before the schools were closed due to COVID 19, findings showed that 82.6% of the children regularly attended school during this evaluation. Compared with the baseline and MTE, there was an increase from 37.2% of children who regularly attended school at baseline to 80.5% during MTE Percent of children 5–17 years who progressed in school during the last year - For children who progressed in school, findings showed an increase from 78.9% of children 5–17 years who progressed in school during the last year at baseline, to 81.5% during the MTE and then to 82% during the end-line. Percent of children <5 years of age who recently engaged in stimulation - Compared with baseline, MTE and evaluation, the results showed that 93.2% of the children were recently engaged in stimulation during the baseline. This percentage dropped to 60% at MTE and increased to 87.4 % during the end-line. Caregivers who agree that harsh physical punishment is an appropriate means of discipline or control children - The percentage of caregivers who agreed that harsh physical punishment is an appropriate means of discipline or control for children were 38.5% at baseline and 32.4% at MTE. Compared to the baseline, the percentage reduced to 33% during the end-line evaluation. Percent of households able to access money to pay for unexpected expenses - Comparative evaluation results on the percentage of households able to access money to pay for unexpected expenses revealed that there was an increase from 50.9% at baseline to 58.5% at MTE and 63.8% during the end-line. Gender issues in the ZAMFAM project - 69.7% of the respondents reported that both husband and spouse participate in making daily household purchases decisions while 10.7% said husbands and 19.6% said women were responsible making such decisions. 86.7% disagreed that a husband can beat a wife if she goes out without telling him while 87% also disagreed to the assertion that a husband can beat a with if she’s not looking after the children. Page 9 of 57 Knowledge of HIV/AIDS - 95.3% of the caregivers reported that they received HIV testing services while 89.3% reported that they had received information on HIV for children to stay safe. - During MTE, 33.7% of the adolescents knew their HIV status which increased to 82.9% during the evaluation. In addition, 84.7% of the adolescents knew where to access HIV tests at baseline. Project Equity - Cross-cutting issues - Through community outreach services, the project therefore reached out to marginalized groups in Zambia who would otherwise had no access to health due to gaps in service provision. Engaging both boys and girls in children’s platforms/structures such as clubs, among others helped both boys and girls to effectively participate in the project’s activities and to present their views without any form of discrimination. Project Impact - The project has contributed to strengthening referral systems through improved tracking and follow up of children living with HIV using volunteers, reduction in stigma and discrimination at family and community levels, development of platforms for sharing experiences, opening up conversations on HIV health issues at community level as well as promoting collective action. - This project worked to improve families’ capacity to care for and support children living with and affected by HIV. The project further worked to strengthen the resilience of the targeted families and strengthened their economy through the training and activities in Action Groups, which included Internal Saving and Lending, Pass on Gift system of livestock and legumes, entrepreneurship and business training and linking to markets Lessons Learned The end of project evaluation established that the building of capacity of community members and health workers through training and awareness utilising (facility based) and outreach activities was critical in reaching caregivers with knowledge of HIV care and support for children. These interventions contributed to positively changing attitudes towards children living with HIV and improving the identification and referral of children to health and education services. In addition, the strengthening and utilisation of community and district level milt-sectoral committees and the engagement of actors from different Government Departments, NGOs and community leaders such as headmen, neighbourhood health committee members, CWACs and religious leaders and other implementing partners made it possible to reach more children and strengthen the implementation of project activities for impact through co-learning. More so, the provision of Block grants to selected schools helped in ensuring that vulnerable children had access to education as schools waived off school fees for vulnerable children thereby facilitating equitable access to education among OVCs. Further, the organizing of families in Action Groups brought people together to act collectively and support their children while the integration of Savings and Internal Lending in Communities (SILC) provided opportunities to households to save and borrow money on more flexible terms, and use the funds to meet basic needs including the needs of children. In addition, the implementations of the “Pass on Gift” activities helped to strengthen resilience and food security for the families of the registered OVC. This was made stronger through the formation of pass on gift committees in the target communities and the training and on-going support of the committees ensured adherence to the set rules of the pass on system. Page 10 of 57 Sustainability of project activities - The engagement of different stakeholders including government departments (Social Welfare, Community Development, Health and Education) will ensure that the activities and achievements gained will be sustained beyond the project. - The capacity building interventions that the ZAMFAM project provided to staff from different government departments such as health, education and community development as well as community structures such as CHWs, CWACs and Para-vets will ensure that the knowledge and skills to support OVCs will continue as well as interventions for supporting Action Groups are sustained. - The use of existing community structures such as CHWs, CWACs, Action Groups, SILCs and Neighbourhood Health Committees (NHCs) will ensure that the support provided to children and caregivers are integrated within local structures and health facilities for enhanced access to HIV care and support for children living with or affected by HIV. - The pass – on – gifts given to Action Group members – legume seeds, goats, chickens￾provides the possibility of scaling up empowerment activities, sustainable economic development and food security. - The popularisation of the available referral services has resulted into appreciation of services, an outcome which will promote sustainable referral process in the communities. - Engagement of stakeholders through Child Protection Committees promoted linkages on child protection issues thereby creating a sustainable case management system. Recommendations The end line evaluation generated recommendation for consideration. These include: 1). Supporting health facility staff and established structures to continue with horizontal transfer of health skills at community level through Community Action Groups and local health platforms, 2). Integrating the household life skills and livelihood interventions into community development and social welfare routine activities. 3). Government departments and DAPP should further the collaboration among the Ministries of Education, Health, and Community Development through technical working committees. 4). In line with the objective on developing family resilience, there is need for DAPP and Government to further support the development of guidelines on empowerment activities for resilience to disasters or pandemics such as the current COVID- 19 Page 11 of 57 1.0. Introduction 1.1 Background By 2018, around 48,000 adults and 5,400 children became newly infected with HIV in Zambia1 . Among children, new infections are slowly decreasing from 8,800 children in 2010 to 5400 in 2018. Zambia has a generalized HIV epidemic with 11.3% of adults living with HIV 2018, a slight reduction from 13% in 20102 . According to the 2016 Zambia Population Based HIV Impact Assessment (ZAMPHIA, Report 2019), 12.0% of persons aged 15 – 59 years are living with HIV (9.3% among adult males, 14.6% among adult females) while HIV prevalence among children under 15 years was estimated to be 1.1%. Adolescent girls and young women aged 15-24 have an incidence rate of 1.07% compared to .08% of men ages 15- 24. As of 2019, 87% of people living with HIV were aware of their status, and 89% on treatment while 75% were virally suppressed3 . The HIV epidemic affects children disproportionately. The Joint United Nations Programme on HIV/AIDS (UNAIDS) estimates that 15 million children have lost one or both parents to the disease, and many more are living with sick caregivers. This leaves children vulnerable to a range of poor outcomes, including dropping out of school, early marriage, transactional sex, and HIV infection. Infected children are less likely to access HIV treatment than adults. Community-based programs for orphans and vulnerable children (OVC) are important links for families to health and social services ensuring that caregivers are able to meet the needs of children, that children at risk are tested for HIV, that HIV-positive children access and adhere to treatment, and that adolescent girls stay safe, in school, and on track to reach their potential. Zambia is commended for staying on track in its pursuit of epidemic control by 2020 as the joint partnership of Zambia, PEPFAR, Global fund and others is on course to achieve 95-95- 95 at a national level and 95-95-95 in each age-stratified gender group by geographic catchment area. Efforts have been directed towards the over 600,000 children who have been orphaned by AIDS, and 100,000 newly infected with the HIV virus (USAID - Zambia_COP-2020_Part￾1-of-Planning-Letter) 4 . Providing care and support for these children is one of the biggest challenges Zambia faces today. In partnership with the Zambian government, the United States is committed to help meet this challenge by investing resources to enhance the quality of OVC services in Zambia. To help address the challenges of OVCs and families in Zambia, DAPP implemented the ZAMFAM project using a community-based approach and worked primarily through community and government structures, faith-based organizations (FBOs), schools, community-based organizations (CBOs), and private structures. The project targeted to address four key areas that impact families’ ability to meet the needs of OVC. 1.2 The ZAMFAM Project Zambia Family South-Central (ZAMFAM SC) project had a total funding of twenty-four million, four hundred and eleven thousand and twenty-three United States Dollars (USD 24,411,023). The main goal of the project was to reach out to125, 000 Orphans and Vulnerable 1 UNAIDS 'AIDSinfo' (accessed August 2019) 2 WHO ‘Country statistics: Zambia’ (accessed October 2018) 3 UNAIDS, 20218 4 COP 2020 Page 12 of 57 Children (OVC) annually with services that would ensure improved care and resilience of vulnerable populations. The target population were children living with or affected by HIV and their families in fourteen (14) implementation districts which include; Livingstone, Kalomo, Choma, Namwala, Sinazongwe, Monze, Mazabuka and Chikankanta in Southern Province and Kapiri-Mposhi, Kabwe, Chisamba, Mkushi , Mumbwa and Chibombo in Central Province 1.3. Project Purpose and Results 1.3.1 Main Project Purpose The main purpose of the ZAMFAM project was to improve the care and resilience of orphans and vulnerable children (OVC) in the Central and Southern Provinces of Zambia by supporting, protecting, and strengthening the capacity of children, families, and communities. This was implemented by conducting various activities at individual, family and community, district, and central levels. 1.3.2 ZAMFAM planned results areas The project activities were linked the four result areas as follows: Result 1: Child wellbeing status measurably improved due to provision and accessing of quality care and support services Result 2: Capacity of government and community structures to care for and support children and adolescents living with, affected by and/or vulnerable to HIV measurably increased: Result 3: Resilience of households to care for children and adolescents living with, affected by and/or vulnerable to HIV measurably increased Result 4: Reduced new infections among adolescents as a result of behavior change communication to the adolescents and their families Result 5: Shared learning and evidence base to improve programming and inform policy and program investment strengthened. 1.3.3 ZAMFAM Planned Project Activities Result 1: Child wellbeing status measurably improved due to provision and accessing of quality nutrition, food security, immunization and early child hood care and support services. Under this result area, the following were the planned specific activities to be carried out; 1.1 Identification of child abuse cases and provision of support to those living with or effected with HIV 1.2 Identify and initiate on treatment HIV positive children and adolescents and enhance adherence support to HIV positive children and adolescents 1.3 Promote AIDS/HIV positive living through structured teen clubs and trios for improved viral load suppression 1.4 Improve WASH (encourage all families to have 5 basic needs (latrine/ toilet, hand wash, drink clean water, kitchen hygiene, bathing facilities and nutrition support) 1.5 Support school re-enrollment and progression Result 2: Capacity of government and community structures to care for and support children and adolescents living with, affected by and/or vulnerable to HIV measurably increased. For this result area, the following were the planned specific activities to be carried out; Page 13 of 57 2.1 Participate as an active player in all government networking committees such as DWAC and Child Welfare Protection Committees. 2.2 Engage with local leaders (Headmen, Political leaders and Faith-based leaders) to ensure their full involvement and ownership. 2.3 Train and mentor CHWs to provide HIV services to HIV+ children and CWAC members to carry out individual case management of children including case planning and follow up of the OVC Result 3: Resilience of households to care for children and adolescents living with, affected by and/or vulnerable to HIV measurably increased. 3.1 Supporting the Action Groups to make their own action plans to bring about positive change for the families and children 3.2 Support Action Groups in graduation process through individual case management. 3.3 Ensure that the Pass on Gifts (Chickens, Goats, Legumes and Cash) initiative is operational among the group members. 3.4 Facilitate for trainings in agriculture and marketing in order to improve crop production, horticulture and livestock rearing and marketing among the Action Groups. 3.5 Support urban groups in entrepreneurship and business management including saving groups among the care givers. 3.6 Train HIV Positive adolescents and youth between 17-20 years of age and household members in Market-linked entrepreneurship, business skills and production Result 4: Reduced new infections among adolescents because of behavior change communication to the adolescents and their families Key proposed activities under this result area include: 4.1 HIV prevention activities through sensitization and promoting safer sexual behavior including delaying sex, abstain, use condoms through social behavior change messages 4.2 Mobilizing and sensitizing sexually active adolescents on consistent and correct use of condoms. 4.3 Sensitize adolescents and caregivers about PrEP; 4.4 Mobilize adolescent boys to take up Voluntary Male Medical Circumcision. 4.5 Carry out HIV testing campaigns and ensure those tested HIV positives are linked to treatment and virally suppressed resulting in reduced risk for them infecting others Result 5: Shared learning and evidence base to improve programming and inform policy and program investment strengthened. Under this result area, the ZAMFAM planned to conducted the following activities 5.1 5.2 Conduct studies to portray best practices related to OVC programming. 5.3 Conduct radio programs aimed at reaching the communities in the targeted districts with important messages and disseminate best practices from the project 5.4 Implement an effective monitoring and evaluation system. This evaluation, therefore, documented the overall project performance towards set project goals, objectives, and outcomes and drew out lessons useful for future OVC programming. Page 14 of 57 This evaluation was commissioned to assess the performance of the project activities implemented. The key deliverables that were set in this evaluation included: 1. Evaluation design and development of the inception report 2. Final evaluation tools and guidelines and field work and data collection 3. First and second draft evaluation reports 4. Final evaluation report 1.4 Evaluation Purpose 1.4.1 Main Objective The main objective of this evaluation was to document the overall project performance towards set project goals, objectives, and outcomes and draw out lessons useful for future OVC programming. Further, this evaluation generated insight into the lessons learned from the project for improving DAPP and partners’ work and organisational learning, program replication and wider stakeholder information dissemination. 1.4.2 Specific Objectives The end of project evaluation will specifically focus on the following objectives: 1. Showcase overall ZAMFAM SC performance in comparison with the baseline and MTE 2. Obtain feedback from the supported communities and other stakeholders 3. Provide strategic and concrete evidence on the relevance, results, processes, and resource utilization for the project. 4. Identify lessons and good practices from the implementation of ZAMFAM SC, and opportunities to improve overall OVC programming among other stakeholders. 1.4.3 Evaluation Focus and framework This end line evaluation will focus four key areas: Evaluation criteria Key issues to explore Relevance  Relevance to the priorities of DAPP, Partners and donor strategies  Relevance to the priorities of the direct target group including priorities of children Efficiency and Value for money  Management set up  Timely delivery of outputs  The extent to which resources were allocated in line with project results Effectiveness  What strategies were used that led to the attainment of the project results  Outcomes achieved qualitative and quantitative (mainly intended but also unintended) at four levels: o Individual, organisational, community and policy  Change as experienced by children and community members to enhance accountability Sustainability  Likelihood that outcomes have been internalised/institutionalised and will continue to be persist/built at: Page 15 of 57 2.0. Methodology 2.1 Evaluation Design This evaluation utilized both quantitative and qualitative methods to obtain in-depth understanding of how the ZAMFAM SC project interventions contributed to the wellbeing of children, families and communities. This evaluation was a “before and after” study design without a comparison group that compared the ZAMFAM SC Project Baseline, Mid-Term and End line results. The evaluation was designed to determine the extent to which the achievements have taken place in the targeted districts and among the target population. This design involved measuring the outcome of interest before and after the program implementation as well as engaging program beneficiaries in reviewing and understanding how the change or lack of it happened. The design for the data collection tools followed the program theory and log frame as designed at the start on the programs. 2.2 Data Collection Tools and techniques Questionnaire Interviews: The evaluation used the household caregiver survey tool, Survey tool for children 0-9 years, and a survey tool for adolescents 10-17 years. Systematic desk-based review: These were used to review of secondary data using project documents and other materials related to the project5 . Focus Group Discussions (FGD): FGDs were done with caregivers and adolescents to assess the extent to which the outcomes of the project were achieved relative to the project design. Key Informant Interviews (KIIs)/In-Depth Interviews (IDIs): There were used to discuss project performance with stakeholders, and implementers for project intervention approaches, partnerships, barriers and successes. 2.3 Sample and Sampling Process 2.3.1 Quantitative sample Based on the sampling targets provided by DAPP, a three-stage stratified cluster design was used to select a sample of 700, households from the 8 participating districts. Six districts were old districts i.e. (Kapiri Mposhi, Chibombo, Livingstone, Kabwe, Mazabuka, Choma) that started implementation from project implementation from project inception while two districts i.e. (Mkushi and Kalomo) started implementation in November 2019. The first stage involved a proportional sampling of the districts and secondly, project sites were selected and their respective population of project targets from which the sampling frame was drawn. Lastly, based on the target sample provided in the sampling frame, individual households and participants were proportionately sampled from each sampled project site. This was represented by 350 households in Central province (representing 25 in Mkushi, 75 in Kapiri Mposhi, 125 in Kabwe and 125 in Chibombo). In Addition, 350 households were sampled in southern province (representing 25 in Kalomo, 75 in Livingstone, 125 in Mazabuka and 125 in Choma). The probability, proportional to size sampling (PPS) technique was used to identify the 28 project sites. Households were proportionately selected from each sampled site based on the population. This sampling technique was used because different districts had varying population sizes and the selection was proportional to the population size. The sampling for children 0-9 years old and adolescents 10-17 years old in the participating households was 5Bryman, 2012 Page 16 of 57 based on their participation in the project. In addition, children enrolled in the project were selected purposively based on their availability during the data collection period. 2.3.2 Qualitative sample Qualitative data were collected using 18 FGDs (9 with children and 9 with caregivers). Using information provided by the project staff in consultation with Child Care Volunteers (CCVs), caregivers in Action Groups, teachers, health care workers and other Community Health Workers (CHWs) Community Welfare Assistant Committees (CWACs), Para Vets and the District Stakeholders (District Health Officers (DHOs) District Education Boards Secretary (DEBS), Community Development and Social Welfare Staff were selected based on their availability. These participated in FGDs, KIIs/IDIs in all the districts. FGDs had a total of 126 participants while 53 participants were interviewed through KII)/IDIs – and these included 18 district officials from government departments (DHO, DEBS, Community Development and Social Welfare), 11 teachers, 7 health workers, 7 community health workers, 6 CWACs, and 4 Para vets. In addition, 6 ZAMFAM project staff in the targeted districts were also interviewed. 2.3.3 Evaluation Data Collection Summary TABLE 1: SUMMARY OF END LINE RESPONDENTS District Category Target Achieved Percent Chibombo Household Questionnaires 125 125 100 FGDs 2 2 100 KIIs/IDIs 6 8 133 Kabwe Household Questionnaires 125 126 99 FGDs 2 3 67 KIIs/IDIs 6 7 86 Kapiri Household Questionnaires 75 71 106 FGDs 2 2 100 KIIs/IDIs 6 6 100 Livingstone Household Questionnaires 75 74 101 FGDs 2 2 100 KIIs/IDIs 6 6 100 Mazabuka Household Questionnaires 125 118 106 FGDs 2 3 67 KIIs/IDIs 7 8 88 Choma Household Questionnaires 125 119 105 FGDs 2 2 100 KIIs/IDIs 7 8 88 Overall Household Questionnaires 650 633 98 FGDs 12 14 113 KIIs/IDIs 38 42 113 Source: End line Survey Page 17 of 57 Table 2: Summary Of Evaluation Components Study areas Central, Southern provinces: DAPP implementation areas Study population ZAMFAM SC enrolled beneficiaries, specifically: Primary caregivers (with children -0-17 years enrolled in the project) OVC aged 0–9 years (through caregiver interview) OVC aged 10–17 years self-response through data collectors and FGDs Minimum sample size Targeted 700 Ttotal OVC Hhouseholds in two provinces, Targeted 700 Primary Ccaregivers & OVC aged 0–17 years Method & location of survey Interviewer-administered survey questionnaire, FGD and KII/IDIs conducted either at Ccaregivers/OVC’s home or central place Study instruments Caregiver Qquestionnaire OVC 0–9 years Qquestionnaires: administered to Ccaregiver 10–17 years; administered directly to the child Undernourishment measured by obtaining the mid-upper arm circumference (MUAC) for children 6–59 months Timing Field data collection was conducted simultaneously using two (2) teams. Team 1 was in southern province (Kalomo, Choma, Livingstone and Mazabuka and team two in central province (Kapiri, Kabwe, Chibombo and Mkushi). Field work took place between 24th September – 3 rd October, 2020 Figure 1: Sample Characteristics between MTE and End-line 2.4 Data Analysis Quantitative data analysis and validation were done after exporting the datasets from excel spreadsheet to Statistical Package for Social Sciences (SPSS) version 25. Data was analyzed using SPSS software in which frequencies were generated. Other tests such as the descriptive and correlation tests were conducted in SPSS. Indicator detail sheets from the compendium of indicators were used as reference points to analyze the data. Emphasis was given to indicators drawn from the objectives set in the ToR. 6 27 665 341 139 19 65 8 30 683 253 250 18 53 0 100 200 300 400 500 600 700 800 Districts Operational Areas Households Children (0-9 years) Children (10-17 years) FGDs KII Sample Characteristics between MTE and End-line MTR Sample size Endline Sample size Page 18 of 57 Meanwhile, qualitative analysis and presentation were done through content and thematic analysis. This methodology was used to identify emerging themes from the FGDs, KIIs and IDIs. This was also used to explain the observed patterns in the quantitative data for triangulation of findings. All the data audio and notes were transcribed. Some interviews which were conducted in the local language were translated later to English. 2.5 Quality Control Strategy During the evaluation, we placed a premium value on the credibility of findings and the quality of data collected. Thus, data collectors were trained to ensure that high-quality data sets were gathered. The one-day training on the survey methodology, survey questions, and interview techniques were done in readiness for data collection. Our data collection methodology was shared with DAPP-ZAMFAM Team for review, comments and sign-off. In the field, the supervisors monitored the completeness and accuracy of the data in the entire study process. Data cleaning and error checks were done using the tablets. 2.6 Response to COVID 19 To ensure the protection of participants and researchers from COVID 19, recommended COVID-19 guidelines were observed. Firstly, participants were selected in such a way that minimized risk, protected (but did not exclude) vulnerable populations, maximized social value and collaborative partnerships, in a way that did not jeopardize the scientific validity of the baseline. Before and during fieldwork, PRIM-Zambia sensitized data collectors and communities about COVID-19. PRIM-Zambia communicated to data collection teams on how to protect themselves and surrounding communities. PRIM ensured that all data collectors and participants had access to sanitary supplies such as soap, hand sanitizer gel, and masks. Further, data collectors were trained in the data collection protocol that ensured participant's health and safety such as maintaining greater than 2m when conducting interviews, where necessary, with participants allowed to opt-out of interviews. 2.7 Limitations The baseline was undertaken with some limitations which included: - The escalating numbers of COVID 19 cases in Zambia, provided a limitation to effectively carry out the fieldwork as most data collectors were more careful in interacting with participants. To overcome this challenge, protective materials (masks and hand sanitizers) was bought for both the data collectors and participants for all the target individuals. In some cases, the baseline team faced a limitation of the non-availability of some participants especially children who participated in the project. In all the districts, fewer children (0-9 years and 10-17 years) were not available as they were reported to be either in school or out of homes. To manage this, the consultants communicated the situation with DAPP and ZAMFAM Management team to ensure that children and caregivers were available. Further, data collectors followed participants in their households and waited long periods to conduct interviews and in some cases, additional participants were sampled from among the ZAMFAM enrolled children to carter for the ones not available. Page 19 of 57 3.0 Evaluation Findings 3.1 Introduction This report presents consolidated findings from the evaluation of the ZAMFAM SC project which was conducted in two (2) provinces: Central province representing (Kapiri Mposhi, Kabwe and Chibombo) and in southern province representing (Livingstone, Mazabuka and Choma). DAPP through the ZAMFAM SC project interventions addressed issues of improving child wellbeing status through the provision of quality care and support services, increasing resilience of households to care for children and adolescents living with, affected by and/or vulnerable to HIV, increasing capacity of government and community structures to care for and support children and adolescents living with, affected by and/or vulnerable to HIV. This section, begins by presenting the participant demographic data, then presents the findings following the ZAMFAM project objectives and the nine (9) PEFPAR Essential Indicators as in the table below: Table 3: PEPFAR Essential Indicators for the Evaluation Essential Indicator Indicator EI 1 Percent of children whose primary caregiver knows the child’s HIV status EI 2 Percent of children <5 years of age who are undernourished as measured by MUAC EI 3 Percent of children too sick to participate in daily activities EI 4 Percent of children who have a birth certificate, observed or self-reported EI 5 Percent of children aged 5–17 years regularly attending school EI 6 Percent of children aged 5–17 years who progressed in school during the last year EI 7 Percent of children <5 years of age who recently engaged in stimulating activities with any household member over 15 years of age EI 8 Percent of caregivers who agree that harsh physical punishment is an appropriate means of discipline or control in the home or school EI 9 Percent of households able to access money to pay for unexpected household expenses Source: Mbizvo et al. (2018) 3.2 Participant demographic data The purpose of this chapter is to provide a summary of some demographic and socioeconomic characteristics of the population in the households sampled in the evaluation. For the purpose of the evaluation, a household was defined as a person or a group of persons, related or unrelated, who live together and share common cooking and eating arrangements with children -0-17 years enrolled in the project. Household age structure was analysed to understand a breakdown of different age groups in the evaluation. Participants demographic data by gender: Findings showed that a total of 633 caregivers participated in the evaluation. Of these 199 representing (31.5%) were male and 434 (representing (68.5%) were female. Compared with Baseline and MTE data, the evaluation showed that there was an increase in the number of male’s who participated in the surveys from 8.6% during the baseline, to 13.6% at midterm and 31.4% during the end of project evaluation. Page 20 of 57 In addition, the percentage of female participants has been reducing from 91.4% at baseline to 84.4% and midterm and 68.6% during the evaluation. These findings could imply that more men are getting involved providing care and support to OVCs in their households as compared to the initial stages of the project. Figure 2: Caregiver participant by gender Average age of caregivers: At baseline the average age of caregivers who participated in the survey was 43 years while at MTE, the average age rose to 44 years. During the end-line evaluation, the average age of the caregivers was 46 years. These findings demonstrate that older caregivers were getting more involved in OVC caregiving compared to previous years when this responsibility was much more left to relatively young people. These results were triangulated with results from qualitative interviews in FGDs and KII/IDIs. Figure 3: Average age of caregivers during the baseline, MTE and End-line During the end line, age desegregations showed that 261 of the care givers were above 51 years representing (41.30%) while 175 of them were between 41 to 50 years representing (27.70%) and 144 (representing 22.70%) were between 31 to 40 years. Further, 44 and 13 of the participants were either between 21 to 30 years or 11 to 20 years respectively. Of the 633 caregivers interviewed, 89 representing (14.1%) have not attended any education while 281 representing (44.5%) had primary (between grades 1 to 7) education and 252 representing (39.8%) had secondary level of education (grades 8 to 12). Very few caregivers (0.4%) and 0 50 100 Baseline Mid-Term End-line 91,4% 86,4% 68,6% 8,6% 13,6% 31,4% Female Male 43 45 47 41 42 43 44 45 46 47 48 Baseline MTR Endline Average age of Caregivers Page 21 of 57 (1.2%) had either a vocational (skills based short training) or tertiary (College/University) education qualification respectively. Figure 4: Caregiver participant % by age grade Marital relationships have been seen to influence the burden of caring for children. This evaluation sought to establish the marital status of the caregivers. Findings showed that 384 participates representing (60.6%) were married while 194 representing (30.7%) were widowed. Further, findings showed that (5.7%) have never been married or were divorced or separated as seen in the table 5 below. Table 4: Participants Marital status Participants Marital status Frequency % Valid Married 414 (n/683) 60.6 Cohabiting (but not married) 9 (n/683) 1.3 Never been married Divorced or separated 39 (n/683) 5.7 Widowed 210 (n/683) 30.7 Total 683 (n/683) 100.0 Further, finding showed that a total of 503 children participated in the evaluation. Of these, 245 were male and 258 were female. In addition, 253 were aged between 0-9 years while 250 were aged 10-17 years of which (117/250 representing 43.6% were males and 141(n/250) representing (56.4%) were females. Figure 5: Children who participated in the evaluation 1,90% 6,40% 22,70% 27,70% 41,30% 0,00% 10,00% 20,00% 30,00% 40,00% 50,00% 11 to 20 Years 21 to 30 Years 31 to 40 41 to 50 Years 51+ Years Caregiver participant % by age grade Page 22 of 57 3.3 The Relevance of the ZAMFAM Project 3.3.1. Relevance to international, national, district and community goals Care and support for children infected or affected by HIV: The ZAMFAM interventions build on global efforts on strengthening care and support for children infected or affected by HIV. The United Nations General Assembly Special Session on HIV/AIDS in June 2001 provided quality global leadership, awareness and support for overcoming the pandemic, and advocated for increased attention towards affected children. In the Special Session’s Declaration of Commitment, countries and partner organizations agreed to: “By 2005 to implement national policies and strategies to: build and strengthen governmental, family and community capacities to provide a supportive environment for orphans and girls and boys infected and affected by HIV/AIDS. The United Nations Children’s Fund (UNICEF) was instrumental in spearheading the formation of the Global Partners Forum for Children Affected by HIV/AIDS (GPFC) as a focal point for advocacy, dialogue, and prioritizing of action items. Orphans and Vulnerable Children (OVC) programs remain central to achieving an AIDS-free generation, contributing to the achievement of the goals of the Accelerating Children’s HIV/AIDS Treatment (ACT) Initiative. These efforts are in line with the core principle of SDGs 17, and that of AIDS response. Further, the ZAMFAM interventions were in line with Sustainable Development Goals (SDGs) such as ending hunger, achieving food security and improved food security for all6 . Further, the project was relevant to the WHO goal of 95:95:95 in which the emphasis is on ensuring that 95%of the people living with HIV/AIDS know their status, 95% of those knowing their status are on treatment and 95% of those on treatment have their viral load suppressed. This project worked to enhance screening, testing and reduction in loss to follow￾up among children living with HIV. Relevance to National Priorities: In Zambia, the ZAMFAM interventions were in line with the USAID – PEPFAR initiative of retaining people living with HIV (PLHIV) on treatment using client centred approaches and preventing new HIV infections through evidence-based prevention programs. The Government of the Republic of Zambia (GRZ) is committed to ending HIV by 2030 and is using the 90/90/90 Fast Track Strategy to measure epidemic control. As such, the government of Zambia, through the Ministry of Health has developed and rolled out National Health Strategy Plan and National HIV and AIDS Strategic Framework 2017 – 6 http://www.sdgfund.org/mdgs-sdgs. 136 117 253 109 141 250 0 50 100 150 200 250 300 Male Female Total Children who participated in the evaluation 0-9 years 10-17 years Page 23 of 57 2021. In addition, the Ministry of General Education revised its school curriculum in 2013 which included integration of comprehensive sexuality education in all public schools. The Ministry of Community Development and Social Services is also providing basic social protection services and programmes offering social assistance and promotional services (livelihood and empowerment) to the poor and vulnerable people in society. District Health Staff talked to reported that: “Before the project, we used to have difficulties in ensuring that children are tested for HIV. This project helped improve HIV testing among children as well as reduced loss-to-follow-up”, (KII – District Health Staff - Kapiri). This project was in with the priorities set in line with the Seventh National Development Plan (SNDP 2017-2021) in which it addressed the plight of an increased number people living below the poverty line which stood at 54 percent in 2015. This was achieved through the project coordinated approach that involved capacity building of community structures (CWACs/ CHWs and Para-vets) who ensured effective resilience building for caregivers. The project also enhanced inter sectoral approach at district levels that ensured effective referrals and linkages from communities to government services. Relevance to USAID Interventions: The ZAMFAM SC project was relevant to the the U.S. Agency for International Development’s (USAID’s) orphans and vulnerable children (OVC) programs. The U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) aim to improve the health and well-being of children living with and affected by HIV. The OVC programs strengthen child and family resilience and contribute to the acceleration of access to HIV/AIDS treatment for children and to the prevention of HIV among adolescent girls and young women. USAID believes that by lessening the impact of HIV and AIDS on children and families, communities are better positioned to work toward an AIDS-free generation. As reflected in its fiscal Year 2015, over 5.5 million orphans and vulnerable children and their families received care and support through PEPFAR-supported programs. OVC programs enable the delivery of vital health and social services that are critical to improving HIV prevention, care, and treatment, thus contributing directly to the UNAIDS 95-95-95 goals and ultimately working toward epidemic control. USAID supports child-cantered, family-focused, community-based, and government-backed OVC programming that targets the full range of OVC needs according to the age and developmental stage of the child. Relevance to DAPP interventions: DAPP works with those in need in a collective process that supports them to effect changes, improve their lives and solve their problems. DAPP believes that poverty and other social vices can be overcome through co-ordinated, community - wide approaches which combine education and literacy, health and hygiene, improved livelihoods and sustainable agriculture. The organisation works with long term sustainable programmes that address needs of families and communities by empowering people to take action to improve their circumstance in a collective and collaborative way. The ZAMFAM project strengthened DAPP in working with caregivers, local communities and other stakeholders (Ministries of Health, Community Development, and Agriculture, Education etc) Page 24 of 57 to provide care and support to OVC. The project activities were in line with DAPP’s strategic plan and priorities. WASH facilities in communities: At the community levels, the stakeholders interviewed stated that the WASH programs responded to the poor WASH facilities in communities. The project aimed at addressing various water and sanitation related problems which the community members in the targeted communities experienced. For example, six key informant respondents reported that the project was very relevant as it addressed one of the major challenges in the rural communities of lack of access to hygiene and sanitation information as well as practices. They indicated that: Before the ZAMFAM, our yards and surrounding communities used to be very dirty which posed as a health hazard to the children and community members” … “the project taught us how to maintain our environments clean as well as keeping the children clean and smart”, (FGD with caregivers - Kapiri). In all the project sites in Kapiri, Kabwe, Mazabuka and Livingstone districts, caregiver homes were observed to have clean toilets and garbage free yards. Therefore, it can be concluded that the project significantly helped address the problem of lack of information on WASH and poor practices around hygiene and sanitation. Quality and Learning Outcomes in School: In the districts visited, access to teaching and learning materials and equipment was a fundamental barrier to quality learning in all schools. Findings demonstrate that the Block Grants provided to schools by the ZAMFAM project, were in line with the Ministry of General Education (MoGE) Education and Skills Sector Plan 2017- 2021 and the 7NDP which aims at ensuring that no citizen is left behind, particularly with regard to accessing education services. Findings showed that the education interventions supplemented government efforts by building on initiatives aimed at improving the learning environments, such as desks, books and black boards, all of which contributed towards improved school attendance and literacy for OVCs. “You know, most of our school have limited learning materials such as books and desks, this project helped in supplying these materials which helped in improving the quality of learning environment for the leaners” KII – DEBS staff - Choma Community engagement in project implementation: To effectively implement the project, respondents from all the participating districts developed a community engagement plan which addressed pertinent issues of limited community engagement in improving, health, education and the wellbeing of children affected by and infected with HIV. At the community level, CHWs, CWACS, CCVs, Para vets and community mobilisers were engaged to support project interventions. Parents, teachers and health workers also participated in different interventions. Interviews with community members showed that community awareness and sensitization activities through door to door campaigns and community meetings helped enhance Page 25 of 57 community knowledge of HIV as well as access to health services, increased food security, improved livelihood and sanitation as well as education outcomes. “Through door to door campaigns and the use of pictures it was easy to convey message on HIV and good health practices” ….” Promotion of Community Action Groups also made it fascinating to attend project activities and also promoted effective parents’ participation in the health and education of children.” FGD - Caregivers - Kabwe. Although there is increased community and parental participation in improving the wellbeing of children, a few parents reported that myths, misconceptions and cultural barriers on sexuality barriers and time constraints still affect discussions with parents on sexual and reproductive health issues. Young people interviewed, indicated that some parents do not have adequate time for their children and even if they are with their children, they are not open to play and or have discussions with their children regarding sensitive issues. “Sometimes, it’s difficult to talk to parents about HIV because some issues about sexuality may come in. Now as you discussing sexuality issues with parents is still considered as a taboo in our community.” FGD-Children - Choma Resilience building in families: Findings in this evaluation established that the ZAMFAM project activity was aligned to addressing serious gaps in the capacity of families to manage adversities as a result of the HIV pandemic. According to the International Food Policy Research Institute’s 2016 Global Hunger Index, hunger in Zambia has only gotten worse. The data from the report suggests that Zambia is currently the third hungriest nation in the world, with 47.8 percent of the population undernourished, a 40 percent or higher rate of stunting in children five or younger and a mortality rate of 6.4 percent in children five or younger. This project worked to improve families’ capacity to care for and support children living with and affected by HIV. The project worked to strengthen the resilience of the targeted families and strengthened their economy through the training and activities in Action Groups, which included Internal Saving and Lending, Pass on Gift system of livestock and legumes, entrepreneurship and business training and linking to markets. Families that participated in the evaluation reported that: “The project had helped them to have enough food to care for the children in their households as well as raise money to pay for hospital bills for children and buy for them cloths.” FGD – Caregivers Chibombo It was established that Community Action and Saving Groups were the foundation for results achieved in the ZAMFAM project. These groups consist of one representative from each of the benefiting families. The groups provided a platform for peer support, information and sharing of best practices, discussing and challenging each other to adopt healthy, sound and productive practices and members supported one another with internal saving and lending activities which Page 26 of 57 the families needed. In addition, families received pass-on-gifts through the ZAMFAM. The gifts consisted of goats, chickens and crop input mainly for legumes in rural communities including cash for income generating activities in urban communities. 3.4 The Project Effectiveness 3.4.1 Introduction This section presents the project in line with the ZAMFAM project objectives and PEPFAR essential indicators. The report starts with the presentation of the project achievements against the set objectives and further presents the findings in line with the PEPFAR essential indicators. Further, some indicators are compared with the baseline and the Mid-term evaluation results to demonstrate the success in the life of the project. The overall goal of the objective was to improve the well-being of children and young people in terms of averages of social conditions encountered. 3.4.2 Evaluation results against set objectives Strategic Objective 1: Child wellbeing status measurably improved due to provision and accessing of quality care and support services The purpose of this end of project evaluation was to document the overall project performance towards set project goals, objectives, and outcomes. Findings show that the objective of improving child wellbeing status through the provision and accessing of quality care and support services was achieved. Secondary data demonstrate that the ZAMFAM SC supported 11,413 HIV positive OVCs. Of those supported, 90.8 % of HIV positive children on the program cumulatively reported viral load suppression of with 100% self-reported adherence. Through its interventions, families were empowered to take care of vulnerable children through the establishment of 3,021 Community Actions Groups (CAGs). Activities in the CAGs helped to enhance caregiver’s knowledge of and practices in parental skills, health, economic strengthening activities, child protection skills, care and support. Further, the evaluation established that all ZAMFAM enrolled children were provided with quality HIV care and support services including screening activities to ensure that children that were at risk were tested and caregivers knew their status, while those found positive were linked to HIV services. District Health Offices staff talked to reported that: “Community mobilization and engagement of community volunteers as well as CHWs and Health Staff played a critical role in ensuring that children and families at risk were screened for HIV. This helped our district to increase the number of children with known HIV status and are on treatment.” KII-DHO staff Kabwe In collaboration with other organizations, CHWs, CWACs and health facility staff, children between the ages of 10-17 living with HIV were encouraged to join support groups through which they drew inspiration, energy and strength to cope up with challenges related to their HIV status. In line with the children’s educational needs, secondary results showed that 21,630 OVCs received education services. The project also collaborated with the Ministry of General Education to improve the learning environment for the children and provided educational support through wavering of school fees. Interviews with the DEBS staff indicated that: Page 27 of 57 “We are very happy, ZAMFAM supported the Ministry of General Education with improved capacity to manage OVCs in schools by supporting improvements in the learning environment for the children by providing educational materials that made the wavering of school fees possible.” KII-DEBS staff Kabwe In all the school visited during the evaluation, it was confirmed that desks, exercise books, black boards and text books were distributed which led to schools supporting all the ZAMFAM enrolled children with free education. The project also supported the youth with trainings in self-help groups and in entrepreneurship skills and supported with entrepreneurship and income generation activities. These activities led families to be better prepared and resilient by enhancing their ability to generate income and reduce HIV related vulnerabilities and other economic shocks. Strategic Objective 2: Capacity of government and community structures to care for and support children and adolescents living with, affected by and/or vulnerable to HIV increased The evaluation findings showed that the project strengthened the capacity of government and community structures for them to be able to improve the quality of care for and support children and adolescents living with, affected by and/or vulnerable to HIV. Results showed that the project strengthened the coordination of activities as well as referrals through supporting district stakeholders’ meetings, as well as enhancing networking and coordination on OVC support and care. It was reported by the DHO staff talked to that: The ZAMFAM project helped to coordinate District HIV/AIDS Coordinating Committee (DHAC) meetings as a means of continuously engaging our different stakeholders in OVC programming to strengthen our capacity, linkages and referrals between partners” KII – Livingstone – DHO Staff The project also strengthened the capacity of sub-grantees to ensure that the capacity of Action Group leaders, CWAC's and CHW's improve their ability to perform their roles effectively and efficiently. The community meetings contributed towards improved nutrition and food security through promoting diversification in farming practices such as fish farming and providing training on how farmers can respond to climate change. At district level, the project networked with districts and sub-district structures through the government line ministries namely Ministry of Health, Ministry of Community Development and Social Services, Ministry of Education as well as Ministry of Local Government and Housing. Findings also showed that the project interacted with the District Welfare Assistance Committees (DWAC) under the Ministry of Community Development and Social Services, Neighborhood Health Committees (NHCs) and many more other district structures such as the District Child Protection Committees that support the plight of orphans and vulnerable children. In an interview with social welfare staff, it was reported that at community level, the project strengthened the involvement of local leaders who played a critical role in the coordination of activities. Page 28 of 57 Strategic Objectives 3: Resilience of households to care for children and adolescents living with, affected by and/or vulnerable to HIV measurably increased In order to increase resilience of households to care for children and adolescents living with, affected by and/or vulnerable to HIV, findings showed that the project focused on providing households with opportunities to actively cope with daily struggles whilst enabling resilience for households to care for children and adolescents living with/affected and/or vulnerable to HIV. Findings showed that this project objective was attained by strengthening the knowledge and skills of caregivers through Action Groups and supporting them with pass on gifts of small livestock and legumes, providing cash gifts as well as strengthening the Savings and Internal Lending Groups (SILC). Community Development Staff at district level reported that the project supported their interventions as follows: “The ZAMFAM project has helped us in strengthening the income generating activities. It used to difficult for us to mobilize communities, but through the project vulnerable groups were mobilized and we worked to strengthen their livelihood” KII – Community Development Staff - Chibombo Discussions with Agricultural Extension Officers indicated that the project worked with agricultural extension officers who played a vital role in strengthening caregivers knowledge and skills in crop management to ensure proper crop management for high yields. Para-vets were also engaged and supported caregivers with continued monitoring of the health of the livestock that the project supported to ensure that risk of loss of animals from preventable diseases is mitigated. These collaborative efforts between government line ministries, the sub￾district level, and other implementing partners were vital in ensuring sustainability of the projects activities beyond the life of the project. Strategic Objective 4 - Reduced new infections among adolescents as a result of behavior change communication to the adolescents and their families. This evaluation established that this objective was achieved as the project supported the adolescents with knowledge of and information on HIV prevention, treatment, care and support. This was done through the establishment of Support Groups for OVC living with HIV as well as the creation of linkages between adolescent support groups and health facilities. For instance, secondary data review showed that the ZAMFAM project successfully managed to establish 320 clubs targeting OVC aged 10 to 17 years of age. The clubs were connected to the health facilities for strong linkages between adolescents, CHWs and health workers. In addition, the ZAMFAM project collaborated with different stakeholders such as NZP+, Action Groups and health facility staff in providing a social foundation for OVC living with HIV, from which they drew inspiration, energy and strength to cope up with challenges related to their HIV status. The groups allowed members to be sharing information on life skills as well as to participate in sport and cultural activities. Support clubs were also promoted using sport and cultural activities as a forum for them to act as champions in the fight against HIV by reaching Page 29 of 57 other youth with information on behavior change communication. Participants in FGDs reported that: “Through clubs we are able to learn about sexuality issues, HIV, STIs and how to report people who violet our rights”.. We also have the opportunity to meet people like health workers who personally support use.” KII – adolescents - Choma Findings from FGDs with adolescents further established that adolescents were also engaged in meetings in which they discussed issues concerning then importance of HIV counselling and testing, adherence to treatment adherence and how to support each other to address issues of stigma and discrimination, positive living and child rights. In this regard, open interactions with adolescents through life skills training and sports provided an opportunity for the adolescents living with HIV to be supported and adopt behavior change communication strategies to enhance their well-being. Strategic Objective 5 - Shared learning and evidence base to improve programming and inform policy and program investment strengthened. Findings showed that this objective was met through the development of the Monitoring and Evaluation Plan (MEP) which guided the routine implementation of project interventions. The review of documents in the districts visited, established that the primary data sources were in place that helped in tracking project activities and reporting on the activities. Findings showed that the available data sources generated adequate results for sharing with stakeholders during district meetings, radio programs and dialogue meetings that were conducted in the communities in which feedback from the local leaders were also shared with the ZAMFAM members. The project had dedicated staff that managed project learning, and documentations for decision making. The project was also informed by the reviews at baseline and mid-term which provided for the utilisation of a well-coordinated evidence based. 3.4.3 PEFPAR Essential Indicators (EI) for the evaluation 3.4.1.1 EI 1: % of primary caregiver who know the child’s HIV status The ZAMFAM project objective of increasing resilience of households to care for children and adolescents living with, affected by and/or vulnerable to HIV was measured. One key indicator which was measured under this was knowing the child’s HIV status which is a critical first step in linking OVC to HIV prevention, care, and treatment programs and services (Measure Evaluation 2015b). Findings showed that at baseline, 52.8% of caregivers knew about HIV status of the OVCs who were under their care. This indicator increased during MTE to 98.6% and at End line to 99.0% as shown in Figure 6 below. Page 30 of 57 Figure 6: Caregivers who know the HIV status of the Child Community sensitization through participatory health education strategies such as radio programs and meetings positively shaped uptake of services; changes people’s knowledge and attitude (mindset) towards children with living with HIV and increases parents’ support/ care towards such children. These findings were triangulated with findings from qualitative discussions with participants in FGDs and KIIs and IDIs. It was reported that increased interaction with community based volunteers (CHWs, CCV and Action Groups) helped to enhance and sustain high levels of awareness about the importance of knowing the HIV status of children and how to care for children living with HIV and AIDS. Interviews with health facility staff demonstrate that the project contributed to strengthened referral systems through improved tracking and follow up of children living with HIV. This change happened because community health workers were able to reach out to as many people possible in their households. It was reported that household visits made people to freely do HIV testing as it guaranteed them confidentiality. Health Workers talked to in Kapiri Mposhi reported that: “the integration of CHWs in the project activities ensured that caregivers and children were directly linked to the services provided at the health facility such as HIV testing, access to ART and follow-up checks”. Interviews with health workers showed that such processes triggered a new culture within the health systems that appreciated and valued efficient referral systems. Health care workers at different levels reported that the new system had also helped in improving adherence to HIV treatment. Picture 1: Community awareness on OVC care and support 52,80% 98,60% 99,00% 0,00% 20,00% 40,00% 60,00% 80,00% 100,00% 120,00% Baseline MTR Endline % of caregivers who know HIV status their child Page 31 of 57 “CHWs have been very helpful in teaching us on how to care for and manage the condition of children living with HIV”,-FGD caregivers Kabwe “Knowing the HIV status of my child help me to seek encouragement and support from my colleagues and neighbors” FGD-caregivers - Chibombo Interviews at district levels showed that the project had helped to strengthen the community health approach that the Ministry of Health is pursuing by building the capacity of health workers and community health workers who are critical to enhancing HIV testing, enrolment of patients into ART and enhancing follow-up to care. “Initially, we used to have problems with the enrolment of patients into care, the ZAMFAM project trained us and our staff at family and community level to undertake screening, index testing, enroll those found positive and follow-up to ensure that they adhere to treatment.” KII – DHO Staff - Mazabuka Secondary results showed that ZAMFAM SC through CHWs supported HIV positive OVC through monthly visits and providing age-appropriate HIV treatment counseling, as well as PREP information to HIV negative partners. The CHWs also provided positive living counseling support, substance abuse counseling and access to other social and health related services through referrals. Desk review of project documents also showed that the project engaged 924 CHWs with each supporting an average of 5 – 20 HIV positive OVCs. 3.4.1.2 EI 2: % of children under 5 years who are undernourished as measured by MUAC This evaluation also measured the percentage of children less than 5 years of age who are undernourished as measured by MUAC. Undernourishment is measured as a percentage of children that were undernourished at the time of the survey. This indicator was measured anthropometrically among children aged 6 to 59 months. The indicator captures the reserves of muscle and fat in the body, which are depleted when a child is acutely or chronically Page 32 of 57 undernourished. All the children that measured less than 125mm, as specified in PEPFAR guidelines were undernourished. Evaluation findings were compared with the 2016 baseline and 2019 MTE results which shows a decrease in the number of children measures less than 125mm. Results showed a reduction in the percentage of children between 6 -59 months who are undernourished as measured by MUAC from 3.7% at baseline to 13.4% at MTE and 0% at end line. This entails that for all the children who participated in the evaluation, no child was found to be undernourished. As seen in Figure 7: In comparison to the baseline, MTE and end-line results, the evaluation results are comparable with the MTE results as they had similar samples while the baseline had a larger sample which makes the results statistically difficult to be compared. Findings shows that the project supported households with nutritional activities such as gardening, cooking and demonstrations. Figure 7: Children between 6-59 months who are undernourished “Some of us received seeds that helped us to have a backyard garden. We were taught on how to cultivate the gardens, plant high value nutritious crops as well as feed the children.” FGD – caregivers - Mazabuka Interviews with agricultural extension officers demonstrated that the project collaborated with the department officers to monitor crop management for good yields. In addition, Para Vets were also engaged to monitor health of animals owned by beneficiaries to ensure that risk of loss of animals from preventable diseases is lessened to prevent disruption of the livestock pass on activities to the next beneficiaries. These interventions helped households to have adequate food supplies for improved household nutrition. 3.4.1.3 Percent of children too sick to participate in daily activities This evaluation assessed, through caregivers, the number of children who were too sick to patriciate in any daily activities in the past 12 months prior to the evaluation. This was important as health during childhood is an important indicator of human development. Findings showed that the percent of children who were reported to have been too sick to participate in daily activities two weeks preceding the evaluation reduced from 36.5% at baseline to 13% during the MTE and 12.8% during the end of project evaluation as seen in the figure below: Page 33 of 57 Figure 8: Children too sick to participate in daily activities Caregivers in FGDs indicated that children living with HIV had several health needs that usually make them sick to participate in activities and play with other children. They stated that: “Through action groups, we have learnt that if a child is over-sleeping and not able to play, then, a caregiver should pay attention to the health condition of the child, because a child is expected to be active.” FGD with caregivers - Livingstone. It was reported that children are generally viewed as healthy when they are assessed by adult standards, and there has been a great deal of progress in reducing childhood death and diseases. However, ZAMFAM encouraged caregivers through Action Groups on the importance of assessments of children living with HIV by professional health workers. Interviews with caregivers showed that many of the children living with HIV were getting sick before the project as they were not aware of their status and were not on ART while those on ART were not adhering to treatment. Caregivers who had children living with HIV reported that: “Before, the project, children were getting sick --- sometimes you would wonder why. But now, we have learnt that improved nutrition, and good feeding practices, and general care for children contributes to their wellbeing.” FGD with caregivers - Chibombo In addition to the 3 core HIV, nutrition and child health PEPFAR Essential Indicators, this evaluation further measured PEPFAR MER Core Indicators, which include: The percentage of diarrhoeal cases among children, children who experienced fever in past 2 weeks preceding the surveys as shown below: Diarrhoea Cases: This evaluation measured the percentage of Children under 5 years who had diarrhoea in past 2 weeks: The evaluation compared the baseline, MTE and End-line results for the percentage of children who had diarrhoea in the last 2 weeks preceding the survey. Findings showed that the diarrhoeal cases were 33.4% at baseline and dropped to 14.7% at MTE. At the end-line, findings showed that only 11% of the children had diarrhoea. 36,50% 13% 12,80% 0,00% 10,00% 20,00% 30,00% 40,00% Baseline MTR Endline Comparative results for children too sick to participated in daily activities Page 34 of 57 Figure 9: % of children who had diarrhoea in the last 2 weeks These findings showed a drop from baseline to MTE and further to the end-line. These findings were triangulated with data from FGDs and KII/IDIs with caregivers and stakeholders who said that the reduction was partly due to the teachings on the important of maintaining hygiene and sanitation standards. The teachings included washing hands after using the toilets, maintaining proper disposal of kitchen and other household solid wastes, use of toilets when defecating and the importance of having a toilet in the household as well as washing hands using tip tap with either soap or ash after using the toilet or before and after eating. “We used to have a lot of diarrhoea cases here before ZAMFAM taught us about how to wash hands before eating or feeding children. DAPP has helped us to be health and keep children away from the clinics.” FGD – caregivers - Mazabuka “The problem is that some people who did not receive this information have not adopted these health practices which still affects every one of us in the community.” FGD – caregivers – Livingstone During implementation period, families worked on a number of activities to improve on their health in line with WASH which included: construction of new latrine, rehabilitation of old latrine, digging of Rubbish pits, putting up of tip taps and dish racks. A number of families met WASH basic needs and were spreading the message to their neighbors who were not ZAMFAM beneficiaries and had no knowledge on basics of WASH. Children with fever in past 2 weeks: This evaluation assessed the percentage of children who had fever in the last 2 weeks preceding the surveys. Comparative data showed that there was a drop in the proportion of children who recorded to have fever from 43.20% at baseline in 2016 to 23.20% at MTE in 2018 and 22% during the end line in 2020. 33,40% 14,70% 11,00% 0,00% 5,00% 10,00% 15,00% 20,00% 25,00% 30,00% 35,00% 40,00% Baseline MTR Endline % of Children under 5 years who had diarrhoea in past 2 weeks Page 35 of 57 Table 5: PEPFAR MER Core Indicators Indicator BM (%) MTE (%) End line Diarrhoea in past 2 weeks 33.4% 14.7% 11% Fever in past 2 weeks 43.2% 23.2% 22% children too sick to participate in daily activities 36.5% 13% 12.8% Source: Baseline, MTE and End line field Survey 2018 and Mbizvo, et al. 2018 3.4.1.4 EI 3: Percent of children who have a birth certificate, observed or self-reported The concern for the need to have all births registered by 2030 is evident in targets of 16.9 and 17.19 of the Sustainable Development Goals (SDGs). This evaluation assessed this indicator which was important given the need to protect all children because a child who is not registered is in danger of being being denied of his or her rights such as the right to an official identity, a recognized name and a nationality. In Zambia, issuance of a birth certificate is a legal requirement for children and it is a critical indicator of the rights of the child and confirming their right to access public services from the government such as health and education. Comparative ZAMFAM SC results for the baseline, MTE and end line showed that there was an increase in the number of children having birth certificates from 9.5% at baseline to 22.2% at MTE and 27.2% during the end line. This positive change (increase in children obtaining of birth certificates) was partly attributed to ZAMFAM awareness activities on the importance of obtaining birth certificates. Figure 10: Children who have a birth certificate These results were triangulated with results from FGDs, KII/IDIs in which caregivers and stakeholders expressed the importance of obtaining birth certificates for the children especially orphaned children. For instance, it was reported that; “Through ZAMFAM most parents know that the birth certificate is an official right for children. Having the certificate helps children to have an identity, a recognised name and a nationality which are important for accessing services and 9,50% 22,20% 27,20% 0,00% 5,00% 10,00% 15,00% 20,00% 25,00% 30,00% Baseline MTR Endline Percentage of Children who have a birth certificates Page 36 of 57 opportunities including employment. I know that with this knowledge, parents will continue helping their children acquire the certificates.” KII – Social Welfare and Community Development staff. Most (86%) of the caregivers talked to during FGDs also acknowledged the importance of having a birth certificate. They indicated that through the ZAMFAM project, they were able to know that having a birth certificate helps service providers to know the age of the child and provide appropriate services. Nonetheless, although ZAMFAM has contributed to this effort, some caregivers, complained that the process for obtaining the birth certificates for children from government authorities was cumbersome. They reported that the process was not a one off activity as it required one to visit the registration office several times with referrals to other departments like health (hospital records), the police in case of processes for orphaned children among others. Interviews with six district authorities under the Social Welfare Department from six districts acknowledged the role that information from the project played in promoting acceptability of birth certificates in the community. They further reported that the process for issuing registration certificates required more investments -both financial and human resource -to ensure that more children are registered. 3.4.1.5 Percent of children aged 5–17 years regularly attending school School attendance: This evaluation measured school attendance using the last week children were last in school prior to COVID 19(Last week of March 2020). Those who were in school a week before COVID 19 and did not miss school or missed school with permission before closure were regarded as having a good school attendance. Comparatively, findings between baseline in 2016, MTE 2018 and end line in 2020 in relation to school attendance using the last day a child was last in school before COVID 19 variable showed that, 37.2% of children interviewed, regularly attended school at baseline. This percentage increased to 80.5% during MTE and 82.6% during the end line evaluation for all the six old districts that participated in the evaluation. The increase in the percentage of children regularly attending school a week before COVID 19 school closure was due to the efforts and interventions in the project. Discussions in FGDs and KIIs showed that the capacity building of school staff, the integration of CHWs and CWACs and School Block Grants from the project helped improve school attendance. Teachers interviewed from the 3 schools reported that the school block grants helped the schools to maintain a school friendly environment. Caregivers stated reported that: “We are happy that the project allowed vulnerable children to be in school by giving the school books and desks. Most of our children managed to access education due to this support. Now, many vulnerable children can read and write because of the grant.” FGD – caregivers Kabwe. For some teachers, the increase in the number of children regularly attending school was attributed to the support children are receiving from their caregivers. Firstly, teachers indicated that PTA members were trained in maintaining school infrastructure which has led into their participation in school governance which is helping to maintains the school. In addition, caregivers demonstrated willingness to engage teachers on the welfare of their children Page 37 of 57 including reporting to teachers’ every time their children were sick as well as having an interest to know the welfare of their children in school. Findings further demonstrated that children appreciated the fact that they were free to attain education without being sent home due to non￾payment of school fees. Children currently enrolled in school: This indicator measured children’s access to a structured learning environment by tracking those who are enrolled and attending school at the time of the evaluation. Findings showed that the percentage of pupils enrolled in schools during the baseline was 77.7% and 96.6% at MTE. During the end line, findings showed a reduction to 89.4%. This reduction could be attributed to COVID- 19 which made some parents not to allow children into school at the beginning of the term. Figure 11: Children currently enrolled and are attending school These findings were triangulated with qualitative results which demonstrated that several interventions helped children to be enrolled in school. For instance, interviews with teachers indicated that interventions that focused on back to school campaigns helped children to get back to school. For caregivers, activities in the Action Groups and the Pass-on Gifts, and participation in the Savings groups helped then to have adequate finances to support children with necessary school requirements. More so, caregivers stated that: “the ZAMFAM trained us and we appreciate the trainings and awareness messaging on parental care and child rights that we received, which helped us have an, increased appreciation of the importance of education.” FGD caregivers – Kapiti Mposhi 3.4.1.6 Percent of children 5–17 years who progressed in school during the last year Children progressed in school: Timely progression through school is an important measure for school performance, completion and the onset of other life transitions for adolescents. This was important because grade repetition leads to children getting over-age for their grade. This evaluation compared the evaluation results with the baseline and MTE results. For children who progressed in school, findings showed an increase from 78.9% of children 5–17 years who progressed in school during the last year before the baseline, to 81.5% during the MTE and them to 82% during the end line. Discussions with school teachers and the DEBS, indicated that the ZAMFAM project worked in collaboration with Ministry of General Education to 77,70% 96,60% 89,40% 0,00% 20,00% 40,00% 60,00% 80,00% 100,00% 120,00% Baseline MTE Endline % of Children aged 5-17 years currently enrolled in school 37,20% 80,50% 82,50% 0,00% 20,00% 40,00% 60,00% 80,00% 100,00% Baseline MTE Endline % of Children aged 5-17 years regulary attending school Page 38 of 57 improve the learning environment for the children and provided educational support through wavering of school fees. Figure 12: Children 5-17 years who progressed in school Further, it was reported that the ZAMFAM project implemented school block grant activities and supported schools with desks, exercise books, black boards and text books in order to address the identified needs of the children. Further, caregivers talked to stated that: “children were now healthier because we are able to provide for them good and nutritious diets, they are on treatment and are better able to attained school” FGD – caregivers – Kapiri Mposhi. Qualitative interviews with learners indicated that the interventions implemented in the school by the ZAMFAM project were beneficial to enhancing their school progression. They reported that with the support of the project, they were receiving more support from their caregivers, teachers and colleagues in schools. For instance, they stated that: “Because of the blackboards in classrooms and books in the libraries we now know how to read making it easy to progress to higher grades” …., for some “the desks provided in classrooms are making learning effective.” FGDs – Leaners The ZAMFAM project further promoted the participation of young people, through teen clubs in the communities for young people in the community as the clubs in schools are currently only open to those in school. Young people recommended that opening up more clubs in areas not covered by ZAMFAM would enable many young people to enhance their decision making skills including the ability to say no to early sex and marriages. They stated that ZAMFAM should have introduced motivation sessions for children at community level to improve their motivation towards education, as well as assertiveness and negotiation skills. 3.4.1.7 Percent of children <5 years of age who recently engaged in stimulation During the evaluation, households were asked if children are cared for in a loving, safe, family and community environment with safe places to play. The evaluation sought to assess the percentage of children under 5 years who whether they recently engaged in stimulating 78,90% 81,50% 82% 77,00% 78,00% 79,00% 80,00% 81,00% 82,00% 83,00% Baseline MTE Endline Percent of children 5–17 years who progressed in school during the last year Page 39 of 57 activities with any household member over 15 years of age. The results showed that 93.2% of the children were recently engaged in stimulation during the baseline. This percentage dropped to 60% at MTE and increased to 87.4 % during the end line evaluation. The percentage increase between the MTE and End line was attributed increased awareness among caregivers by the CCVs and CWACs on the importance of playful parenting and child stimulations emphasised by the project. Figure 13: Children under 5 who recently engaged in stimulation Findings from qualitative data demonstrated the value for effective connection between children and their parents. Respondents agreed that it was important for children to feel loved by siblings and caregivers and to be able to talk to, play with and sing songs with children for their health growth and development. There was considerable emphasis that the project engaged caregivers and community members on respecting children, with most respondents indicating that this helped them to understand and support the growth of children through playful parenting and stimulation. This was acknowledged by children. For instance, children reported that: “As children in this community we feel loved, because parents spend some time playing with us. Playing with children is good if you are not in school or doing other work, especially within the household setting”. FGD – children-10-17. 3.4.1.8 Caregivers who agree that harsh physical punishment is an appropriate means of discipline or control children A history of exposure to harsh physical discipline has been linked to negative outcomes for children, ranging from conduct disorder to depression and low self-esteem. This evaluation assessed the percentage of caregivers agreeing to claims that harsh punishment is appropriate discipline to be given to a child. Harsh punishment includes slapping children or using a harsh language or psychologically maltreating children. Findings showed that the percentage of 93,20% 60% 87,40% 0,00% 10,00% 20,00% 30,00% 40,00% 50,00% 60,00% 70,00% 80,00% 90,00% 100,00% Baseline MTR Endline Percent of children <5 years of age who recently engaged in stimulation Page 40 of 57 caregivers who agreed to this assertion was 38.5% at baseline and 32.4% at MTE. This number was 30.5% during the end line evaluation. Figure 14: Caregivers who agree that harsh punishment is appropriate for the child Results showed a decrease in the number of caregivers who agreed that harsh punishment is an appropriate means of disciplining children. An important predictor of whether parents will use corporal punishment for a specific misbehavior is the type of goal they have in disciplining their child. When parents have short-term socialization goals or parent centered goals, they are more likely to use power-assertive techniques such as corporal punishment than when they have child centered or long-term goals 3.4.1.9 Percent of households able to access money to pay for unexpected expenses The change in resilient livelihoods among parents and caregivers to provide well for their children through encouraging enterprise development and food security was measured. This is necessary because improved financial inclusiveness, especially for rural communities, is a model that transforms the livelihoods of many vulnerable households. In addition, the empowerment of people with knowledge on how to create savings as a group in a safe, suitable and flexible way is a prerequisite for accumulation of productive assets, which allows vulnerable households to acquire assets as well as send their children to school. This indicator targeted at measuring the availability of and access to credit among households. This evaluation compared this indicator on households with access to credit as measured during the 2016 baseline and 2018 MTE. Comparative evaluation results on the percentage of households able to access money to pay for unexpected expenses revealed that there was an increase from 50.9% at baseline to 58.5% at MTE and 63.8% during the end line. Figure 15: Households access to money for paying unexpected expenses 38,40% 32,40% 30,50% 0,00% 10,00% 20,00% 30,00% 40,00% 50,00% Baseline MTE Endline Caregivers who agree that harsh physical punishment is an appropriate 50,90% 58,50% 63,80% 0,00% 10,00% 20,00% 30,00% 40,00% 50,00% 60,00% 70,00% Baseline MTR Endline Percent of households able to access money to pay for unexpected expenses Page 41 of 57 This increase in productive assets was partly attributed to the ZAMFAM interventions that supported households with pass-on gifts like chickens, goats and seeds. Evidence from focus group discussions with community members also indicated that many community members had access to credit as about 60% of those who participated in FGDs indicated that they had access to credit from different sources such as loans from the savings groups, sale of crops and livestock. They indicated that: “We are able to acquire loans to support us with farming inputs” …, “some of us belong to savings groups and cooperatives where we are able to get loans to support our households” FGD – Caregivers Kapiri The proportion of parents or caregivers that are able to provide well for their children was measured. This indicator was a proxy for poverty and vulnerability in which if parents or caregivers are unable to provide important basic items for each child, that child is considered vulnerable. It is a means of measuring whether economic gains at the household level actually translate into provision for children, for their well-being. Findings showed that 91% of the caregivers who participated in the evaluation were able to provide for the school requirements while 89.5% said they were able provide for the medical requirements to their children in addition to what the ZAMFAM was already providing. Figure 16: Caregivers able to provide for their children These findings demonstrated a positive change in the dynamics in the way caregivers provided for their children in which fewer of them (9%) reported that they still had problems in providing for the school requisites and 10.5% medical services to children. Household food insecure (ever no food in past 4 weeks): The ability of households to have year round access to sufficient food is an important measure of socio-economic status. In the evaluation, respondents were asked to recall the number of days in the last on month preceding the evaluation in which they did not have sufficient food for the household needs. Results showed a reduction in the times when households had limited or no food supply which moved 91% 9% 89,50% 10,50% 0% 20% 40% 60% 80% 100% Yes No Yes No Caregivers able to provide for the school requirements Caregivers able provide for the medical requirements % of caregivers able to provide for their children Page 42 of 57 from 74.0% at baseline to 65.6% during the MTE and 51.5% during the evaluation as seen in the figure below. Figure 17: Households with food insecure Interviews with household members indicates that, households realised that food security exists when all people, at all times, have physical and economic access to sufficient, safe and nutritious food that meets their dietary needs and food preferences: “We have learnt how to store food for consumption for the whole year – we no longer sell most of the produce immediately after harvesting just to have money”,…..”And when we sell some crops, we save the money to help provide for our families. This lesson on the importance of saving is good for financial sustainability.” FGD – caregivers. These findings demonstrate that 82% of the caregivers talked to in FGDs are able to produce enough food for their household consumption, but that some sell the harvest and end up starving their households. During FGDs and direct observation by evaluation team, it was noted that caregivers had improved knowledge of modern farming methods obtained through the project with a few (4) households visited had active back yard gardens teeming with a variety of crops growing. Findings further showed that 17 out of 26 community members interviewed (17 females and 9 male participants), showed knowledge of the need of crop diversification in efforts to enhance household food security. Households with small back yards (especially in urban areas) were growing their vegetables in disused containers. In addition, this evaluation established the number of households that owned livestock in the targeted areas. This was important because engagement of households in livestock activities is a major empowerment activity for rural and vulnerable population in Zambia. Evaluation findings showed that there was an increase in the number of households owning livestock from 47.7% at baseline to 50.7% during the Mid-Term Evaluation. The number of households increased to 85.6% during the endline evaluation. 74% 65,60% 51,50% 0% 10% 20% 30% 40% 50% 60% 70% 80% Baseline MTE Endline % of households with food insecure (ever no food in the past 4 weeks Page 43 of 57 Figure 18: Household that own goats and chicken ( livestock) Interviews with caregivers indicated that this indicator was important as it strengthened their knowledge and skills in livestock keeping through Action Groups. The caregivers received the pass on gifts of small livestock and legumes, provided with cash gifts as well as supporting their participation in Savings and Internal Lending Groups (SILC). They explained that: “We have lots of people here being involved in rearing small animals such as goats and chickens with the support from the project and Ministry of Agriculture. People are able to raise income to expand the business and also meet the basic needs of children. I know some people that have built houses from these business.” FGD – caregivers. At district and community levels, 6 of the District Agricultural Officers including Agricultural Extension Staff reported that the project also worked with agricultural extension officers who played a vital role in monitoring legume fields of families to ensure proper crop management for good yields. Para-vets also continued to monitor health of animals owned by beneficiaries to ensure that risk of loss of animals from preventable diseases is mitigated. The survival of animals ensures that there is continuity of the livestock pass on activities to the next beneficiary. 3.4.2 Gender and other themes The evaluation endeavored to establish individual and group decision-making powers and the influence women have in changing the direction of the decision to their opinion and feeling. Findings showed that 69.7% of the respondents reported that make both husband and wife participate in making daily household purchases decisions while 10.7% said husbands these decisions and 19.6% said women were responsible making such decisions. In addition, the 71.2% of the households said that both husband and wife were responsible for making decisions on what to do with the money that wife earns for her work as seen in the table below. 47,70% 50,70% 85,60% 0,00% 20,00% 40,00% 60,00% 80,00% 100,00% Baseline MTE Endline % of households own livestock Page 44 of 57 Decision making Other indicators b) Making daily household purchases? c) Deciding what to do with the money she earns for her work? ) Deciding how many children to have? Husband 10.7% 7.6% 13.0% Wife 19.6% 21.2% 11.7% Both 69.7% 71.2% 75.3% Provincial level results on decision making on family planning were compared with secondary results from the 2018 Zambia Demographic and Health Survey. Recognizing that the methodologies and sample sizes were different, the comparison was meant to document the trends between the Edline and national results. Findings showed that 81.2% of the respondents indicated that both women and husbands make decisions on family planning compared to Figure 19: Decision making on family planning 79.4% of the ZDHS national data. In addition, 12% said men make decision in the ZDHS 2018 report compared to 11.5% during the ZAMFAM endline. Unlike in central province, in southern province had 77.1% of the participants reported that both men and women make decisions on family planning compared to 81% of some response in the ZDHS 2018 report. More so, 6.7% in the ZDHS compared to 14.5% during the endline who said men make decisions, compared to 11.9% in the ZDHS and 8.4% in the endline who said women do make such decisions. In relation to decision making over what to do with the money women earn in central province, the ZDHS 2018 showed that 30.2% said women do make decision while 9.3% said men and 60.5 said both men and women make such decisions. In the endline evaluation 7.2% said women, 21.8% said men and 71% said both make decisions over what to do with the money women earn. In Southern Province, findings from the ZDHS showed that 30.6%, 28.5% and 41.0% said women, men and both members make decisions over how the money earned can be used respectively. In this evaluation, 7.80%, 21.40% and 70.80% said women, men and both members make decisions over how the money earned is used respectively. These results 12,00% 11,50% 6,70% 14,50% 7,80% 7,30% 11,90% 8,40% 79,40% 81,20% 81,00% 77,10% 0,00% 10,00% 20,00% 30,00% 40,00% 50,00% 60,00% 70,00% 80,00% 90,00% ZDHS Endline ZDHS Endline Central Province Southern Provence Decision making on family planning Men Women Both Page 45 of 57 demonstrated that in southern province, gendered power relation are higher than in central province. To establish the key gender issues imbedded in culture and practice, the evaluation asked respondents four sets of question on whether there are special circumstances when men can beat their wife on given instances as shown below. Figure 20: Acceptability of husband to beat the wife As seen in the figure above, 86.7% disagreed that a husband can beat a wife if she goes out without telling him while 87% also disagreed to the assertion that a husband can beat a wife if she’s not looking after the children. Comparatively, these results were common across the two participating provinces and districts. For instance, 87.3% of the participants in central province disagreed with the assertion that it is acceptable for a husband to beat his wife once she goes out without telling him. Whereas 86.1% disagreed with this statement in southern province. For those who disagreed with the assertion that a woman should be beaten if she does not look after the children, in central provinces it was 87.6% while in southern province it was 86.4%. Findings from FGDs and IDIs indicates that women are appreciate the importance of being involved in decision making, and their participation in business activities. In IDIs, participants reported that this change was attributed to the trainings conducted in the project that focused on female leaders with an emphasis on ensuring that that Action Groups and Support Groups had the representation of males and females 3.4.4 Adolescents Knowledge and practice of HIV – 10-17 years Youth with comprehensive knowledge of HIV/AIDS: HIV epidemics are perpetuated through primarily sexual transmission and blood or fluid contacts of infection to successive generations of young people. In all the districts visited, the HIV interventions were implemented to contribute to the reduction of the spread of HIV and alleviate the impact of AIDS on those infected and affected by the disease i.e. the children and their families through promoting prevention, care, support and treatment. Sound knowledge about HIV is an essential pre-requisite albeit, often an insufficient condition—for adoption of behaviours that reduce the risk of HIV transmission. The evaluation measured the percentage of youth aged 10–17 years who both correctly identify areas were HIV testing is conducted. Findings shows that majority 11,10% 86,70% 12,60% 87,00% 10,20% 89,30% 11,10% 88,40% 0,00% 10,00% 20,00% 30,00% 40,00% 50,00% 60,00% 70,00% 80,00% 90,00% 100,00% Agree Disagree Agree Disagree Agree Disagree Agree Disagree She goes out without telling him She is not looking after their children She argues with him She refuses to have sex with him Is it acceeptable for a husband to beat his wife if: Page 46 of 57 (85%) of the adolescent knew of where people can access HIV testing compared to 15% of the adolescents who reported that they did not know where people test for HIV as seen in the figure below. Figure 21: Adolescent knowledge of were to access HIV testing Secondary results showed that the ZAMFAM SC supported 739 adolescents, mainly female, with vocational training skills. Further, findings showed that the project established teen clubs that supported the HIV+ adolescents with access to ART and adherence to treatment as well as youth clubs that were connected to the health facilities for awareness and support with effective HIV information. The groups also allowed members to be sharing information on life skills as well as sport and cultural activities. HIV Treatment Adherence for positive children: This evaluation also assessed how HIV positive children were treated among peers. This was important because stigma and discrimination of those living with HIV has psychosocial effects on the lives of those infected by the pandemic. These results were compared with MTE results. Findings showed that there was a decrease in the number of adolescents who reported that HIV+ children were treated unkindly by other children from 33.7% during the MTE to 28.2% during the evaluation as shown below: Figure 22: Children reporting that those living with HIV are treated unkindly by other peers Yes 85% No 15% CHILD KNOWS WHERE PEOPLE CAN TEST FOR HIV 33,70% 66,30% 28,20% 71,80% 0,00% 10,00% 20,00% 30,00% 40,00% 50,00% 60,00% 70,00% 80,00% Yes No HIV Positive Children are treated unkindly by other children MTE Endline Page 47 of 57 Establish / strengthen Teen Clubs: Findings demonstrate that the ZAMFAM project supported HIV+ children with integrated support through teen clubs, that conducted group discussions, sport and cultural activities as forums for them to act as champions in the fight against HIV by reaching other youth with information on behaviour change communication. ZAMFAM SC successfully managed to establish 320 clubs. These results were triangulated with findings from FGDs which showed that youth had comprehensive knowledge of HIV/AIDS as stated below: “Before we joined the clubs for children in this area, we did not know where to report if people are talking bad of us, ---, community awareness has also helped to educate people about children living with HIV and the support we need.” FGD – Children in Kabwe These results demonstrate the increased awareness in adolescents getting HIV tests and being aware of the results. At the sometime, there is a reduction in stigma and discrimination for the adolescents living with HIV among peers. Picture 1: adolescents teen club discussing Health and HIV issues Support to children: As seen in the picture, the ZAMFAM SC collaborated with NZP+ to provide support to groups for children and adolescents aged 10 to 18 years of age in (“Teen Clubs”). Adolescents talked to indicated that they draw inspiration, energy and strength from peers in the clubs to cope with challenges related to their HIV status. They indicated that; “we discuss with friends issues of to interact and support colleagues living with HIV and avoid stigma and discrimination” These findings demonstrate the need to enhance teen clubs among adolescents. Preventions: Parents also confirmed that the engagement of adolescents in teen clubs is helping the children to build their strength to report any abuse they face just because they are living with HIV. Children in FGDs reported that Ten Clubs are essential in learning more about how to prevent re-infections as well as infecting others. They indicated that: “with the information we share in in teen clubs, we are more aware of our Sexual and Reproductive Health (SRH), and life skill through sport” FGD – Children - Livingstone. Secondary data also showed that support was provided to clubs using sport and cultural activities as a forum for them to act as champions in the fight against HIV on behaviour change. Secondary data showed that 106,558 OVCs completed HIV prevention intervention while 3,000 were trained as champions in HIV prevention and development of risk reduction. In addition, the project engaged caregivers on intervention to support adolescent sexuality interventions. Children Page 48 of 57 talked to reported that: “at least these days parents stopped shouting at us if we share with them our school work on comprehensive sexuality education, these days they support us even with work on the some” Treatment: Discussions with project staff indicated that activities in the teen clubs included adherence support among teens. This was also confirmed by the District Health Staff in Choma District who indicated that: “the ZAMFAM project helped to link children to health facilities through teen clubs”, Adolescents talked to in Kabwe District also stated that: “we are able to receive ART drugs on time and without missing because, in this teen club, we encourage each other to visit the health facilities were ART was being given.”. Further, the project conducted HIV screening of all children/adolescents to ensure that exposed children/ adolescents were tested, and if found positive, linked to HIV services. The screening and testing actions were also meant to ensure that all the registered OVCs knew the HIV status, which also contributed to the attainment of the first 95% target of people knowing their HIV status. More so, CHWs and health facility staff and Community Care Volunteers (CCVs) supported the testing and conducted home visits to HIV positive children for monitoring their viral load suppression. Secondary date shows that an average of 91% children with viral load suppression compared to 75% national level average 4.5 Indicator tracking sheet This evaluation documented the indicator achievement rate. Table 6: Indicator achievement tracking table # Indicator 2016 2018 2020 BM Values (%) MTE Values (%) End-line results Old Districts 1 Percent of children who have a birth certificate 9.5% 22.2% 27% 2 Percent of children regularly attending school 37.2% 80.5% 82.6% 3 Percent of children who progressed in school during the last year 78.9% 81.5% 82% 4 Percent of children <5 years of age who recently engaged in stimulating activities with any household member over 15 years of age 93.2% 60% 87.4% 5 Percent of caregivers who agree that harsh physical punishment is an appropriate means of discipline or control in the home or school 38.5% 32.4% 30.5% 6 Percent of households able to access money to pay for unexpected household expenses 50.9% 58.5% 63.8% 7 Percent of children whose primary caregiver knows the child’s HIV status 52.8% 98.6% 99% 8 Percent of children <5 years of age who are undernourished 3.7% 13.4% 0% 9 Percent of children too sick to participate in daily activities 36.5% 13% 12.8% Page 49 of 57 3.7. Project Efficiency The ZAMFAM investment was justified as it has generated positive progress towards meeting the set targets namely increasing HIV testing, care and support by enhancing resilience of households to care for children and adolescents living with, affected by and/or vulnerable to HIV, improving child wellbeing, increasing capacity of government and community structures to care for and support OVCs, and strengthening shared learning and evidence on OVC care. Evaluation findings further established that the management process was a centralised management style, which helped to attain value for money since it promoted accountability with regard to distribution and utilisation of resources and materials such as school block grants, training of implementers and stakeholders. In addition, the use of community participatory methodologies which included engagement of volunteers, community mobilisers and other local engagement approaches was a cost saving measure. In addition, this approach helped to broaden the coverage of HIV awareness, care and support thus enhancing the project’s efficiency and community ownership. Joint meetings between the community and service providers (such as CHWs, CWACs and CCVs and Para vets) enabled the provision of feedback from multiple stakeholders during one monitoring visit thus saving transport costs and time. In addition, it was reported that stakeholder meetings increased awareness by the community regarding child protection, education and health as well as the challenges that the service providers face and in general resulted into improved communication. Further, the evaluation showed that the project team adhered to the planned activities and strategies. As demonstrated in the effectiveness section, the activities that were implemented were in line with those that were stipulated in log frame. The project was able to adhere to the activities through coordination meetings and monitoring sessions. Analysis of the project management structure showed that the project had a lean management structure with a flat administrative hierarchy. This management process worked for quick decision-making and effective participation of the staff. The project implementation process was decentralized with different stakeholders such as nurses, CHWs and community leaders implementing different components of the project based on their strength and local presence of the partners in the sites. 3.8. Project equity - cross-cutting issues The equity analysis focused on the degree to which the results of the intervention were equitably distributed such as the extent to which the project reached the most vulnerable and marginalized children as well as the extent to which the project addressed gender equality. Messages reaching all people through the adoption of an inclusive health education model: The use community outreach activities ensured that information was provided to both in and out of school children. In addition, the use of radio messages ensured inclusiveness with regards to provision of health education and life skills messages as even those who are not educated could access the messages on health. Further, provision of school support enabled the most vulnerable children to access education services. Reaching out to the most vulnerable in the community by bringing services closer: According to the baseline report, prior to the intervention, awareness levels of and access to HIV services Page 50 of 57 were low partly due to long distances to health facilities. The project therefore reached out to marginalized groups in Zambia who would otherwise had no access to health services due gaps in service provision. Increased awareness on gender breaking patriarchal decision making barriers: In a patriarchal society like Zambia, women and children have limited ability to make individual decisions regarding health seeking. This project through promoting awareness and counselling at family level through Action Groups helped men to understand the importance of prioritizing both women and men making decision making. The results (both qualitative and quantitative) for example show that there were many households that reported making decisions together than in those cases where men or women led the decision-making process. This change was attributed to the trainings conducted in the project that focused on female leaders with an emphasis on ensuring that that Action Groups and Support Groups had the representation of males and females Engagement of both boys and girls: Engaging both boys and girls in children’s platforms/structures such as clubs, among others helped both boys and girls to effectively participate in the project’s activities and to present their views without any form of discrimination. To achieve this, the leadership of the clubs has to also be gender sensitive, and also trained in the importance and value of child participation in program implementation. Currently, both boys and girls were accorded equal opportunity to assume leadership in the clubs. Routine capturing of data on age and gender was done through the attendance registers for different program activities. The community structures had a deliberate policy within the project to promote the voices of the girl-child in the program activities. Non-Discrimination and Inclusion: All persons with different socio-economic statuses are provided equal opportunity to participate in the school, HIV and other empowerment activities. The project was found to be non-discriminatory in its activities and which included both boys and girls from different socio-economic backgrounds and encourages both abled and non￾disabled children to participate in the activities. 3.9. Project impact Strengthened referral system: The project contributed to strengthened referral systems through improved tracking and follow up of children living with HIV. This change happened because community health workers were able to reach out to as many people possible in their households. It was reported that household visits made people to freely do HIV testing as it guaranteed them confidentiality. Health Workers talked to in Kapiri Mposhi reported that: “the integration of CHWs in the project activities ensured that caregivers and children were directly linked to the services provided at the health facility such as HIV testing, access to ART and follow-up checks.” Interviews with health workers showed that such processes triggered a new culture within the health systems that appreciated and valued efficient referral systems. Health care workers at different levels reported that the new system had also helped in improving adherence to HIV treatment. Reduction in stigma and discrimination at family and community levels: In the FGDs, parents reported a reduction in misinformation and myths on HIV due to campaigns. This had Page 51 of 57 in turn helped reduce discrimination against children living with HIV at family and community levels. It was reported that people were now more than before aware the benefits of knowing and disclosing ones’ HIV status and adhering to HIV treatment. Development of platforms for sharing experiences: Through the district and community level meetings, health workers, community volunteers and project staff were able to share their experiences and learnings. At community level, Actions Groups and Teen Clubs were established and strengthened to ensure that information was shared among members. Similarly, the Multi-sectoral coordination committees at district level involving health, social welfare and education among others created an environment for sharing and adopting positive experiences with regards how to provide quality health HIV services and care and support for OVCs. Health workers also appreciated the reflective meetings they held with project staff and community project mobilisers, they stated that: “In the meetings, colleagues shared narratives of how parents have come back to them to express their gratitude of how our services have positively impacted on their children. Such sharing makes us even work harder to provide more services to many people.” (KII, health worker). Opening up conversations on HIV health issues at community level: The interviews showed that the activities in the community had helped in developing interest in discussions on HIV and care for children living with HIV at community level. The community leaders noted that issues of HIV health matters, the education of girls and discussions around GBV issues were now part of the discussions in community meetings. This inclusion was made possible due to engagement of community leaders in the implementation process of life skills, education services and health services. It was suggested that such discussions would continue in creating demand for these services in the community beyond the project period. Promoting collective action: The project helped trigger social movement towards collectiveness in promoting resilience of households to care for children and adolescents living with, affected by and/or vulnerable. In the FGDs, examples of such collective action included cases where neighbours had been able to engage and support others to take their children to access HIV services, and also encourage children who had dropped out of school to go back to school. These positive changes were possible through education provided by Action Groups as well as through the pass on gifts managed by the groups which helped the family meet the educational needs of the children and also contributed towards building resilience at family level. Improved school performance: The support that was provided to schools in the form of learning materials, desks and block grants in exchange for waiver of school fees for identified OVCs helped in improving the school learning environment and alleviating the burden to pay school fees. This situation motivated learners to concentrate in class and excel academically. For some teachers, the increase in the number of children regularly attending school was also attributed to the support children were receiving from their caregivers who had demonstrated willingness to engage teachers on the welfare of their children. These activities will continue Page 52 of 57 as the project built the capacity in PTA members, caregivers and other stakeholders to initiate and manage these activities. Improved socio-economic welfare: The savings groups and business opportunities that were promoted and supported by the project helped households to meet the educational, nutritional and other needs of OVCs. A few households reported being able to build houses from the businesses. 3.10. Lessons Learned This section identifies the key lessons that can be used to guide future strategies, projects and broader development lessons.  Engage CHWs members in Case Management. The engagement of CHWs and CWACs provided a platform for the ZAMFAM SC project to identify and provide the needed services to the OVCs. This was enhanced by conducting review meetings with CHWs, CWACs and Community Mobilisers, which further promoted client follow-up and skills building particularly for the purpose of group reinforcement skills and case management. The adoption of this model subsequently increased up-take of health services among the community thus contributing towards responsive and resilient HIV services. ● Factoring in continuous refresher training for CHWs is important in reaching to as many CHWs as possible. Responses in the evaluation, showed that the current number of CHWs trained is not adequate to cover the vast geographical areas in their communities thus the need for the Government to integrate and support training programmes for CHWs. ● Providing outreach activities through teen clubs, sports was essential for adolescents in learning more about HIV information especially how to prevent themselves from re￾infections as well as infecting others. In addition, the strengthening of Action Groups has been an effective and accessible method for improving community health as it improves the identification of and referral of children to health and education services. This process is effective as it is possible to reach out to as many children as possible within a short time. ● Community sensitization through participatory health education strategies such as radio programs and meetings positively shapes uptake of services; changes people’s knowledge and attitude (mindset) towards children with living with HIV and increases parents’ support/ care towards such children. ● The project model of simultaneously promoting multi-sectoral coordination and working with different stakeholders helped in strengthening implementation and coordination of structures at district and levels. This multilevel focus was vital as OVC needs emanate from different sources. Findings showed that the project worked with government (e.g. District Welfare Assistance Committees, District medical office, health centres, community development and Agriculture) all of which helped to stimulate response to the needs of caregivers and children. ● The establishment and strengthening of community structures (e.g. Action Groups, Community Welfare Assistance Committees, Neighbourhood health committees, Page 53 of 57 parents and teachers’ associations, traditional leadership, elected civic leadership and community mobilisers) was found to be necessary in providing care and support to the OVC and people living with HIV and AIDS. This collaborative approach was vital in refining and ensuring that the existing project approaches and strategies were appropriate for the needs of the OVCs. ● The provision of block grants to selected schools was vital in ensuring that vulnerable children were in schools as in return, school fees of targeted school pupils were waived thereby promoting equitable access to education as even vulnerable children could easily access education. Findings showed that the approach was good in that it provided double benefit the OVCs and the schools that benefited as opposed to the project just providing bursaries the children. ● Integration of Savings and Internal Lending in Communities (SILC) provides opportunities to households to save and borrow money on more flexible terms, and use the funds to meet basic needs including the needs of children. In all districts and communities that were visited, savings groups had been developed, and members were saving cash amounts and giving small loans to their members which they paid back with small interest. ● The integration of pass-on gifts to households was critical in ensuring that household livelihoods were improved. Findings showed that the household economic challenges were also addressed through for example empowering some pass-on gifts to some Action Groups with vegetable seeds and legumes seeds. For legumes, recipients harvested and passed - on the seeds to other group members. Other groups were also operating small businesses such as selling second-hand clothes, broiler chicken rearing which were financed by pass on grants given to group members. Qualitative interviews, observations and quantitative data showed that these activities helped in improving household income and food security. Parents also reported using some of the funds to meet educational needs. 3.8. Sustainability of Project Activities  The full engagement of different stakeholders including government departments (Social Welfare, Veterinary, Community Development, Health and Education) through training, multi-sectoral collaboration and linkages will ensure that the activities and achievements gained are sustained beyond the project. Interviews with the Department of Community Development staff in 4 districts indicated that: “We are happy the ZAMFAM has helped us established a multi--sectoral committee at district level to support OVCs, we will build on the gains made and continue with the activities”  The establishment and strengthening of community structures (e.g. Action Groups, Community Welfare Assistance Committees, Neighborhood health committees, parents and teachers’ associations, traditional leadership, elected civic leadership and community mobilisers) will continue to ensure that the provision of care and support to the OVC and people living with HIV and AIDS continue even after the project.  The use of existing community structures such as CHWs, CWACs, VAGs, SILCs and NHCs will ensure that the support provided to children and caregivers are integrated Page 54 of 57 within local structures and health facilities for enhanced access to HIV care and support for children living with or affected by HIV. Interviews with 8 CHWs and 7 CWACs talked reported that: “We are happy that the project trained us to link children to health facilities and other care services, we will use the approaches learnt to continue ensuring that the knowledge and skills to support OVCs continue’.  The pass – on – gifts given to Action Group members – legume seeds, goats, chickens￾provides the possibility of scaling up empowerment activities, as well as strengthening sustainable economic development and food security. Caregivers reported that: “we are happy now with the knowledge and skills in managing the livestock, we will continue utilising the knowledge we have and ask the para-vets were we have a gap as they live with us for help”. In addition, the popularisation of coordinated planning and support to OVCs through Action Groups and the coordination of monthly review and planning meetings for Action Group will ensure that care and support to OVCs is continued. The popularisation of the available referral services has resulted into appreciation of services, an outcome which will promote sustainable referral process in the communities.  The project model of stimulating multi-sectoral coordination, capacity building and working with different stakeholders will ensure that implementation and coordination structures at district levels continue to support interventions for OVCs. Findings showed that the project worked with government (e.g. District Welfare Assistance Committees, District medical office, health centres, community development and Agriculture) which all will continue to stimulate response to the needs of caregivers and children. 4.0. Conclusion The evaluation showed that the project was effectively implemented. This was demonstrated by increased resilience of households to care for children and adolescents living with, affected by and/or vulnerable to HIV, improving child wellbeing, increasing capacity of government such as schools and community development as well as community structures to care for and support OVCs. These changes were possible due to improved awareness by the community on HIV issues and use of community structures like clubs, VAGs and SILC to provide sustainable platforms for generating income and providing support and care to OVCs. In addition, availability of trained CHWs and health staff meant that patients could be tested locally a situation which was cheaper for the community and in particular vulnerable people such as women. Increased knowledge also helped in addressing gender barriers by making both male and female be involved in decision making. The training of Community health workersresulted into improved interaction with children and support to caregivers within their communities. Whereas participation in coordination meetings created a platform for stakeholders to share their experiences thus enabling co-learning aimed at improving service provision. Finally, development of a referral network helped children to complete all stages of the referral systems and also turn up for reviews thus enabling successful prevention and management of loss to follow-up for linkage back into care. 4.1. Recommendations The end line evaluation generated recommendation for consideration. These include: Page 55 of 57  Findings showed that the engagement of adolescents in teen clubs was critical in ensuring that HIV prevention, care and treatment for adolescents was improved. It is recommended that DAPP should link these teen clubs as well as the data base for the HIV positive children to other ongoing HIV interventions. This will help the children to continue receiving the health, education and support services.  DAPP working with established district stakeholders should consider supporting health facility staff and other established structures to continue with horizontal transfer of health skills at community level through Community Action Groups and local health platforms that will ensure children living with or affected by HIV are supported.  DAPP working with other key stakeholders to continue integrating the household life skills and livelihood interventions into community development and social welfare routine activities. This will ensure that the resilience of households and vulnerable children are sustained even after the ZAMFAM SC project.  Strengthen district and community level multi-sectoral coordination targeting Ministries of Education, Health, and Community Development and other key stakeholders through developed or developing technical working committees at national, provincial and district levels to ensure sustained and coordinated delivery of HIV and child protection interventions.  Findings showed that the ZAMFAM SC used the self-help groups approach to economic strengthening through income generating activities and block grants to ensure that children’s education was being supported. It is recommended that government departments such as the Ministry of Community Development and Social Welfare Services (MCDSWS) should follow up on self-help groups at community level through district offices and identification of market linkages. The departments should further engage the groups to explore means of registering the them with Registrar of societies to enable the groups receive further support and linkages to other initiatives as registered legal entities.  Given that more girls than boys still drop out of school, there is need for the Ministry of Education as well as Community Development and Social Services to continue sensitizing parents on the importance of girl child education.  There is need to develop sustainability guidelines on empowerment activities that covers building of household resilience to disasters or pandemics such as the current COVID- 19. The guidelines should be rolled out to community action groups and community livelihood groups, child care volunteers and para vets. Page 56 of 57 Annex 0-4 Years 5-9 years 10-14 years 15-17 Years Total Male Female Male Female Male Female Male Female Male Female N % N % N % N % N % N % N % N % N % N % Children who have a birth certificate, observed or self-reported Yes - - - - - - - - 17 11 25 16.1 10 12.2 12 14.6 27 11.4 37 15.6 No - - - - - - - - 51 32.9 62 40 22 26.8 38 46.5 73 30.8 100 42.2 Children aged 5–17 years regularly attending school (Not Skipping School) Yes - - - - 18 8.5 22 10.4 5 3.2 3 1.9 9 11 21 25.6 32 7.2 46 10.2 No - - - - 93 43.9 79 37.3 63 40.6 84 54.2 23 28 29 35.4 179 39.9 192 42.8 children 5–17 years who progressed in school during the last year Yes - - - - - - - - 36 23.2 54 34.8 11 13.4 16 19.5 47 19.8 70 29.5 No - - - - - - - - 32 20.6 33 21.3 21 21.6 34 41.5 53 22.4 67 28.3 Children <5 years of age who recently engaged in stimulation Yes 19 42.2 17 37.8 - - - - - - - - - - - - 19 42.2 17 37.8 No 5 11.1 4 8.9 - - - - - - - - - - - - 5 11.1 4 8.9 Caregivers who agree that harsh physical punishment is an appropriate means of discipline or control children Yes - - - - - - - - - - - - - - - - 69 10.9 124 19.6 No - - - - - - - - - - - - - - - - 136 21.5 304 48 0-4 Years 5-9 years 10-14 years 15-17 Years Total Male Female Male Female Male Female Male Female Male Female N % N % N % N % N % N % N % N % N % N % Primary caregiver who know the child’s HIV status Yes - - - - - - - - - - - - - - - - 203 32 468 67 No - - - - - - - - - - - - - - - - 2 1 0 0 Children under 5 years who are undernourished as measured by MUAC Yes 0 0 0 0 - - - - - - - - - - - - 0 0 0 0 No 25 56 20 44 - - - - - - - - - - - - 25 56 20 44 Yes 7 15.6 0 0 6 2.8 18 8.8 7 4.5 14 9.0 5 6.1 6 7.3 25 5.1 38 7.7 No 17 37.8 21 46.7 105 49.5 83 39.2 61 39.4 73 47.1 27 32.9 44 53.7 210 42.5 221 44.7 Children under 5 who had Diarrhoea Yes 2 4.4 1 2.2 - - - - - - - - - - - - 2 4.4 1 2.2 No 22 48.9 20 44.4 - - - - - - - - - - - - 22 48.9 20 44.4 Children with fever in past 2 weeks Yes 2 4.4 2 4.4 - - - - - - - - - - - - 2 4.4 2 4.4 No 22 48.9 19 42.2 - - - - - - - - - - - - 22 48.9 19 42.2 Page 57 of 57 Households able to access money to pay for unexpected expenses Yes - - - - - - - - - - - - - - - - 130 20.5 270 42.7 No - - - - - - - - - - - - - - - - 75 11.8 158 25