EVALUATION MID-TERM PERFORMANCE EVALUATION OF THE USAID/TANZANIA KIZAZI KIPYA “NEW GENERATIONS” PROGRAM April 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by the ME&A Data for Development Program. USAID/TANZANIA MID-TERM PERFORMANCE EVALUATION OF THE KIZAZI KIPYA “NEW GENERATIONS” PROJECT Submitted: April 10, 2019 Prepared by: Terence Beney, Team Lead, NORC Jake Laden, Evaluation Advisor, NORC Gerald Usika, Survey Specialist and Evaluation Manager, NORC Nasson Konga, Monitoring and Evaluation Specialist, NORC Research supported by: Julie Tumbo, Subject Matter Expert Charles Mlwande, Subject Matter Expert Zena Mabeyo, Social Welfare Expert Bahati Tenga, Researcher Daud Siwalaze, M&E Specialist Submitted by: David Hughes, Acting Chief of Party, ME&A USAID/Tanzania Data for Development Program Plot 122 Mwai Kibaki Road, Mikocheni A, Dar es Salaam, Tanzania Email: Dhughes@engl.com Cover Photo: Community Case Workers in Bukoba by Daud Siwalaze. Credit: Data for Development DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT The USAID Data for Development Project (Data for Development) conducted a mid-term performance evaluation of the Kizazi Kipya Project implemented by Pact and its sub-grantees. The project’s goal is to improve the well-being of one million HIV-affected orphans and vulnerable children (OVC) and their families. Key expected results include improved health, nutrition, education, protection, livelihoods and psychosocial outcomes, as well as strengthening the HIV continuum of care among HIV positive OVC, youth, and their families. The evaluation’s purpose is to assess project performance, analyze the effectiveness of its approaches, assess its prospects for sustainability, and help the Mission to identify best practices and lessons learned. The project helped strengthen the national case management system and contributed to the continuum of care for HIV positive children and adolescents; however, there remains a need to optimize the referral to care and treatment system and scale-up services to support treatment adherence and viral suppression. The project contributed to household resilience and women’s empowerment through WORTH+ savings and loan groups, which show promising sustainability. Violence against children prevention and parent/guardian training contributed to changing caregiver behaviors towards children. Conversely, challenges remain in victim response and resolution of gender-based violence cases. Case management improved access to services: addressing service delivery needs locally, increasing the frequency of client referrals to services, and smoothing the referral process. However, challenges remain to the implementation and sustainability of the community volunteer approach. Ongoing donor commitments and increased government commitment are needed to bring the national case management system to scale and sustain it after project completion. CONTENTS EXECUTIVE SUMMARY................................................................................................................................................... i EVALUATION PURPOSE AND EVALUATION QUESTIONS ...................................................................... i PROJECT BACKGROUND ..................................................................................................................................... i EVALUATION DESIGN AND METHODS.......................................................................................................... i KEY FINDINGS AND CONCLUSIONS............................................................................................................... i RECOMMENDATIONS........................................................................................................................................... v 1.0 EVALUATION PURPOSE AND EVALUATION QUESTIONS...................................................................... 1 1.1 EVALUATION PURPOSE ........................................................................................................................... 1 1.2 EVALUATION QUESTIONS...................................................................................................................... 1 2.0 PROJECT BACKGROUND..................................................................................................................................... 2 2.1 GOALS AND OBJECTIVES ........................................................................................................................ 2 2.2 TARGET AREAS AND GROUPS.............................................................................................................. 2 3.0 EVALUATION METHODS AND LIMITATIONS .............................................................................................. 3 3.1 SITE SELECTION........................................................................................................................................... 3 3.2 DATA COLLECTION METHODS........................................................................................................... 3 3.3 DATA ANALYSIS METHODS ................................................................................................................... 4 3.4 METHODOLOGICAL LIMITATIONS..................................................................................................... 4 4.0 FINDINGS AND CONCLUSIONS ....................................................................................................................... 4 4.1 EQ1: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN STRENGTHENING THE CONTINUUM OF CARE FOR HIV POSITIVE CHILDREN AND ADOLESCENTS?.............................. 4 4.3 EQ1.2: WHAT FACTORS EXPLAIN THE ACHIEVEMENT OR UNDER-ACHIEVEMENT OF THESE RESULTS?....................................................................................................................................................... 7 4.4 EQ 2: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN CONTRIBUTING TO HOUSEHOLD RESILIENCE AND WOMEN’S EMPOWERMENT THROUGH ECONOMIC STRENGTHENING ACTIVITIES? .............................................................................................................................................................. 10 4.5 EQ2.1: TO WHAT EXTENT HAS KIZAZI KIPYA CONTRIBUTED TO HOUSEHOLD RESILIENCE AND WOMEN’S EMPOWERMENT THROUGH ECONOMIC STRENGTHENING ACTIVITIES?.............................................................................................................................................................. 11 4.6 EQ2.2: WHAT FACTORS EXPLAIN THE ACHIEVEMENT OR UNDERACHIEVEMENT OF THESE RESULTS?..................................................................................................................................................... 14 4.7 EQ3: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN CONTRIBUTING TO VIOLENCE AGAINST CHILDREN PREVENTION AND RESPONSE?............................................................................ 14 4.8 EQ3.1 TO WHAT EXTENT HAS KIZAZI KIPYA CONTRIBUTED TO VIOLENCE AGAINST CHILDREN PREVENTION AND RESPONSE?............................................................................ 15 4.9 EQ3.2: WHAT FACTORS EXPLAIN THE ACHIEVEMENT OR UNDER-ACHIEVEMENT OF THESE RESULTS?..................................................................................................................................................... 17 4.10 EQ4: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN CONTRIBUTING TO HIV PREVENTION AMONG ADOLESCENTS, AND ADOLESCENT GIRLS IN PARTICULAR?............. 20 4.11 EQ4.1: TO WHAT EXTENT HAS Kizazi Kipya CONTRIBUTED TO HIV PREVENTION AMONG ADOLESCENTS, AND ADOLESCENT GIRLS IN PARTICULAR?.......................................... 21 4.12 EQ4.2: WHAT FACTORS EXPLAIN THE ACHIEVEMENT OR UNDER-ACHIEVEMENT OF THESE RESULTS?..................................................................................................................................................... 23 4.13 EQ5: TO WHAT EXTENT DID THE CASE MANAGEMENT SYSTEM CONTRIBUTE TO THE ACHIEVEMENT OF RESULTS? .................................................................................................................. 24 4.14 EQ 6: WHAT ARE KIZAZI KIPYA’S PROSPECTS FOR SUSTAINABILITY?............................... 26 5.0 RECOMMENDATIONS.......................................................................................................................................... 29 ANNEXES ......................................................................................................................................................................... 32 ANNEX I: EVALUATION STATEMENT OF WORK .................................................................................... 33 ANNEX II: BIBLIOGRAPHY ................................................................................................................................. 40 ANNEX III: KII AND FGD DISCUSSION GUIDES ........................................................................................ 42 ANNEX IV: KII, FGD, AND CCW SURVEY SAMPLING METHODOLOGY......................................... 73 ANNEX V: CCW SURVEY QUESTIONNAIRE............................................................................................... 76 ANNEX VI: CCW SURVEY HIGHLIGHTS TABLES ...................................................................................... 84 ANNEX VII: KIIS and fgds by stakeholders and locations.............................................................................. 92 ANNEX VIII: LIST OF kii PARTICIPANTS........................................................................................................ 95 ANNEX IX: necessity of prioritizing hiv positive children and adolescents.............................................. 98 ANNEX X: PEPFAR OVC Graduation benchmarks.................................................................................... 102 ANNEX Xii: CASE STUDY ON THE SUSTAINABILITY OF WORTH+ GROUPS ........................... 103 ANNEX Xii: DISCLOSURE OF ANY CONFLICTS OF INTEREST......................................................... 106 LIST OF TABLES Table 1: Key Characteristics of VLSGs....................................................................................................................... 12 Table 2: OVC Funds, Socials Funds, and CRMC in Project Supported VSLGs, Disaggregated by Cluster as of FY 2018 .................................................................................................................................................................... 13 Table 4: Furaha Caregiver Behavior Toward Children (Waves 1, 2, and 3 Q1 FY 2019)............................. 16 Table 5: Furaha Caregiver Parenting Attitudes (Waves 1, 2, and 3 Up to Q1 FY 2019)............................... 16 Table 6: Prevention Specific Interventions Delivered Through the Teen Club Platform .............................. 21 Table 7: Statistically Significant Improvement in Communication between Furaha Caregivers and Youth .............................................................................................................................................................................................. 22 HIV Diagnosis, Treatment and Viral Suppression Estimates, COP18.............................................................. 101 HES Cumulative up to Q4, FY 2018 ........................................................................................................................ 104 LIST OF FIGURES Figure 2: Number of Beneficiaries Counted for the TZ_ECON Indicator and Corresponding Percentage of the Target Value for a Given Quarter................................................................................................................... 11 Figure 3: Distribution of Referrals Across Service Categories............................................................................. 25 HIV Prevalence by Sex....................................................................................................................................................99 Viral Load Suppression by Sex................................................................................................................................... 100 HES Cumulative up to Q4, FY 2018 ........................................................................................................................ 104 ACRONYMS Acronym Description AGYW Adolescent Girls and Young Women AIDS Acquired Immune Deficiency Syndrome AOR Agreement Officer Representative ART Antiretroviral Therapy ARV Antiretroviral HIV Drugs ASRH Adolescent Sexual and Reproductive Health CC City Council CCW Community Case Worker CDO Community Development Officer CHAC Council HIV and AIDS Coordinator CHMT Council Health Management Team CHSSP Community Health and Social Welfare Systems Strengthening Program CHW Community Health Worker CLHIV Children Living with HIV COCODA Community Concerns of Orphans and Development Association COP Country Operational Plan CPT Child Protection Team CRMC Community Resource Mobilization Committee CSO Civil Society Organization CTC Care and Treatment Clinic DACC District AIDS Control Coordinator DC District Council DCDO District Community Development Officer DHIS District Health Information System DMO District Medical Officer DREAMS Determined, Resilient, Empowered, AIDS-free, Mentored and Safe DSWO District Social Welfare Officer EQ Evaluation Question ES Economic Strengthening EGPAF Elizabeth Glaser Pediatric AIDS Foundation ET Evaluation Team FGD Focus Group Discussion FY Fiscal Year GBV Gender-Based Violence GLBL Girls Lets Be Leaders GoT Government of Tanzania HIV Human Immunodeficiency Virus HJF Henry M. Jackson Foundation IP Implementing Partner IR Intermediate Result JSI John Snow, Inc. KII Key Informant Interview LGA Local Government Authority LV Livelihood Volunteer Acronym Description M&E Monitoring and Evaluation MAC Multi-Sectoral AIDS Committee MC Municipal Council MER Monitoring, Evaluation and Reporting MIS Management Information System MOCSO Mwanza Outreach Care and Support Organization MOHCDGEC Ministry of Health, Community Development, Gender, Elderly, and Children MVC Most Vulnerable Children MVCC Most Vulnerable Children’s Committee MUAC Mid-Upper Arm Circumference NACS Nutrition Assessment, Counseling, and Support NICMS National Integrated Case Management System NIMR National Institute for Medical Research OVC Orphans and Vulnerable Children PEPFAR President’s Emergency Plan for AIDS Relief PLHIV People Living with HIV PO-RALG President’s Office for Regional Administration and Local Government Q Quarter SOP Standard Operating Procedure SWO Social Welfare Officer TACAIDS Tanzania Commission for AIDS TASAF Tanzanian Social Action Fund TC Town Council TDHS Tanzania Demographic and Health Survey UNAIDS Joint United Nations Program on HIV/AIDS UNICEF United Nations Children’s Fund USAID United States Agency for International Development USSD Unstructured Supplementary Service Data VAC Violence Against Children VAWC Violence Against Women and Children VAWC-PC Violence Against Women and Children Protection Committee VLS Viral Loan Suppression VSLG Village Savings and Loans Group WSA Whole School Activity WSWO Ward Social Welfare Officer i EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of this mid-term performance evaluation of the United States Agency for International Development (USAID) Kizazi Kipya: USAID New Generations Project is to assess its performance and achievements, analyze the effectiveness of various approaches used by the project, and assess prospects for sustainability. The evaluation sought to answer the six evaluation questions (EQ) found below under Key Findings and Conclusions. PROJECT BACKGROUND Kizazi Kipya is a five-year project (July 2016-July 2021) implemented by Pact and its subgrantees. Kizazi Kipya’s planned outcomes include better financial resources for parents and caregivers of orphans and vulnerable children (OVC), as well as improved access to health and Human Immunodeficiency Virus (HIV) services for children and adolescents. Its goal is to improve the well-being of one million HIV-affected OVC and their families. Key expected results include improved health, nutrition, education, protection, livelihoods, psychosocial outcomes, and a strengthening of the HIV continuum of care among HIV positive OVC, youth, and their parents/caregivers. The project works towards the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) goal of 90-90-90, where 90 percent of people know their HIV status, 90 percent of those positive receive antiretroviral therapy (ART) and 90 percent of those receiving ART achieve viral suppression. EVALUATION DESIGN AND METHODS The evaluation used a mixed-methods data collection design consisting of five qualitative and quantitative data collection methods: 1) document review, 2) key informant interviews and group interviews, 3) focus group discussions, 4) community case worker (CCW) survey, and 5) review of Kizazi Kipya performance data. Six sites were selected purposively for the evaluation with priority given to regions with high numbers of project-supported civil society organizations (CSOs); districts/councils active activities during 2017 and 2018; districts providing a balance of urban and rural locations; and CSOs focused exclusively on case management and those providing case management and additional services/interventions. KEY FINDINGS AND CONCLUSIONS EQ1: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN STRENGTHENING THE CONTINUUM OF CARE FOR HIV POSITIVE CHILDREN AND ADOLESCENTS? • The project is appropriately designed to contribute to epidemic control by delivering services that support adherence and suppression, but its efficacy in case finding is hindered by inherent design limitations—it is not optimized as a case finding activity, nor is it aligned with Government of Tanzania (GoT) structures to optimally support adherence monitoring. • Referrals from care and treatment clinics (CTCs) have yet to be optimized due to procedures not yet systematized or consistently implemented. These limitations are ultimately demonstrated by the fact that while there are up to 180,000 children living with HIV (CLHIV) in Tanzania, less than 11,000 are enrolled in Kizazi Kipya. • The sheer volume of services delivered, combined with the fact that delivery is case managed, suggests that the continuum of care for HIV positive children enrolled has been strengthened. • There remain difficulties determining the extent to which services have been layered for enrolled children. For instance, is it possible to distinguish between the proportions of CLHIV according ii to the number of services received? Also, the national case management system is still at a nascent stage and the effectiveness of CCWs is subject to multiple risks. EQ2: HOWEFFECTIVE HAS KIZAZI KIPYA BEEN IN CONTRIBUTING TO HOUSEHOLD RESILIENCE AND WOMEN’S EMPOWERMENT THROUGH ECONOMIC STRENGTHENING (ES) ACTIVITIES? • The number of village savings and loans groups (VSLGs) has increased steadily over the project’s first two years, 1 and the number of VSLGs with either an OVC or a social fund has grown rapidly reaching 91 percent of project-supported VSLGs by the end of Fiscal Year (FY) 2018. In addition, the total cumulative value of these two funds has nearly tripled during FY 2018. • The project made progress fostering the development of Community Resource Mobilization Committees (CRMCs), which reached 15 percent by the end of FY 2018.2 Over FY 2018, 10,259 OVC benefited from CRMC resources and 5,072 benefited from OVC funds. • The Southern Highland geographic cluster lagged behind its peers in terms of implementation of OVC, social funds, and CRMCs by the end of FY 2018. In addition, the project did not meet its Year 2 target for VSLGs supported, reaching 76 percent of that goal by the end of FY 2018. • OVC parents and guardians were positive and enthusiastic about the VSLGs. Most were part of a group, and in most cases, there were people from multiple groups operating in the same area. • Some enabling factors, such as resources for livelihood volunteers (LVs), remain a challenge. Allowances received by LVs were just enough to cover transportation costs for regular visits, but sometimes they had to use their own money or turn down an emergency visit because they could not afford to travel. Similar problems were noted regarding referrals, whether it was the extra travel costs for the LVs themselves or the household requesting travel allowances. EQ3: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN CONTRIBUTING TO VIOLENCE AGAINST CHILDREN PREVENTION AND RESPONSE? • Prevention and training strategies to address violence again children (VAC) and gender-based violence (GBV) have helped change caregiver behavior towards children and parenting. Response to VAC via CCWs has improved knowledge on where to report such incidences. People feel safe to report or seek advice from CCWs who belong to their community. • Challenges remain in the resolution of GBV cases. This includes perpetrators that are powerful or close relatives and can result in long, drawn out court cases and challenges with geographic coverage, especially in rural areas. Cases are challenged by community norms to settle incidences within the community or family and not report formally to the district Child Protection Team (CPT) or other authorities. Some CCWs follow protocols and use the GBV and child abuse incident reporting form and VAC service register for follow-up linkages with the police gender desk, district social welfare officers, CPT or violence against women and children committee (VAWC); others succumb to community pressure to settle cases through families. • High workloads are a challenge for CCWs; there are not enough trained CCWs in the wards and villages. CCW sustainability is in question due to reported challenges such as low monthly stipends and increased workloads. • Determined, Resilient, Empowered, AIDS-free, Mentored, and Safe (DREAMS) participants feel that interventions should be extended to adolescent boys/young men as they are also at risk of being perpetrators or victims of GBV. Participants further stressed that HIV/AIDS affects both 1 From FY16 where KK inherited VSLGs from Pamoja Tuwale (6,305) till FY18 (9,028) 2 Pact Kizazi Kipya Information Handout. iii genders, so when the boys learn how to protect themselves against violence, it will be helpful. Currently, Kizazi Kipya is targeting both male and female OVC, and Huru facilitators include boys and girls in their sextual reproductive health training classes. • Job aids developed by Kizazi Kipya in collaboration with Tulonge Afya have proven to be useful for youth and children behavior change and communication. • Kizazi Kipya has introduced several technological innovations, which have enhanced its data management processes and procedures. To simplify referrals tracking, it introduced a bi￾directional referral system. It also developed a District Health Information System 2 database linked to an unstructured supplementary service data (USSD) platform, which allows real-time data collection and submission via any type of phones and real-time payments of monthly stipends to CCWs. EQ4: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN CONTRIBUTING TO HIV PREVENTION AMONG ADOLESCENTS, AND ADOLESCENT GIRLS IN PARTICULAR? • Kizazi Kipya has been monitoring outcomes of its DREAMS focused prevention curricula, Huru and Furaha. Huru includes a behavior change education component for adolescent girls and young women aged 4-18 for HIV/AIDs prevention in and outside of the school setting. Huru’s pre- and post-test results on HIV knowledge acquisition are compelling. Of the 20 items tested, 18 showed a statistically significant improvement, indicating a potentially effective prevention intervention. • The Furaha parenting pre- and post-test results also demonstrate statistically significant improvements in the behaviors of caregivers and children. Of particular interest to prevention is the quality of communication between caregivers and youth on sexual behavior topics. • Several challenges were identified for school-based curriculum: • The curriculum targeting specific ages of out-of-school OVC girls, of which Pact does not have sufficient numbers enrolled in each council to organize into groups in their own communities. • The curriculum requires intensive training and monitoring of peer educators who facilitate the sessions, but the CSOs themselves first require more capacity-building with regards to recruitment of appropriate youth facilitators and designing youth-centered programming. • The project has not established which girls to link to social behavior change communication sessions delivered under Sauti to avoid duplication of delivery in councils where both work. • The difficulty of tracking layering is exacerbated for prevention interventions and DREAMS in particular because services are delivered by multiple activities. EQ5: TO WHAT EXTENT DID THE CASE MANAGEMENT SYSTEM CONTRIBUTE TO THE ACHIEVEMENT OF RESULTS? • Case management improves access to services by mobilizing and sensitizing community actors to jointly address service local delivery needs and by increasing the frequency of client referrals to services and smoothing the referral process. • Case management has contributed to improved service quality, primarily by augmenting the social workforce with a trained cadre of workers who, in turn, introduce effective practices and catalyze quality improvement by raising quality issues with relevant authorities. • Case management should improve the effectiveness of services if the case plans are implemented and outcomes concluded. Currently, tracking case completion is a challenge as information from referrals is not linked with data from CTCs and is not shared with CCWs or the implementing partner (IP) to enable follow-up. iv EQ6: WHAT ARE KIZAZI KIPYA’S PROSPECTS FOR SUSTAINABILITY? • National commitment and institutionalization are satisfactory in the sense that the system for supporting OVC case management has become formalized under the national integrated case management system (NICMS) and is being reinforced by bodies such as the Tanzania Commission for AIDS (TACAIDS) and President’s Office for Regional Administration and Local Government (PO-RALG). The national/central government demonstrates internalization of the NICMS in its plans and activities. PO-RALG officials and other government staff play pivotal roles in cascading training and adherence to national case management guidance. • Kizazi Kipya’s design links to the four domains under the OVC benchmarks: 1) healthy, 2) stable, 3) safe, and 4) schooled. Kizazi Kipya has started planning graduation of households and to devise ways to measure these benchmarks and will follow these graduation benchmarks moving forward. • High levels of motivation among CCWs and lead CCWs encourages optimism. Motivation to continue working comes from a strong sense of mission and is less driven by financial incentives. • Financial and resource challenges affecting CCWs’ ability to implement in project-supported councils threaten the sustainability of the case management effort over the long term. Furthermore, many CCWs find their stipends and travel reimbursement inadequate to cover the costs to serve households and keep themselves afloat financially. • As Kizazi Kipya’s sister program, the Community Health and Social Welfare Systems Strengthening Program (CHSSP) closes, it is questionable whether the cascading of training will continue. PO￾RALG is emphasizing social welfare and OVC, and there are PO-RALG master trainers qualified to provide case management training in the long-term; however, few to no initial or follow-on trainings have taken place under GoT funding alone, and it will likely require USAID funding to continue. • While social welfare officers and other supervisors are on GoT payroll, their ability to reach CCWs requires travel stipends and material costs not covered by the GoT so far. • CTC staff, community health workers (CHW), and CCWs are able to effectively link OVC to service providers, making it easier for CTCs to get clients through home visits and linking them to follow-ups. Moreover, communities are more informed on the importance of HIV testing, GBV prevention, caring families, and child safety. • There are self-sustaining prospects of WORTH+ groups. WORTH+ groups enable members to work with their small savings and not push them beyond their means. This ensures sustainability in that members cannot get over indebted. Most WORTH groups, moreover, have registered formally with local government authorities, which qualifies them for government support, and are linking participants to the Tanzanian Social Action Fund (TASAF) and health insurance. • DREAM facilitators will continue to educate girls during the regular class sessions and not during extra time and Saturdays. Due to funding limitations, they will not be able to provide scholastic materials, sanitary pads, and other goods that Kizazi Kipya used to provide. • In some councils, government social welfare departments are working to hand over the Kizazi Kipya-supported OVC to education departments to ensure continuity after Kizazi Kipya. The district governments have also mobilized village governments to start fundraising to support OVC. • The NICMS has contributed to institutional and budgetary commitments; however, gaps remain in the GoT’s ability to be self-reliant and contribute to OVC and people living with HIV (PLHIV) households. v RECOMMENDATIONS • Review geographic limitations to targets to allow for services to HIV positive children falling within the facility but outside of project catchment area. • Provide more intensive coordination with facilities and care and treatment partners to facilitate enrollment of HIV positive children. Also, implement a constant presence of senior cadre at facilities to oversee completion of referrals and participate in clinical tasks, such as switch committees, along with regular case conferencing including facility staff. • Develop custom indicators to improve the continuum of care, including a viral load suppression indicator for OVC partners and a linkage rate indicator for care and treatment partners. • Include young men in DREAMS interventions to better understand what GBV is and protect themselves and girls against violence and HIV infections. • Work with the GoT to increase the monthly CCW stipend and to decrease their dropout rate. • Provide more nutritional information for HIV positive OVC, especially those on ART. • Develop a DREAMS tracker for all DREAMS partners in PEPFAR Tanzania to use. Record level data should be uploaded in a system that is hosted by one IP that has strategic information capacity so that the technical support for the system is consistent and data quality assessments can be conducted by that IP. This system would make tracking of layering possible for the entire PEPFAR DREAMS program, as well as additional analysis of aggregated DREAMs data. Models of such an arrangement are available in other PEPFAR countries, e.g., Uganda. • Revisit targeting and costing models to find the resources for scale-up. Although there are other prevention efforts, curricula-based prevention efforts are essential for broader efficacy. The scale of implementation in Kizazi Kipya is questionable and should be revisited. • Develop strategies for improving the commitments of local actors to ensure treatment and care as well as case management over the long-term. PO-RALG and Kizazi Kipya should encourage local government authority (LGA) officials to establish and fund their own council level committees’ activities. • For health service providers to take over HIV treatment and care, LGAs must fund and sustain the practices promoted by Kizazi Kipya. Also, ownership and involvement of GoT officials at ward and village levels are vital. • GoT or other donor commitments will be needed to sustain CCW stipends, travel, printing, and maintenance of tools and referral forms. Other operational requirements will be needed for CCW supervisors and social welfare officers to provide continued supportive supervision. • Succession planning needs to be implemented and mainstreamed. The likelihood that the GoT or other donors maintain the costs of structural operations for case management will increase once Kizazi Kipya begins to work with stakeholders in this effort. • USAID should encourage the GoT’s national commitments to funding in an effort to sustain the NICMS once the project ends. Leveraging other donor support and continuing PEPFAR support will be required in the intermediate term. 1 1.0 EVALUATION PURPOSE AND EVALUATION QUESTIONS 1.1 EVALUATION PURPOSE The purpose of this mid-term performance evaluation of the United States Agency for International Development (USAID) Kizazi Kipya project is to 1) assess the extent to which project performance is on track to achieve expected outcomes and analyze the effectiveness of it approaches and 2) document good practices, gaps/limitations, and lessons learned. Primary audiences for the evaluation include the Mission’s technical office, Pact, and Pact’s partners so as to inform the second half of project implementation and the design of the follow-on orphans and vulnerable children (OVC) support project. 1.2 EVALUATION QUESTIONS The evaluation seeks to answer the following six evaluation questions (EQs) and related sub-questions.3 EQ1: How effective has Kizazi Kipya been in strengthening the continuum of care for HIV positive children and adolescents? ∉ EQ1.1: To what extent has Kizazi Kipya strengthened the continuum of care for HIV positive children and adolescents? ∉ EQ1.2: What factors explain the achievement or under-achievement of these results? EQ2: How effective has Kizazi Kipya been in contributing to household resilience and women’s empowerment through economic strengthening (ES) activities? ∉ EQ2.1: To what extent has Kizazi Kipya contributed to household resilience and women’s empowerment through ES activities? ∉ EQ2.2: What factors explain the achievement or under-achievement of these results? EQ3: How effective has Kizazi Kipya been in contributing to violence against children prevention and response? ∉ EQ3.1: To what extent has Kizazi Kipya contributed to violence against children prevention and response? ∉ EQ3.2: What factors explain the achievement or under-achievement of these results? EQ4: How effective has Kizazi Kipya been in contributing to HIV prevention among adolescents, and adolescent girls in particular?4 ∉ EQ4.1: To what extent has Kizazi Kipya contributed to HIV prevention among adolescents and, in particular, adolescent girls? ∉ EQ4.2: What factors explain the achievement or under-achievement of these results? EQ5: To what extent did the case management system contribute to the achievement of results? EQ6: What are Kizazi Kipya’s prospects for sustainability? 3 USAID Kizazi Kipya Technical Brief. 4 *One of three DREAMS councils where Kizazi Kipya is active: Kyela DC, Mbeya CC, and Temeke, Dar es Salaam. 2 2.0 PROJECT BACKGROUND 2.1 GOALS AND OBJECTIVES The Kizazi Kipya “New Generations” project is a five-year (July 05, 2016-July 04, 2021) project implemented by Pact and its subgrantees. The project’s goal is to improve the well-being of one million Human Immunodeficiency Virus (HIV)-affected OVC and their families. Key expected results include improved health, nutrition, education, protection, livelihoods, and psychosocial outcomes, as well as strengthening the HIV continuum of care among HIV positive OVC, youth, and their parents/caregivers. The project responds to the need in Tanzania of strengthening and operationalizing the national integrated case management system (NICMS) at the local district, ward, and village levels. It builds off a prior project in Tanzania, Pamoja Tuwalee, to work towards the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) goal of 90-90-90, where 90 percent of people know their HIV status, 90 percent of those positive are receiving antiretroviral therapy (ART), and 90 percent of those receiving ART achieve viral suppression. Kizazi Kipya collaborates with the Youth Department under the Prime Minister’s Office at the policy level and the President’s Office for Regional Administration and Local Government (PO-RALG) at the implementation level to deliver a package of adolescent and youth services at council, ward, village, and school levels. The project provides sub-grants to civil society organizations (CSOs) who work with the local government authorities (LGAs) to implement activities.5 Pact’s Kizazi Kipya partners include the John Snow International (JSI) Research & Training Institute, USAID/ Community Health and Social Welfare Systems Strengthening Program (CHSSP), the Elizabeth Glaser Pediatric AIDS Foundation (EGPAF), Huru International, Railway Children Africa, Boresha Afya (Deloitte￾Southern Highlands, Management and Development for Health-Central Kagera and Dar es Salaam, FHI 360-Northern and Lake Zone), Henry M. Jackson Foundation (HJF) (non-consortium), and the Ifakara Health Institute. 2.2 TARGET AREAS AND GROUPS Pact’s original geographic coverage was 130 councils, which included 42 scale-up councils and 88 sustained councils. The plan was to transition off from all sustained councils by September 30, 2017. In 2017, PEPFAR changed scale-up councils, previously known as Tier 1a and 1b, to PEPFAR priority councils. From Fiscal Year (FY) 2018-FY 2021 (years two to five), Pact will work in 79 PEPFAR priority councils of which 67 had beneficiaries enrolled into care at the end of FY 2017. In FY 2017, Pact worked in a total of 131 councils, a combination of scale-up (Tier 1a and Tier 1b) and sustained (Tier 2) councils. In FY 2017, Pact wrote Standard Operating Procedures (SOPs) to guide CSOs to transition beneficiaries in all Tier 2 (sustained) councils off PEPFAR support before September 30, 2017. Pact inherited 62 councils from the Pamoja Tuwalee OVC project that were either Tier 2 councils or neither Tier 1 nor Tier 2 councils; however, Pact did not plan to work in any of these 62 councils from FY 2018 onwards. By the end of FY 2017, Pact was implementing OVC activities in 67 Tier 1 councils. 5 UNAIDS country factsheets, 2017, accessed April 2019 http://www.unaids.org/en/regionscountries/countries/unitedrepublicoftanzania/ 3 3.0 EVALUATION METHODS AND LIMITATIONS The information below describes the evaluation methodology, including data collection methods, sampling methods, and methodological limitations. Primary fieldwork/interviews for the evaluation took place from February 4-March 8, 2019. 3.1 SITE SELECTION Six locations were selected purposively for the evaluation fieldwork. Priority was given to regions with high numbers of CSOs supported by Kizazi Kipya and districts/councils that had active activities during the first half of the project in both 2017 and 2018. Districts were selected to provide a balance between urban and rural CSOs focused exclusively on case management and those that provide case management and additional services/interventions, and Determined, Resilient, Empowered, AIDS-free, Mentored and Safe (DREAMS) and non-DREAMS locations. Based on the above criteria, final sites selected for the evaluation included Dar es Salaam and Dodoma (to capture national stakeholders); Temeke Municipal Council (MC), Mbeya City Council (CC) (two of the six DREAMS Districts); and Njombe Town Council (TC) in Njombe, Nyamagana MC in Mwanza, and Bukoba District Council (DC) in Kagera (Non-DREAMS locations). 3.2 DATA COLLECTION METHODS The evaluation used a mixed-methods data collection design consisting the following qualitative and quantitative data collection methods: 1) document review, 2) key informant interviews and group interviews (KIIs), 3) focus group discussions (FGDs), 4) community case worker (CCW) survey, and 5) review of Kizazi Kipya performance data. Each method is described below. Document Review: The evaluation team (ET) reviewed project documents and other documents related to health and social welfare. Pact provided some documents, and the ET found others during its own literature search. (See Annex II for a comprehensive bibliography.) Key Informant Interviews: The ET conducted 36 KIIs with 121 individuals, including 79 women and 42 men, at the national, regional, and district levels, including one KII with USAID technical staff; seven KIIs with Pact staff and collaborating partners; three KIIs with Government of Tanzania (GoT) staff from the Tanzania Commission for AIDS (TACAIDS) and PO-RALG at the national level; and 26 KIIs with district health and social welfare authorities and sub-grantee CSOs. Local health and social welfare actors interviewed included Council Health Management Team (CHMT), District Medical Officer (DMO), Council HIV and AIDs Coordinator (CHAC), District AIDs Control Coordinator (DACC), District Social Welfare Officers (DSWO), Community Development Officers (CDO), Child Protection Teams (CPT)/Most Vulnerable Children’s Committees (MVCC), Facility Care and Treatment Clinics (CTCs) in￾charge, and CCW supervisors (WSWOs and CDOs). (See Annex IV for the KII and FGD discussion guides, Annex VII for a breakdown of KII and FGD participants by stakeholder and location, and Annex VIII for a detailed list of the KIIs completed). Focus Group Discussions: The ET conducted 14 FGDs with 147 individuals, including 99 women and 48 men at the ward and village level, including five FGDs with CCWs and lead CCWs, two with adolescent girls and young women (AGYW) in DREAMS, one with livelihood volunteers (LVs), and six with parents and guardians in WORTH+ economic empowerment groups. CCW Survey: The ET worked through its local research partner, Utafiti, to conduct a phone survey of a representative sample of 269 (148 female and 121 male) CCWs. The survey instrument covered all elements of CCW training related to CHSSP and the casework implemented by the same CCWs under 4 Kizazi Kipya. (See Annex IV for more on the KII, FGD, and CCW survey sampling methodology; Annex V for a copy of the CCW survey questionnaire; and Annex VI for a summary of the CCW survey results.) 3.3 DATA ANALYSIS METHODS To analyze the qualitative data, the ET conducted a content analysis using the Dedoose qualitative analysis software. The ET used thematic coding to identify and clarify patterns in the data among the codes allowing it to draw inferences from the data by systematically identifying themes and sub-themes and assessing their relative importance in answering the EQs. To analyze the CCW survey data, the ET used Microsoft Excel to analyze and visualize data using descriptive statistics. 3.4 METHODOLOGICAL LIMITATIONS The above evaluation methodology involved the following potential limitations: 1) reliance on, at times, unverifiable perceptions and anecdotes by key informants; 2) use of unverified Kizazi Kipya performance monitoring data; 3) recall bias by key informants commenting on past events; 4) halo bias by key informants incentivized to under-report socially undesirable answers or refrain from criticizing colleagues or people on whom they depend for support; 5) sampling bias given the purposive (non-random) selection of key informants for KIIs and FGDs and that the CCW survey sampling frame was limited to those CCWs who had access to a tele/mobile phone; and 6) the difficulty assessing the sustainability of project interventions that were ongoing at the time of the evaluation fieldwork. To mitigate the above limitations, the ET sought to verify findings from one data source and method with findings from other data sources and methods through triangulation. The ET also worked closely with Kizazi Kipya and Utafiti to implement sound qualitative and quantitative sampling methods and to impose strict data collection quality control methods. To address limitations related to evaluating project sustainability, the ET asked probing questions related to sustainability wherever relevant. This allowed the ET to estimate the “likelihood” of sustainability and the factors contributing to this likelihood. 4.0 FINDINGS AND CONCLUSIONS 4.1 EQ1: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN STRENGTHENING THE CONTINUUM OF CARE FOR HIV POSITIVE CHILDREN AND ADOLESCENTS? Findings on evaluation questions 1, 2, 3, and 5 are all pertinent to assessing the activity’s strengthening of the continuum of care for HIV positive children and adolescents, in so far as performance in each technical area effects all beneficiaries, including this sub-population. The evaluation responds to EQ 1 considering the extent to which the project successfully identifies and enrols these beneficiaries, how well it provides them with a package of needed services, and whether it facilitates the achievement of HIV-related health outcomes. (See Annex IX for background information affirming the necessity for targeting HIV positive children and adolescents.). 4.2 EQ1.1: TO WHAT EXTENT HAS KIZAZI KIPYA STRENGTHENED THE CONTINUUM OF CARE FOR HIV POSITIVE CHILDREN AND ADOLESCENTS? Identifying and Enrolling HIV Positive Children and Adolescents Kizazi Kipya engages with high-volume CTCs to identify and enroll beneficiaries. It is one of the two key enrollment platforms; the other being recruiting beneficiaries through engagement with community-level structures. While the former yields a higher proportion of people living with HIV (PLHIV) and children living with HIV (CLHIV) clients for enrollment, provided the HIV status of those referred from the facility to the OVC project is the prioritized screening criteria, the latter allows the project to contribute to 5 finding new cases, provided community engagement is targeted and screening for risk of HIV infection is effective. Neither of these modalities is as yet yielding satisfactory results, however, as demonstrated by project performance on the OVC_HIVSTAT indicator (see Figure 1). Of the 484,758 OVC below 18 years served in FY18 Q4, 444,652 (92 percent) reported their HIV status to the IP. Of these, 14,251 (3 percent) disclosed being HIV positive, 302,403 (68 percent) reported being HIV negative, and 127,998 (29 percent) reported no status. The OVC who reported their status as negative, together with those for whom a test is not indicated, make up 74 percent of those 18 years and under enrolled in the program. Even taking into account the possibility of additional CLHIV within the large (and problematic) proportion of beneficiaries reporting no status, it is apparent that, with only 3 percent of beneficiaries being HIV positive, Kizazi Kipya is neither finding nor enrolling what might be considered a significant proportion of the estimated 180,000 CLHIV. The reasons contributing to the limited case finding and enrolment of CLHIV, as well as steps Kizazi Kipya is taking to address these limitations, are discussed in the following section. Figure 1: Kizazi Kipya OVC_HIVSTAT Result FY18 Q4 Further Implications of the OVC_HIVSTAT Results The large proportion of beneficiaries with undisclosed status in FY18 Q4 is a typical and persistent feature of the indicator results across multiple reporting periods. This information deficit undermines the effectiveness of case planning and service delivery to both HIV negative beneficiaries (for whom prevention interventions should be prioritized), as well as HIV positive beneficiaries (for whom enrollment and retention in treatment, and adherence for viral load suppression, should feature prominently in their case management). Potential reasons for this result relate to beneficiary behavior, implementation or case management practices, and structural limitations. Fear of stigma remains a constraint on health seeking behavior, including testing, and it is conceivable that many beneficiaries have never been tested, do not know their status and have nothing to report, and choose this option when reporting on their status. Fear of stigma would also limit voluntary disclosure, 6 and it is likely that some PLHIV enrolled in Kizazi Kipya are deliberately withholding their status from CCWs. If the project is not emphasizing the necessity of declared status for effective case management, CSOs and the social workforce are less likely to expend the effort to obtain it from beneficiaries who do not volunteer the information easily. This may influence how the HIV Risk, Service and Adherence Assessment tool is administered. CCWs are also limited in the extent to which they can encourage disclosure because, according to the National Guidelines for the Management of HIV and AIDS, disclosure support must be conducted by trained health care workers. Addressing this limitation would require substantially increased cooperation between Kizazi Kipya, care and treatment partners, and facilities at local level. Of those who reported their status as HIV positive, 80 percent are on ART, 8 percent are not on ART, while 12 percent did not report their ART status. Kizazi Kipya is integrating this information into case planning, focusing on ensuring close follow up of those not on ART. Providing a Package of Needed Services A strengthened continuum of care implies that CLHIV are receiving a package of needed services, as determined by the vulnerability assessment conducted of the household and documented in that household’s case plan. Implementation of the case plan could be reported as the layering of services for individual beneficiaries in each of the OVC service categories (healthy, stable, safe, and schooled). Unfortunately, the layering of services remains a problematic feature to report, further complicated by the requirement to filter by a particular descriptor, such as HIV status. Currently, report layering would require a considerable data mining effort. A key difficulty is the absence of a unique identifier for each beneficiary. While the project does generate unique identifiers, the algorithm employed to do so has not been successful in generating exclusive identifiers, and duplicates have compromised the integrity of the Kizazi Kipya management information system (MIS), although to a limited degree. Despite these difficulties, an observation can be deduced about layering, based on aggregated counts of services delivered. Layering is built into targeting for the key services measured by custom indicators, namely economic strengthening (TZ_ECON) and nutrition assessment, counseling, and support (NACS) or nutrition assessment, counseling and support (TZ_NUT). The TZ_ECON target is 80 percent of the OVC_SERV target for caregivers and 80 percent of the OVC_SERV target for those 15 to 17 years. Similarly, the OVC_NUT target is 70 percent of OVC_SERV under 18. Home visits for case management also routinely incorporate referrals for participation in savings groups and NACS, activities related to these services. With these parameters, and in the absence of routine reporting on layering, it is fair to assume that, should the targets for these indicators be met, layering of these services is occurring for this sub-population and most OVC_SERV. Home visits should also routinely include early childhood development and positive parenting education for caregivers, although at the close of FY18 both of these activities were in a pre-implementation phase. Psycho-social support is also noted in project documentation as being provided during home visits, but it is not clear what this entails. Kizazi Kipya also implements two activities tailored to this sub-population, namely strengthening volunteer cadre’s skills to support parents of CLHIV and supporting CLHIV to access group-based psycho-social support. While the training of CCWs to support parents of CLHIV exceeded the annual target by the close of FY18, coverage was limited to 28 districts and the training of 1,599 CCWs in total (approximately 10 percent of those mobilized by Kizazi Kipya). Given these parameters, there are likely to be gaps in the delivery of this service. At the close of FY18, adherence support had been extremely limited, with a total of 488 OVC and 213 CLHIV having been provided with linkage to disclosure support and ART adherence education, respectively. The project anticipates that increasing the identification of CLHIV through 7 OVC_HIVSTAT, combined with accelerated training of CCWs in adherence support, will improve these numbers. Achieving Care and Treatment Outcomes Kizazi Kipya issued 68,416 health referrals by the close of FY18, 44,654 (65 percent) of which were for HIV services, and of the latter, 54 percent were completed. While these numbers are not disaggregated by HIV status, they do reflect an active pursuit of health outcomes for project beneficiaries. Once they have been identified and enrolled, the key outcome for CLHIV is viral load suppression (VLS). While it is assumed that informed case management and the layering of services contributes to viral load suppression, it is not an outcome currently tracked by Kizazi Kipya. In spite of the absence of direct data, and despite the inconsistent layering of services, evidence from the KIIs and FGDs suggests that key health outcomes for this beneficiary group, namely knowing their status and being virally suppressed, are being realized for at least some of its members (4/7 FGDs, 3/5 KIIs with CSOs, 3/5KIIs with CTCs in-charge, 2/4 KIIs with DMOs). Kizazi Kipya’s contribution to identifying new positives can be affirmed based on routine tracking of screening for a referral to testing and completed referrals (241 new positives in FY18). However, the project’s role in contributing to VLS cannot be credibly confirmed at this stage, as the viral load test data of HIV positive beneficiaries is not being routinely tracked. 4.3 EQ1.2: WHAT FACTORS EXPLAIN THE ACHIEVEMENT OR UNDER￾ACHIEVEMENT OF THESE RESULTS? Factors Enabling Results for HIV Positive Children and Adolescents Data points to four clusters of practices associated with improved continuum of care for CLHIV: enhanced enrollment from CTCs, enhanced enrollment from communities, strengthening caregiver support to CLHIV, and promoting behavior change among CLHIVs. Enhancing Enrollment from CTCs Although enrollment of CLHIVs from CTCs is not yet optimal, it is the crucial practice for enrolling CLHIVs. Key informants described a number of practices they considered effective for enrolling from the CTC platform. Collaboration with care and treatment partners, EGPAF in particular, has eased the access to facilities (1/1 KII with EGPAF; 2/5 KIIs with Pact; 1/5 KIIs with CTCs in-charge; 1/4 KIIs with DMOs; 1/5 KII with CSOs). “…[O]n the part of the addition of enrollment platforms where we have added the CTC as another platform, this helped to identify the parents with HIV and their children and through this, there has been an increase in the number of beneficiaries enrolled in the project.” KII18_KK_NjombeTC_CSO COCODA With access to the facility, subsequent innovations could be introduced to strengthen linkages and by extension enrollment. The innovations most frequently associated with enhanced enrollment include the assignment of a focal person to each facility (1/1 KII with EGPAF; 3/5 KIIs with Pact; 1/5 KIIs with CTCs in-charge) and putting bi-directional referrals in place (1/1 KII with EGPAF; 2/5 KIIs with Pact; 3/5 KIIs with CTCs in-charge). The efficacy of the bi-directional referrals is due in part to coaching facility staff on applying the criteria for identifying OVC clients for referral to Kizazi Kipya (3/5KIIs with Pact; 3/5 KIIs with CTCs in-charge). Enhancing Enrollment from Communities Kizazi Kipya enrolls the larger proportion of its beneficiaries directly through the community. In addition to engaging with local government and civil structures, the project expanded platforms for beneficiary enrollment to target specifically PLHIV support groups, clients of community-based health and social 8 welfare providers who are also lead CCWs/CCWs, abused children, malnourished children, orphans, and schools. The first two, in particular, potentially improve the probability of finding new CLHIV. Kizazi Kipya offers home-based testing to identify those at high risk of being HIV infected. Home testing offers an attractive entry-point for this high-risk group which is still reserved in its health seeking behavior due to stigma and lack of knowledge (2/5 FGDs with OVC caregivers; 2/5 KIIs with CSOs). “How I joined the Kizazi Kipya project, there were CCWs who passed at our home to provide health service and testing, at home there was a patient (my sister); she was regularly sick, today fever, tomorrow sick!! The CCWs visited us and gave us a referral to go for HIV testing and the results confirmed HIV positive. So, she got the service and was linked to medical service, now she goes on using medicine.” FGD05_KK_MbeyaCC_Parents_Caregivers CCWs are the cadre on which successful implementation of Kizazi Kipya hinges, and have been successful in identifying new HIV cases, although to a limited extent. “The project has enabled us to discover more children with HIV through the help of the CCWs who visit them at their households and bring them to the health centers and through this we have found a lot of infected children.” KII20_KK_NjombeTc_CTC in-charge Case finding by CCWs has been strengthened over the implementation period by equipping them with an improved screening tool. The tool was initially piloted in six councils on a sample of 1,028 project clients and subsequently rolled out in 28 councils and administered to 74,824 clients. Yield from the tool ranged from 3 percent to 4.6 percent. While CCW training in case management has been extensive, their skills in technical areas, particularly in providing services to CLHIV and support to their caregivers, required supplementation. The supplementary training enhanced their efficacy substantially, according to interview respondents (2/5 KIIs with Pact; 1/1 KII with EGPAF; 3/5 KIIs with CSOs). Supporting Caregivers of CLHIV While interventions to support caregivers of HIV-positive children and adolescents are integrated across multiple interventions (e.g., economic support and linking to health services), the project also recognizes the need for specific parenting skills related to the HIV-specific needs of caregivers. In addition, CCWs conduct weekly case management of families with children and/or youth living with HIV to ensure that their unique health, nutrition, protection, and socio-emotional care needs are met and to improve HIV clinical outcomes. Specific support to caregivers of CLHIV is associated with improved health outcomes in KII and FGD responses. “When my wife used to go for testing, she always found out that her viral load count was high, and the same applied to me. The kid was also very weak, but the Lord gave us these providers and they told us that the child wasn’t taking the medication as directed. He then started to take the medicines in the morning and evening. And then he had good adherence on treatment and right now he is in good health too.” FGD04_KK_Bukoba DC_Parents & Guardians Tanzania’s National Guidelines for the Management of HIV and AIDS direct that disclosure be facilitated or conducted by trained health care workers. Kizazi Kipya, through its trained CCWs, helps caregivers understand the importance of disclosure and provide referrals to facilities that conduct disclosure counseling. Disclosure support is a critical psycho-social service that alleviates what caregivers experience as an enormous burden to bear. “There was this boy at our school who was HIV positive. When the mother came, she started crying that she failed to tell her son the truth about his health. What brought that child to the hospital, and he received care and treatment. Now he is doing well and is now in secondary school.” KII08_KK_Mbeya CC_DREAM Huru facilitators 9 Factors Constraining Results for HIV Positive Children and Adolescents A number of constraints limit the capacity of the project to strengthen the continuum of care for HIV positive children and adolescents. Inheriting beneficiaries from the preceding mechanism (Pamoja Tuwalee) limited the available slots for enrollment, while the urgency to fill targets makes the intention of enrolling from facilities and identifying new cases difficult to sustain. Geographic limits further complicate the targeting of HIV positive children, because clients will often fall outside of a facility’s catchment area. However, the key limitations are structural and inherent in the OVC platform. Kizazi Kipya is designed to enable the OVC platform to be built in line with the NICMS. Case Management implies a sustained intervention with a household. It requires a comprehensive assessment of household members’ needs. Once an assessment is made, the CCW is committed to address those needs. Assistance is documented as a written care plan. Case management takes time, which conflicts with the requirements of two HIV specific PEPFAR priorities: pediatric case finding and retention and adherence support. Pediatric HIV case finding requires doing an HIV risk assessment a priori rather than after enrollment. Kizazi Kipya does an HIV risk assessment as part of its core package, after enrollment of an at-risk family; therefore, after the commitment to case management services is already made. If an HIV test is warranted, then the CCW writes HIV testing into the care plan. Regardless of the children’s HIV status, the household remains in case management until everyone in the household meets the mandatory graduation criteria (as defined by PEPFAR in Monitoring, Evaluation, and Reporting (MER) 2.3 guidance), which takes time to achieve. Case finding requires a continuous effort, which OVC programs are not designed for by virtue of their limited enrollment targets. PEPFAR expects OVC projects to enroll CLHIV to support adherence/retention. Three constraints, however, make this a challenging expectation: 1. While CTCs are a major project enrollment platform, procedures are not optimized and have yet to result in a significant proportion of CLHIVs being linked to the project. 2. The household focused case management model imposes a substantial burden on CCWs, who are obligated to deliver strictly defined services to all members of the household. Therefore, while the project does provide adherence support to CLHIV, it is also committed to address all vulnerabilities in the case plan, requiring time and effort that may otherwise be devoted to optimizing adherence support to one family member. 3. PEPFAR requires that home-based care services should merge into the OVC project. But the GoT already has a home-based care platform delivered by volunteers (not CCWs), with a specific government curriculum (not part of the CCW curriculum). So, merging home-based care into the OVC project is sensible, but confusing in the Tanzanian context where two separate systems operate in parallel. CONCLUSIONS While HIV prevalence among Tanzanian children is below the national rate, their poor levels of viral load suppression accentuate the necessity for OVC programming to find and enroll missing cases. Although the project is appropriately designed to contribute to epidemic control by delivering services that support adherence and suppression, its efficacy in terms of case finding is hindered by inherent design limitations in that it is not optimized as a case finding project, nor is it aligned with GoT structures to optimally support adherence monitoring. Referrals from CTCs have yet to be optimized due to procedures not yet systematized or consistently implemented. These limitations are ultimately demonstrated by the fact that, while there are up to 180,000 CLHIV in Tanzania, less than 11,000 are enrolled in Kizazi Kipya. The sheer volume of services delivered, combined with the fact that delivery is case managed, suggests that the continuum of care for enrolled HIV positive children has been strengthened. However, the difficulties determining the extent to which services have been layered for those children (i.e., being able to easily track and distinguish between the proportions of CLHIV according to the number of services 10 each received) necessarily qualifies any definitive conclusion on this evaluation question. The difficulties with unique identifiers and the limited ability to extract layering queries form the MIS makes layering analysis challenging and unreliable. The qualified conclusion is further justified by the fact that the national case management system is nascent and that the effectiveness of CCWs is subject to multiple risks. 4.4 EQ 2: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN CONTRIBUTING TO HOUSEHOLD RESILIENCE AND WOMEN’S EMPOWERMENT THROUGH ECONOMIC STRENGTHENING ACTIVITIES? Kizazi Kipya offers ES activities that primarily target OVC caregivers, but also offers certain activities specifically tailored for older OVC.6 The project categorizes OVC households into one of four categories based on data from the family and child asset assessment that is part of project enrollment. In order of most to least vulnerable, these categories are “provision,” “protection,” “production,” and “promotion.” Small groups of women in the lower categories are formed to save together and make low-interest loans to one another that help them launch or expand small businesses. The core project ES activity of the project is the strengthening and creating village savings and loans groups (VSLGs). Besides providing basic material to operate the VSLGs, partner CSOs and LVs also provide technical support on topics such as record-keeping, loan management, and group management. The WORTH Yetu model used by Kizazi Kipya enhances the traditional VSLG approach so as to serve and support the project’s overall objectives. For example, VSLGs are encouraged to create a fund to pay for OVC secondary school fees, primary and secondary school education kits, and food and health services. The WORTH Yetu model also includes training on social responsibility and inclusion principles. Agents also assist VSLGs to create community resource mobilization committees (CRMCs) tasked with mapping local resources and raising resources to support “provision” OVC households in the community. In addition, “provision” OVC households are linked by the project to existing consumption support services, primarily the Tanzania Social Action Fund (TASAF.) To foster the diversification and growth of the income generating activities undertaken by VSLG members, the project also provides each group with five Selling Made Simple books and links mature groups to credit from government and bank sources. The project also conducts market assessments in certain councils and provides business development support and startup kits to OVC caregivers. To complement these activities, Kizazi Kipya laid the groundwork to offer household training sessions beginning in FY 2019 focused on improving members’ financial literacy and money management. These trainings aim to help households use scarce resources more effectively, shift from reactive to proactive household budgetary decision-making, and choose the financial services that best meet their needs. Older OVC and youth living and working on the streets are targeted by other ES activities to prepare them to enter the workforce or become entrepreneurs. The principal activity is the provision of vocational training scholarships for out-of-school girls and boys aged 15-19. Early on, this activity was focused on DREAMS but later expanded to other councils. This scholarship program supports individuals to select and enroll in the course of their choice, monitors their attendance, and provides a start-up kit on completion. Concurrent with the vocational training, these youth also receive soft skills training through bi-weekly sessions on starting and growing a business, the use of technology for business, etc. Depending on the council, either the International Labor Organization’s Start and Improve your Business curriculum or FEMA Magazines are used. Another training session is provided for six months to those OVC that have completed their vocational training focused on financial literacy and information and communications 6 The values reported for Q2 of FY 2017 are from the Q3 FY 2017 quarterly report, which appears to recalculate the Q2 figures using the new geographic scope and target value. The original values were largely in line with that of Q1 of FY 2017. 11 technology. Kizazi Kipya also had plans to support and link CSOs with potential government and private institutions to support OVC with employability skills, career choices, and marketing of youth products. However, the departure of Restless Development from the consortium forced the project to idle this effort for a time. A revision to the project’s implementation strategy has subsequently been adopted, with the previous sub-partner acting in an advisory role. 4.5 EQ2.1: TO WHAT EXTENT HAS KIZAZI KIPYA CONTRIBUTED TO HOUSEHOLD RESILIENCE AND WOMEN’S EMPOWERMENT THROUGH ECONOMIC STRENGTHENING ACTIVITIES? Figure 2 shows project results for the PEPFAR indicator TZ_ECON; 7 however, there are several important caveats associated with this chart. First, the number of councils covered by the project changed dramatically between Q2 and Q3 of FY 2017,8, 9 and the project indicated that counting was problematic until Q4 of FY 2017 when an improved VSLG data collection tool was rolled out along with coaching and training. Second, the enrollment of beneficiaries in the WORTH Yetu groups (as opposed to the original VSLGs often inherited from Pamoja Tuwalee) started in Q4 of FY 2018. Third, the target value changed significantly over time, even after the transition just mentioned. 10 Nonetheless, looking at the quarters where the indicator definition appears to have remained stable, the project did significantly increase the number of beneficiaries receiving ES services. It also exceeded its (admittedly revised downward) target for Q1 of FY 2019. Figure 2: Number of Beneficiaries Counted for the TZ_ECON Indicator and Corresponding Percentage of the Target Value for a Given Quarter Given the above caveats, it is more instructive to look at the evolution of the key ES activities over time. As previously noted, VSLGs are a critical component of ES activities and some, albeit spotty, data exists regarding their evolution and progression. Table 1 presents key characteristics of the VLSGs at key project stages: when they were inherited from Pamoja Tuwalee, at the end of FY 2017, and at the end of FY 2018. The data is not always comparable, and in particular project reporting did not disaggregate between OVC 7 Indeed, the TZ_ECON target started at 494,947 for Q1 and Q2 of FY 2017, declined to 159,077 for Q3 and Q4 of FY 2017, then climbed to 230,499 for FY 2018 before declining to 154,709 for Q1 of FY 2019. 8 While the number of project-supported VSLGs with a CRMC is unknown for FY 2017, this activity did not seem to have started yet, so in all likelihood there were no such groups in place by Q4 of FY 2017. 9 Demographic and Health Survey and Malaria Indicator Survey 2015-16. Final Report. Tanzania, December 2016. 10 Violence against Children in Tanzania: Findings from a National Survey, 2009. Summary Report on the Prevalence of Sexual, Physical and Emotional Violence, Context of Sexual Violence, and Health and Behavioral Consequences of Violence Experienced in Childhood. Dar es Salaam, Tanzania: UNICEF Tanzania, Division of Violence Prevention, National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, and Muhimbili. University of Health and Allied Sciences, 2011. 12 funds and social funds until FY 2018. Cumulative monetary values reported for FY 2017 and FY 2018 are cumulative for the fiscal year. Table 1: Key Characteristics of VLSGs Characteristics Inherited from Pamoja Tuwalee Q4 FY 2017 Q4 FY 2018 Number of VSLGs 6,305 7,490 9,028 Number with CRMCs - Unknown 1,387 Number with OVC funds - 1,115* 5,072 Number with social funds - 3,163 Cumulative total savings (Tsh) Unknown 15,021,437,106 13,108,109,481 Cumulative value of CRMC (Tsh) - Unknown 179,671,332 Cumulative value of OVC funds (Tsh) - 522,936,661* 818,774,679 Cumulative value of social funds (Tsh) - 622,005,196 * Values for the OVC and social funds are reported jointly for these time periods and cannot be disaggregated into each category. The key points in Table 1 are that the number of VSLGs has increased steadily over the first two years of the project, and the number of VSLGs groups with either an OVC or a social fund has grown rapidly and reached 91 percent of all supported VSLGs by the end of FY 2018. In addition, the total cumulative value of these two funds nearly tripled between during FY 2018 (even if the per-VSLG average cumulative value is significantly lower than in FY 2017). The project has also made progress in fostering the development of CRMCs within these groups, which reached 15 percent by the end of FY 2018.11 Over all four quarters of FY 2018, the project reported that 10,259 OVC benefited from CRMC resources and 5,072 benefited from OVC funds. That said, the Southern Highland geographic cluster lagged behind its peers in terms of implementation of OVC, social funds, and CRMCs by the end of FY 2018. In addition, the project did not meet its Year 2 target number of VSLGs supported (11,834), and by the end of FY 2018 had reached 76 percent of that goal. OVC parents and guardians in all five FGDs were enthusiastic about the groups. However, across FGDs, and even within them, the specifics of the VSLG operations varied along several dimensions, including the minimum required contribution, the frequency of meetings, how often the money from the saving fund was distributed to members, and whether members had received some material support, such as chicken projects (3/5 FGDs) or pig projects (1/5 FGDs). In all five FGDs, participants noted the value of learning how to save and how to operate the group (record-keeping, etc.) Typically, leaders were trained by CSOs and in turn trained other group members. Group governance was not always mentioned but seems to be standard and promoted by the project or CSOs. In one FGD, participants praised the value of books about self-help and running the groups, but this did not come up in other discussions. Although not mentioned in all FGDs, VSLGs were officially registered, which provided valuable benefits. For example, in Mwanza it allowed access to local government loans, but none of the FGD participants reported being linked to local government or banks loans, although the project reported that 119 groups had received loans and been linked. In Njombe, participants noted that official registration had helped them to receive community recognition. The idea that these groups were safe places for vulnerable people where everyone is treated the same and encourages and helps each other came across in three of the five FGDs. 11 Drawn from Pact’s 2017 and 2018 Annual Reports. 13 In two FGDs, participants also noted that the training they received encouraged them to open a bank account. FGD participants in Dar Salaam and Mwanza indicated that their group had a community or OVC fund (3/6 FGDs with WORTH groups, also see Table 2). The extent of these innovations in savings groups is reflected in Table 2 below. However, in all FGDs participants expressed the perceived importance of helping OVC in the community, regardless of whether their caregivers were part of the groups. Social contributions were most often mentioned in terms of buying school supplies and clothing for OVC. In three of the five FGDs, participants reported being able to send their children to school and/or vocational education as a result of the groups, typically through the saving and lending activities and, in one case, through the OVC/community fund. By Q4 of FY 2018, the project reported that 34 percent of participating households received TASAF support, totaling 35,353 in Q1 FY 2019. Table 2: OVC Funds, Socials Funds, and CRMC in Project Supported VSLGs, Disaggregated by Cluster as of FY 2018 Clusters Number of Project￾Supported VSLG in Q4 Number and Percentage of VSLGs with CRMC Cumulative Value of the CRMC Number of OVC Who Benefitted From the CRMC Number and Percentage of VSLGs with OVC Funds Cumulative Value of the OVC Funds Number of OVC Who Benefitted from OVC Funds Number of VSLGs with Social Funds Cumulative Value of the Social Funds Central 1,194 170 (14%) 16,581,900 821 724 (61%) 134,864,604 8,194 609 (51%) 132,843,732 Coastal 1,778 367 (21%) 48,585,024 1,528 1,213 (68%) 297,577,116 11,211 1,031 (58%) 319,693,257 Lake 1,960 288 (15%) 22,707,500 2,449 1,078 (55%) 138,866,898 27,682 458 (23%) 48,950,272 Southern 1,267 219 (17%) 19,979,908 836 811 (64%) 98,545,960 16,998 379 (30%) 43,747,822 Southern Highland 2,163 220 (10%) 59,404,600 3,662 851 (39%) 118,604,600 19,649 357 (17%) 31,045,874 Western 666 123 (18%) 12,412,400 963 395 (59%) 30,315,501 11,469 331 (50%) 45,724,239 Total 9,028 1,387 (15%) 179,671,332 10,259 5,072 (56%) 818,774,679 95,203 3,165 (35%) 622,005,196 The link between income generating activities and VSLGs was clearly made by direct beneficiaries. In four of the five FGDs, participants praised the groups for their ability to start and successfully run a business (agriculture, charcoal, sewing, welding, etc.), which in turn enabled them to care for their family and participate more fully in the groups via more money saved and greater social contributions. By Q4 of FY 2018, the project reported 11,719 VSLG members (53 percent of the target 22,111) and 17,286 caregivers in six councils who were provided with startup kits. Across the FGDs, participants reported the misuse of loans was rare given that they typically made joint lending decisions and asked for details about the underlying need. In addition, they reported being visited often (typically monthly) by LVs or project staff who would observe a meeting and look at their records. At the same time, LVs and project staff would dispense advice on how to improve the groups, which participants indicated they welcomed. The contribution of ES activities to household resilience is not systematically tracked. Nonetheless, FGD participants did report resilience related benefits, including increased means to pay school fees (6/6 FGDs with WORTH groups), medical related expenses (6/6 FGDs with WORTH groups), supplement nutrition (3/6 FGDs with WORTH groups) and manage shocks (2/6 FGDs with WORTH groups). Women participants also spoke about increased say in household expenditure decisions (2/6 FGDs with WORTH groups) and generally considered participation to be empowering and beneficial. The FGD with LVs in Kagera yielded broad agreement that VSLG training covered many key topics, including group organization, book-keeping, follow-up visits, leadership committees, fund management, 14 and dividend distribution. They uniformly praised the quality of training facilitators noting they were patient, willing to answer questions, and taught in an interactive way. LVs noted that the training not only helped them be effective in their role but also built their confidence and leadership. One noted that it inspired her to educate others in society beyond her role, and that she derived satisfaction from seeing something small grow. Nonetheless, a few challenges were raised during these discussions, including the fact that they had to cover a 400-page book in a week which required a lot of extra studying, and that the training allowance was too small. 4.6 EQ2.2: WHAT FACTORS EXPLAIN THE ACHIEVEMENT OR UNDERACHIEVEMENT OF THESE RESULTS? Factors Constraining Contribution to Household Resilience and Women’s Economic Empowerment LVs reported that the allowances they received during their regular operations were just enough to cover transportation costs, and sometimes they had to use their own pocket money or turn down an emergency visit because they cannot afford to travel. Similar problems were noted regarding referrals, whether it was the extra travel costs for the LVs themselves or the household requesting an allowance to travel to the various administrative bodies. As a result, LVs noted that they often have had to abandon referrals mid￾way. Part of the challenge is that it is not always easy for them to establish whether there is a valid case and where the case should be brought (for example, in case of inheritance). This challenge could be addressed via the provision of formal legal aid, which USAID already supports in some of these regions. Another challenge related to travel is the distance and bad weather, which means they spend a lot of time on the road unable to do other things for themselves and their households. LVs also noted that the forms they use to report to the project could use some improvements, as there are several of them and they are somewhat duplicative. Also, the notebooks provided to groups are small and hard to read and complete, making the group leaders’ and the LVs’ work more difficult than necessary. CONCLUSIONS While the contribution of ES activities to resilience and women’s empowerment is not systematically tracked, the qualitative data suggests that it is potentially substantial. Individual benefit varies, although all FGD participants claimed to have benefited at least to some extent. The WORTH model is thoroughly grounded in evidence and substantial effort is invested in ensuring its implementation fidelity, which accounts for much of its achievements. 4.7 EQ3: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN CONTRIBUTING TO VIOLENCE AGAINST CHILDREN PREVENTION AND RESPONSE? Violence against women and children (VAWC) is an area of focus for the GoT, as seen in its Second National Costed Plan of Action for Most Vulnerable Children and the National Action Plan to End Violence Against Women and Children in Tanzania. Despite this, 27 percent of girls in Tanzania have given birth by the age of 18, 31.1 percent are married, and 25.8 percent have experienced sexual violence. According to the 2015-16 Tanzania Demographic and Health Survey and Malaria Indicator Survey (2015-16 TDHS￾Malaria Indicator Survey12), 40 percent of women age 15-49 have experienced physical violence, and 17 percent have experienced sexual violence. Although the incidence of violence is higher among married women, particularly formerly married women, 16 percent of never-married women have also experienced physical violence, and 9 percent have experienced sexual violence. Additionally, there has been no change in women’s experience of either physical violence or emotional violence since the 2010 TDHS. As it was in the 2010 TDHS, spousal violence is much higher in the mainland than in Zanzibar. 12 PEPFAR MER Indicators, Appendix E: Global OVC Graduation Benchmarks Matrix, updated 7/06/2018. 15 Data on violence against children from the United Nations Children’s Fund (UNICEF) August 2011 report13 reveal that violence against children is a serious problem in Tanzania: nearly three in 10 females and approximately one in seven males in Tanzania have experienced sexual violence prior to the age of 18. In addition, almost three-quarters of both females and males have experienced physical violence prior to 18 by an adult or intimate partner, and one-quarter have experienced emotional violence by an adult during prior to turning 18. Even though the UNICEF data is outdated, it highlights that only about one of eight females and less than one of 20 males received services for sexual violence prior to age 18, and about 16 percent of females and males who experienced sexual violence prior to age 18 would have liked additional services. Kizazi Kipya has implemented the following violence prevention and response interventions that can be grouped into three categories shown in Table 3. Table 3: Kizazi Kipya Violence Prevention and Response Interventions Frameworks, Manuals, and Curriculums on Prevention Developing a national parenting framework in preventing violence, abuse, and harmful social practices against children and operationalizing it at the LGA and community levels. Adapting the Stepping Stones curriculum Tanzania’s context for AGYW, a gender-transformative HIV-prevention intervention for reducing sexual risk-taking and intimate partner violence. Piloting the Furaha (adapted Sinovuyo) Caring Families for Parents and Teens curriculum in five councils (Temeke DC, Mbeya CC, Iringa DC, Kyela DC, and Morogoro MC) to reduce community-based risk of violence, neglect, and abuse and improve positive parenting. Building economic resilience to reduce vulnerability to violence against children (VAC) by strengthening households via the worth groups. Training Training on the National Parenting Education Manual for Families to strengthen the capacity of caregivers in positive parenting and non-violent discipline techniques and building a protective environment for their children with district community development officers (DCDOs), CSO staff, lead CCWs, and CCWs. Collaborating with CHSSP to identify councils and wards for conducting Child Protection Committee trainings and working with the WAMATA organization and Amana OSC to ensure the referral of VAC cases. Response Systems Positioning CCWs to identify cases of violence, abuse, neglect, and exploitation and refer them DSWOs and CPTs, as per the GoT’s national child protection regulations. Working with stakeholders to ensure that OVC and their household members can access available child protection services, including one-stop centers for violence against children and gender-based violence (GBV) services or, where the systems and/or structures do not exist, ensuring that CSOs, DSWOs, police, and other stakeholders have basic skills to address child protection issues. Tracking abuse cases by CSOs via the GBV and Child Abuse Incident Reporting Form and the VAC/GBV Service Register to ensure that CCWs have the ability to identify abuse cases and provide relevant support (including referrals to the DSWOs, Police Gender and Children Desk, One Stop Centers and or members of Protection Committees). 4.8 EQ3.1 TO WHAT EXTENT HAS KIZAZI KIPYA CONTRIBUTED TO VIOLENCE AGAINST CHILDREN PREVENTION AND RESPONSE? Prevention of Violence Against Children Kizazi Kipya has focused on community sensitization by raising awareness and training MVCC/CPT members and CCWs on VAWC, including how to identify signs for VAC. Using a cascade strategy, these have then trained communities and parents/caregivers of the OVC or most vulnerable children (MVC). In interviews, members of the MVCC/CPT stated that they now know their roles and responsibilities and 13 See ADS 201mah, USAID Evaluation Report Requirements and the Evaluation Report Review Checklist from the Evaluation Toolkit for additional guidance. 16 are able to undertake them effectively (3/3 KIIs with CPT/MVCC). The project has trained 204 facilitators (85 males, 119 females) in seven councils and mobilized 4,100 caregivers (Kizazi Kipya FY 2018 Report). In three of four KIIs, DMOs noted the role CCWs have played in educating community members on VAC. “On preventing violence against children, Kizazi Kipya works with CCWs at the community, capacitating them with VAC education, including how to handle, help and/or link the victim to service.” KII, District Official As seen in Tables 4 and 5 below, the increased education and training provided to parents has changed caregiver behavior towards the child and parenting attitudes. The number of caregivers who state “I never” punished my child in the last four weeks increased from 52 percent in the pre-test to 87.5 percent in the post-test. Similarly, those saying that “no” physical discipline is needed for child growth increased from 62.8 percent in the pre-test to 92.2 percent post-test. Table 4: Furaha Caregiver Behavior Toward Children (Waves 1, 2, and 3 Q1 FY 2019) Variable Category Pre-Test (n=7,525) Post-Test (n=7,525) Percentage n Percentage n P-Value Number of times I punished my child in last four weeks Never 52.0 3,913 87.5 6,582 <0.001 Once 25.3 1,907 8.7 657 <0.001 Twice 12.4 933 2.4 177 <0.001 ≥ 3 10.3 772 1.4 109 <0.001 Number of times I said things that upset my child Never 42.8 3,217 83.6 6,294 <0.001 Once 26.9 2,021 10.4 786 <0.001 Twice 14.8 1,114 3.8 283 <0.001 ≥ 3 15.6 1,173 2.2 162 <0.001 Source: Kizazi Kipya Project Data Table 5: Furaha Caregiver Parenting Attitudes (Waves 1, 2, and 3 Up to Q1 FY 2019) Variable Category Pre-Test Post-test p-value Percentage n Percentage n Childcare takes more time and energy than what I have to give Yes 57.4 4,319 24.4 1,836 <0.001 No 29.1 2,188 37.0 2,783 <0.001 Not sure 13.5 1,018 38.6 2,906 <0.001 Children cause stress Yes 56.6 4,262 22.3 1,675 <0.001 No 30.4 2,286 73.5 5,530 <0.001 Not sure 13.0 977 4.3 320 <0.001 Physical discipline is needed for child growth Yes 24.2 1,821 5.3 398 <0.001 No 62.8 4,729 92.2 6,939 <0.001 Not sure 13.0 975 2.5 188 <0.001 Source: Kizazi Kipya Project Data Strengthening Violence Against Women and Children Committees Kizazi Kipya is working with CHSSP to collaborate with the GoT in its implementation of the National Plan of Action to End Violence against Women and Children (2017/18-2021/22) (NPA-VAWC), which aims to prevent and respond to all forms of VAWC through comprehensive multi-sectoral collaboration at all levels. CHSSP established VAWC committees (previously called MVCC/CPTs) within LGAs, and Kizazi Kipya is strengthening the capacities of these committees to reinforce the government commitment to provide effective leadership for eliminating all forms of violence to women and children. Prevention Programs Targeting Children Kizazi Kipya is also collaborating with the DREAMS project, which is working to sensitize and train children on GBV and other forms of VAC, understanding what abuse entails, detecting the signs, and learning who they can report to if they have been abused. Kizazi Kipya interventions, like establishing children clubs and training children in schools, has increased their self-esteem and seems to have made a significant change 17 in reporting since sexual GBV to children has been a common form of VAC and has been lagging in reporting as compared to other forms of violence (1/5 KIIs with Pact staff). “Teachers have been educated on how to educate children through children clubs at school. Also, we have found that their parents have abused their children and/or very close relative, so, sometimes due to awareness these children tend to bring their fellow child who has been abused and link to us.” KII, GoT Response to Violence Against Children To ensure violence and abuse cases are addressed, Kizazi Kipya is working with VAWC committees, which consist of stakeholders like the police, medical doctors, social welfare officers (SWOs), religious leaders, and other community representatives. A variety of stakeholders reported that CCWs have been very helpful in supporting the victims of the violence. Given the challenge that most forms of VAC are done by close relatives, a variety of stakeholders reported that CCWs now know how to probe and identify violence and provide a helping hand to the victims. As one CPT/MVCC noted “…for instance, in my ward, there were several cases of violence against children, and it was a burden for sure; but after the establishment of the NICMS and the presence of CCWs at the community, identification of violence cases helped us to know the environment where these acts are done.” Others stated that previously, victims—especially women and children—remained silent because they did not know where to report such incidences (KII, CTC staff; KII, CSO). There has been an increase in VAWC referrals during the course of the project. However, referrals to violence response related services remain limited and difficult to monitor to case completion. During Q4 FY 2018, LCWs/CCWs linked 627 (male 329 and female 298) OVC and youth to child protection service providers across the project councils for cases of abuse, neglect, and exploitation. As of the close of FY 2018, 565 children under 18 had been linked to child protection service providers, while 62 beneficiaries older than 18 had been linked to GBV service providers. 4.9 EQ3.2: WHAT FACTORS EXPLAIN THE ACHIEVEMENT OR UNDER￾ACHIEVEMENT OF THESE RESULTS? Factors Explaining Achievement of Results Increased community awareness of VAC and GBV: Stakeholders reported that education provided by CCWs during household visits has been instrumental in parent/caregivers, children and other family members understanding child protection and violence against women (1/5 KIIs with CPT/MVCC, 1/1 KIIs with project consortium partner, 1/1 KIIs with AGYW DREAMS). Likewise, training provided to different community-level actors has improved their knowledge regarding VAC and GBV issues and their responsibilities. As noted by a DMO, “Thanks to Kizazi Kipya project, the awareness of VAC issues is imparted from the individual child to parents/caregiver and family members and to the community as a whole. This has increased reporting of VAC incidences from all three sources. About 80 percent of VAC incidents in my area is from project efforts, and this has been possible mainly because of the awareness provided to the community.” The project has also increased the community’s collective to respond to VAC issues, unlike before when community members depended on the government. Examples cited include community members reporting and burning down places prone to sexual harassment and increased use of toll-free number to anonymously report VAC incidences (2/4 KIIs with DMOs, 1/3 KIIs with CPTs/MVCCs). Use of CCWs: CPT/MVCC noted that the use of CCWs in identification and reporting of VAC and GBV issues has improved the services they provide to households. As CCWs come from the same community, people feel safer confiding in them. As noted by one CPT/MVCC member, “There are some incidences that would have not reached our committee if they were not brought to us by CCWs.” 18 Effective coordination and linkages of community actors in response to VAC and GBV issues: VAWC committee members said that the project has helped them engage actors in the community in the prevention and response to VAC and GBV issues (2/3 KIIs with CPTs/MVCCs). These multidisciplinary community actors cover three major areas to which VAC and GBV cases can be linked: law enforcement, physical support, and biomedical interventions. The composition of the VAWC committee has expanded to accommodate actors from other disciplines necessary to provide victim services. To ensure effective and efficient stakeholder linkages, the project facilitated the development of a community mapping document to be used by CCWs for effective stakeholder coordination 3/5 KIIs with CSOs). Improved record keeping and tracking of VAC and GBV issues: The provision of tools and equipment to LGAs has improved the recording and tracking of VAC and GBV issues from the community to the National level. GoT officials received office equipment such as computers and furniture (2/2 KIIs with GoT officials). According to one GoT official, “We didn’t have computers in our district councils before making it hard to keep data. The support has improved data accessibility, which is really a major strength for us.” The project also provided motorcycles and bicycles for district and ward officials, easing the transportation burden and improving their ability to provide supportive supervision. Stakeholders also mentioned the form used by CCWs in tracking VAC issues in households, in children’s clubs, and in schools as an essential tool in monitoring VAC and GBV issues (1/3 KIIs with CPTs/MVCCs, 1/4 KIIs with DMOs). Finally, in three of five KIIs, Pact officials mentioned that the project equipped CCWs and other community actors—including case management officers from sub-grantee CSOs and teachers at the Huru project—with job aids on VAC as well as a parenting job aid for disciplining children. Improved referral system: Improved education and knowledge of community members enabled them to know where to go to address issues related to VAC and GBV. The project referral forms and the escorted referrals to health facilities for cases that need immediate action also helped ensure that children received timely and appropriate services (2/5 KIIs with CPTs, 1/5 KIIs with CSOs, 3/5 KIIs with Pact). Factor Explaining Under-Achievement of Results Existing community norms: Community norms of not exposing family problems have led to a tendency of resolving VAC and GBV issues within the community instead of taking them to the authorities. In addition, the fact that in many cases perpetrators are a powerful person or a close relative in the family has made VAC and GBV reporting more difficult, and one of the reasons many cases that are referred to a higher level get dropped due to the lack of evidence. (1/2 KIIs with the GoT, 2/4 KIIs with DMOs, 2/5 KIIs with CTC in-charge, 2/3 KIIs with CPTs/MVCCs). Long duration of case follow-up in courts: DMOs, CSO sub-grantees, and CPTs/MVCCs reported that a long duration of court cases follow-up distorts the initial evidence collected and causes victims to rethink the intention of seeking justice. One DMO explained that the need for sufficient evidence often results in a long duration of the court process and in several cases adjournments before judgment. This discourages victims and witnesses from continuing with case follow-up, and they decide to drop the case. Dropouts are more prevalent in GBV cases, since the management of VAC issues is through formal structures. However, prolonged time for investigation often distorts the initial evidence, thus denying justice to the victim. Challenges with geographical coverage: The physical and geographical location of the target population is another constraining factor with the challenge being more severe in rural areas. Timely engagement in these communities in VAC and GBV cases has been a problem, and this persists when referrals are provided to the victim for the case to be managed at the district level. In all cases, victims do not have money for transport and accommodation to attend the referral case (2/4 KIIs with DMOs, 1/5 KIIs with CSOs). 19 “The community we serve is between 24 km to 87 km away from the district center. Our beneficiaries are vulnerable people and the majority of them are OVC who need to be escorted. It is obvious that the victim won’t be able to attend the court.” KII, CSO Sub-Grantee Insufficient human resources: DMOs, CSOs’ sub-grantees, and Kizazi Kipya consortium partners noted a lack of skilled personnel to coordinate management of VAC and GBV cases. According to one DMO regulations stipulate that there should be at least two DSWOs at the LGA level, while each ward should have at least one SWO; if not a CDO cadre can assume the role. In two of four KIIs, DMOs reported having only one DSWO with no SWO cadre; while in another KII, DMOs reported having neither a SWO nor CDO cadre at the ward level. Insufficient VAC and GBV screening tools: CSOs cited the need to have CCWs use a VAC and GBV screening tool. They have trained the CCWs to use this tool, but its printing is still underway. Non-resolution of GBV cases: Despite project efforts to address GBV in collaboration with the GoT, challenges remain. These include a prevalence of GBV due to the traditional practice of inherited widow, premature closure of GBV cases outside the legal system due to corrupt government officials, community norms protecting perpetrators by negotiating directly with the family of GBV survivors, and the non￾reporting of cases because perpetrators threaten the survivors (1/5 KIIs with Pact, 1/1 KIIs with IPs, 2/5 KIIs with CSOs, and 2/5 FGDs with OVC guardians/parents). “The challenges are about the community wanting to resolve the issue within the community instead of moving it to the authorities. Perpetrators are sometimes protected by community norms or cultural ways; a lot of communities settle cases through families. Some CCWs follow their protocol but others succumb to the community pressure. In some cases, incidences are being handled at the ward or village level but are not being report to the district CPT.” KII, Pact Exclusion of males from DREAMS interventions: According to DREAMS participants, interventions should be extended to adolescent boys/young men, who are also at risk of being perpetrators or victims of GBV, thus enabling them to protect themselves and/or protect girls against GBV. It will also help them identify which acts are GBV and which are not, as sometimes they commit gender violence without knowing. Participants further stressed that HIV/AIDS affects both genders, which increased the need for boys to learn how to protect themselves from violence (1/1 FGDs with AGYW DREAMS, 1/3 KIIs with CPTs/MVCCs, 1/1 KIIs with Huru Facilitator, 1/1 KIIs with Furaha Facilitator/DREAMS teachers). Currently, Kizazi Kipya is targeting both male and female OVC, and Huru facilitators include boys and girls in their training classes on sexual reproductive health. Externality of behavior change communication to children and youth due to parenting skills: Job AIDS developed by Kizazi Kipya in collaboration with Tulonge Afya—which integrates parenting education in home visits to parents, guardians, and caregivers—has proved effective in affecting youth and children behavior change and communication. Moreover, parenting messages have been provided to teen clubs complementing the messages provided to their caregivers by CCWs during home visits. (1/5 KIIs with CSOs, 1/5 KIIs with Pact, 1/1 KII with Huru facilitators, 1/4 KIIs with DMOs, 1/5 FGDs with OVC guardians/parents). High workload on CCWs due to insufficient staff: CCW dropouts due to low monthly stipends has increased the workload on remaining CCWs. If Kizazi Kipya had enough trained CCWs in the villages, more OVC and their households would have been identified and served (2/5 KIIs with CTCs in-charge). One GoT official reported an 80 percent shortage of skilled staff on case management and resultant unmet need. She recommended that district executive directors should recategorize government staff to mitigate the shortage of social welfare staff for case management while urging development partners to recruit on behalf of the government, similar to what UNICEF is doing in Kigoma. She also stated that district executive directors should find ways to retain staff using internal sources of revenues. Effective Kizazi Kipya approaches: One DMO noted that even though Kizazi Kipya is not providing handouts/material support, it has increased community awareness and follow up on OVC. Several key 20 informants reported having joint annual planning and budgeting meetings with Kizazi Kipya (1/1 KII with DREAMS Coordinator, 1/2 KIIs with the GoT, 1/1 KII with IPs, 2/4 KIIs with DMOs), while others mentioned the collaboration between Kizazi Kipya and other USAID activities, such as Boresha Afya, Tulonge Afya, Sauti, and CHSSP (2/5 KIIs with CSOs). “Sauti, for example. gives us the list of the girls 15-24 they’re working with. CCWs to go to the households and do the screening and enrollment” (KII Pact). Finally, Kizazi Kipya has introduced several technological innovations, which has enhanced its data management processes and procedures. These include a bi-directional referral system that simplifies referrals tracking and a District Health Information System 2 (DHIS2) database that allows real-time data collection and submission via any type of phone. By linking these two systems, the project has facilitated real-time stipend payments to CCWs (1/1 KIIs with EGPAF-Boresha Afya, 3/5 KIIs with CSOs). Nutrition has not been addressed enough: In Q4 of FY18, LCWs/CCWs provided nutrition-related assessments using mid-upper arm circumference (MUAC) tapes and nutrition counseling and referrals to enrolled beneficiaries. Of the 100,142 OVC under age five who were nutritionally assessed with MUAC tapes, 1,684 (2 percent) had severe malnutrition, 6,796 (7 percent) had moderate malnutrition, and 91,662 (92 percent) had no malnutrition. Food security remains an unattended area for most of the Kizazi Kipya beneficiaries. Kizazi Kipya community volunteers have been trained in NACS and are expected to continue providing nutrition-related assessments using mid-arm upper circumference tapes and nutrition counseling and referrals. However, the deficit in service provision is linking beneficiaries to food provision. FGD respondents reported cases where nutrition remains inadequate and “most of the OVC don’t have enough food and when they are using antiretroviral HIV drugs (ARV), the nutrition status becomes more delicate” (3/5 KIIs with CSOs, 1/4 KIIs with DMOs, 1/1 KIIs with the Huru facilitator, 1/5 KIIs with CSOs, 2/5 KIIs with CTC in-charge). “However, generally the problem facing vulnerable children, especially orphans, is that they don’t get enough food. The project should consider nutrition/food to the vulnerable children. Nutrition is a very important aspect in addressing the needs of vulnerable children.” KII, Huru Facilitators CONCLUSIONS The project’s prevention and training strategies to address VAC and GBV has changed caregiver attitudes and behavior towards parenting and children, and its work with children in schools has led to increased GBV reporting. Response to VAC via CCWs has improved community knowledge on where to report such incidences. Since CCWs are local community members, people feel safer reporting or seeking advice from them. Challenges remain in resolution of GBV cases, however. These includes perpetrators that are powerful or close relatives who either strike a deal with government officials or directly with the victim’s family, long drawn out court cases, and challenges with geographic distance, especially in rural areas. 4.10 EQ4: HOW EFFECTIVE HAS KIZAZI KIPYA BEEN IN CONTRIBUTING TO HIV PREVENTION AMONG ADOLESCENTS, AND ADOLESCENT GIRLS IN PARTICULAR? OVC programs contribute to prevention outcomes through structural interventions addressing systems and economic strengthening, and prevention specific interventions including education support for in￾school youth, livelihoods support for out of school youth, and behavior change interventions in the form of curriculum-based messaging, delivered through group platforms such as teen clubs and parenting groups. Household economic strengthening has been discussed in a preceding section. This section focuses on Kizazi Kipya’s livelihoods support, education support and behavior change interventions. 21 4.11 EQ4.1: TO WHAT EXTENT HAS KIZAZI KIPYA CONTRIBUTED TO HIV PREVENTION AMONG ADOLESCENTS, AND ADOLESCENT GIRLS IN PARTICULAR? Overview of Prevention Activities Kizazi Kipya works to establish or strengthen teen clubs for youth aged 10-14 years and 15-19 years, in the latter case also including out-of-school youth, to deliver HIV knowledge interventions, adolescent sexual and reproductive health messaging and linkage to services (ASRH), life skills development, and livelihoods opportunities. A summary of project interventions for each age group is presented in Table 6. Table 6: Prevention Specific Interventions Delivered Through the Teen Club Platform Prevention Specific Interventions 10-14 Years 15-19 Years 15-19 Years In-School Out of School ASRH Outreach x x x Girls Lets Be Leaders (Life Skills & HIV Prevention) . . . HURU (Menstruation Support and HIV Prevention) x . . Behavior Change Curricula (Stepping Stones Derivative) . x x SIYB (ILO Curriculum) . . .x Education Support x . . Vocational Training . . x Furaha Parenting Curriculum (Sinovuyo Derivative) . . . Performance Against Targets A summary of Kizazi Kipya’s results relative to its performance targets is presented below. • By the end of FY 2017, the project, together with EGAPF, had mobilized and oriented health care workers, developed standard operating procedures, and implemented a bi-directional referral system to support ASRH in teen clubs. At the end of Quarter 2 of FY 2018, Kizazi Kipya reported that 127 project-supported teen clubs had received outreach ASRH, representing 1,963, or 25 percent, of youth aged 10-19 years. • By the end of FY 2017, the project reported supporting 294 teen clubs across 10 districts, representing 22 percent of the 1,365 target for the year. At the end of Quarter 2 of FY 2018, the project was supporting 256 teen clubs, which was 35 percent of the FY 2018 target. • By the end of FY 2017, the Girls Let’s be Leaders (GLBL) curriculum had been adapted for Kizazi Kipya teen clubs, and NPEs and CPEs had been trained to deliver the curriculum. At the end Quarter 2 of FY 2018, the project was delivering the GLBL life skills and HIV prevention curriculum in 159 teen clubs, which represented 59 percent of the FY 2018 target. • By the end of Quarter 2 of FY 2018, Pact’s DREAMS coordinators had trained 90 teachers on the HURU curriculum in Temeke MC (14 schools), Kyela DC (16 schools) and Mbeya CC (15 schools) to establish HURU clubs, administer the HURU pre-test, and deliver HURU’s SRH educational sessions in Quarter 3. At the close of Quarter 1 of FY 2019, 980 girls had received the curriculum. In addition to the Huru curriculum, this group will also have received subsidies in the form of school materials and uniforms to complete their education support package. • After delays in receiving the DREAMs modification to its agreement, Pact was in a position to appoint DREAMs consultants at the end of Quarter 4 of FY 2017. However, as of Quarter 3 of FY 2018, the adapted Stepping Stones curriculum for Tanzania was not ready for roll-out, and the strategic approach was under review. 22 • The Start and Improve Your Business curriculum was rolled out to youth receiving vocational scholarships once funding issues were resolved. This activity was linked with OVC plus up funds. By the end of Quarter 1 of FY 2019, 331 youth had received vocational training. • The project supported districts to implement the Whole School Approach. As of end of Quarter 1 of FY 2019, Kizazi Kipya’s was supporting 410 schools, with over 16,000 learners registered. Of these schools, 383 (93 percent) had developed School Development Plans, a key milestone in Whole School Activity (WSA) implementation. With the exception of the WSA number, the project’s prevention outreach is modest in the light of the substantial OVC_SERV target. In addition, these prevention interventions have been confronted with significant constraints to performance, an observation explored in more detail in the section following. Huru and Furaha Performance on Intended Outcomes Kizazi Kipya has been monitoring outcomes on two of its DREAMS focused prevention curricula, namely Huru and Furaha. Huru’s pre- and post-test results on HIV knowledge acquisition are compelling. Of the 20 items tested, 18 returned a statistically significant improvement, indicating a potentially effective intervention for the prevention portfolio. The Furaha parenting pre- and post-test results similarly demonstrated statistically significant improvements in reported behaviors by caregivers and children alike. Of particular interest to prevention outcomes is the quality of communication between caregivers and youth on sexual behavior topics (see Table 7). Table 7: Statistically Significant Improvement in Communication between Furaha Caregivers and Youth Areas of Improvement Never Discussed Difficult to Discuss Discussed Pre￾Test Post￾Test Pre￾Test Post￾Test Pre￾Test Post￾Test Caregiver Communication with child about puberty and growing up 37.9% 7.5% 38.6% 7.9% 23.5% 84.5% Communication with child about safe sex practices and contraceptive methods 37.5% 8.1% 40.2% 8.3% 22.2% 83.6% Communication with child about relationship with adults, i.e., sugar daddy/sugar mommy 38.3% 7.3% 40.2% 8.3% 21.7% 85.0% Child Communication with caregiver about puberty and growing up 35.8% 7.2% 42.5% 8.1% 21.7% 84.7% Communication with caregiver about safe sex practices and contraceptive methods 37.3% 7.6% 42.6% 8.8% 20.1% 83.6% Communication with caregiver about relationship with adults, i.e., sugar daddy/sugar mommy 37.1% 7.4% 44.0% 7.3% 18.9% 85.2% 23 4.12 EQ4.2: WHAT FACTORS EXPLAIN THE ACHIEVEMENT OR UNDER￾ACHIEVEMENT OF THESE RESULTS? As of Quarter 1 of FY 2018, Restless Development was no longer eligible to receive U.S. government funding, and Pact terminated its contract. Pact and Restless Development worked together to ensure a smooth transition, which included CSOs absorbing NPEs and Pact hiring Restless Development’s four field staff. Additionally, Pact met with several potential replacements for Restless Development in the consortium; however, no organization had the capacity to replace the previous partner. In addition, CCWs’ high drop-out rates and capacity issues implementing out-of-school teen clubs compelled the project to review the teen club model and explore cost-sharing options with its previous partner. Kizazi Kipya was set to implement a curriculum for youth aged 15-19 based on several evidenced-based sources, including Stepping Stones, and adapted to the Tanzania context by Jhpiego. However, using the curriculum for OVC programming presented several challenges that Pact that it is still trying to resolve. These challenges include the following: • The curriculum targets specific ages of out of school OVC girls, but Pact does not have sufficient numbers enrolled in each council to organize them into groups in their own communities. • The curriculum requires intensive training and monitoring of peer educators who facilitate the sessions, but the CSOs themselves first require more capacity-building with regards to recruitment of appropriate youth facilitators and designing youth-centered programming. • Kizazi Kipya has an equal number of boys as girls enrolled and thus would need to identify an equivalent evidence-based curriculum for boys to be delivered separately from girls. The lack of integration would require more resources and could potentially be inefficient to take to scale. • The project is yet to establish which girls can be linked to social behavior change communication sessions delivered under Sauti to avoid duplication of delivery in councils where both work. Both Huru and Furaha are derivatives of evidence-based curricula, with minor adaptations to the implementation context. Material fidelity is high, which may explain the strong performance on the above results. There are, however, emerging implementation fidelity issues for Huru that need to be addressed if the results are to be sustained. The foremost of these are challenges related to the cascade training model: “Not all the teachers who attended training are able to teach their fellow teachers. Others are not confident. You find that these trainings have not been transmitted to other teachers. You know other teachers are the number one child abusers, so they also need this education.” KII08_KK_Mbeya CC_DREAM Huru facilitators The older youth enrolled in DREAMS acknowledged the acquisition of HIV prevention knowledge and changes in sexual and reproductive behavior; however, the programming benefits most frequently cited by participant youth were economic knowledge and skills in such areas as saving and entrepreneurship, as noted by this DREAMS participant, “We have learned different skills that we did not have before the project for instance, hair dressing, cake making, how to manufacture liquid soap.” Notwithstanding, DREAMS participants also consistently observed a lack of programming for their male peers, as captured in the following quote, “It is time for them also to be educated. It is very hard for us to convince young men…I think they should educate young men, if we meet in forums, I think we are going to share experiences.’ CONCLUSIONS Kizazi Kipya implements a portfolio of youth interventions that demonstrate effectiveness for those participating youth. However, with the exception of WSA interventions, the scale is very limited. Furthermore, the issues discussed in the case management section of this report regarding the tracking of layering apply, in particular to the DREAMS interventions. The difficulty of tracking layering is exacerbated for prevention interventions and DREAMS in particular because services are delivered by multiple 24 interventions. Add to these issues the difficulties in rolling out the interventions, and it would appear that youth interventions are currently the most challenging to project performance. 4.13 EQ5: TO WHAT EXTENT DID THE CASE MANAGEMENT SYSTEM CONTRIBUTE TO THE ACHIEVEMENT OF RESULTS? Kizazi Kipya has aligned its own case management approach with the NICMS. In response to the evaluation question, data from the CCW survey, KIIs, and FGDs, as well as a review of relevant documents, were used to inform a consideration of the extent to which the case management system has improved access to services and the quality and effectiveness of those services. Improving Access to Services Case management is intended to improve access to services by accurately assessing client needs and successfully linking them to the appropriate services. Data from the CCW survey indicates that nearly 90 percent of CCWs either agreed or strongly agreed that case management training improved beneficiaries’ access to services. Another 96 percent of CCW Survey respondents agreed or strongly agreed that they are able to refer beneficiaries to services providers, 79 percent agreed that beneficiaries receive services once the referral is made, and 67 percent agreed that the services meet clients’ needs. These perceptions are endorsed by data from other stakeholders, who suggested that access to services has been improved because case management has intensified referrals and because they are accompanied by information either from CCWs or referral forms, “We had these services before the Kizazi Kipya project. However, I must admit that the project…has a great organized team and it has succeeded to going down deep into the community to identify these vulnerable households and their children.” KII10_KK_MbeyaCC_DSWO-DCDO The increased frequency of referrals alluded to in the above statement is also supported by the project data, which demonstrates increased frequency in completed referrals by the project over the implementation period. Case management has not only increased the frequency of referrals but imposed an order that has substantially improved the concerted delivery of services across multiple providers. “Before the whole referral process was tough. If the parent/ guardian and their family had a problem, they had to go to a certain office directly, and others didn’t know where to start, what process to follow and what to do after the problem has happened. But through the project people have been educated and services have been brought close that if they get a problem, the [CCW] will direct them to a specific place, and you will get such service. It reduces the time for a person to find a service and for their problem to be solved.” KII20_KK_NjombeTc_CTC In Charge Improving Quality of Services CMOs/CMCs support CCWs with care planning skills via routine supportive supervision, mentorship and in-service training during monthly meetings. The link between case management training and implementation and the quality of services accessed is indirect, but there is some evidence suggesting the service quality is improving. One of the ways in which case management strengthens quality is through the effective practices introduced by a trained workforce. These practices include case conferencing and the consistent implementation of case management procedures by the trained workforce. “After they received training, they know all the procedures to follow, they know where to start from. Before … these systems were available, but they had no good flow as they have now. Now the procedure is that if the case starts from point A, it has to go to point B. These systems therefore have been improved.” KII01_KK_Bukoba_DMO, DSWO and DACC The substantial value added by a trained workforce is a persistent theme in the KIIs and FGDs. Additionally, nearly 80 percent of respondents to the CCW survey said they applied skills from training frequently on the job, and 84 percent believed that the training has helped them to coordinate referrals. 25 Key informants remarked on the importance of both specific technical training and continuous training to sustain skills and to prepare new workers for service. Key informants also describe how the trained social workforce cadre catalyzes quality improvement through implementing case management practices. “The quality of services has also improved because of the community workers, especially CCWs and CHWs. When they find a sick person who needs services, they link her with services. In case she doesn’t get the expected services, they give us the report, and we can work on it.” KII17_KK_Njombe TC_DSWO_DCDO Despite the significant augmentation of the social welfare workforce through Kizazi Kipya, the human resource need is still not saturated, as noted by this DMO from Bukoba, “There is still lack of human resources to make sure that these people are served well with services…we don’t have enough technical support at the village level.” Improving the Effectiveness of Referral Services Approximately 90 percent of CCW Survey respondents agreed or strongly agreed that the CCW training improved effectiveness of referral systems in the community, appoint also noted consistently noted in the KIIs and FGDs, such as this DSWO in Bukoba, “The referral forms that they provide to CCW are the ones that help us to know that there is a great progress because they do also report on them. He knows how many he has identified, how many are doing well, how many are absentees, how many he should make a follow up.” There is also some indication that the utilization of data to track and direct implementation is also taking hold. The most valid measure of effectiveness would be a graduation rate for cases in PEPFAR activities, based on the new MER benchmarks. Kizazi Kipya is currently preparing to track progress towards graduation against benchmarks, so this data is not yet available. For the NICMS, a similar indicator tracking achievement of case management outcomes would provide a measure of effectiveness of social welfare services. What can arguably serve as a proxy indicator of effectiveness at this stage is the distribution of referrals across service categories, such as the one presented in Figure 3. Figure 3: Distribution of Referrals Across Service Categories The distribution in Figure 3 is clearly skewed towards linkages to health services, which may be attributable in part to Kizazi Kipya being a PEPFAR project with an emphasis on contributing to epidemic control. It 26 could also indicate that there is a services deficit generally in Tanzania, and there are simply insufficient options for completed referrals. This latter hypothesis is supported by data from the KIIs and FGDs, as the following quote illustrates: “The issue of adequacy of service provision depends on which services are being provided. When we talk of medical services, they do all receive these services because the health facilities are close to the community. But when we come to vulnerable environments…the challenge could be what if the whole village needs assistance? Because the village has about fifteen vulnerable households in need of such assistance and has three women groups able to support three vulnerable households out of fifteen. The government also might be able to help two vulnerable households, but you can see there that we still have a shortage in this area.” KII01_KK_Bukoba_DMO, DSWO and DACC CONCLUSIONS There is evidence indicating that case management has contributed to project results by improving access, quality, and effectiveness of service delivery. By systematizing and coordinating service provision, case management imposes an order that facilitates completed referrals and improves layering. While the NICMS framework does not include quality standards as yet, quality improves because a trained workforce begins introducing effective practices and insisting on quality by bringing lapses to the attention of the authorities. The supportive supervision provided to CCWs also introduces the basis of continuous quality improvement into the system. And while the effectiveness of case management is not being tracked yet, the development of relevant indicators is underway. The important supplement to tracking case completion will be the documenting of availability of services to which cases could be linked for referral. 4.14 EQ 6: WHAT ARE KIZAZI KIPYA’S PROSPECTS FOR SUSTAINABILITY? Institutionalization and Continuation of Kizazi Kipya Practices in Supporting the National Case Management System National commitment and institutionalization are satisfactory in the sense that the system for supporting OVC case management has become formalized under the NICMS, is being reinforced by TACAIDS and PO-RALG, and shows signs internalization in government plans and activities at all levels. PO-RALG officials and other sub-national government staff play pivotal roles in cascading training and adherence to the national case management guidance. A PO-RALG official reported that it has adopted Kizazi Kipya activities and is directing councils to include the NICMS in LGA budgeting. He further reported that the government has produced a CD with case management tools and guidelines and distributed it to all the councils to reproduce. Further, clear guidance has been released to all partners who want to invest in OVC projects to follow the national guideline for NICMS. At least one PO-RALG official believes the NICMS will continue after Kizazi Kipya and other USAID support concludes, “Kizazi Kipya project activities will be operationalized into our daily government activities for budget allocation and for implementation. Therefore, the activities and practices will continue.” Sustainability in PEPFAR for OVC is closely related to the graduation benchmarks in PEPFAR MER indicator guidance. Kizazi Kipya’s design does link to the four domains under the OVC benchmarks including: 1) healthy, 2) stable, 3) safe, and 4) schooled. Kizazi Kipya has started planning the graduation of households and ways to measure these benchmarks; however, the extent to which the benchmarks in each domain have been met across Kizazi Kipya councils and households is currently unknown. Kizazi Kipya will follow these graduation benchmarks moving forward.14 It is important to note that these 14 According to PEPFAR MER Indicators, Appendix E, “Graduation occurs when a child and caregiver enrolled in a PEPFAR OVC program are deemed to have become more stable and no longer in need of OVC project-provided services. For caregivers and children 17 or under 4 to be counted as graduated, all child and all caregiver beneficiaries in a household must meet ALL applicable (age and HIV status specific) graduation benchmarks established by PEPFAR 27 benchmarks are new and, according to Pact, a challenge to meet in most OVC households, let alone in the aggregate at the council level. (See Annex X for more on PEPFAR OVC graduation benchmarks.) Sustainability of Case Management and Support Structures The sustainability of the NICMS depends greatly on the ability of the system to sustain HIV and OVC support structures, such as multi-sectoral AIDS committees (MACs) and violence against women and children protection committees (VAWC-PCs), as well as the volunteer corps of CCWs and the SWOs and others who provide supportive supervision and mentoring to case managers. CSOs, CPTs, and DC officials reported that case management practices are being internalized in LGA day￾to-day activities (1/5 KIIs with CSOs, 4/5 KIIS with DSWOs, 2/4 KIIS with CPTs). CSOs identified a number of factor contributing to the sustainability of the NICMS, including the spirit of volunteerism; committed leadership; strong relations with partners, LGAs, and beneficiaries; and service providers at the grassroots committed to working with CSOs. However, additional capacity-building on financial management, strategic plan development, and fundraising strategies would enable them to formulate strategies to reduce donor dependence and sustain NICMS activities after Kizazi Kipya (3/5 KIIs with CSO staff). Human Resources for LGA Support and Supervision for Case Management Stakeholders reported various strategies in place to ensure that the NICMS is sustained and has enough human resources to resolve personnel shortages in social welfare departments. To do this, PO-RALG is implementing the categorization of LGA staff and shifting of responsibility. For example, district executive directors are identifying other staff, such as teachers, extension officers, nurses etc., to train them to serve as CCW supervisors (1/2 KIIs with PO-RALG). Nonetheless, human resources for supportive supervision and the shortage of qualified SWOs beyond the district level poses risks for continuing casework at the ward and village levels. Turnover and transfers of government staff at the district level further threaten the sustainability of functioning health and child protection committees. While there are structural challenges that could affect sustainability, that case managers are now part of a formalized national system with standards and formal guidance bodes well for sustainability. LGA staff reported coordinating with each other to implement case management activities. As a result of capacity building provided, CCW supervisors at ward level cooperate with CCWs and other ward officers. Moreover, district officials believe that CCWs work and case management activities will continue because it is integrated into the government social welfare systems (3/5 KIIs with DSWOs/DCDOs), “There is sustainability because the system used is also integrated in the government and community in general.” CCWs and Sustainability of Case Management Volunteerism Motivation and attributes of CCWs and lead CCWs gives reason for optimism. An overwhelming majority (86 percent) of surveyed CCWs indicated a high likelihood that they would continue working as volunteers. A majority of CCWs are motivated by their passion for the work (53 percent) or a sense of duty (40.5 percent) and not by financial incentives (cited by 2 percent of CCWs as a motivation). CCW Sustainability Challenges Financial and resource challenges threaten the sustainability of the case management effort over the long term, with nearly 40 percent of surveyed CCWs citing the lack of operational resources, such as transport, as a challenge to sustainability. Another one-quarter of CCWs said the stipends and travel reimbursement were inadequate to cover the costs of serving households and keeping themselves afloat financially. Another 21 percent cited problems with reaching/covering beneficiaries due to lack of funds. Volunteers for improving stability. Additional guidance and tools to facilitate implementation of these global minimum benchmarks is forthcoming.” 28 have to do other livelihood activities to support themselves and their families, which means a reduction from the time they could be conducting case management activities. Sustainability of Continued CCW Training and TA As CHSSP closes it is questionable whether the cascading of training will be continued without CHSSP or Kizazi Kipya support. Kizazi Kipya has begun to fill in technical gaps of CHSSP training and continues to train new CCWs with supplementary training. However, this is not the five-day or 10-day lead CCW training provided under CHSSP. Nationally, there are master trainers from PO-RALG who are qualified to provide case management training in the long term; however, few to no initial or follow-on trainings have taken place under GoT funding alone and will likely require USAID funding to continue. The same can be said for the funding needed for the GoT to continue providing supportive supervision and mentoring to CCWs. While SWOs and other supervisors are on GoT payroll, their ability to reach CCWs requires travel stipends and means material costs (for printing of forms, etc.). These are similar operational constraints mentioned by CCWs and are also important considerations for supervisors. Referrals, VAC, and HIV Prevention and Continuum of Care Government stakeholders reported that, for now, CTC staff, CHWs, and CCWs are able to link OVC to service providers. Moreover, various stakeholders confirm that communities have knowledge on the importance of HIV testing, GBV prevention, caring families, and children safety (2/4 KIIs with MVCCs, 2/5 KIIs with CTC staff). “Through referrals, we provide them with counseling and let them undergo HIV testing and then link them with services. Many clients are identified through community outreach, i.e., at the community they are very active indeed.” KII, CTC Staff The central government has developed guidelines to help district officials mobilize funds to carry on project activities once the donor support ends. A TACAIDS staff member stated, “For these guidelines that we developed, we have topics on how to train these people (council staff) in writing business proposals, how to write project proposals, and how to run these projects.” Various stakeholders also reported that health services will be sustained, as these services are strongly advocated by the government and appear to be permanent (2/5 KIIs with CTC in-charge staff; 1/4 KIIs with CPTs). While there is evidence these practices are being internalized in government facilities, financial constraints may limit the sustainability of treatment and care if USAID or another donor support is not continued. (For more on this, see Annex XI for a case study on the sustainability of WORTH+ groups.) Education and School-Based OVC DREAMs facilitators reported that they will continue to educate girls during the regular class sessions and not during extra time and Saturdays. Due to funding limitations, they will also not able to provide scholastic materials, sanitary pads, and other goods that Kizazi Kipya used to provide (2/2 KIIs with DREAM facilitators). LGA staff reported that, in some councils, the social welfare department is working to hand over the Kizazi Kipya-supported OVC to the education department to ensure continuity after the project. The district government has also mobilized village governments to start fundraising to support OVC. “When the project comes to an end, these children will be supported, since we have sent a request to education departments about the matter, this department now has the responsibility to those children who live under vulnerable conditions who cannot afford those expenses. Through guidelines from the government, they will raise funds from the village to support these children, but also through revenue within that village.” GI, DSWO, DCDO 29 Financial Sustainability While the national trends and NICMS have contributed to institutional and budgetary commitments, gaps remain in the GoT’s ability to be self-reliant in its contribution to OVC and PLHIV households. The share of health sector (all areas of health) budget as a proportion of the total GoT budget has declined from 10.5 percent in 2010/11 to approximately 7 percent in 2017/18. This downward trend—combined with a lack of funding for service delivery, human resources, and information systems—could negatively affect the country’s ability to achieve and sustain support for PLHIV and OVC households. PEPFAR/Tanzania’s investments for the interim are required to ensure continued movement toward epidemic control. The United Nations and World Bank provide a small proportion of additional funding. There are, however, currently no other bilateral donors anticipated in FY 2019 (PEPFAR/Tanzania, April 2018). At the district and community levels, financial constraints threaten the continued functioning and operations of the case management system. Multiple stakeholders from council authorities and facility staff reported that funding to pay for volunteers, communication costs, printing, scholastic materials, transport, and per diems pose major threats to regular normalization of practices promoted by Kizazi Kipya. They suggested that the project link them to other donors after the project ends (4/5 KIIs with DSWOs, 2/4 KIIs with CPTs, 3/5 KIIs with CTC staff). As GoT officials and CTC in charge from different locations reported: “We are not able to pay CCWs. We don’t have a budget for this. These CCWs are only volunteering. Also, the government is not able to print these forms or referral books or buy new ones when they finish.” GI, CPT “I think that the whole system will continue but not in the same way as when the project was present. The project supported the service providers to reach distant households, so when the project ends, such things will also end.” CTC In Charge Staff The likelihood of the NICMS sustaining project practices and achieving the PEPFAR MER OVC graduation benchmarks is very low if funding is not continued by PEPFAR or other donors. Already 20 percent of the PEPFAR budget was reduced for FY 2018 in prevention-based activities. It begs the question as to which other bilateral or multilateral donor support can be leveraged to fill in the gaps that will inevitably be produced USAID’s role in HIV and OVC support is reduced. 5.0 RECOMMENDATIONS Strengthening Continuum of Care for CLHIV Recommendations for USAID and Implementing Partner ● Review the geographic limitations of targets to allow for services to HIV positive children falling within facility catchments areas when accessing services, but outside of geographic coverage where they reside. Flexibility is justified in the light of epidemic control priorities. If necessary, light packages of services could be designed to mitigate the additional cost for extending project infrastructure to these areas. ● Consider allowing for a proportion of targets to remain unfilled as a buffer to accommodate the enrollment of HIV positive children as they are identified across the project’s full implementation and including HEI to support EID results. The buffer would also allow for targets to accommodate facility referrals and would mitigate the limited possibilities of OVC activities for case finding. ● Develop custom indicators to emphasize and drive the OVC platform’s contribution to epidemic control. A key indicator in this regard would be a VLS indicator for OVC partners twinned with a linkage rate indicator for care and treatment partners to track their referrals from facilities to the community platform. 30 ● Facilitate more intensive coordination with facilities and care and treatment partners to facilitate enrollment of HIV positive children. Coordination would include sharing lists of all OVC beneficiaries enrolled with the facility team by curating a list of all HIV positive, under-18 clients, matching that list with OVC partner list, and providing OVC partners with remaining names for future enrollment. It may also be worth considering a constant presence of senior cadre at care and treatment facilities to oversee completion of referrals and participate in clinical tasks, such as switch committees. Regular case conferencing that includes facility staff as participants should also be considered. ● Provide more nutritional information to HIV positive OVC, especially those on antiretroviral HIV drugs (ARVs), and facilitate linkages to food security support, as feasible. Recommendations for Implementing Partner ● Strengthen case management for continuum of care of HIV positives specifically and for intensively managing all clients towards graduation. This will not only strengthen the effectiveness of services but also free up space for continuous enrollment of new positives. ● Analyze the current OVC caseload composition in order to confirm the number and proportions by OVC sub-population type (e.g. CLHIV, children of PLHIV, children of female sex workers, orphans CLWS, etc.), recognizing there will be overlap. Then appropriate sub-population target shifts can be planned. Strengthening Prevention Activities Recommendations for USAID and Implementing Partner ● Find vehicles to supplement SASA! to include young men and boys aged 9-14 years, with a focus on those in school, and community-based interventions to prevent sexual violence and HIV and to better understand GBV, protect themselves against violence and HIV infections, and also protect girls. ● Revisit targeting and costing models to find resources for scale-up. Although there are prevention efforts in addition to those reviewed here, curricula-based prevention efforts are essential for broader efficacy. Kizazi Kipya’s implementation scale is questionable and should be revisited. Strengthening Case Management Recommendations for Implementing Partner ● Strengthen the workforce management of CCWs, including registration and closer tracking of cadres, working with the GoT to increase monthly stipends to decrease CCW dropouts, and institute continuous CCW training with supplementary training for technical areas. ● Strengthen the vulnerability assessment further with validation and weighted scoring of instruments, integrating the strong work already done on HIV-focused assessment ● Improve the MIS and resolve unique identifier issues to ensure error free tracking of layering. Strengthening Prospects for Sustainability Recommendations for USAID and Implementing Partner ● Work to improve the commitments of local actors to ensure case management over the longer term. Regarding the GoT’s commitment to take on financial commitment of the project activities, PO-RALG and Kizazi Kipya should encourage LGA officials to establish and fund their own council level committees’ activities. For health service providers to take over HIV treatment and care, LGAs must fund and sustain the practices promoted by Kizazi Kipya. Also, ownership and involvement of GoT officials at ward and village levels are vital. 31 ● GoT or other donor commitments will be needed to sustain CCW stipends, the costs of travel, printing, and maintenance of tools and referral forms. Other operational requirements will be needed for CCW supervisors and SWOs to provide continued supportive supervision. ● Encourage the GoT’s national commitments to funding in an effort to sustain the NICMS once the project ends. Leveraging other donor support and continuing PEPFAR support will be required in the intermediate term. The likelihood of sustaining the NICMS and achieving the PEPFAR MER OVC graduation benchmarks is low if funding is not continued by PEPFAR or other donors. Recommendations for Implementing Partner ● Succession planning needs to be implemented and mainstreamed. This means integrating Kizazi Kipya service packages into the planning and budgeting tool PlanRep in preparation for the GoT to take on project activities. The likelihood that the GoT or other donors maintain the costs of structural operations or for case management will increase once Kizazi Kipya begins to work with stakeholders in this effort. 32 ANNEXES 33 ANNEX I: EVALUATION STATEMENT OF WORK 34 MID-TERM PERFOMANCE EVALUATION FOR KIZAZI KIPYA ACTIVITY EVALUATION STATEMENT OF WORK I. BACKGROUND INFORMATION A. Identifying Information Project/Activity Title Kizazi Kipya Award Number AID-621-A- 16-00001 Award Dates July 05, 2016- July 04, 2021 Project/Activity Funding Global Health-(HIV/AIDS) (PEPFAR)-$69,134,274 Implementing Organization(s) Pact Inc. Project/Activity AOR Elizabeth Lema B. Development Context 1. Problem or Opportunity Addressed by the Project/Activity Being Evaluated The goal of the cooperative agreement is to improve the well-being of one million HIV-affected orphans and vulnerable children (OVC) and their families. Key expected results include improved health, nutrition, education, protection, livelihoods, and psychosocial outcomes, as well as strengthening the HIV continuum of care among HIV positive OVC, youth, and their parents/caregivers. There are available but underutilized HIV/health and social services that constrain vulnerable children and families’ outcomes. Customized case management to meet HIV-related and other needs of OVC and their families is limited. Coordination and integration of decentralized structures, referral systems, and programs remain inadequate. Limited use and scale up of evidence-based interventions inhibit coverage of high-impact activities that address the specific HIV, well-being, and caregiving needs of OVC. Measurable improvements in these areas hold promise for achieving the nation’s goals for HIV control and response, child and family resilience, gender equity, violence prevention and response, and poverty reduction. 2. Target Areas and Groups Pact’s original geographic coverage was 130 councils of which 42 scale-up councils and 88 sustained councils. The plan was to transition off from all sustained councils by September 30, 2017. In Country Organizational Plan (COP) 2017, PEPFAR changed scale-up councils, previously known as Tier 1a and 1b to PEPFAR priority councils. From FY 2018 – FY 2021 (years two to five), Pact will work in 79 PEPFAR priority councils of which 67 PEPFAR priority councils have beneficiaries enrolled into care at the end of FY 2017, and these councils have FY 2018 OVC_SERV targets as well as 12 additional PEPFAR priority councils with FY 2018 OVC-SERV targets: (11 councils in Shinyanga, Simiyu, and Kigoma Regions plus Tunduma, a council that has not previously received PEPFAR OVC support but has been given targets in FY 2018). In FY 2017, Pact worked in a total of 130 councils, a combination of scale up (Tier 1a and Tier 1b) and sustained (Tier 2) councils. This number is in line with the ceiling of 130 councils advised in the RFA. In FY 2017, Pact wrote Standard Operating Procedures to guide CSOs to transition beneficiaries in all Tier 2 (sustained) councils off PEPFAR support before September 30, 2017. Pact inherited 62 councils from the ended Pamoja Tuwalee OVC project that were either Tier 2 councils or not categorized as either Tier 1 or Tier 2 councils at the time of the award. Pact does not plan to work in any of these 62 councils from FY 2018 onwards. By the end of FY 2017, Pact was implementing OVC activities in 67 Tier 1 councils. The approved COP18 for Tanzania includes OVC-SERV targets for all these 67 councils; therefore, Pact did not plan to transition any beneficiaries in the 67 councils in FY 2017 off of PEPFAR support in FY 2017 or FY 2018. Between October 2017 through September 2021 (FY 2018 – FY 2021), Pact anticipates 35 working in additional councils newly created by the government of Tanzania (exact number to be determined) through the division of some of the same 79 councils into two councils. Kizazi Kipya will only implement in these new councils if the project has existing beneficiaries in wards that were split among the two councils. It is also important to understand that there is a plan to is to modify the award to increase Pact’s geographic coverage councils to 106 upon executed. C. Intended Results of the Project/Activity Being Evaluated Kizazi Kipya’s theory of change focuses on impact, proposing that if Kizazi Kipya enhances the strengths and reduces the deficits of families caring for OVC through assets-based case management and economic strengthening while facilitating cross-sectoral coordination and referral, then there will be increased uptake of highly effective HIV and other essential services leading to reduced HIV incidence, improved family resilience, thriving children, and empowered women and youth. D. Approach and Implementation E. Existing Documents Activity description, project’s work plan, PMP, and M&E plan, Kizazi Kipya baseline data, quarterly and annual performance reports, and project PEPFAR MER OVC Essential Survey outcome indicators. 36 II. EVALUATION RATIONALE A. Evaluation Purpose This midterm performance evaluation is being conducted in order to: 1. Assess the extent to which program performance is on track to achieve expected outcome and analyze the effectiveness of various approaches used by Kizazi Kipya to achieve the four program results and foster sustainability. 2. Document good practices, gaps/limitations, and lessons learned that will inform the Mission's design of the follow-on OVC adolescents care and support program, including critical approaches, focus areas, and interventions. B. Audience and Intended Uses 1. Missions-technical officer – to inform the design of the follow-on or new OVC activity. 2. Government – to contribute to the national guidelines and strategies development. C. Evaluation Questions 1. To what extent is Kizazi Kipya on track to achieving improved child and family well-being (i.e., improved health, nutrition, education, protection, and Household economic strengthening outcomes)? 2. To what extent has Kizazi Kipya contributed to improving the HIV continuum of care among HIV positive children and adolescents and how can this be strengthened in the remaining program period (in collaboration with HIV clinical programs)? 3. To what extent has Kizazi Kipya contributed to HIV prevention among adolescents (with a focus on girls) and how could this be strengthened in the remaining program period? 4. To what extent has Kizazi Kipya contributed to Violence against Children (VAC) prevention and response and how could this be strengthened in the remaining program period? 5. What was the private sector's involvement and contribution to the program? 6. What have been the strengths and weaknesses of USAID's in, and oversight of, Kizazi Kipya? III. EVALUATION DESIGN AND METHODOLOGY A. Evaluation Design This is a midterm performance evaluation that will be comparing differences observed between baseline situation and mid-point of program implementation. It will, therefore, assess performance on predetermined, valid criteria (objectives and expected results) which can be consistently measured over the activity’s life cycle. The comparison will look into facilitating and/or prohibiting factors (internally and externally) that the evaluator will provide information on progress and recommendation to either persuade for its continuity or changing the course of action. The principal source will be reviewing activity progress reports, meeting with stakeholders, and both direct and indirect beneficiaries (informants) 37 B. Data Collection & Analysis Methods Evaluation Questions Suggested Data Source Data Collection Method(s) Sampling or Selection Criteria Data Analysis Method(s) (1) To what extent is Kizazi Kipya on track to achieving improved child and family well￾being (i.e., improved health, nutrition, education, protection, and Household economic strengthening outcomes)? LGAs, IP, and Partners KII, FGD - Qualitative and quantitative (2) To what extent has Kizazi Kipya contributed to improving the HIV continuum of care among HIV positive children and adolescents and how can this be strengthened in the remaining program period (in collaboration with HIV clinical programs)? LGAs, IP, and Partners KII, FGD - Qualitative and quantitative (3) To what extent has Kizazi Kipya contributed to HIV prevention among adolescents (with a focus on girls) and how could this be strengthened in the remaining program period? LGAs, IP, and Partners KII, FGD - Qualitative and quantitative (4) To what extent has Kizazi Kipya contributed to Violence Against Children (VAC) prevention and response and how could this be strengthened in the remaining program period? LGAs, IP, and Partners KII, FGD - Qualitative and quantitative (5) What was the private sector’s involvement and contribution to the program? IP, Partners, and Private organizations KII, FGD - Qualitative and quantitative (6) What have been the strengths and weaknesses of USAID's in, and oversight of, Kizazi Kipya?" LGAs, IP, and Partners KII, FGD - Qualitative and quantitative C. METHODOLOGICAL STRENGTHS AND LIMITATIONS ∉ Language barrier ∉ Ended program spillover effect (can be difficult to have a clear cutline between the effect of the Kizazi Kipya from inherited ended project of Pamoja Tuwalee). IV. EVALUATION PRODUCTS A. Deliverables 1. Inception report with tentative evaluation work plan/schedule indicating which step/activity should occur and when to ensure that the evaluation team fully understands the assignment approach. The work plan and schedule should be approved by USAID prior to implementation. 2. Evaluation design and methodology clearly articulating how the evaluation questions will be answered during the pre-evaluation meeting. 38 3. The second briefing will be held immediately following site visits and data analysis to make sure that all questions have been addressed. 4. A debrief meeting with a PowerPoint Presentation of the key findings, issues, and recommendations. 5. The draft report. The written report should clearly describe findings, conclusions and recommendations (using the report format provided in “Reporting Requirements” below). Community care team Tanzania team will provide comment on the draft report within 14 working days of submission. 6. A final report that incorporates the team’s responses to Mission comments and suggestions. 7. Final report format – Executive summary, Methodology, Findings, Analysis of the findings of each research question. Thirty (30)-page limit. B. Reporting & Submission Guidelines Criteria to Ensure Quality of the Evaluation Report Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report.15 ∉ Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. ∉ Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. ∉ The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. ∉ Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. ∉ The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline need to be agreed upon in writing by the technical officer. ∉ Evaluation methodology should be explained in detail and sources of information properly identified and will be included as an Annex in the final report. ∉ Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ∉ Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. ∉ Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. ∉ If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. ∉ If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific, with defined responsibility for the action. 15 This is the role or job title of each individual participating. Do not document participant names in this table. 39 V. TEAM COMPOSITION, ROLES, AND RESPONSIBILITIES OF TEAM MEMBERS The Mission proposes a team composition of Team Leader/Program Development and Gender, M&E, and OVC experts. All team members should have the following qualifications: ∉ Master’s degree or higher level of education in a relevant technical area, experience working in OVC program is desirable. ∉ Knowledge, skills, and experience with USAID contracting and reporting requirements; policies and initiatives; and tools, such as performance monitoring plans (PMPs) and results frameworks. ∉ Advanced written and oral communications skills in English. Team Leader/Program Development: Should have a minimum of 10 years of experience in public health, with technical knowledge and experience with OVC intervention, preferably community-based support. The team leader will be identified by the research firm/contractor and approved by USAID prior to the start of evaluation activities. The team leader will be responsible for (1) managing the team’s activities, (2) ensuring that all deliverables are met in a timely manner, (3) serving as a liaison between the Mission and the evaluation team, and (4) leading briefings and presentations. In addition, the team leader must have the following qualifications: ∉ Excellent skills in planning, facilitation, and consensus building; ∉ Demonstrated experience leading an evaluation team; ∉ Excellent interpersonal skills; ∉ Excellent skills in project management; and ∉ Excellent organizational skills and ability to keep to a timeline. Monitoring and Evaluation Expert: One person, preferably locally-based, with relevant professional qualifications and skills and at least five years of experience in USAID M&E procedures and project and organizational management. This person should also have strong knowledge, skills, and experience in qualitative and quantitative evaluation tools, as well as skills in community program designing and monitoring. The Mission M&E specialist can participate in the evaluation exercise. Gender Expert: This person should have a post graduate degree in gender. S/he should have at least five years of experience in gender programming, i.e., analysis, planning, mainstreaming, and evaluating. OVC Expert: S/he should be knowledgeable in OVC programming, program assessment, and evaluation methodologies of community-based programs and work plan development and training in information monitoring systems. S/he should have extensive experience, and demonstrate state-of-the-art knowledge, in conducting OVC programmatic designing and evaluations/assessments. VI. EVALUATION MANAGEMENT A. Logistics Outline specific kinds of support USAID and the implementing organization(s) will provide. Specify any additional logistical roles or responsibilities it expects the evaluation team to fulfill or to provide (e.g., interpreters, translators, surveyors). Clarify any specialized management arrangements, such as in the case of joint evaluations. Implementing partner will only provide basic support required including organizing appointments for planned visits. The regional based teams expect to provide logistical support and visit arrangements. B. Tasks and Scheduling Dates and Timeline – Report required (September 1-20, 2018). 40 ANNEX II: BIBLIOGRAPHY 41 Kizazi Kipya Contract, Cooperative Agreement No. AID-621-A-16-0000, mod, 2017. USAID Kizazi Kipya Year 2, Quarter 3 Report, April 1, 2018 to June 30, 2018. USAID Kizazi Kipya Year 2, Quarter 2 Report, January 1, 2018 to March 31, 2018. USAID Kizazi Kipya Quarterly Progress Report for FY 2017 Quarter 2, January 1, 2017 to March 31, 2017. USAID Kizazi Kipya Quarterly Progress Report for FY 2017 Quarter 3, April 1, 2017 to June 30, 2017. USAID Kizazi Kipya Quarterly and Annual Progress Report for FY 2017 Quarter 4 and annual, July 1, 2017 to September 30, 2017. USAID Kizazi Kipya Year 2, Quarter 4 and Annual Progress Report, July 1 to September 30, 2018 and October 2017 to September 2018. Kizazi Kipya Project, Tanzania, PEPFAR MER OVC Essential Survey Outcome Indicators, March 2018. Pact Kizazi Kipya Information Handout, 2018. Technical Brief: USAID Kizazi Kipya, Support for the National Five-Year Development Plan (2016/17 2020/21) through capability development of adolescents and youth, November 2017. Technical Brief: USAID Kizazi Kipya, Support for implementation of the National Integrated Case Management Framework, November 2017. Technical Brief: USAID Kizazi Kipya Support for implementation of the National Plan of Action to End Violence Against Women and Children, November 2017. 42 ANNEX III: KII AND FGD DISCUSSION GUIDES 43 GROUP INTERVIEW/KII PROTOCOL: IMPLEMENTING PARTNER OR SUB-GRANTEE AT LOCAL LEVEL Introduction and Consent Date (dd/mm/yyyy): . Interviewer’s name: . Number of participants: . Designations16 of the participants (Enter each participant’s designation separated by a comma): . Start time (24 h clock, hh:mm): . End time (24 h clock, hh:mm): . To start the interview, please read the following script: Introduction and purpose of the study Hello and thank you for agreeing to speak with us. My name is_______________ (interview name) and this is my colleague__________________. We work with the Data for Development project, a USAID￾funded platform that seeks to improve the quality and use of data in decision-making in Tanzania. We are undertaking an evaluation for USAID to understand what has worked well, what did not work as well, and what has been sustained from the USAID-funded Kizazi Kipya Project implemented by Pact and the Community Health and Social Welfare Systems Strengthening Project, or CHSSP, implemented by John Snow that together aim to improve the well-being of people living with HIV, adolescent girls and young women, and most vulnerable children and their families. Description of study procedures If you agree to be interviewed, you will be asked to share your experiences with and views about the two projects. During our conversation we would like to take notes and record the conversation. The recording and the notes are to make sure that the evaluators have heard and understood what you have shared with us today. Your ideas, together with other data, will be used to produce a report and not otherwise. This report will be presented to USAID for decision-making. Your name will not be included in the report, but the ideas you share might be. Our discussion today will take 45-60 minutes. Confidentiality The recording and the notes are strictly confidential. Only the evaluation team will listen to this recording or read the notes. The recordings and the notes will be kept in a secure location and all electronic information will be coded and secured. The recordings and notes will be destroyed after the project is completed. Your privacy will be protected; we will not include any information in any report that would make it possible to identify you. Please note that we cannot guarantee full confidentiality because of the group setting, as we cannot ensure that participants will not disclose any information shared during the group interview. Once again, we ask that what we discuss today remains here with us. Benefits of participating in this study There is no direct benefit to you being in this study, but we hope that the results of our study will help improve the project implementation and the important services it provides. You will receive no compensation for participating in this interview. Right to refuse or withdraw The decision to participate in this study is entirely up to you. You may refuse to take part in the study at 16 This is the role or job title of each individual participating. Do not document participant names in this table. 44 any point in time, and you have the right not to answer any single question, as well as to withdraw from the discussion at any point. You will not be penalized if you choose to not answer or withdraw from this discussion. Right to ask questions and report concerns You have the right to ask questions about this evaluation and to have those questions answered by me before, during, or after the interview. Do you have questions for me at this time? If you have any further questions about the study, feel free to contact Gerald Usika through [gusika@engl.com] or by phone at [+255 756180413]. This evaluation has been reviewed and approved by National Institute for Medical Research ethical committee. If you have any questions concerning the study, please contact NIMR at +255222121400. Consent Do you agree to participate in this study? Do I have your permission turn on the tape recorder and begin the interview? 1. Yes 2. No We will use this protocol for any government cadre, with the exception of clinical staff at facilities and DREAMS coordinators where we encounter them. The following protocol is to be used with technical implementing staff at local level, whether the respondent(s) is (are) working directly for an implementing partner in the consortium or a local sub￾grantee. The Implementer’s Role and Relationship with Kizazi Kipya A.1 Describe the work your CSO is doing in the Kizazi Kipya project? Elaborations and probes: Who are your beneficiaries? What services are you delivering directly? Which services are you linking beneficiaries to? A.2 How did this CSO became a partner to Kizazi Kipya? Elaborations and probes: For example, were you approached by a Kizazi Kipya implementer or a local government authority to join the project? Did you have to submit a proposal or go through a selection process? A.3 Can you describe how your relationship with Kizazi Kipya is managed? Elaborations and probes: For example, is there a contract or MOU in place? What and how frequently do you report to Kizazi Kipya? What kind of support does Kizazi Kipya provide you with? A.4 Has working with Kizazi Kipya led to changes that have strengthened your CSO? Elaborations and probes: For example, have you had to introduce policies and practices to improve governance or financial management? What other changes have been introduced and would you say these have improved the CSO? In what way? Referrals and Linkages B.1 How do you work with local government, community structures, and health facilities to do your work? Elaborations and probes: For example, do you coordinate with them to identify clients and/or deliver services? How do you do so? Are there sufficient government resources and services to link clients to? 45 Do you have examples of how coordinating with local government and community structures has improved project performance? B.2 How do you work with other CSOs and NGOs to deliver services you need to link clients to? Elaborations and probes: For example, do you coordinate with them to identify clients and/or deliver services? How do you do so? Are there enough CSOs and other services to meet the needs of Kizazi Kipya clients? Do you have examples of how coordinating with other CSOs and projects has improved project performance? B.3 Are you able to link victims of violence to services that successfully provide them with the assistance they need? Elaborations and probes: For example, are you able to link them to the health services that meet their needs? Are child protection services helpful effective? Are the police helpful and effective? What about legal services and the courts? What are the challenges? Case Management and Provision of Services C.1 We are very interested in how your volunteers manage to deliver services. Would you describe the work the volunteers are doing and how they are supported? Elaborations and probes: In addition to Community Case Workers, what other volunteers from your organization deliver services to vulnerable households? How do you manage these volunteers and how do they go about their work? How are they compensated? How do you coordinate your volunteers with other community workers such as Community Health Workers? What challenges do they face in doing their work? C.2 What strategies have worked particularly well in the implementation of project activities? Elaborations and probes: ● How successful have you been in identifying and enrolling HIV positive children into services? What strategies have worked? ● How successful have you been in supporting households to become resilient economically? Would you say that these activities empowering women? What strategies have worked particularly well, and what challenges remain? ● Have you been working to prevent violence against women and children in the community? What strategies have worked well, and what challenges remain? ● Would you say your work is helping to prevent HIV infections in your community? What strategies work well for prevention? ● Have you been working to help children stay in school? What strategies work well for this? C.3 Are you implementing DREAMS activities or activities that are exclusively for adolescent girls and young women? What do these activities consist of? How do you go about recruiting AGYW for these activities? How do you ensure that the participating AGYW get all the services in the package? C.4 We are also very interested in how well the case management of vulnerable households is working. Are you managing to implement the national case management system successfully through the CCWs? ∉ What has helped to implement case management successfully? For example, training, the provision of tools, supportive supervision? 46 ∉ What challenges do you continue to face in implementing the national case management system? How can case management be improved? C.4 How is data about your work collected, managed, and used? Elaborations and probes: What kind of data is collected? What tools are you using? Are there registers or case management files? How is this data brought together and submitted to Kizazi Kipya? How do you use the data? C.5 How do you manage client confidentiality? Elaborations and probes: Are there any SOPs or guidelines for ensuring the confidentiality of the clients? What are the challenges of maintaining confidentiality in close communities such as the one you work in? How successful are you at maintaining confidentiality? Can you describe any issues that have arisen and that you’ve had to manage related to the confidentiality of clients? Continued Challenges and Lesson Learned D.1 Considering all these things we’ve discussed—how the volunteers are working, how case management is working, how successfully we are supporting households with each of the services—what challenges do you continue to face and how can they be addressed? D.2 What lessons have been learned so far that can be used to improve Kizazi Kipya and improve activities like this in the future? Sustainability E.1 Do you think the provision of services to vulnerable households will continue after the Kizazi Kipya project ends? Elaborations and probes: For example, to what extent are the practices the project has put in place to improve access and quality of services likely to be continued by stakeholders after Kizazi Kipya closes? Are there any institutions that could play the role the project has taken on after Kizazi Kipya exits? Would there be funding from other sources for such activities? E.2 What should Kizazi Kipya do differently to ensure the sustained delivery of quality services to vulnerable households after it closes? Elaborations and probes: What efforts should the project be making now? Are there efforts that other stakeholders could be making to improve prospects for sustainability? Conclusion We would like to thank you all for participating in this conversation. Sometimes these conversations can be difficult and sometimes you attend when it is difficult or inconvenient for you. We truly appreciate your effort and the valuable information you’ve given us. We’ve learned a lot from you and we will use this information to help SAUTI improve what they do. If you have any questions for us, please feel free to ask. 47 FGD GUIDE FOR AGYW IN DREAMS Introduction and Consent Date (dd/mm/yyyy): . Interviewer’s name: . Implementing Partner Name: . Number of participants: . Age of each participant: . Start time (24 h clock, hh:mm): . End time (24 h clock, hh:mm): . To start the interview, please read the following script: Introduction and purpose of the study Hello and thank you for agreeing to speak with us. My name is_______________ (interview name) and this is my colleague__________________. We work with the Data for Development project, a USAID￾funded platform that seeks to improve the quality and use of data in decision-making in Tanzania. We are undertaking an evaluation for USAID to understand what has worked well, what did not work as well, and what has made the most difference in the USAID-funded Kizazi Kipya Project implemented by Pact and the Community Health and Social Welfare Systems Strengthening Project, or CHSSP, implemented by John Snow. The projects aim to improve the well-being of people living with HIV, adolescent girls and young women, and most vulnerable children and their families. We are, thus, very happy that you have agreed to meet with us to tell us about your experiences. Description of study procedures If you agree to be in this focus group discussion, you will be asked to share your experiences with and views about the two projects. During our conversation we would like to take notes and record the conversation. The recording and the notes are to make sure that the evaluators have heard and understood what you have shared with us today. Your ideas, together with other data, will be used to produce a report and not otherwise. This report will be presented to USAID for decision-making. Your name will not be included in the report, but the ideas you share might be. Our discussion today will take 45-60 minutes. Confidentiality The recording and the notes are strictly confidential. Only the evaluation team will listen to this recording or read the notes. The recordings and the notes will be kept in a secure location and all electronic information will be coded and secured. The recordings and notes will be destroyed after the project is completed. Your privacy will be protected; we will not include any information in any report that would make it possible to identify you. Please note that we cannot guarantee full confidentiality because of the group setting, as we cannot ensure that participants will not disclose any information shared during the group interview. Once again, we ask that what we discuss today remains here with us. Benefits of participating in this study There is no direct benefit to you being in this study, but we hope that the results of our study will help improve the project implementation and the important services it provides. You will receive no compensation for participating in this interview. Right to refuse or withdraw The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any point in time, and you have the right not to answer any single question, as well as to withdraw from the discussion at any point. You will not be penalized if you choose to not answer or withdraw from this 48 discussion. Right to ask questions and report concerns You have the right to ask questions about this evaluation and to have those questions answered by me before, during, or after the interview. Do you have questions for me at this time? If you have any further questions about the study, feel free to contact Gerald Usika through [gusika@engl.com] or by phone at [+255 756180413]. This evaluation has been reviewed and approved by National Institute for Medical Research ethical committee. If you have any questions concerning the study, please contact NIMR at +255222121400. Consent Do you agree to participate in this study? Do I have your permission turn on the tape recorder and begin the interview? 1. Yes 2. No Let’s begin by talking about how you got involved in the project, then we can talk about your experiences of the project. 1. How did you become involved in DREAMS? (Probe: Who introduced you and/or where were recruited from—school, community, clinic?). 2. What made you decide to join? What was interesting to you about the program? 3. Have you learned anything or gained anything from being part of DREAMS? If so, what were the most important things you’ve learned or skills you acquired from being in DREAMS? 4. What has been the most important part of the DREAMS program for you? (Probe: education support, safe spaces and being together with other girls, having a mentor, economic strengthening activities, PrEP, accessing family planning, HIV and GBV prevention education, other). Why was this the most important? 5. Has anything changed in your life because of DREAMS? Can you explain what that is and how it has changed? 6. Are there issues in your life or in your community that DREAMS is not addressing, that should be addressed? What are these? How could they be addressed? 7. How would you recommend improving DREAMS? -----------------Thank you for your time-------------------- 49 FGD GUIDE FOR COMMUNITY CASE WORKERS Let’s begin by talking about the Case Management training you received from CHSSP? 1. What topics did the training cover? 2. How useful was the training? In answering this question, please consider such things as the knowledge/expertise of the instructors, the quality of the instruction, the quality and usefulness of training resources, and the relevance of the topics covered to your job as a community case worker. 3. What were the most important things you learned or skills you acquired from the training as they apply to your job as a community case worker? 4. To what extent has the training you received contributed to improving your ability to provide community health, social welfare, and HIV prevention and treatment services to key populations, particularly people living with HIV, adolescent girls and young women, and most vulnerable children? In answering this question, please describe how you have applied what you learned from the training in your job as a community case worker. Please cite specific examples. 5. What challenges do you face in applying what you learned in the training to your job as a community case worker? How did CHSSP help you address these challenges? What more could it do? (Note to interviewer: Among other things, probe to find out about the type and frequency of follow-up support received since the Case Management training.) 6. How likely is it that you will be able to apply what you learned from the training to your job as a community case worker over the long run? Why? How can this likelihood be improved? 7. What were the strengths of weaknesses of the case management training you received from CHSSP? How could this training be improved? Now, let’s talk about your work as a community case worker as supported by the Kizazi Kipya project. 8. Please describe your work as a community case worker? What does a CCW do? 9. What support do you receive in your work as a community case worker from Kizazi Kipya? 10. How useful has this support been? In particular, how has it affected your capacity and performance as a community case worker? 11. What are the greatest challenges you face in your work as a community case worker? To what extent does Kizazi Kipya support address these challenges? What more could it do? 12. What are the strengths of weaknesses of the support you receive from Kizazi Kipya? How could this support be improved? 13. Can you describe the services you help people access? Please describe the services provided by Kizazi Kipya and the services provided by other organizations or government structures that you link people to. 14. Are the people you assist in your work as a community case worker accessing and using the services to which you referred them? What factors help explain why they are or are not accessing and using these services? 15. For those people who do access and use the services to which you referred them, what is the quality of care or services they are receiving? What factors explain the quality of care or services they are receiving? 16. Are there sufficient services available in the community for you to be able to link vulnerable households to meet their needs? What are the gaps? 50 17. Overall, how have the CHSSP and Kizazi Kipya projects contributed to improving the local system for providing health, social welfare, and HIV prevention and treatment services to key populations, both with regards to the case management system that you are involved in and in other ways you may be aware of? 18. How long do you plan to continue working as a community case worker? What factors make you want to work for a longer period? For a shorter period? What could CHSSP, Kizazi Kipya, or someone else do to make you want to keep working as a community case worker over the long term? -----------------Thank you for your time-------------------- 51 GI GUIDE FOR HURU FACILITATORS IN DREAMS 1. How did you become involved in DREAMS? (Probe: Who introduced you and/or where were recruited from—school, community?) What made you decide to join? What was interesting to you about the program? 2. What do you do (in detail)? Are you delivering a curriculum for prevention/DREAMS? Which one? How? 3. Which girls are being reached? The most vulnerable? How are they recruited? How do you keep track of the girls completing it? In what setting is it delivered? What other activities do you do with the girls ? 4. Have you been trained to do what you are doing? Who trained you? Are you being supervised? Have you learned anything or gained anything from being part of DREAMS? If so, what were the most important things you’ve learned or skills you acquired from being in DREAMS? 5. What has been the most important part of the DREAMS program for you? (Probe: education support, safe spaces and being together with other girls, having a mentor, economic strengthening activities, PrEP, accessing family planning, HIV and GBV prevention education, other.) Why was this the most important? 6. Has anything changed in your day to day role because of DREAMS? Can you explain what that is and how it has changed? 7. What are the results you observe for the girls? 8. Are there other activities in the community that support outcomes for AGYW, e.g., are there activities with parents, community leaders, etc.? 9. Are there issues in your community that DREAMS is not addressing, that should be addressed? What are these? How could they be addressed? 10. What challenges do you face and how do you address them? How would you recommend improving DREAMS? 11. Do you see any sustainability of the activities you are doing to support DREAM girls? Why and why not? -----------------Thank you for your time-------------------- 52 GI GUIDE FOR CARE AND TREATMENT IPS Introduction and Consent Date (dd/mm/yyyy): - Interviewer’s name: - Number of participants: - Designations17 of the participants (Enter each participant’s designation separated by a comma): - Start time (24 h clock, hh:mm): - End time (24 h clock, hh:mm): - To start the interview, please read the following script: Introduction and purpose of the study Hello and thank you for agreeing to speak with us. My name is_______________ (interview name) and this is my colleague__________________. We work with the Data for Development project, a USAID￾funded platform that seeks to improve the quality and use of data in decision-making in Tanzania. We are undertaking an evaluation for USAID to understand what has worked well, what did not work as well, and what has been sustained from the USAID-funded Kizazi Kipya Project implemented by Pact and the Community Health and Social Welfare Systems Strengthening Project, or CHSSP, implemented by John Snow that together aim to improve the well-being of people living with HIV, adolescent girls and young women, and most vulnerable children and their families. Description of study procedures If you agree to be interviewed, you will be asked to share your experiences with and views about the two projects. During our conversation we would like to take notes and record the conversation. The recording and the notes are to make sure that the evaluators have heard and understood what you have shared with us today. Your ideas, together with other data, will be used to produce a report and not otherwise. This report will be presented to USAID for decision-making. Your name will not be included in the report, but the ideas you share might be. Our discussion today will take 45-60 minutes. Confidentiality The recording and the notes are strictly confidential. Only the evaluation team will listen to this recording or read the notes. The recordings and the notes will be kept in a secure location and all electronic information will be coded and secured. The recordings and notes will be destroyed after the project is completed. Your privacy will be protected; we will not include any information in any report that would make it possible to identify you. Please note that we cannot guarantee full confidentiality because of the group setting, as we cannot ensure that participants will not disclose any information shared during the group interview. Once again, we ask that what we discuss today remains here with us. Benefits of participating in this study There is no direct benefit to you being in this study, but we hope that the results of our study will help improve the project implementation and the important services it provides. You will receive no compensation for participating in this interview. Right to refuse or withdraw The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any point in time, and you have the right not to answer any single question, as well as to withdraw from 17 This is the role or job title of each individual participating. Do not document participant names in this table. 53 the discussion at any point. You will not be penalized if you choose to not answer or withdraw from this discussion. Right to ask questions and report concerns You have the right to ask questions about this evaluation and to have those questions answered by me before, during, or after the interview. Do you have questions for me at this time? If you have any further questions about the study, feel free to contact Gerald Usika through [gusika@engl.com] or by phone at [+255 756180413]. This evaluation has been reviewed and approved by National Institute for Medical Research ethical committee. If you have any questions concerning the study, please contact NIMR at +255222121400. Consent Do you agree to participate in this study? Do I have your permission turn on the tape recorder and begin the interview? 1. Yes 2. No Exposure to Kizazi Kipya 1. How does the Kizazi Kipya project support you and your role in providing care and treatment services? For example, does the project coordinate with you to achieve testing, treatment, and viral load suppression targets? 2. How closely do you work with the Kizazi Kipya project? For example, do you meet with Kizazi Kipya staff to jointly plan for linking people to services? Do you discuss and try to help specific households by linking clients from the facility to Kizazi Kipya services? Contribution of Kizazi Kipya and Case Management 3. Has the Kizazi Kipya project improved the services provided to vulnerable households and their children? If it has, can you explain how? a. Access probes: For example, does Kizazi Kipya provide services directly to households that weren’t available before or that no other project provides? Does Kizazi Kipya make it easier to link clients to services provided by other organizations? b. Quality probes: Has Kizazi Kipya supported other organizations to improve the quality of the services they provide? 4. One of Kizazi Kipya’s key objectives is to improve case management of vulnerable households. Has the Kizazi Kipya project strengthened case management in your communities? If yes, can you explain how? a. For example, was there a case management system in place before Kizazi Kipya? What is the difference between the earlier system and the system in place now? b. Resource probes: Are there more personnel involved in case management because of Kizazi Kipya support? Are the personnel involved adequately trained and supervised? Are the personnel better equipped with case management tools? Are there more resources available to do case management? c. Quality probes: Are the most vulnerable households being identified and reached with services? Are households being linked to services they need? Are we able to better track 54 the progress of households out of vulnerability because of improvements in case management? 5. A key assumption in case management is that there are enough quality services in the community that vulnerable households can be linked to. Are there sufficient services in your communities to be able to meet the needs of the vulnerable households and children you identify? If not, then what are the gaps? 6. How can case management be improved? Achieving Results 7. Has the Kizazi Kipya project made it easier to find HIV positive children and deliver services to them? Can you explain your answer? Was it more difficult in the past to find the HIV positive children? What is Kizazi Kipya doing now that makes it easier? 8. Has the Kizazi Kipya project improved referrals and the linking of vulnerable households and children to health services other than HIV services? Can you explain your answer? What is the project doing specifically that makes referring and linking to health services better? 9. Has the Kizazi Kipya project improved referrals and the linking of vulnerable households and children from health services to other social welfare services? Can you explain your answer? What is the project doing specifically that makes referring and linking to other services better? 10. Does the Kizazi Kipya project strengthen the economic situation of vulnerable households and women in particular? Can you explain your answer? How does the Kizazi Kipya project go about improving the economic situation of vulnerable households? 11. Does the Kizazi Kipya project help women and children who are victims of violence? Does the project help to prevent violence against women and children? Can you explain your answer? What is the project doing specifically? What are the main challenges in dealing with violence against women and children in your community? Is the project dealing with these challenges successfully? 12. Does the Kizazi Kipya project help prevent the spread of HIV, with young people and young women in particular? Can you explain your answer? Sustainability 13. From your perspective, will the contributions of the Kizazi Kipya last after the project closes? For example, will the case management practices introduced through these projects continue? Can you provide some examples of what will continue? What will you not be able to accomplish after these projects end? 55 GI GUIDE FOR CTC STAFF Introduction and Consent Date (dd/mm/yyyy): - Interviewer’s name: - Number of participants: - Designations18 of the participants (Enter each participant’s designation separated by a comma): - Start time (24 h clock, hh:mm): - End time (24 h clock, hh:mm): - To start the interview, please read the following script: Introduction and purpose of the study Hello and thank you for agreeing to speak with us. My name is_______________ (interview name) and this is my colleague__________________. We work with the Data for Development project, a USAID￾funded platform that seeks to improve the quality and use of data in decision-making in Tanzania. We are undertaking an evaluation for USAID to understand what has worked well, what did not work as well, and what has been sustained from the USAID-funded Kizazi Kipya Project implemented by Pact and the Community Health and Social Welfare Systems Strengthening Project, or CHSSP, implemented by John Snow that together aim to improve the well-being of people living with HIV, adolescent girls and young women, and most vulnerable children and their families. Description of study procedures If you agree to be interviewed, you will be asked to share your experiences with and views about the two projects. During our conversation we would like to take notes and record the conversation. The recording and the notes are to make sure that the evaluators have heard and understood what you have shared with us today. Your ideas, together with other data, will be used to produce a report and not otherwise. This report will be presented to USAID for decision-making. Your name will not be included in the report, but the ideas you share might be. Our discussion today will take 45-60 minutes. Confidentiality The recording and the notes are strictly confidential. Only the evaluation team will listen to this recording or read the notes. The recordings and the notes will be kept in a secure location and all electronic information will be coded and secured. The recordings and notes will be destroyed after the project is completed. Your privacy will be protected; we will not include any information in any report that would make it possible to identify you. Please note that we cannot guarantee full confidentiality because of the group setting, as we cannot ensure that participants will not disclose any information shared during the group interview. Once again, we ask that what we discuss today remains here with us. Benefits of participating in this study There is no direct benefit to you being in this study, but we hope that the results of our study will help improve the project implementation and the important services it provides. You will receive no compensation for participating in this interview. Right to refuse or withdraw The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any point in time, and you have the right not to answer any single question, as well as to withdraw from 18 This is the role or job title of each individual participating. Do not document participant names in this table. 56 the discussion at any point. You will not be penalized if you choose to not answer or withdraw from this discussion. Right to ask questions and report concerns You have the right to ask questions about this evaluation and to have those questions answered by me before, during, or after the interview. Do you have questions for me at this time? If you have any further questions about the study, feel free to contact Gerald Usika through [gusika@engl.com] or by phone at [+255 756180413]. This evaluation has been reviewed and approved by National Institute for Medical Research ethical committee. If you have any questions concerning the study, please contact NIMR at +255222121400. Consent Do you agree to participate in this study? Do I have your permission turn on the tape recorder and begin the interview? 1. Yes 2. No Exposure to Kizazi Kipya 1. How familiar are you with the Kizazi Kipya project? Can you explain what you think its objectives are? What services does it provide? 2. How does the Kizazi Kipya project support you and your role in providing services to households and the children effected by HIV or at risk of infection? For example, does the project coordinate with your facility to for testing, enrolling people in treatment, and supporting adherence of clients? 3. How closely do you and your colleagues work with the Kizazi Kipya project? For example, do you meet with Kizazi Kipya staff to jointly plan for linking people to services? Do you discuss and try to help specific households by linking clients from the facility to Kizazi Kipya services? Contribution of Kizazi Kipya and Case Management 4. Has the Kizazi Kipya project improved the services provided to vulnerable households and their children? If it has, can you explain how? a. Access probes: For example, does Kizazi Kipya provide services directly to households that weren’t available before or that no other project provides? Does Kizazi Kipya make it easier for households to get services by linking them to services provided by other organizations? b. Quality probes: Has Kizazi Kipya supported other organizations to improve the quality of the services they provide? 5. One of Kizazi Kipya’s key objectives is to improve case management of vulnerable households. Has the Kizazi Kipya project strengthened case management in your communities? If yes, can you explain how? a. For example, was there a case management system in place before Kizazi Kipya? What is the difference between the earlier system and the system in place now? b. Resource probes: Are there more personnel involved in case management because of Kizazi Kipya support? Are the personnel involved adequately trained and supervised? Are 57 the personnel better equipped with case management tools? Are there more resources available to do case management? c. Quality probes: Are the most vulnerable households being identified and reached with services? Are households being linked to services they need? Are we able to better track the progress of households out of vulnerability because of improvements in case management? 6. A key assumption in case management is that there are enough quality services in the community that vulnerable households can be linked to. Are there sufficient services in your communities to be able to meet the needs of the vulnerable households and children you identify? If not, then what are the gaps? 7. How can case management be improved? Achieving Results 8. Has the Kizazi Kipya project made it easier to find HIV positive children and deliver services to them? Can you explain your answer? Was it more difficult in the past to find the HIV positive children? What is Kizazi Kipya doing now that makes it easier? 9. Has the Kizazi Kipya project improved referrals and the linking of vulnerable households and children to health services other than HIV services? Can you explain your answer? What is the project doing specifically that makes referring and linking to health services better? 10. Has the Kizazi Kipya project improved referrals and the linking of vulnerable households and children from health services to other social welfare services? Can you explain your answer? What is the project doing specifically that makes referring and linking to other services better? 11. Does the Kizazi Kipya project strengthen the economic situation of vulnerable households and women in particular? Can you explain your answer? How does the Kizazi Kipya project go about improving the economic situation of vulnerable households? 12. Does the Kizazi Kipya project help women and children who are victims of violence? Does the project help to prevent violence against women and children? Can you explain your answer? What is the project doing specifically? What are the main challenges in dealing with violence against women and children in your community? Is the project dealing with these challenges successfully? 13. Does the Kizazi Kipya project help prevent the spread of HIV, with young people and young women in particular? Can you explain your answer? Sustainability 14. From your perspective, will the contributions of the Kizazi Kipya last after the project closes? For example, will the case management practices introduced through these projects continue? Can you provide some examples of what will continue? What will you not be able to accomplish after these projects end? 58 GI GUIDE FOR DREAMS COORDINATORS IN LGAS Introduction and Consent Date (dd/mm/yyyy): - Interviewer’s name: - Number of participants: - Designations19 of the participants (Enter each participant’s designation separated by a comma): - Start time (24 h clock, hh:mm): - End time (24 h clock, hh:mm): - To start the interview, please read the following script: Introduction and purpose of the study Hello and thank you for agreeing to speak with us. My name is_______________ (interview name) and this is my colleague__________________. We work with the Data for Development project, a USAID￾funded platform that seeks to improve the quality and use of data in decision-making in Tanzania. We are undertaking an evaluation for USAID to understand what has worked well, what did not work as well, and what has been sustained from the USAID-funded Kizazi Kipya Project implemented by Pact and the Community Health and Social Welfare Systems Strengthening Project, or CHSSP, implemented by John Snow that together aim to improve the well-being of people living with HIV, adolescent girls and young women, and most vulnerable children and their families. Description of study procedures If you agree to be interviewed, you will be asked to share your experiences with and views about the two projects. During our conversation we would like to take notes and record the conversation. The recording and the notes are to make sure that the evaluators have heard and understood what you have shared with us today. Your ideas, together with other data, will be used to produce a report and not otherwise. This report will be presented to USAID for decision-making. Your name will not be included in the report, but the ideas you share might be. Our discussion today will take 45-60 minutes. Confidentiality The recording and the notes are strictly confidential. Only the evaluation team will listen to this recording or read the notes. The recordings and the notes will be kept in a secure location and all electronic information will be coded and secured. The recordings and notes will be destroyed after the project is completed. Your privacy will be protected; we will not include any information in any report that would make it possible to identify you. Please note that we cannot guarantee full confidentiality because of the group setting, as we cannot ensure that participants will not disclose any information shared during the group interview. Once again, we ask that what we discuss today remains here with us. Benefits of participating in this study There is no direct benefit to you being in this study, but we hope that the results of our study will help improve the project implementation and the important services it provides. You will receive no compensation for participating in this interview. Right to refuse or withdraw The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any point in time, and you have the right not to answer any single question, as well as to withdraw from 19 This is the role or job title of each individual participating. Do not document participant names in this table. 59 the discussion at any point. You will not be penalized if you choose to not answer or withdraw from this discussion. Right to ask questions and report concerns You have the right to ask questions about this evaluation and to have those questions answered by me before, during, or after the interview. Do you have questions for me at this time? If you have any further questions about the study, feel free to contact Gerald Usika through [gusika@engl.com] or by phone at [+255 756180413]. This evaluation has been reviewed and approved by National Institute for Medical Research ethical committee. If you have any questions concerning the study, please contact NIMR at +255222121400. Consent Do you agree to participate in this study? Do I have your permission turn on the tape recorder and begin the interview? 1. Yes 2. No Exposure to Kizazi Kipya 1. Please describe your role in providing services to AGYW. 2. How familiar are you with the Kizazi Kipya project? Can you explain what you think its objectives are? What services does it provide? 3. How does the Kizazi Kipya project support you and your role in providing services to AGYW? For example, do you get any technical support such as training from the project? Do you receive any material support such as funds for transport from the project? 4. How closely do you and your colleagues work with the Kizazi Kipya project? For example, do you meet with Kizazi Kipya staff meet to jointly plan for DREAM activities? Contribution of Kizazi Kipya and Case Management 5. Has the Kizazi Kipya project improved the services provided to AGYW? If it has, can you explain how? a. Access probes: For example, does Kizazi Kipya provide services that weren’t available before or that no other project provides? Does Kizazi Kipya make it easier for AGYW to get services by linking them to services provided by other organizations? b. Quality probes: Has Kizazi Kipya supported other organizations to improve the quality of the services they provide? 6. One of Kizazi Kipya’s key objectives is to improve case management of AGYW. Has the Kizazi Kipya project strengthened case management in your communities? If yes, can you explain how? a. For example, was there a case management system in place before Kizazi Kipya? What is the difference between the earlier system and the system in place now? b. Resource probes: Are there more personnel involved in case management because of Kizazi Kipya support? Are the personnel involved adequately trained and supervised? Are the personnel better equipped with case management tools? Are there more resources available to do case management? 60 c. Quality probes: Are the most vulnerable AGYW being identified and reached with services? Are AGYW being linked to services they need? Are we able to better track the layering of services to AGYW? 7. A key assumption in case management is that there are enough quality services in the community that AGYW can be linked to. Are there sufficient services in your communities to be able to meet the needs of the AGYW you enroll in DREAMS? If not, then what are the gaps? 8. How can case management be improved? Achieving Results 9. Has the Kizazi Kipya project made it easier to find HIV positive AGYW and deliver services to them? Can you explain your answer? Was it more difficult in the past to find the HIV positive children? What is Kizazi Kipya doing now that makes it easier? 10. Has the Kizazi Kipya project improved referrals and the linking of AGYW health services? Can you explain your answer? What is the project doing specifically that makes referring and linking to health services better? 11. Does the Kizazi Kipya project strengthen the economic situation of AGYW? Can you explain your answer? How does the Kizazi Kipya project go about improving the economic situation of AGYW? To what extent is it improved? Do AGYW have better prospects as a result of Kizazi Kipya’s assistance? What is working well? 12. Does the Kizazi Kipya project help women and children who are victims of violence? Does the project help to prevent violence against women and children? Can you explain your answer? What is the project doing specifically? What are the main challenges in dealing with violence against women and children in your community? Is the project dealing with these challenges successfully? 13. Does the Kizazi Kipya project help prevent the spread of HIV among AGYW? Can you explain your answer? Sustainability 14. From your perspective, will the contributions of Kizazi Kipya last after the project closes? For example, will the case management practices introduced through these projects continue? Can you provide some examples of what will continue? What will you not be able to accomplish after these projects end? 61 GI GUIDE FOR LGA STRUCTURES AND LOCAL SYSTEM ACTORS Introduction and Consent Date (dd/mm/yyyy): - Interviewer’s name: - Number of participants: - Designations20 of the participants (Enter each participant’s designation separated by a comma): - Start time (24 h clock, hh:mm): - End time (24 h clock, hh:mm): - To start the interview, please read the following script: Introduction and purpose of the study Hello and thank you for agreeing to speak with us. My name is_______________ (interview name) and this is my colleague__________________. We work with the Data for Development project, a USAID￾funded platform that seeks to improve the quality and use of data in decision-making in Tanzania. We are undertaking an evaluation for USAID to understand what has worked well, what did not work as well, and what has been sustained from the USAID-funded Kizazi Kipya Project implemented by Pact and the Community Health and Social Welfare Systems Strengthening Project, or CHSSP, implemented by John Snow that together aim to improve the well-being of people living with HIV, adolescent girls and young women, and most vulnerable children and their families. Description of study procedures If you agree to be interviewed, you will be asked to share your experiences with and views about the two projects. During our conversation we would like to take notes and record the conversation. The recording and the notes are to make sure that the evaluators have heard and understood what you have shared with us today. Your ideas, together with other data, will be used to produce a report and not otherwise. This report will be presented to USAID for decision-making. Your name will not be included in the report, but the ideas you share might be. Our discussion today will take 45-60 minutes. Confidentiality The recording and the notes are strictly confidential. Only the evaluation team will listen to this recording or read the notes. The recordings and the notes will be kept in a secure location and all electronic information will be coded and secured. The recordings and notes will be destroyed after the project is completed. Your privacy will be protected; we will not include any information in any report that would make it possible to identify you. Please note that we cannot guarantee full confidentiality because of the group setting, as we cannot ensure that participants will not disclose any information shared during the group interview. Once again, we ask that what we discuss today remains here with us. Benefits of participating in this study There is no direct benefit to you being in this study, but we hope that the results of our study will help improve the project implementation and the important services it provides. You will receive no compensation for participating in this interview. Right to refuse or withdraw The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any point in time, and you have the right not to answer any single question, as well as to withdraw from 20 This is the role or job title of each individual participating. Do not document participant names in this table. 62 the discussion at any point. You will not be penalized if you choose to not answer or withdraw from this discussion. Right to ask questions and report concerns You have the right to ask questions about this evaluation and to have those questions answered by me before, during, or after the interview. Do you have questions for me at this time? If you have any further questions about the study, feel free to contact Gerald Usika through [gusika@engl.com] or by phone at [+255 756180413]. This evaluation has been reviewed and approved by National Institute for Medical Research ethical committee. If you have any questions concerning the study, please contact NIMR at +255222121400. Consent Do you agree to participate in this study? Do I have your permission turn on the tape recorder and begin the interview? 1. Yes 2. No We will use this protocol for any government cadre, with the exception of clinical staff at facilities and DREAMS coordinators where we encounter them. Exposure to Kizazi Kipya 1. Please describe your role in providing services and support to vulnerable households and the children in their care. 2. How familiar are you with the Kizazi Kipya project? Can you explain what you think its objectives are? What services does it provide? 3. How does the Kizazi Kipya project support you and your role in providing services to vulnerable households and the children they care for? For example, do you get any technical support such as training from the project? Do you receive any material support such as funds for transport from the project? 4. How closely do you and your colleagues in government work with the Kizazi Kipya project? For example, do you meet with Kizazi Kipya staff to jointly plan for providing services? Do you discuss and try to help specific households? Contribution of Kizazi Kipya and Case Management 5. Has the Kizazi Kipya project improved the services provided to vulnerable households and their children? If it has, can you explain how? a. Access probes: For example, does Kizazi Kipya provide services that weren’t available before or that no other project provides? Does Kizazi Kipya make it easier for households to get services by linking them to services provided by other organizations? b. Quality probes: Has Kizazi Kipya supported other organizations to improve the quality of the services they provide? 6. One of Kizazi Kipya’s key objectives is to improve case management of vulnerable households. Has the Kizazi Kipya project strengthened case management in your communities? If yes, can you explain how? a. For example, was there a case management system in place before Kizazi Kipya? What is the difference between the earlier system and the system in place now? 63 b. Resource probes: Are there more personnel involved in case management because of Kizazi Kipya support? Are the personnel involved adequately trained and supervised? Are the personnel better equipped with case management tools? Are there more resources available to do case management? c. Quality probes: Are the most vulnerable households being identified and reached with services? Are households being linked to services they need? Are we able to better track the progress of households out of vulnerability because of improvements in case management? 7. A key assumption in case management is that there are enough quality services in the community that vulnerable households can be linked to. Are there sufficient services in your communities to be able to meet the needs of the vulnerable households and children you identify? If not, then what are the gaps? 8. How can case management be improved? Achieving Results 9. Has the Kizazi Kipya project made it easier to find HIV positive children and deliver services to them? Can you explain your answer? Was it more difficult in the past to find the HIV positive children? What is Kizazi Kipya doing now that makes it easier? 10. Has the Kizazi Kipya project improved referrals and the linking of vulnerable households and children to health services? Can you explain your answer? What is the project doing specifically that makes referring and linking to health services better? 11. Does the Kizazi Kipya project strengthen the economic situation of vulnerable households and women in particular? Can you explain your answer? How does the Kizazi Kipya project go about improving the economic situation of vulnerable households? To what extent is it improved; can households meet their basic needs with Kizazi Kipya’s assistance or do they need to do more? What is working well? 12. Does the Kizazi Kipya project help women and children who are victims of violence? Does the project help to prevent violence against women and children? Can you explain your answer? What is the project doing specifically? What are the main challenges in dealing with violence against women and children in your community? Is the project dealing with these challenges successfully? 13. Does the Kizazi Kipya project help prevent the spread of HIV, with young people and young women in particular? Can you explain your answer? Sustainability 14. From your perspective, will the contributions of the Kizazi Kipya and CHSSP projects last after the project closes? For example, will the case management practices introduced through these projects continue? Can you provide some examples of what will continue? What will you not be able to accomplish after these projects end? 64 FGD PROTOCAL: PARENTS/GUARDIANS Introduction and Consent Date (dd/mm/yyyy): - Interviewer’s name: - Implementing Partner Name: - Number of participants: - Designations21 of the participants (Enter each participant’s designation separated by a comma): - Start time (24 h clock, hh:mm): - End time (24 h clock, hh:mm): - To start the interview, please read the following script: Introduction and purpose of the study Hello and thank you for agreeing to speak with us. My name is_______________ (interview name) and this is my colleague__________________. We work with the Data for Development project, a USAID￾funded platform that seeks to improve the quality and use of data in decision-making in Tanzania. We are undertaking an evaluation for USAID to understand what has worked well, what did not work as well, and what has been sustained from the USAID-funded Kizazi Kipya activity that was implemented from July 05, 2016-July 04, 2021 to improve the well-being of HIV-affected orphans and vulnerable children (OVC) and their families. The project is implemented by Pact. Description of study procedures If you agree to be interviewed, you will be asked to share your experiences with and views about the Kizazi Kipya project. During our conversation we would like to take notes and record the conversation. The recording and the notes are to make sure that the evaluators have heard and understood what you have shared with us today. Your ideas, together with other data, will be used to produce a report and not otherwise. This report will be presented to USAID for decision-making. Your name will not be included in the report, but the ideas you share might be. Our discussion today will take 45-60 minutes. Confidentiality The recording and the notes are strictly confidential. Only the evaluation team will listen to this recording or read the notes. The recordings and the notes will be kept in a secure location and all electronic information will be coded and secured. The recordings and notes will be destroyed after the project is completed. Your privacy will be protected; we will not include any information in any report that would make it possible to identify you. Please note that we cannot guarantee full confidentiality because of the group setting, as we cannot ensure that participants will not disclose any information shared during the group interview. Once again, we ask that what we discuss today remains here with us. Benefits of participating in this study There is no direct benefit to you being in this study, but we hope that the results of our study will help improve the project implementation and the important services it provides. You will receive no compensation for participating in this interview. Right to refuse or withdraw The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any point in time, and you have the right not to answer any single question, as well as to withdraw from 21 This is the role or job title of each individual participating. Do not document participant names in this table. 65 the discussion at any point. You will not be penalized if you choose to not answer or withdraw from this discussion. Right to ask questions and report concerns You have the right to ask questions about this evaluation and to have those questions answered by me before, during, or after the interview. Do you have questions for me at this time? If you have any further questions about the study feel free to contact Gerald Usika through [gusika@engl.com] or by phone at [+255 756180413]. This evaluation has been reviewed and approved by NIMR ethical committee. If you have any questions concerning the study, please contact NIMR at +255222121400. Consent Do you agree to participate in this study? Do I have your permission turn on the tape recorder and begin the interview? 1. Yes 2. No 1. How did you become involved in Kizazi Kipya? 2. Which activities of Kizazi Kipya have you been involved in? a. Referrals and linkages to specific health services. b. Involvement in WORTH groups and other income generating activities. c. Any education-related support. d. Referrals to any other social welfare services, and with violence response in particular. 3. How have these activities and services helped your family? What changes have these activities brought about for you and your household? Let’s talk about each one. a. Are people in the household healthier? Have they changed health behaviors? Go for testing? Take their medication regularly? Use contraception? b. Do they have money to pay for expenses? Have the managed to use money from savings group to generate further income? c. Have they received any education support—school materials, subsidies or bursaries, enrolled in TVET? d. Do they know of people who have been victims of violence that Kizazi Kipya has helped? Does the program educate the community on violence against women and children? 4. Describe how each of these activities work. For example, when you get linked to a health service, how does that happen? How do the savings groups work? 5. Let’s talk a little more about how the Kizazi Kipya program works with you and your household. a. How often do you receive a visit at home from Kizazi Kipya? b. What happens at these visits? c. Does the person from Kizazi Kipya help you get services that are not health services? How does the person help you get the services? 66 d. Does the person from Kizazi Kipya who supports you ever talk about graduation? What is discussed? e. Do you know of anyone who has “graduated”? Are they better off now that they have graduated from the program? 6. Do you have any suggestions about how each of these activities might be improved? And what about the program as a whole, how could it be improved? a. A program like this can’t do everything, but are there very urgent challenges that it could help with that it isn’t helping with right now? 7. Let’s talk again about changes in your homes and in the community that have come about because of Kizazi Kipya. Are there changes that are permanent and will remain, even if Kizazi Kipya is no longer here? What changes will not last without Kizazi Kipya? -----------------------------Thank you----------------------------- 67 GI GUIDE FOR CARE AND TREATMENT IPS Introduction and Consent Date (dd/mm/yyyy): - Interviewer’s name: - Number of participants: - Designations22 of the participants (Enter each participant’s designation separated by a comma): - Start time (24 h clock, hh:mm): - End time (24 h clock, hh:mm): - To start the interview, please read the following script: Introduction and purpose of the study Hello and thank you for agreeing to speak with us. My name is_______________ (interview name) and this is my colleague__________________. We work with the Data for Development project, a USAID￾funded platform that seeks to improve the quality and use of data in decision-making in Tanzania. We are undertaking an evaluation for USAID to understand what has worked well, what did not work as well, and what has been sustained from the USAID-funded Kizazi Kipya Project implemented by Pact and the Community Health and Social Welfare Systems Strengthening Project, or CHSSP, implemented by John Snow that together aim to improve the well-being of people living with HIV, adolescent girls and young women, and most vulnerable children and their families. Description of study procedures If you agree to be interviewed, you will be asked to share your experiences with and views about the two projects. During our conversation we would like to take notes and record the conversation. The recording and the notes are to make sure that the evaluators have heard and understood what you have shared with us today. Your ideas, together with other data, will be used to produce a report and not otherwise. This report will be presented to USAID for decision-making. Your name will not be included in the report, but the ideas you share might be. Our discussion today will take 45-60 minutes. Confidentiality The recording and the notes are strictly confidential. Only the evaluation team will listen to this recording or read the notes. The recordings and the notes will be kept in a secure location and all electronic information will be coded and secured. The recordings and notes will be destroyed after the project is completed. Your privacy will be protected; we will not include any information in any report that would make it possible to identify you. Please note that we cannot guarantee full confidentiality because of the group setting, as we cannot ensure that participants will not disclose any information shared during the group interview. Once again, we ask that what we discuss today remains here with us. Benefits of participating in this study There is no direct benefit to you being in this study, but we hope that the results of our study will help improve the project implementation and the important services it provides. You will receive no compensation for participating in this interview. Right to refuse or withdraw The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any point in time, and you have the right not to answer any single question, as well as to withdraw from 22 This is the role or job title of each individual participating. Do not document participant names in this table. 68 the discussion at any point. You will not be penalized if you choose to not answer or withdraw from this discussion. Right to ask questions and report concerns You have the right to ask questions about this evaluation and to have those questions answered by me before, during, or after the interview. Do you have questions for me at this time? If you have any further questions about the study, feel free to contact Gerald Usika through [gusika@engl.com] or by phone at [+255 756180413]. This evaluation has been reviewed and approved by National Institute for Medical Research ethical committee. If you have any questions concerning the study, please contact NIMR at +255222121400. Consent Do you agree to participate in this study? Do I have your permission turn on the tape recorder and begin the interview? 1. Yes 2. No Exposure to Kizazi Kipya 1. How does the Kizazi Kipya project support you and your role in providing care and treatment services? For example, does the project coordinate with you to achieve testing, treatment, and viral load suppression targets? 2. How closely do you work with the Kizazi Kipya project? For example, do you meet with Kizazi Kipya staff meet to jointly plan for linking people to services? Do you discuss and try to help specific households by linking clients from the facility to Kizazi Kipya services? Contribution of Kizazi Kipya and Case Management 3. Has the Kizazi Kipya project improved the services provided to vulnerable households and their children? If it has, can you explain how? a. Access probes: For example, does Kizazi Kipya provide services directly to households that weren’t available before or that no other project provides? Does Kizazi Kipya make it easier to link clients to services provided by other organizations? b. Quality probes: Has Kizazi Kipya supported other organizations to improve the quality of the services they provide? 4. One of Kizazi Kipya’s key objectives is to improve case management of vulnerable households. Has the Kizazi Kipya project strengthened case management in your communities? If yes, can you explain how? a. For example, was there a case management system in place before Kizazi Kipya? What is the difference between the earlier system and the system in place now? b. Resource probes: Are there more personnel involved in case management because of Kizazi Kipya support? Are the personnel involved adequately trained and supervised? Are the personnel better equipped with case management tools? Are there more resources available to do case management? c. Quality probes: Are the most vulnerable households being identified and reached with services? Are households being linked to services they need? Are we able to better track 69 the progress of households out of vulnerability because of improvements in case management? 5. A key assumption in case management is that there are enough quality services in the community that vulnerable households can be linked to. Are there sufficient services in your communities to be able to meet the needs of the vulnerable households and children you identify? If not, then what are the gaps? 6. How can case management be improved? Achieving Results 7. Has the Kizazi Kipya project made it easier to find HIV positive children and deliver services to them? Can you explain your answer? Was it more difficult in the past to find the HIV positive children? What is Kizazi Kipya doing now that makes it easier? 8. Has the Kizazi Kipya project improved referrals and the linking of vulnerable households and children to health services other than HIV services? Can you explain your answer? What is the project doing specifically that makes referring and linking to health services better? 9. Has the Kizazi Kipya project improved referrals and the linking of vulnerable households and children from health services to other social welfare services? Can you explain your answer? What is the project doing specifically that makes referring and linking to other services better? 10. Does the Kizazi Kipya project strengthen the economic situation of vulnerable households and women in particular? Can you explain your answer? How does the Kizazi Kipya project go about improving the economic situation of vulnerable households? 11. Does the Kizazi Kipya project help women and children who are victims of violence? Does the project help to prevent violence against women and children? Can you explain your answer? What is the project doing specifically? What are the main challenges in dealing with violence against women and children in your community? Is the project dealing with these challenges successfully? 12. Does the Kizazi Kipya project help prevent the spread of HIV, with young people and young women in particular? Can you explain your answer? Sustainability 13. From your perspective, will the contributions of the Kizazi Kipya last after the project closes? For example, will the case management practices introduced through these projects continue? Can you provide some examples of what will continue? What will you not be able to accomplish after these projects end? 70 GI GUIDE FOR DREAMS COORDINATORS IN LGAS Introduction and Consent Date (dd/mm/yyyy): - Interviewer’s name: - Implementing Partner Name: - Number of participants: - Designations23 of the participants (Enter each participant’s designation separated by a comma): - Start time (24 h clock, hh:mm): - End time (24 h clock, hh:mm): - To start the interview, please read the following script: Introduction and purpose of the study Hello and thank you for agreeing to speak with us. My name is_______________ (interview name) and this is my colleague__________________. We work with the Data for Development project, a USAID￾funded platform that seeks to improve the quality and use of data in decision-making in Tanzania. We are undertaking an evaluation for USAID to understand what has worked well, what did not work as well, and what has been sustained from the USAID-funded Kizazi Kipya activity that was implemented from July 05, 2016-July 04, 2021 to improve the well-being of HIV-affected orphans and vulnerable children (OVC) and their families. The project is implemented by Pact. Description of study procedures If you agree to be interviewed, you will be asked to share your experiences with and views about the Kizazi Kipya project. During our conversation we would like to take notes and record the conversation. The recording and the notes are to make sure that the evaluators have heard and understood what you have shared with us today. Your ideas, together with other data, will be used to produce a report and not otherwise. This report will be presented to USAID for decision-making. Your name will not be included in the report, but the ideas you share might be. Our discussion today will take 45-60 minutes. Confidentiality The recording and the notes are strictly confidential. Only the evaluation team will listen to this recording or read the notes. The recordings and the notes will be kept in a secure location and all electronic information will be coded and secured. The recordings and notes will be destroyed after the project is completed. Your privacy will be protected; we will not include any information in any report that would make it possible to identify you. Please note that we cannot guarantee full confidentiality because of the group setting, as we cannot ensure that participants will not disclose any information shared during the group interview. Once again, we ask that what we discuss today remains here with us. Benefits of participating in this study There is no direct benefit to you being in this study, but we hope that the results of our study will help improve the project implementation and the important services it provides. You will receive no compensation for participating in this interview. Right to refuse or withdraw The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any point in time, and you have the right not to answer any single question, as well as to withdraw from 23 This is the role or job title of each individual participating. Do not document participant names in this table. 71 the discussion at any point. You will not be penalized if you choose to not answer or withdraw from this discussion. Right to ask questions and report concerns You have the right to ask questions about this evaluation and to have those questions answered by me before, during, or after the interview. Do you have questions for me at this time? If you have any further questions about the study feel free to contact Gerald Usika through [gusika@engl.com] or by phone at [+255 756180413]. This evaluation has been reviewed and approved by NIMR ethical committee. If you have any questions concerning the study, please contact NIMR at +255222121400. Consent Do you agree to participate in this study? Do I have your permission turn on the tape recorder and begin the interview? 1. Yes 2. No Je, ninaruhusiwa kuwasha kinasa sauti na kuanza mahojiano? 1. Ndiyo 2. Hapana Exposure to Kizazi Kipya 1. Please describe your role in providing services to AGYW. 2. How familiar are you with the Kizazi Kipya project? Can you explain what you think its objectives are? What services does it provide? 3. How does the Kizazi Kipya project support you and your role in providing services to AGYW? For example, do you get any technical support such as training from the project? Do you receive any material support such as funds for transport from the project? 4. How closely do you and your colleagues work with the Kizazi Kipya project? For example, do you meet with Kizazi Kipya staff meet to jointly plan for DREAM activities? Contribution of Kizazi Kipya and Case Management 5. Has the Kizazi Kipya project improved the services provided to AGYW? If it has, can you explain how? a. Access probes: For example, does Kizazi Kipya provide services that weren’t available before or that no other project provides? Does Kizazi Kipya make it easier for AGYW to get services by linking them to services provided by other organizations? b. Quality probes: Has Kizazi Kipya supported other organizations to improve the quality of the services they provide? 6. One of Kizazi Kipya’s key objectives is to improve case management of AGYW. Has the Kizazi Kipya project strengthened case management in your communities? If yes, can you explain how? a. For example, was there a case management system in place before Kizazi Kipya? What is the difference between the earlier system and the system in place now? 72 b. Resource probes: Are there more personnel involved in case management because of Kizazi Kipya support? Are the personnel involved adequately trained and supervised? Are the personnel better equipped with case management tools? Are there more resources available to do case management? c. Quality probes: Are the most vulnerable AGYW being identified and reached with services? Are AGYW being linked to services they need? Are we able to better track the layering of services to AGYW? 7. A key assumption in case management is that there are enough quality services in the community that AGYW can be linked to. Are there sufficient services in your communities to be able to meet the needs of AGYW you enroll in DREAMS? If not, then what are the gaps? 8. How can case management be improved? Achieving Results 9. Has the Kizazi Kipya project made it easier to find HIV positive AGYW and deliver services to them? Can you explain your answer? Was it more difficult in the past to find the HIV positive children? What is Kizazi Kipya doing now that makes it easier? 10. Has the Kizazi Kipya project improved referrals and the linking of AGYW health services? Can you explain your answer? What is the project doing specifically that makes referring and linking to health services better? 11. Does the Kizazi Kipya project strengthen the economic situation of AGYW? Can you explain your answer? How does the Kizazi Kipya project go about improving the economic situation of AGYW? To what extent is it improved; do AGYW have better prospects as a result of Kizazi Kipya’s assistance? What is working well? 12. Does the Kizazi Kipya project help women and children who are victims of violence? Does the project help to prevent violence against women and children? Can you explain your answer? What is the project doing specifically? What are the main challenges in dealing with violence against women and children in your community? Is the project dealing with these challenges successfully? 13. Does the Kizazi Kipya project help prevent the spread of HIV among AGYW? Can you explain your answer? Sustainability 14. From your perspective, will the contributions of Kizazi Kipya last after the project closes? For example, will the case management practices introduced through these projects continue? Can you provide some examples of what will continue? What will you not be able to accomplish after these projects end? 73 ANNEX IV: KII, FGD, AND CCW SURVEY SAMPLING METHODOLOGY 74 KII and FGD Sampling Methodology To sample national stakeholders, six locations were selected. These were Dar es Salaam, Dodoma, Temeke MC in Dar es Salaam, Mbeya CC in Mbeya, Njombe TC in Njombe, Nyamagana MC in Mwanza, and Bukoba DC in Kagera. Sites were selected purposively with the following in mind: • The ET selected similar locations to where Kizazi Kipya activities took place and considered the number of supported CSOs in the region. Discussions with the Mission and JSI were conducted to verify how active JSI interventions were in these sites. • The selected districts provided a balance between urban and rural locations and a balance between districts with CSOs providing case management and those providing both case management and HIV treatment. Both KIIs and FGDs at regional and district levels were conducted in Swahili by Swahili-speaking facilitators at locations accessible to participants. In national-level KIIs, where the KII participants spoke fluent English, the ET conducted the interview in English. All KIIs and FGDs at the regional and district levels were recorded (with the prior, informed consent by participants) and were transcribed and translated by Utafiti, Data for Development’s local research partner. The ET worked closely with JSI, USAID/Tanzania, and (as appropriate) Pact to select specific KII participants from the different stakeholder groups. Selection was made using purposive, or non￾probability, sampling methods. The main goal of the purposive sampling was to focus on particular characteristics of the target population that were of interest, which would best enable the ET to answer the EQs. The purposive sampling method was appropriate in this case because, due to the specialized knowledge they possessed, there was a limited number of people who could serve as primary data sources. The ET sought to select 10-15 FGD participants from districts and wards that provided a balance across the wards where CHSSP was active. This included both focus wards (where all activities and follow-up training were provided) and non-focus wards (where training and quarterly follow-up were provided). CCW Survey Sampling Methodology The CCW survey sampling frame consisted of the list of active CCWs trained in case management by either Kizazi Kipya or CHSSP and who were currently implementing the integrated case management model under Kizazi Kipya and for whom the program(s) had a working telephone/mobile phone number. From this list, the ET selected a random sample of 269 CCWs, plus a randomly selected set of 269 replacements for non-responses across all CHSSP/Kizazi Kipya regions and councils. After three unsuccessful attempts to reach a selected CCW, survey enumerators selected the next name on the randomly selected replacement list and continued this process until 269 surveys were complete. Sample size calculations indicated that to achieve a 90 percent confidence level with a 5 percent margin of error, a target sample size of 269 CCWs was required—given the 24,692 CCWs in the total CCW population. Enumerators captured survey responses on a tablet using a Survey Monkey form, which helped facilitate more accurate data capture and allowed survey results to be uploaded immediately to a web-based data repository. The survey was conducted telephonically by eight enumerators with prior experience in survey data collection. The enumerators were trained in data collection methods, ethics, and in how to use the specific survey instrument. Their training included a practicum supervised by Data for Development staff. Quality control was provided through a field control form used by the subcontractor’s quality control staff. The form captured the status of each call and verified the rationale for selecting replacements. Data for Development staff monitored the data on the back end and recommended cleaning and corrections to the subcontractor’s team. The completed responses in the survey dataset were further cleaned and checked for quality and consistency before the ET analyzed them. 75 The 269 CCWs who responded to the survey came from 22 districts. Of these 148 (55 percent) were females and 121 (45 percent) were males.24 In terms of the highest level of education completed, approximately one-third of the survey respondents had completed primary school and approximately one￾fifth had completed secondary school, some secondary school, or some primary school. The majority of CCWs completed primary school, while another 22 percent completed a secondary school education. Less than 7 percent of CCWs have a diploma or an advanced degree. A total of 57 percent of CCWs said that they had been trained by Pact/Kizazi Kipya (in technical skills areas) while another 45 percent said they had been trained by JSI/ CHSSP (in comprehensive case management). 24 Regions covered by the CCW survey (with the number of respondents in parentheses) include: Arusha (4), Dar es Salaam (50), Dodoma (1), Geita (15), Iringa (12), Kagera (13), Katavi (1), Kigoma (2), Kilimanjaro (11), Mara (4), Mbeya (42), Mjini Magharibi (1), Morogoro (10), Mwanza (18), Njombe (4), Pwani (13), Rukwa (2), Ruvuma (19), Shinyanga (10), Singida (5), Tabora (24), and Tanga (8). 76 ANNEX V: CCW SURVEY QUESTIONNAIRE 77 PERFORMANCE EVALUATION OF THE USAID KIZAZI KIPYA (KK) AND COMMUNITY HEALTH SYSTEM STRENGTHERNING PROGRAM (CHSSP) SURVEY INSTRUMENT FOR COMMUNITY CASE WORKERS (CCW) PRE-INTERVIEW FIELD CONTROL [ENUMERATOR, COMPLETE THIS SECTION BEFORE THE INTERVIEW] ENUMERATOR Enumerator select your name from the list below. [PROG: Program list of enumerators] CCWID [PROG: MIN OF CCW1 MAX OF CCW534] Enter the CCW ID. REGION ENUMERATOR: ENTER the region that the CCW is coming from. [PROG: Program drop-down with regions] COUNCIL ENUMERATOR: ENTER the Council that the CCW is coming from. [PROG: Program drop-down with regions] WARD ENUMERATOR: ENTER the WARD that the CCW is coming from. [PROG: Program drop-down with regions] VILLAGE ENUMERATOR: ENTER the VILLAGE that the CCW is coming from. [PROG: Program drop-down with regions] INTRODUCTION AND CONSENT [ENUMERATOR, TO START THE INTERVIEW, PLEASE READ THE FOLLOWING SCRIPT] Hello and thank you for agreeing to speak with us. My name is_______________ (interview name). We work with the Data for Development project, a USAID-funded project that helps improve the use of data for decision-making in Tanzania. We are evaluating the Kizazi Kipya and Community Health and Social Welfare Systems Strengthening, or CHSSP, projects that were implemented by Pact and JSI. USAID would like to understand what has worked well in these projects, and what did not work as well. In the questions I’m about to ask you we will be mainly focusing on your work as community case worker and the case management training you received under CHSSP. Your answers will not be presented with your name and will be used for reporting. This report will be presented to USAID for decision-making. Our interview today will take not more than 45 minutes. The decision to participate in this study is entirely voluntary and you have the right not to answer any single question, as well as to skip one at any point. If you have any further questions about the survey feel free to contact Gerald Usika through [gusika@engl.com] or by phone at [+255 756180413]. CONSENT. Do you agree to participate in this survey? PROG: COLLECT TIMESTAMP AFTER THIS QUESTION IS ANSWERED. 78 1 YES 0 NO PROG: IF ANSWER IS NO, THEN GO TO THE END; IF ANSWER IS YES, CONTINUE WITH QUESTIONNAIRE. DEMOGRAPHICS [PROG: COLLECT TIMESTAMP] AGE [PROG: DON’T KNOW=998, REFUSE=999] [PROG: MIN=1, MAX=120] How old are you [on your last birthday]? • [Numeric Response] SEX [ENUMERATOR, PLEASE OBSERVE AND RECORD THE GENDER OF THE RESPONDENT – DO NOT ASK] 1. Male 2. Female CCW TITTLE [PROG: REGULAR CCW, LEAD CCW, DON’T KNOW=998, REFUSE=999] What level of formal education have you completed? i. No education ii. Some primary iii. Completed primary iv. Some secondary v. Completed secondary vi. Vocational training vii. University How old are you [on your last birthday]? • [Numeric Response] MAIN QUESTIONAIRE 1. Did you participate in any training related to Case Management? PROG: IF ANSWER IS NO, THEN GO TO THE END; IF ANSWER IS YES, CONTINUE WITH QUESTIONNAIRE. i. YES ii. NO iii. Don’t know 2. What were the qualifications for you to be selected as a community case worker? [ENUMERATOR, PLEASE TICK ALL THAT APPLY – DO NOT ASK THE OPTIONS] i. Knows how to read and write ii. Level of education iii. Previous experience as a community worker iv. Resident of that particular village 79 v. Vetted or recommended by the local village council vi. All of the above vii. Other (please specify) viii. Don’t know 3. Who trained you? [ENUMERATOR, PLEASE TICK ALL THAT APPLY – DO NOT ASK THE OPTIONS] i. Social welfare officers ii. MOHCDGEC/PO-RALG Staff iii. Pact/KK iv. JSI/CHSSP v. Institute of Social Work (ISW) vi. Others, specify vii. Don’t know 4. What topics were you trained on [ENUMERATOR, PLEASE TICK ALL THAT APPLY – DO NOT ASK THE OPTIONS] i. Identification of OVC and their families ii. Referral provision iii. Linking OVC and their families with services iv. Follow-up with OVC and their families v. Child development vi. Effects of neglect and abuse vii. Basic child and family assessment viii. Assessment of HIV risks, services and adherence for children and adolescents (0-18 years) ix. Child protection x. Counseling skills xi. Principles of case management xii. Coordinating structures for KK/CHSSP xiii. Processes/steps of case management xiv. Standard operation procedures guiding the implementation of KK/CHSSP xv. Roles and responsibilities of the government and key stakeholders xvi. The use of case management tools xvii. Don’t know PROG: IF ANSWER IS DON’T KNOW, THEN GO TO QUESTION 6. xviii. Other, specify 5. Are you applying the knowledge and skills acquired from the case management training? i. Yes, frequently ii. Yes, sometimes iii. Yes, rarely iv. No, not at all v. Don’t know PROG: IF ANSWER IS NO, SKIP TO QUESTION 7. IF THE ANSWER IS YES AND DON’T KNOW, CONTINUE WITH QUESTION 6 6. Of the topics you were trained on, which did you use in your work? [ENUMERATOR, PLEASE TICK ALL THAT APPLY – DO NOT ASK THE OPTIONS] i. Identification of OVC and their families ii. Referral provision iii. Linking OVC and their families with services 80 iv. Follow up of the OVC and their families v. Child development vi. Effects of neglect and abuse vii. Basic child and family assessment viii. Assessment of HIV risks, services and adherence for children and adolescents (0-18 years) ix. Child protection x. Counseling skills xi. Principles of case management xii. Coordinating structures for the KK/CHSSP xiii. Processes/Steps of Case Management xiv. Standard Operation Procedures guiding the implementation of KK/CHSSP xv. Roles and Responsibilities of the government and key stakeholders xvi. The use of case management tools xvii. Don’t know xviii. Other, specify xix. None of these were useful [ENUMERATOR, PLEASE ASK THE BELOW QUESTION] ……………………………………………………………………………. 7. If you are not applying some of the things that you were trained on, why not? [ENUMERATOR, PLEASE TICK ALL THAT APPLY – DO NOT ASK THE OPTIONS] i. It wasn’t practical for me ii. Lack of support from management iii. I haven’t found time iv. Lack of confidence v. Lack of funding or resources vi. Lack of transportation vii. Busy with other required activities/Too little time viii. Other reasons, please specify (limited open field) ix. Don’t know 8. What other topics do you need training in? (Open-Mention) 9. Has the case management training you received helped you coordinate referrals and services for your clients from different government departments and other sources (e.g., protection, justice, social welfare, HIV/AIDS, Health and Education)? [ENUMERATOR, PROMPT THE RESPONSES] i. Yes, to a large extent ii. Yes, to some extent iii. Yes, a little bit iv. No, not at all v. The training has made it worse vi. Don’t know Why do you say so? (open-ended question) 10. Is there any support from the Lead CCW, Social Welfare Officer (SWO) or other staff to ensure that the CCW is able to care for, support and protect beneficiaries? i. Yes, a lot of support ii. Yes, some support iii. No, not much iv. Don’t know 81 Please rate the following statements on a scale of 1 to 5, with 1 representing strongly agree and 5 representing strongly disagree. 11. There are enough social welfare service providers to protect, care and support the people living with HIV, adolescent girls and young women, most vulnerable children and their families. [ENUMERATOR, PROMPT THE RESPONSES] i. Strongly agree ii. Agree iii. Neither agree nor disagree iv. Disagree v. Strongly disagree vi. Don’t know 12. I am able to successfully refer beneficiaries to providers so that they receive all the services they need. [ENUMERATOR, PROMPT THE RESPONSES] i. Strongly agree ii. Agree iii. Neither agree nor disagree iv. Disagree v. Strongly disagree vi. Don’t know 13. Once I refer beneficiaries to providers, they are able to receive the referred services. [ENUMERATOR, PROMPT THE RESPONSES] i. Strongly agree ii. Agree iii. Neither agree nor disagree iv. Disagree v. Strongly agree vi. Don’t know Why do you say so? (open-ended question) 14. The referred services provided to beneficiaries do a good job meeting their needs or addressing their problems. [ENUMERATOR, PROMPT THE RESPONSES] vii. Strongly agree viii. Agree ix. Neither agree nor disagree x. Disagree xi. Strongly agree xii. Don’t know Why do you say so? (open-ended question) 15. The case management training I received through the CHSSP project has improved access to social and health services for the groups of beneficiaries I serve. i. Strongly agree ii. Agree iii. Neither agree nor disagree iv. Disagree v. Strongly disagree 82 vi. Don’t know Why do you say so? (open-ended question) 16. Because of the case management training I received through the CHSSP project, we now refer and link clients to better quality services. i. Strongly agree ii. Agree iii. Neither agree nor disagree iv. Disagree v. Strongly disagree vi. Don’t know 17. The case management training I received through the CHSSP project has improved the effectiveness of referral systems in our community. i. Strongly agree ii. Agree iii. Neither agree nor disagree iv. Disagree v. Strongly disagree vi. Don’t know 18. The case management training I received through the CHSSP project has improved HIV prevention among the beneficiaries I serve. i. Strongly agree ii. Agree iii. Neither agree nor disagree iv. Disagree v. Strongly disagree vi. Don’t know Why do you say so? (open-ended question) 19. The case management training I received through the CHSSP project has improved violence against children prevention and response. i. Strongly agree ii. Agree iii. Neither agree nor disagree iv. Disagree v. Strongly disagree vi. Don’t know Why do you say so? (open-ended question) 20. What challenges do you face in doing your job? [ENUMERATOR, PLEASE TICK ALL THAT APPLY – DO NOT ASK THE OPTIONS] i. Lack of transport to do my work ii. Wide coverage, hard to reach all iii. Too many households to visit per CCW, hard to reach all iv. The payment is not enough v. Not getting enough support from the Lead CCW 83 vi. Not getting enough support from the SWO vii. Poor handling of cases by service providers beneficiaries are referred to viii. Beneficiaries not giving required cooperation ix. Other, specify x. Don’t know 21. How likely is it that you will keep working as a community case worker over the long run after CHSSP project support has ended? [ENUMERATOR, PROMPT THE RESPONSES] i. Very likely ii. Somewhat likely iii. Not very likely iv. Not at all likely v. Don’t know PROG: IF ANSWER IS VERY LIKELY OR SOMEWHAT LIKELY, THEN GO TO QUESTION 19. IF ANSWER NOT VERY LIKELY OR NOT AT ALL LIKELY, GO TO QUESTION 20. IF ANSWERS DON’T KNOW, THEN END SURVEY. 22. What motivates you to keep working as a community case worker? [ENUMERATOR, PLEASE TICK ALL THAT APPLY – DO NOT ASK THE OPTIONS] i. Because it is my passion and I’m committed to doing my work ii. Part of my day to day duties iii. I am motivated by the payment I get from the project iv. The number of households I am serving fits my ability v. I am getting enough support from the Lead CCW vi. I am getting enough support from the SWO vii. Referral authorities handles cases very well viii. Beneficiaries provide enough cooperation ix. Other, please specify (Open field) x. Don’t know 23. What challenges do you face in continuing working as a community case worker over the long run after USAID program support has ended? [ENUMERATOR, PLEASE TICK ALL THAT APPLY – DO NOT ASK THE OPTIONS] i. Lack of transport to do my work ii. Wide coverage, hard to reach all iii. Too many households to visit per CCW, hard to reach all iv. The payment is not enough v. Not getting enough support from the Lead CCW vi. Not getting enough support from the SWO vii. Poor handling of cases by the referral authorities viii. Beneficiaries not giving required cooperation ix. Other, specify x. Don’t know 84 ANNEX VI: CCW SURVEY HIGHLIGHTS TABLES 85 CCW Demographics Of the 269 CCWs surveyed a majority (55 percent) were females. Of the CCWs trained the highest number were from Dar es Salaam and Mbeya regions (19 percent and 16 percent respectively). Sampled Regions Female Male Total Percentage (N=269) Arusha 4 0 4 1% Dar es Salaam 37 13 50 19% Dodoma 1 1 0% Geita 4 11 15 6% Iringa 4 8 12 4% Kagera 8 5 13 5% Katavi 1 1 0% Kigoma 1 1 2 1% Kilimanjaro 8 3 11 4% Mara 3 1 4 1% Mbeya 24 18 42 16% Mjini Magharibi 1 1 0% Morogoro 4 6 10 4% Mwanza 12 6 18 7% Njombe 2 2 4 1% Pwani 5 8 13 5% Rukwa 1 1 2 1% Ruvuma 9 10 19 7% Shinyanga 3 7 10 4% Singida 4 1 5 2% Tabora 12 12 24 9% Tanga 2 6 8 3% Grand Total 148 121 269 100% The majority of CCWs completed primary school with 22 percent who completed a secondary school education. Less than 7 percent have a diploma or advanced degree. CCW Level of Education Number of Respondents Percentage University 8 3.0% Certificate 4 1.5% Diploma 6 2.2% Completed secondary 58 21.6% Some secondary 50 18.6% Completed primary 79 29.4% Some primary 59 21.9% Vocational training 5 1.9% Grand Total 269 86 Training Received Fifty-seven (57) percent of CCWs said that they had been trained by Pact/Kizazi Kipya while another 45 percent said they had been trained by JSI/ CHSSP. Who Trained You? Multiple Responses Percentage (N=269) Social welfare officers 3 1.1% MOHCDGEC/PO-RALG Staff 6 2.2% Pact/Kizazi Kipya 154 57.2% JSI/CHSSP 120 44.6% Institute of Social Work (ISW) 1 0.4% All CCWs received training on identification of OVC and their families. Training on assessment of HIV risks, services, and adherence to treatment was cited among a majority of CCWs (64 percent). Roughly one-third received training on 1) the effects of neglect and abuse, 2) linking OVC and their families to services, and 3) child development topics. Over 20 percent surveyed received training on child protection or counseling skills. Topics Multiple Responses Percentage (N=269) Identification of OVC and their families 178 66.2% Difficulties and their families 43 16.0% Referral provision 64 75.2 Linking OVC and their families with services 79 29.4% Child development 78 29.0% Effects of neglect and abuse 86 32.0% Basic child and family assessment 26 9.7% Children’s foundations 9 3.3% Assessment of HIV risks, services, and adherence for children and adolescents (0-18 years) 172 63.9% Child protection 60 22.3% Counseling skills 56 20.8% Principles of case management 27 10.0% Coordinating structures for Kizazi Kipya/CHSSP 4 1.5% Processes/steps of case management 19 7.1% Standard operation procedures guiding the implementation of Kizazi Kipya/CHSSP 5 1.9% Roles and responsibilities of the government and key stakeholders. 2 0.7% The use of case management tools 25 9.3% 87 Training Benefits and Application of Knowledge and Skills Nearly 80 percent of CCWs say they applied skills from training frequently on the job. If Applied Skills Multiple Responses Percentage (N=268) Don’t know 1 0.4% No, not at all 4 1.5% Yes, rarely 38 14.2% Yes, sometimes 11 4.1% Yes, frequently 214 79.9% Kagera and Mwanza had especially high application of KSAs (both over 90 percent). Regions Don’t Know No, Not at All Yes, Rarely Yes, Sometimes Yes, Frequently Grand Total Percentage Yes, Frequently Arusha 4 4 100.0% Dar es Salaam 2 11 2 35 50 70.0% Dodoma 1 1 100.0% Geita 2 13 15 86.7% Iringa 1 2 9 12 75.0% Kagera 1 12 13 92.3% Katavi 1 1 100.0% Kigoma 2 2 100.0% Kilimanjaro 1 2 8 11 72.7% Mara 4 4 100.0% Mbeya 9 33 42 78.6% Mjini Magharibi 1 1 100.0% Morogoro 1 2 7 10 70.0% Mwanza 1 17 18 94.4% Njombe 1 3 4 75.0% Pwani 1 2 10 13 76.9% Rukwa 1 1 2 50.0% Ruvuma 1 2 16 19 84.2% Shinyanga 1 8 9 88.9% Singida 1 4 5 80.0% Tabora 2 2 20 24 83.3% Tanga 1 2 5 8 62.5% Grand Total 1 4 38 11 214 268 79.9% Roughly 84 percent believe to some or a large extent that the training helped them to coordinate referrals. Training Helped Coordinate Referrals Multiple Responses Percentage (N=126) Don’t know 1 0.38% No, not at all 10 3.76% Yes, a little bit 32 12.03% Yes, to some extent 97 36.47% Yes, to a large extent 126 47.37% 88 Over 96 percent agree or strongly agree that they are able to successfully refer beneficiaries to providers. Able to Successfully Refer Beneficiaries to Providers Multiple Responses Percentage (N=267) Don’t know 1 0.37% Strongly disagree 2 0.75% Disagree 4 1.50% Neither agree nor disagree 3 1.12% Agree 130 48.69% Strongly agree 127 47.57% Over 79 percent agree that beneficiaries are able to receive services once they have made a referral. Able to Receive Services Once They Have Made a Referral Multiple Responses Percentage (N=263) Don’t know 4 1.50% Strongly disagree 3 1.12% Disagree 18 6.74% Neither agree nor disagree 26 9.74% Agree 100 37.45% Strongly agree 112 41.95% Over two thirds of CCWs (67 percent) agree that the referred services meet clients’ needs. Referred Services Meet Clients’ Needs Multiple Responses Percentage (N=265) Don’t know 6 2.25% Strongly disagree 6 2.25% Disagree 31 11.61% Neither agree nor disagree 44 16.48% Agree 118 44.19% Strongly agree 60 22.47% The majority of CCWs (nearly 78 percent) agreed that the CCW training had contributed to improved HIV prevention among the beneficiaries served. CCW Training Had Contributed to Improved HIV Prevention Multiple Responses Percentage (N=255) Don’t know 2 0.75% Strongly disagree 3 1.12% Disagree 23 8.61% Neither agree nor disagree 19 7.12% Agree 128 47.94% Strongly agree 80 29.96% 89 Over half (50.8 percent) agree that case management training improved beneficiaries’ access to services and another 37.6 percent strongly agree. Case Management Training Improved Beneficiaries’ Access to Services Multiple Responses Percentage (N=266) Don’t know 3 1.1% Strongly disagree 0 0.0% Disagree 10 3.8% Neither agree nor disagree 18 6.8% Agree 135 50.8% Strongly agree 100 37.6% Just under one-half (49.1 percent) agree that the case management training CCWs received has improved linkages to better quality services and another 46.4 percent strongly agree. Case Management Training Received Has Improved Linkages to Better Quality Services Multiple Responses Percentage (N=267) Don’t know 0 0.0% Strongly disagree 2 0.7% Disagree 1 0.4% Neither agree nor disagree 9 3.4% Agree 131 49.1% Strongly agree 124 46.4% The majority (51.7 percent) agree the CCW training improved effectiveness of referral systems in the community and another 39.7 percent strongly agree. CCW Training Improved Effectiveness of Referral Systems Multiple Responses Percentage (N=267) Don’t know 8 3.0% Strongly disagree 3 1.1% Disagree 2 0.7% Neither agree nor disagree 10 3.7% Agree 138 51.7% Strongly agree 106 39.7% Nearly half (48.9 percent) agree that CCW training led to improved VAC prevention and response; in contrast, another 42.5 percent strongly disagree that it has. Whether the Training Led to Improved VAC Prevention and Response Multiple Responses Percentage (N=260) Don't know 6 2.3% Strongly disagree 113 42.5% Disagree 4 1.5% Neither agree nor disagree 10 3.8% Agree 130 48.9% Strongly agree 3 1.1% 90 Sustainability and Success Factors Key Drivers and Support Over 93 percent believe they have enough support from their supervisors and leads. Receive Enough Support from Their Supervisors and Leads Multiple Responses Percentage (N=267) Don’t know 2 0.75% No, not much 16 5.99% Yes, some support 94 35.21% Yes, a lot of support 155 58.05% Over 80 percent of CCWs agree that there are enough social welfare providers to support OVC and PLHIV. There Are Enough Social Welfare Providers to Support OVC and PLHIV Multiple Responses Percentage (N=266) Don’t know 1 0.37% Strongly disagree 8 3.00% Disagree 23 8.61% Neither agree nor disagree 18 6.74% Agree 136 50.94% Strongly agree 80 29.96% Motivation and Likelihood of Continuing Work The broad majority (86 percent) feel it is very likely they will continue to work as CCWs. Continue to Work as CCWs Multiple Responses Percentage (N=267) Don’t know 1 0.4% Not at all likely 8 3.0% Not Sure 8 3.0% Somewhat likely 21 7.9% Very likely 229 85.8% A majority are motivated by their passion for the work (53 percent) or a sense of duty (40.5 percent) and are not motivated by the financial incentive (only cited among 2 percent as a motivation). Motivation Multiple Responses Percentage (N=269) Because it is my passion and I’m committed to doing my work 142 52.8% Part of my day to day duties 109 40.5% I am motivated by the payment I get from the project 5 1.9% The number of households I am serving fits my ability 16 5.9% I am getting enough support from the Lead CCW 5 1.9% I am getting enough support from the SWO 3 1.1% Referral authorities handles cases very well 2 0.7% Beneficiaries provide enough cooperation 29 10.8% 91 Implementation Challenges Lack of cooperation from beneficiaries (43 percent) and lack of transport (40 percent) were the most commonly cited challenges. Nearly one-quarter cited that payment was not enough. Another 21 percent cited problems with reaching/covering beneficiaries (links to transport). Implementation Challenges Multiple Responses Percentage (N=269) Lack of transport to do my work 107 39.8% Wide coverage, hard to reach all 56 20.8% Too many households to visit per CCW, hard to reach all 23 8.6% The payment is not enough 65 24.2% Not getting enough support from the Lead CCW 9 3.3% Not getting enough support from the SWO 10 3.7% Poor handling of cases by the service providers beneficiaries are referred to 11 4.1% Beneficiaries not giving required cooperation 116 43.1% Don’t know 1 0.4% Sustainability Challenges Lack of resources—including lack of transport—was one of the most reported challenges among 32 percent with 17.5 percent saying that their compensation is not enough. Another common challenge is lack of cooperation from beneficiaries (cited by 11.5 percent). Challenges That Might Keep Me from Continuing to Work as a Caseworker Multiple Responses Percentage (N=269) Lack of transport to do my work 87 32.3% Wide coverage, hard to reach all 22 8.2% Too many households to visit per CCW, hard to reach all 7 2.6% The payment is not enough 47 17.5% Not getting enough support from the Lead CCW 8 3.0% Not getting enough support from the SWO 6 2.2% Poor handling of cases by the referral authorities 12 4.5% Beneficiaries not giving required cooperation 31 11.5% Don’t know 23 8.6% 92 ANNEX VII: KIIS AND FGDS BY STAKEHOLDERS AND LOCATIONS 93 Data Collection Activities National (Dar and Dodoma) and Temeke MC (Dar*) Mbeya CC25 (Mbeya*) Njombe TC (NJombe) Nyamagana MC (Mwanza) Bukoba DC (Kagera) Total National Level KIIs Kizazi Kipya and partners￾Pact, JSI, EGPAF 6 . . . . 6 GoT (PO-RALG and TACAIDS) 3 . . . . 3 USAID technical staff (GI w/AORs and OVC Specialist) 1 . . . . 1 Total National KIIs 10 District and Local KIIs DREAMS Coordinator 1 . . . . 1 Council Health Management Team (CHMT) (DMO, CHAC, DACC, DSWO, CDO) . 1 1 1 1 4 CPT/MVCC 1 1 . . 1 3 Facility CTC in Charges 1 1 1 1 1 5 CCW Supervisors (WSWOs and CDOs) . 1 1 1 1 4 DREAMS Facilitators 1 1 . . . 2 PEPFAR Care and Treatment Implementing Partners (IPs) Sub-grantees (Boresha Afya￾Deloitte-Njombe and HJF in Mbeya) . 1 1 . . 2 Sub-grantees: CARITAS￾Mbeya, TADEPA-Kagera, Community Concerns of Orphans and Development Association (COCODA)- Njombe, Pastoral Activities and Services for People with AIDS (PASADA)-Dar and Mwanza Outreach Care and Support Organization (MOCSO)-Mwanza. 1 1 1 1 1 5 Total District Level KIIs 5 7 5 4 5 26 Total KIIs 36 25 All those <15 estimated not on treatment, plus proportions of 15 to 25-year-olds estimates for both males and females, moderated by THIS prevalence. 94 Data Collection Activities National (Dar and Dodoma) and Temeke MC (Dar*) Mbeya CC25 (Mbeya*) Njombe TC (NJombe) Nyamagana MC (Mwanza) Bukoba DC (Kagera) Total Community Level Focus Groups OVC Parents & Guardians WORTH+ Members of saving and lending groups (combined) 1 1 1 2 1 6 CCWs and Lead CCWs 1 1 1 1 1 5 LVs . . . . 1 1 DREAMS AGYW 1 1 . . . 2 Total FGDs 3 3 2 3 3 14 95 ANNEX VIII: LIST OF KII PARTICIPANTS 96 Interview Date District Region Stakeholder Type Interview Type (FGD, GI, KII) # of Participants # Female # Male Facilitator (Initials) 2/13/2019 Bukoba DC Kagera CCWs FGDs 15 6 9 DS & BT 2/15/2019 Bukoba DC Kagera OVC Guardians/Parents FGDs 14 6 8 DS & BT 2/15/2019 Bukoba DC Kagera Livelihood Volunteers FGDs 10 6 4 DS & BT 2/12/2019 Bukoba DC Kagera DMO, DSWO DAC KIIs 1 1 0 DS & BT 2/14/2019 Bukoba DC Kagera CTC In charge KIIs 2 1 1 DS & BT 2/14/2019 Bukoba DC Kagera CPT/MVCC KIIs 8 3 5 DS & BT 2/13/2019 Bukoba DC Kagera CCW Supervisors (WCDO/WSWO) KIIs 5 3 2 DS & BT 2/13/2019 Bukoba DC Kagera CSO-TADEPA KIIs 8 3 5 DS & BT 2/12/2019 Mbeya CC Mbeya CCW Supervisors (WCDO/WSWO) FGDs 5 5 0 GU, MK, JT 2/14/2019 Mbeya CC Mbeya OVC Guardians/Parents FGDs 10 10 0 GU, MK, JT 2/12/2019 Mbeya CC Mbeya AGYW Dream Girls FGDs 9 9 0 GU, MK, JT 2/14/2019 Mbeya CC Mbeya CCWs FGDs 8 5 3 GU, MK, JT 2/15/2019 Mbeya CC Mbeya CTC In charge KIIs 5 5 0 GU, MK, JT 2/11/2019 Mbeya CC Mbeya CPT/MVCC KIIs 3 2 1 GU, MK, JT 2/13/2019 Mbeya CC Mbeya Huru Facilitators (Dream teachers) KIIs 6 5 1 GU, MK, JT 2/15/2019 Mbeya CC Mbeya DSWO, DMT, CDO (CHMT) KIIs 2 2 0 GU, MK, JT 2/13/2019 Mbeya CC Mbeya CSO-CARITAS KIIs 4 3 1 GU, MK, JT 2/11/2019 Mbeya CC Mbeya C&T, DREAM IP_KIHUMBE KIIs 1 0 1 GU, MK, JT 2/22/2019 National Dar es Salaam Pact KIIs 2 2 0 JL, TB 3/4/2019 National Dar es Salaam USAID KIIs 2 1 1 JL, TB 2/21/2019 National Dar es Salaam Pact-Continuum of care KIIs 1 1 0 JL, TB 2/22/2019 National Dar es Salaam Pact-Economic Strengthening KIIs 2 1 1 JL, BT 2/26/2019 National Dar es Salaam TACAIDS KIIs 1 0 1 GW, JL, NK 2/22/2019 National Dar es Salaam Pact-VAC and GBV KIIs 4 3 1 JL, BT 2/25/2019 National Dar es Salaam CHSSP/Case Management KIIs 2 1 1 JL, BT 2/25/2019 National Dar es Salaam EGPAF-Boresha Afya KIIs 1 0 1 GU, NK 1/16/2019 National Dodoma PO-RALG Social Welfare Department KIIs 1 1 0 GU, DS 3/7/2019 National Dodoma PO-RALG Social Welfare Department KIIs 1 0 1 GU 97 Interview Date District Region Stakeholder Type Interview Type (FGD, GI, KII) # of Participants # Female # Male Facilitator (Initials) 2/21/2019 Njombe TC Njombe OVC Guardians/Parents FGDs 10 9 1 GU, MK, JT 2/18/2019 Njombe TC Njombe CCWs FGDs 10 7 3 GU, MK, JT 2/22/2019 Njombe TC Njombe CTC In charge KIIs 4 3 1 GU, MK, JT 2/20/2019 Njombe TC Njombe CCW Supervisors (WCDO/WSWO) KIIs 3 3 0 GU, MK, JT 2/19/2019 Njombe TC Njombe DSWO, DCDO KIIs 2 1 1 GU, MK, JT 2/21/2019 Njombe TC Njombe CSO-COCODA KIIs 6 3 3 GU, MK, JT 2/21/2019 Njombe TC Njombe SHISO_ BA CT Partner KIIs 2 1 1 GU, MK, JT 2/8/2019 Nyamagana MC Mwanza CCWs FGDs 15 12 3 NK & BT 2/6/2019 Nyamagana MC Mwanza OVC Guardians/Parents FGDs 9 9 0 NK & BT 2/8/2019 Nyamagana MC Mwanza Members of S&L Groups FGDs 11 6 5 NK & BT 2/7/2019 Nyamagana MC Mwanza CTC In charge KIIs 1 1 0 NK & BT 2/5/2019 Nyamagana MC Mwanza CHAC, DCDO, DSWO KIIs 3 2 1 NK & BT 2/5/2019 Nyamagana MC Mwanza CCW Supervisors (WCDO/WSWO) KIIs 8 8 0 NK & BT 2/5/2019 Nyamagana MC Mwanza CSO-MOCSO KIIs 3 1 2 NK & BT 2/15/2019 Temeke MC Dar es Salaam CTC In charge KIIs 5 3 2 AA 2/11/2019 Temeke MC Dar es Salaam CPT/MVCC KIIs 5 4 1 AA 2/13/2019 Temeke MC Dar es Salaam Furaha Facilitators (Dream teachers) KIIs 6 5 1 SM 2/12/2019 Temeke MC Dar es Salaam AGYW Dream Girls FGDs 5 5 0 AA 2/13/2019 Temeke MC Dar es Salaam CSO-PASSADA KIIs 3 0 3 TB 2/11/2019 Temeke MC Dar es Salaam DREAM COORDINATOR KIIs 1 0 1 SM 2/14/2019 Temeke MC Dar es Salaam OVC Guardians/Parents FGDs 9 6 3 AA 2/14/2019 Temeke MC Dar es Salaam CCWs FGDs 12 3 9 AA Total 50 50 50 266 177 89 98 ANNEX IX: NECESSITY OF PRIORITIZING HIV POSITIVE CHILDREN AND ADOLESCENTS 99 According to the HIV Impact Survey 2016-2017 (THIS), the prevalence of HIV among Tanzanians aged 15 to 64 years is 5.0 percent (6.5 percent among females and 3.5 percent among males), which corresponds to an approximate total of 1.4 million people living with HIV (PLHIV) across that age spectrum. However, prevalence is substantially lower in younger age categories, dropping to 1.4 percent among 15 to 24-year￾olds (2.1 percent among females and 0.6 percent among males) and as low as 0.4 percent among 0 to 14- year-olds. A spike marks the transition from lower youth prevalence to higher adult prevalence in the 20 to 24 years age category for females, from 1 percent in the preceding age category to 3.4 percent. The significant disparity in HIV prevalence between males and females at this mark persists through all age categories but is most pronounced among younger adults (ages 20 to 39), where it is more than double that of males in the same age groups. HIV Prevalence by Sex Note: Error bars represent 95 percent confidence interval. Source: THIS 2016-2017. Viral load suppression (VLS) among HIV-positive Tanzanians ages 15 to 64-years is 52 percent (57.5 percent among females and 41.2 percent among males). Suppression is highest among older adults, with 64.4 percent of females ages 55 to 64-years virally suppressed, and 61.5 percent of males ages 55 to 64- years virally suppressed. However, it is substantially lower in younger age categories, with only 11.7 percent of 0 to 14-year-olds and less than 35 percent of 15 to 24-year-olds virally suppressed. VLS is also markedly lower among younger adults, and among younger adult men in particular, with 50.5 percent of females ages 25 to 34 virally suppressed, and only 25.7 percent of men in the same age group virally suppressed. 100 Viral Load Suppression by Sex Note: Error bars represent 95 percent confidence interval. Source: THIS 2016-2017. The most significant risk to achieving the country’s 90-90-9026 goals by 2020 is that only 52.2 percent of PLHIV ages 15 to 64-years (55.9 percent of HIV-positive females and 45.3 percent of HIV positive males) know their HIV positive status. This proportion would be even lower in the younger age groups, as implied by their poor viral load suppression rates. The risk is exacerbated by the incidence rate of 0.4 percent, or four new infections per 1,000 population annually, with 81,000 new infections per year overall. The Joint United Nations Program on HIV/AIDS (UNAIDS) estimates that approximately 180,000 children between 0 and 19 years of age are living with HIV in Tanzania (lower estimate 130,000; upper estimate 230,000), with only 46 percent enrolled in treatment27 (see Table 2). This approximates the HIV diagnosis, treatment and viral suppression estimates in PEPFAR’s country operational plan for Tanzania, 2018, as well as the UNAIDS 2017 estimates. While there are discrepancies between the three sources (HIV Impact Survey data, the UNAIDS 2017 data, and the Country Operational Plan [COP] 2018 estimates), triangulating them provides the parameters for estimating that in Tanzania there are up to 180,000 children living with HIV, of which up to 98,000 are not yet identified or enrolled in treatment and up to 102,00028 children are not yet virally suppressed. 26 All those not suppressed from PEPFAR estimates, plus proportions of 15 to 25-year-olds not suppressed, moderated by THIS prevalence. 27 By the end of FY 2018, 93 percent of the 176,428 beneficiaries reached with ES services were caregivers. 28 Up to Q2 FY17, the program covered 130 councils comprised of 35 at the scale-up saturation stage, 33 at the scale-up aggressive stage, and 62 at the sustained stage. Starting in Q3 FY 2017 and up to the writing of this report, the program covered 79 councils comprised of 76 in the scale-up saturation stage and 3 at the attained stage. 101 HIV Diagnosis, Treatment and Viral Suppression Estimates, COP18 Epidemiological Data HIV Treatment & Viral Suppression - Total Population HIV Prevalence Estimated Total PLHIV PLHIV Diagnosed On ART ART Coverage Viral Suppression Total Pop 54,199,163 4-7 1,494,324 913,422 913,422 61% 84.0% Pop <15 23,667,667 NA 105,789 54,837 54,837 52% 63.4% Men 15-24 5,291,201 0.6 57,634 14,308 14,308 25% 66.1% Men 25+ 9,328,368 NA 491,334 253,832 253,832 52% 85.0% Women 15-24 6,211,713 2.1 92,004 46,641 46,641 51% 76.4% Women 25+ 10,527,601 NA 747,563 543,804 543,804 73% 86.6% Source: Country Operational Plan, 2018 Priorities for Epidemic Control Based on the profile of the epidemic described above, the implicit priorities for epidemic control in OVC PEPFAR programming is to identify the positive children and link them to treatment, and to support their adherence; promote prevention among adolescent girls and young women; and help find positive males, link them to treatment and support their adherence. Kizazi Kipya is designed to respond to these priorities, as reflected in its results framework and related activities. With respect to identification, enrollment and adherence support to HIV positive children, the important activities are innovations in screening, linkages and referrals; specialized support to caregivers of HIV positive children; and behavior change interventions for disclosure and adherence support. 102 ANNEX X: PEPFAR OVC GRADUATION BENCHMARKS 1. DOMAIN – HEALTHY: 1.1. KEY OBJECTIVE - INCREASE DIAGNOSIS OF HIV INFECTION 1.1.1. BENCHMARK: All children, adolescents, and caregivers in the household have known HIV status or a test is not required based on risk assessment 1.2. KEY OBJECTIVE - INCREASE HIV TREATMENT ADHERENCE, RETENTION AND VIRAL SUPPRESSION 1.2.1. (a) BENCHMARK: All HIV positive children, adolescents and caregivers in the household with a viral load result documented in the medical record and/or laboratory information systems (LIS) have been virally suppressed for the last 12 months.5 OR If viral load testing or viral load testing results are unavailable at clinic treating HIV positive beneficiaries, then: 1.2.1. (b) BENCHMARK: All HIV positive children, adolescents, and caregivers in the household have adhered to treatment for 12 months after initiation of antiretroviral therapy.6 1.3. KEY OBJECTIVE - REDUCE RISK OF HIV INFECTION 1.3.1. BENCHMARK: All adolescents 10-17 years of age in the household have key knowledge about preventing HIV infection 1.4. KEY OBJECTIVE - IMPROVE DEVELOPMENT FOR CHILDREN < 5 YEARS – PARTICULARLY HIV EXPOSED AND INFECTED INFANTS/YOUNG CHILDREN 1.4.1. BENCHMARK: No children < 5 years in the household are undernourished 2. DOMAIN – STABLE 2.1. KEY OBJECTIVE - INCREASE CAREGIVER’S ABILITY TO MEET IMPORTANT FAMILY NEEDS 2.1.1. BENCHMARK: Caregivers are able to access money (without selling productive assets) to pay for school fees and medical costs for children 0-17 3. DOMAIN – SAFE 3.1. KEY OBJECTIVE - REDUCE RISK OF PHYSICAL, EMOTIONAL AND PSYCHOLOGICAL INJURY DUE TO EXPOSURE TO VIOLENCE 3.1.1. BENCHMARK: No children, adolescents, and caregivers in the household report experiences of violence (including physical violence, emotional violence, sexual violence, gender￾based violence, and neglect) in the last six months 3.1.2. BENCHMARK: All children and adolescents in the household are under the care of a stable adult caregiver 4. DOMAIN – SCHOOLED 4.1. KEY OBJECTIVE - INCREASE SCHOOL ATTENDANCE AND PROMOTION 4.1.1. BENCHMARK: All school-age children and adolescents in the household regularly attended school and progressed during the last year 103 ANNEX XI: CASE STUDY ON THE SUSTAINABILITY OF WORTH+ GROUPS 104 There are self-sustaining prospects of WORTH+ groups and the economic strengthening for households. Most WORTH groups have registered formally with LGAs, which qualifies them for receiving government support, including space to convene and linking participants to TASAF and health insurance. LGAs also play a role in mobilizing communities to provide emergency assistance to community members in need. WORTH groups enable members to work with the small savings they have and not push them beyond their means. This is enabled by the rules and bylaws of the groups in setting minimum savings amounts. This ensures sustainability in that by design members cannot get overleveraged in debt to the savings group and as a whole improves the self-reliance of the savings group over the long term (Pact). Pact developed criteria to place households in one of four economic well-being categories: provision, protection, production, or promotion. These categories align with TASAF’s criteria, National Guidelines for Economic Strengthening for MVC Households, and PEPFAR’s OVC_MONEY indicator. The majority of Pact served households are at the level of protection with just over 14 percent at the promotion or production level (see table and figure below). HES Cumulative up to Q4, FY 2018 HES Cumulative up to Q4, FY 2018 Quarter Production Promotion Protection Provision Grand Total FY 2018 Q1 2,411 1,196 24,608 8,847 37,062 FY 2018 Q2 12,146 4,059 43,306 16,600 76,111 FY 2018 Q3 6,936 2,573 38,886 16,469 64,864 FY 2018 Q4 3,543 966 18,784 36,728 60,021 Grand Total 25,036 8,794 125,584 78,644 238,058 While there are good prospects for the sustainability of WORTH+ groups, parents, CSOs, and CPTs reported that it is too early for the project to stop supervisory support enabled by paying LVs. While Pact has proposed that LVs could be supported by the savings produced by the groups, according to communities, savings are not enough to pay LV stipends. “Paying CCW or LVs is a challenge because when I save the money, I will have something in mind that after six months I will have certain amount of money!! Now the notion that I was vulnerable in the community and still I take that money for sustaining my life then gives to another person, to me no it is very difficult.” GI, CPT staff “…[W]e can see sustainability especially on savings and loans. Sustainability is vivid because these groups are not dependent on the project alone…If maybe you teach apparent on nutrition don’t think that the skills will perish after the project exit.” FGD, Parents 105 Beneficiaries also compare Kizazi Kipya with previous project, which did not continue. One CPT noted, “I am worried about the services continuing because we started with WAWATA, everything ended, then Pact/Kizazi Kipya, I have some worries about its continuity.” However, there is a strong believe that the Kizazi Kipya practices will be sustained through same WORTH group’s fundraising strategy (2/5 FGDs with parents/members of saving groups, 2/4 KIIs with CPTs, 2/5 KIIs with CSOs). Community members participating in income generation through WORTH+ groups have shown behavior changes and have internalized formalized practices imparted by the project according to CCWs and CHMTs. Kizazi Kipya, through community groups, has provided education that changed behaviors of children and parents/guardians that could cascade to other community members or the future generation. Stakeholders believe that the formulation of WORTH+ groups and linkage to TASAF will continue, supported by services providers (2/2 KIIs with care and treatment partners, 3/5 KIIs with DSWOs/DCDOs, 3/5 KIIs with CSOs, 1/5 FGDs with parents, 2/4 KIIs with MVCCs). “Currently, we have succeeded to form groups of those children’s guardians and few parents, whereby they generate income which helps them to handle their families.” KII, Care & Treatment Partner “Because they (Kizazi Kipya) have been providing education for people to have self-awareness, they (WORTH+ groups) are capable of being successful and hustle for anything as days go and improve their lives through the support they get.” KII, Care & Treatment Partner Efforts are being made by Pact to institutionalize Kizazi Kipya promoted practices at the district and ward levels. Pact is currently doing sustainability readiness assessment and strengthening the capacity of CDOs to support WORTH+ groups. Pact reported that elements for sustainability are based on member self￾selection and this means that community structures should be able to select their own members, run meetings, and pay stipends for the livelihood volunteers, as opposed to current practice where most things are being done by CSOs (2/5 KIIs with Pact staff). ‘’We have the CDOs in their respective wards. They are custodians who can continue institutionalize Kizazi Kipya practices, we are now introducing fee for service approach. Groups will determine when they need the livelihood volunteer support.’’ GI, Pact Staff 106 ANNEX XII: DISCLOSURE OF ANY CONFLICTS OF INTEREST 107 Name Gary Woller Title Director of Monitoring and Evaluation Organization ME&A Evaluation Position? X Team Leader Team Member Evaluation Award Number (contract or other instrument) - USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Community Health and Social Welfare Systems Strengthening Program I have real or potential conflicts of interest to disclose. Yes X No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4.Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5.Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6.Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. - - I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date March 17, 2019 108 Name Bahati P. Tenga Title Evaluation Expert Organization Mendez England & Associates (ME&A) Evaluation Position Team Leader X Team Member Evaluation Award Number (contract or another instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID Kizazi Kipya Activity I have real or potential conflicts of interest to disclose. Yes X No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 7.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 8.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 9.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 10. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 11. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 12. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. No potential conflicts of interest to disclose. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date December 11, 2018 109 Name Daud Siwalaze Title M&E Specialist Organization NORC at the University of Chicago (Data for Development) Evaluation Position? Team Leader X Team Member Evaluation Award Number (contract or other instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID Kizazi Kipya Activity I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. No conflict of interest to disclose. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date June 15, 2018 110 Name Dr. Zena M. Mabeyo Title OVC Expert Organization USAID/Data for Development Evaluation Position Team Leader X Team member Evaluation Award Number (contract or other instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID Kizazi Kipya Activity I have real or potential conflicts of interest to disclose. Yes X No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 13. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 14. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 15. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 16. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 17. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 18. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. No conflict of interest to disclose. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date November 13, 2018 111 Name Gerald Usika Title Survey Specialist Organization Data for Development Project Evaluation Position Team Leader X Team Member Evaluation Award Number(contract or other instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated(Include project name(s), implementer name(s) and award number(s), if applicable) USAID Kizazi Kipya Activity I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 19. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 20. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 21. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 22. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 23. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 24. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. No conflict of interest to disclose. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 15/09/2018 112 Name Jacob Laden Title Evaluation Advisor Organization NORC At University of Chicago, Data for Development Project Evaluation Position Team Leader X Team Member Evaluation Award Number (contract or another instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID Kizazi Kipya Activity I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 25. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 26. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 27. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 28. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 29. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 30. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. No conflict of interest to disclose. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date June 15, 2018 113 Name Julie Tumbo Title Local HIV and OVC Evaluation Expert Organization ME&A Evaluation Position Team Leader X Team Member Evaluation Award Number (contract or another instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID Kizazi Kipya Activity I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 31. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 32. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 33. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 34. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 35. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 36. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. No conflict of interest to disclose. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date November 15, 2018 114 Name Mkingama Adolph Kapinga Title Public Health and Health System Expert Organization ME&A Evaluation Position USAID/Data for Development Evaluation Award Number (contract or another instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID Kizazi Kipya Activity I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 37. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 38. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 39. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 40. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 41. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 42. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. No any conflict of interest to disclose. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date December 21, 2018 115 Name Nasson Konga Title Monitoring & Evaluation Specialist Organization NORC at the University of Chicago (Data for Development) Evaluation Position Team Leader X Team Member Evaluation Award Number (contract or another instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID Kizazi Kipya Activity I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 43. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 44. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 45. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 46. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 47. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 48. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. No conflict of interest to disclose. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date June 15, 2018 116 Name Madihi, Mlwande Charles Dan Title Child Protection/Rights Evaluation Specialist Organization Data for Development Evaluation Position Team Leader X Team Member Evaluation Award Number (contract or another instrument) AID-OAA-1-15-00024/AID-621-TO-17-00005 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) USAID Kizazi Kipya Activity I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 49. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 50. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 51. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 52. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 53. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 54. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. No conflict of interest to disclose. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date December 21, 2018 117 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523