ENDLINE PERFORMANCE EVALUATION USAID/DCOF’s Family Care First Project in Burundi September 2017, Revised February 2018, Final May 2018 This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago. The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Endline Performance Evaluation USAID/DCOF’s Family Care First Project in Burundi FINAL REPORT (September 2017, Revised February 2018, Final May 2018) Prepared under Contract No.: GS-10F-0033M / AID-OAA-M-13-00013. Tasking N003 Submitted to: USAID/DCOF Submitted by: Ritu Nayyar-Stone (Project Director) Mawadda Damon Gartner (Evaluator) Celina Jensen (Subject Expert) Russell Owen (Research Analyst) Carlos Fierros (Research Analyst) Samantha Downey (Research Assistant) Contractor: NORC at the University of Chicago Attention: Renee Hendley Bethesda, MD 20814 Tel: 301-634-9489: Email: Hendley-Renee@norc.org DISCLAIMER This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago. The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ENDLINE PERFORMANCE EVALUATION | i Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project TABLE OF CONTENTS EXECUTIVE SUMMARY................................................................................................. 1 1.0 EVALUATION PURPOSE & EVALUATION QUESTIONS...................................... 14 1.1 Evaluation Purpose............................................................................................ 14 1.2 Evaluation Questions ......................................................................................... 15 2.0 PROJECT BACKGROUND..................................................................................... 15 2.1 Program Context................................................................................................ 15 2.2 Program Objectives ........................................................................................... 17 2.3 Program Design and Management .................................................................... 18 3.0 EVALUATION METHODS & LIMITATIONS............................................................ 23 3.1 Evaluation management .................................................................................... 23 3.2 Study Design...................................................................................................... 24 3.3 Target Population............................................................................................... 25 3.4 Sampling............................................................................................................ 25 3.5 Limitations.......................................................................................................... 27 4.0 FINDINGS, CONCLUSIONS & RECOMMENDATIONS ......................................... 28 Question 1: Have reintegration methods employed by the projects resulted in stable and sustained placements for children? ........................................................ 28 Question 2: Did the program measurably improve the safety, well-being, and development of highly vulnerable children, particularly those who are living without adequate family care? Did the program impact beyond direct services?..... 33 Question 3: Have prevention methods employed by the projects reduced risks of child/family separation? ........................................................................................... 43 Question 4: Did the program bring out systemic changes at the community, provincial, and national levels that are enabling children to live in family care and preventing inappropriate placements in institutional care? ...................................... 68 Question 5: By project end, to what extent have functioning structures been established that can continue to provide on an ongoing basis adequate case￾management services for children at risk?............................................................... 87 Question 6: Did the project offer models and approaches for expansion, adaptation, and/or replication?................................................................................. 98 Summary Conclusions ........................................................................................... 103 Recommendations for Future Programming Across all Evaluation Questions....... 105 ENDLINE PERFORMANCE EVALUATION | ii Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project TABLE OF TABLES Table 4.1.1: Placement of Children Deinstitutionalized by the Project, at Endline, by Gender and Age Group............................................................................ 30 Table 4.1.2: Rates of Overall Deinstitutionalization, Endline ....................................... 31 Table 4.2.1: Services Received by Families with Deinstitutionalized Children ............ 34 Table 4.2.2: Reasons for Re-separation of Deinstitutionalized Children...................... 36 Table 4.2.3: Well-being Indicators for Child Deinstitutionalized by the Project, Number (percent)..................................................................................... 40 Table 4.3.1: Opinions of Caregiver Ability to Provide Food and Shelter (%)................ 45 Table 4.3.2: Average Monthly Household Cash Income and Household Saving......... 46 Table 4.3.3: Household Expenditure on Children ........................................................ 48 Table 4.3.4: Household Assets.................................................................................... 49 Table 4.3.5: Political Impacts on VSLA Operation and Financial Opportunity ............. 50 Table 4.3.6: Overview of Materials for the Children in VSLA and FSG........................ 52 Table 4.3.7: Of those who Go to School, Number of Children and CWD Going to School 4 or 5 Days of the Previous Week................................................ 53 Table 4.3.8: Health Status of VSLA Children, percent................................................. 54 Table 4.3.9: Health Status of FSG Children, percent................................................... 54 Table 4.3.10: Food Security Status of VSLA and FSG Children, percent...................... 55 Table 4.3.11:Psychosocial Indicators of VSLA Children, percent ................................. 56 Table 4.3.12:Psychosocial Indicators of FSG Children, percent................................... 57 Table 4.3.13:VSLA Safety Indicators for Children, percent........................................... 58 Table 4.3.14: FSG Safety Indicators for Children, percent ............................................ 58 Table 4.3.15: Opinions of Ability to Provide Moral and Spiritual Guidance (%) ............. 59 Table 4.3.16: Caregivers’ Engagement in Community Meetings ................................... 60 Table 4.3.17: Opinions of Support Services Provided by Social Assistants (%)............ 62 Table 4.3.18:Access to Social Assistants ..................................................................... 63 Table 4.3.19:Knowledge of Child Protection Committees (%) ...................................... 64 Table 4.4.1: Placement Decision, NORC Focus Group Discussions........................... 75 Table 4.4.2: Caregivers’ Ability to Share with Children in Their Care .......................... 77 Table 4.4.3: Caregivers’ Perception of the Children’s Ability to Share Important Things ...................................................................................................... 77 Table 4.4.4: Reasons for Placing the Children in RCC (%; Of Those Who Would Consider Placing Child in RCC)............................................................... 81 Table 4.6.1: Desired Support During Second and Third VSLA Cycles ...................... 100 ENDLINE PERFORMANCE EVALUATION | iii Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project REGRESSION TABLES Table R1: Child Safety Regression Results, Pre/Post Comparison, VSLA .............. 42 Table R2: Child Safety Regression Results, Pre/Post Comparison, FSG ................ 43 Table R3: Child Well-being Regression Results, Pre/Post Comparison, VSLA........ 66 Table R4: Child Well-being Regression Results, Pre/Post Comparison, FSG ......... 66 Table R5: Caregiver Knowledge Regression Results, Pre/Post Comparison........... 67 Table R6: Household Economic Strengthening Outcomes ...................................... 68 Table R7: Caregiver Knowledge Regression Results, Pre/Post Comparison........... 86 TABLE OF GRAPHS Graph 1: Percentage of Children Going to School.................................................. 53 Graph 2: Children Who Do Not Get 2 Meals Per Day at Least Once a Week, percent..................................................................................................... 56 Graph 3: Self-Assessment of Knowledge of Issues Important for a Child’s Development............................................................................................ 78 Graph 4: Confidence in the Ability to Find Solution for a Child in Need.................. 79 Graph 5: Caregivers’ Perception on Where Children Are Better Off Living............. 80 TABLE OF FIGURES Figure 1: Organizational Chart of the Child Protection System (National Child Protection Policy 2012-2016)................................................................... 20 ANNEXES ANNEX I: CONCEPT NOTE ANNEX II: RESEARCH QUESTIONS AND INDICATORS ANNEX III: EVALUATION METHODS AND LIMITATIONS ANNEX IV: EVALUATION INSTRUMENTS ANNEX V: SOURCES OF INFORMATION ANNEX VI: CONFLICT OF INTEREST FORMS ENDLINE PERFORMANCE EVALUATION | iv Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project ACRONYMS BID Best Interest Determination CDFC Centre de Développement Familial et Communautaire (Family and Community Development Center) CPCPE Comité Provincial de Coordination de la Protection de l’Enfant (Provincial Committee for Coordination of Child Protection) CPE Comité de Protection de l’Enfant (Child Protection Committee) CWD Children with Disabilites DPO Disabled Persons’ Organization FCF Family Care First Project FGD Focus Group Discussion FSG Family Support Group FTR Family Tracing and Reunification HFC Healing Families and Communities HI Handicap International INGO International Non-Governmental Organization IRC International Rescue Committee KII Key Informant Interview MDPHASG Ministère des Droits de la Personne Humaine, des Affaires Sociales et du Genre (Ministry of Human Rights, Social Affairs, and Gender) N/A Not available (or not applicable, depending on what is meant) NGO Non-Governmental Organization PRC Physical Rehabilitation Center RCC Residential Care Center USAID U.S. Agency for International Development VSLA Village Savings and Loan Association ENDLINE PERFORMANCE EVALUATION | v Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project EXECUTIVE SUMMARY USAID’s Displaced Children and Orphans Fund (DCOF) provides financial and technical assistance to improve the well-being of children outside of family care or at risk of family separation, through direct interventions that affect children and strengthen human and institutional capacities at the family, community, and national levels. USAID/DCOF’s overall goal is to measurably improve the safety, well-being, and development of such children. However, DCOF also gives priority to projects that promise impact beyond direct services, which strengthen local capacity, and offer models and approaches for expansion, adaptation, and/or replication. DCOF has therefore funded the evaluation of a project in Burundi that works in partnership with the government and community-based stakeholders to ensure that children under age 18 are in protective and permanent family care. EVALUATION PURPOSE AND EVALUATION QUESTIONS The Burundi project was implemented from June 2014 to June 2017. The overall aim of the project was to work with the government and community-based stakeholders to ensure that children under age 18 are in protective and permanent family care. IRC and its implementing partners Handicap International and Dutabarane worked with all levels of government in 10 provinces with the objective of mainstreaming family-based child protection approaches and contributing to shifts in fundamental skills, social attitudes, and norms regarding child protection and welfare. The purpose of this performance evaluation report is to compare the situation from baseline to endline and report any improvements over time. However, since this is a performance evaluation and not an impact evaluation with a control or comparison group, the report does not attempt to attribute causality between project interventions and results in the country. The evaluation is based on 6 key questions proposed by USAID/DCOF. The first 3 questions deal with child reintegration; improving the safety, well-being and development of highly vulnerable children; and prevention methods that reduce the risk of child/family separation. The next 3 questions examine if the project has brought about systemic changes in the country that enable children to live in family care and prevent inappropriate placements in institutional care; the sustainability of adequate case￾management services for children at risk; and models/approaches for expansion, adaptation and/or replication. PROJECT BACKGROUND IRC has been working in Burundi for nearly 20 years and participated in and provided technical assistance to the Government of Burundi (GoB) in developing both policy frameworks and their associated action plans applying the Guidelines for Alternative ENDLINE PERFORMANCE EVALUATION | 1 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Care (UN-2009). The Family Care First (FCF, Dushigikire Umwana Mu Muryango) project aimed to prevent unnecessary separation of children from their families and to respond to those children who have already been separated from their families. The project worked within the framework of the national policy on deinstitutionalization, introduced in 2012 by MDPHASG after a national survey on children in institutional care which IRC was instrumental in facilitating. The two main objectives of the project were: (1) reducing unnecessary separation; and (2) placing children in nurturing families. The project targeted families (those at risk of separation and those in which deinstitutionalized children were placed) and the national child protection system through a theory of change leading to the outcomes and objectives listed above. For families, the theory of change assumed that project activities (described below in Section 2.3) of Village Savings and Loan Associations (VSLA), training parents/caregivers in basic business skills to enable parents/caregivers to save and start Income Generating Activities (IGAs) and Family Support Groups (FSG) would create the financial capacity for families to provide for all their children in the home; the Healing Families and Communities (HFC) parental training would improve child-caregiver relationships and social capital1; parental training of children with disabilities, case management, and training of technicians in rehabilitation centers would improve the capacity of these caregivers to meet the needs of children with disabilities; teacher training on inclusive education would allow children with disabilities to attend school with children without disabilities; and raising awareness among the communities would improve support for alternative, family-based care. This would all lead to a decrease in support among caregivers for institutional care. For the national child protection system, the theory of change assumed that building the capacity of the Government of Burundi, Child Protection Committees (Comité de Protection de l’Enfant, CPE), Family and Community Development Centers (Centre de Développement Familial et Communautaire, CDFC), social assistants, and Disabled Persons’ Organizations (DPOs) on family tracing and reunification, case management, referral systems, facilitation of HFC, and building information management systems – coupled with technical support in rolling out a national action plan at the provincial level – would create a national child protection system capable of preventing family separation and supporting the deinstitutionalization process. At the start of the project, FCF aimed to deinstitutionalize 1,800 children living in residential care centers (RCCs) – this number was reduced to 350 children in the last year of the project due to the political and resulting economic crises Burundi experienced around the 2015 presidential elections. IRC also initially aimed to 1 IRC conducted impact evaluations to document that this approach was effective. ENDLINE PERFORMANCE EVALUATION | 2 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project strengthen 11,755 vulnerable caregivers (those whose families were at risk of being separated and caregivers of deinstitutionalized children) through increased access to savings and credit and improved parenting practices – this number was revised down to 9,600 in year two, when IRC set targets in the Performance Monitoring Plan (PMP) for USAID. The 9,600 target was based on updated information that each VSLA would include an average of 20 members, not 25 as originally expected. EVALUATION DESIGN, METHODS AND LIMITATIONS The evaluation comprises of primary data collection led by NORC in the form of key informant interviews (KIIs) and focus group discussions (FGDs) with either the same or similar designation individuals at both baseline and endline. Baseline data collection began in November 2014 with interviews with key government officials; quantitative data was collected from March – April, 2015 and endline data was collected in May 2017. Qualitative data collection consisted of a series of 31 focus groups at baseline and 24 focus groups at endline led by our subcontractor, CARD, and key informant interviews conducted by our subject matter experts, who also conducted field visits to the four provinces of Bujumbura Mairie, Gitega, Ngozi, and Muramvya. The quantitative survey targeted VSLA households from the first cohort of implementation and FSG households. The baseline survey sample was a census of all FSG households (250) and a representative sample of VSLA households (179). The endline sample consisted of all households that responded to the baseline survey, minus those IRC was able to confirm were no longer living in the same location. Additionally, for the VSLA group, the Evaluation Team removed baseline respondents who did not end up participating in the VSLA activities. This generated an endline sample of 153 VSLA households and 239 FSG households. Secondary data used in the evaluation includes data collected by IRC – child well-being indicators for children deinstitutionalized from RCCs. The analysis methodology includes a pre-post analysis of quantitative data as well as content analysis of qualitative data, using NVivo software for detailed FGD summaries. We also ran regressions on key project outcomes for evaluation questions 2, 3, and 4. FINDINGS AND CONCLUSIONS Question 1: Have reintegration methods employed by the projects resulted in stable and sustained placements for children? Findings: ■ 378 children were deinstitutionalized, exceeding the revised project target of 350. About half of those in the project-targeted RCCs were deinstitutionalized. ENDLINE PERFORMANCE EVALUATION | 3 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project ■ IRC completed a family tracing process for 476 children. 90% of those deinstitutionalized remained in their original placements. ■ Family poverty remains the main challenge to reintegration. Conflicts related to land inheritance rights also continue to make reintegration challenging. Conclusions: The project was successful in placement of about 90% of deinstitutionalized children. While the majority of placements were successful, the final target number of children to be reintegrated was reduced to a fraction of the original number. The target number was reduced, in consultation with DCOF, due to the length of time that it took to raise awareness amongst the RCC directors and staff of the importance of deinstitutionalization. This process of reintegration took many months, and, ultimately, was a much more arduous process within the social and political context during and after the 2015 election period. In addition, poverty, the fragile political situation in the country, as well as a disconnect between the Ministry of Interior and the Ministry of Solidarity relating to the issue of deinstitutionalization contributed to an extremely challenging context for the project. Question 2: Did the program measurably improve the safety, well-being, and development of highly vulnerable children, particularly those who are living without adequate family care? Did the program impact beyond direct services? Findings: ■ Families with deinstitutionalized children were supported by the project according to their needs. The well-being of deinstitutionalized children was generally strong by endline. ■ Participation in VSLA was linked to a child 0-8 years old being 15% more likely to have safe and secure relationships with their caregivers, and participation in FSG was linked to a child 0-8 years old being 24% more likely. These positive trends continued for FSG children 9-14 years old, though to a lesser degree. ■ There were 36 cases of re-separation of deinstitutionalized families, 20 of which were children under 18 years of age. The most common reasons for those under 18 years of age was migration of families and poverty. For those 18 and over, the most common reasons were to search for work, live independently, and migration of families. ■ Care of children with disabilities has improved and attitudes toward children with disabilities have grown more positive. Caregivers are better able to care for their children and no longer feel ashamed of their children with disabilities. ■ CPE members and social assistants can play an additional, unplanned role, in supporting families of children with disabilities. FCF successfully piloted an approach ENDLINE PERFORMANCE EVALUATION | 4 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project of training CPEs to raise awareness of and refer families to Physical Rehabilitation Centers (PRCs) and trained them in the follow-up of cases. ■ Despite progress in care and follow-up, disability remains another main reason for separation. Conclusions: FCF targeted two of the main drivers of child separation – poverty and negligent parenting. For deinstitutionalized children, the project did largely achieve positive and supportive placements as their well-being was generally strong. The project positively affected attitudes towards children with disabilities and improved the ability of parents to care for their children who had disabilities. This positive result was attributed by IRC firstly to the role of technical rehabilitation workers who provided the parental training sessions and individual follow-up tools; secondly, to the CPE members and social assistants who supported families with children living with disabilities in their communities; and lastly to the DPOs who worked hard to raise awareness among parents of children with disabilities on the importance of home-based care. However, at the same time, it was apparent that many families are not yet fully equipped to care for children with disabilities, particularly because their homes are not wheelchair-friendly and accessible. Question 3: Have prevention methods employed by the projects reduced risks of child/family separation? Findings: ■ Across the board, caregivers felt that income-generating activities led to greater financial stability, though gender issues arose regarding women controlling money. While cash incomes did not change, more households were likely to have savings after FCF and spending on children’s clothing, education, and health care increased. ■ While the current political climate impacted some VSLA and FSG caregivers, the majority reported that the political circumstances have not affected the operation of their savings groups, nor their ability to earn money. ■ The well-being of children improved in FSG households and on some indicators for VSLA households. In general, children in both FSG and VSLA households saw improvement over the project period in material, health, and psychosocial well-being. ■ Caregivers were more positive in their ability to give moral and spiritual guidance to children in their household at the endline and were more socially engaged. ■ Overall, caregivers have a positive opinion of CPEs and the services provided by social assistants. ENDLINE PERFORMANCE EVALUATION | 5 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Conclusions: The project contributed to greatly increasing the number of households that are saving money, however, it did not increase the average amount of money households saved. Savings actually decreased for VSLA households over the project period, possibly because new savers were saving smaller amounts and bringing down the average, or because of the economic and political situation in the country and the ensuing rise in inflation, costs, and food insecurity. Despite positive results in the numbers of households that were saving, it was not clear that households were materially better off. There was no clear overall increase in assets, but a number of focus groups with FSG and VSLA participants reported that their families were more financially secure and they felt less constrained in their ability to provide food and shelter to their children. Increased spending on children’s clothing, education, and medicine/medical care shows that there is a direct link between livelihoods support and the increase of children’s access to services, such as healthcare and education. Poverty still remains the main challenge to vulnerable families, and coupled with other stressors within the family, such as an ill or deceased parent or child with disabilities, or external factors, such as raising inflation and food insecurity, it reduces the ability of the family to cope. Question 4: Did the program bring out systemic changes at the community, provincial, and national levels that are enabling children to live in family care and preventing inappropriate placements in institutional care? Findings: ■ At endline, there was very little knowledge or awareness of key elements of national policies and strategies, especially at the sub-national level. This lack of awareness among many local authorities resulted in challenges in enforcement of those laws. ■ While the Guidelines for the Alternative Care of Children were not finalized prior to the end of the project, they are being discussed at the national level and a component on foster families has been finalized and is awaiting validation by the Government. ■ Family-based care is viewed as the most appropriate care option for children and there is greater acceptance of family-based care over placement in RCCs. However, at endline there was still some support among stakeholders for placing children in RCCs. ■ The absence of a legal framework around foster care and adoption that addresses inheritance is a barrier to kinship and foster care. ENDLINE PERFORMANCE EVALUATION | 6 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Conclusions: The greatest success of the national child protection system to date, and one recognized by all stakeholders interviewed, is that children living in RCCs in the areas of operation have been identified and registered. Meanwhile, the implementation of the Minimum Standards for Children in Institutions has improved conditions in the RCCs. However, there is very little knowledge or awareness of key elements of national policies and strategies beyond the national level, especially at the sub-national level among those working in the child care and social protection system. The high staff turnover rate and low budget available for CPEs makes it challenging to keep everyone informed of key child protection policies and strategies. This means that not all the laws to protect children are enforced. A greater degree of awareness raising is needed on child welfare issues, including appropriate care placements for children, alternative care, and case management, at the national and sub-national levels. The project began improving communication and collaboration between the Ministry of Human Rights, Social Affairs, and Gender (MDPHASG) and the Ministry of the Interior and Patriotic Training to enforce the Standards, although challenges remain and there is a need for a more harmonized and consolidated approach between the two ministries.2 The project started discussions to commence the drafting of the Guidelines for the Alternative Care of Children at the national level, and even though the drafting of the Guidelines has not yet taken place, the project supported and contributed to the development of a component on foster families. It will be important in the future for the Government and other leading actors in child protection to ensure information about the drafting and contents of the national Guidelines for the Alternative Care of Children is not only widely shared and disseminated to the CDFCs and CPEs at the colline and commune levels, but also that staff members are trained and provide with technical support. Further, well-planned campaigns or awareness-raising initiatives need to be carried out to change public attitudes. A recognized success of the project has been in identifying and registering children in the RCCs and in gaining the support of these RCCs to deinstitutionalize the children in their care. The project has also greatly improved the capacity, knowledge, and awareness of CDFC coordinators. Over the project period, there has been greater acceptance of family-based care over placement in RCCs. While caregivers and almost all interviewed practitioners on the ground from the commune and province level believe that children are better off growing up in a family, the sentiment among stakeholders 2 After completion of the draft of this report, IRC specified that since 2016, the two ministries have exchanged correspondence that the Ministry of Interior and Patriotic Training must consult with the Ministry of Human Rights Social Affairs and Gender before agreeing to approve new RCCs. No new RCCs have been approved since. ENDLINE PERFORMANCE EVALUATION | 7 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project interviewed and among some focus group respondents was that there is still a role for RCCs under certain circumstances. The few who do think it appropriate to place a child in an RCC were largely for reasons of poverty. For children with disabilities, caregivers largely believe the child is better off living in a family although there is still some support among those working within the childcare and protection system for placement of children with disabilities in RCCs. It would be important to continue working closely with these groups in future programs in order to ensure long-term success of deinstitutionalization strategies. Question 5: By project end, to what extent have functioning structures been established that can continue to provide on an ongoing basis adequate case￾management services for children at risk? Findings: ■ FCF increased knowledge and awareness of the CDFC Coordinators on the importance of family-based care and reinforced the capacity of the CPEs. ■ The government currently employs too few social assistants. They feel spread thin and that they have more responsibilities than are reasonable given the demands of the community. ■ CPEs at the colline level do not see an improvement in the child protection system and feel unsupported. ■ Additional training is needed for all child protection actors. ■ FCF has supported the creation and management of case files in addition to follow￾up, but the lack of resources makes it highly likely that case management and follow-up will not be sustained after project end. ■ Standardization of the case management system is still needed. Conclusions: While challenges remain, the project reinforced the capacity of actors responsible for the protection of children at all levels. The project has reinforced the child protection structures and linkages in place at the colline and commune levels. The mapping of available services and subsequent referral mechanism at the colline level helped actors make referrals. Additionally, community members are now more aware of the CPE structure and are better able to take more advantage of the support it offers. Links were also made between the various ministries responsible for protecting children, and while there were challenges in harmonizing the approach of deinstitutionalization amongst the Ministry of Interior and the Ministry of Solidarity in particular, progress was made. The project was not able to address the child protection funding problem, which was outside of its scope. The limited budget allocation at the national level to the social ENDLINE PERFORMANCE EVALUATION | 8 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project service sector in general and designated to the CDFCs and CPEs in particular is constraining the ability of the child care and protection system to function well. This is especially evident for CPEs and social assistants who are not able to ensure the needed follow up of cases, and who generally reported feeling unsupported by officials at the national level. There are too few social assistants, with only one or two per commune, and they lack the materials and resources, such as computers, to work with electronic records. They are not able to print paper forms without the external project support, meaning that they will not be able to keep case files for children in the future unless a greater share of the budget is allocated to child welfare activities or another project is able to cover these costs. The project carried out advocacy activities to encourage a budget line specifically for the protection of children, through which ten additional social workers were recruited. However, this is still not sufficient, and additional funding will be necessary. Additional training and awareness raising is still needed. The knowledge is not being transferred beyond those directly trained by the project and, coupled with high staff turnover, has left CPEs and social assistants feeling insufficiently trained to fully address all their responsibilities. Greater communication and scale down from the national to sub-national level between the various actors would help with knowledge transfer. While the project supported the case management system, it is not at a point where it can continue to function independently. The CDFC was still reliant on IRC to provide the tools and forms for case management and the project was facilitating follow-up visits that are unlikely to be sustained at the same level with the continued lack of funding. There is a need to integrate the CPEs into the system so that they can play a bigger role in following up with families with reunified children or families at risk of separation. Additionally, there is no standard approach to case management at the national level, which is needed to ensure all child protection actors are using the same operating procedures and tools. Question 6: Did the project offer models and approaches for expansion, adaptation, and/or replication? Findings: ■ FCF instituted the importance of family-based care. ■ The awareness raising campaign was a success, CDFCs have increased their capacity and knowledge in case management, a much larger percentage of RCCs now meet the minimum standards, and the project brought together various ministries and partners involved in overseeing the protection of children. ENDLINE PERFORMANCE EVALUATION | 9 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project ■ The project was also able to successfully reintegrate a number of children, the VSLA approach was able to provide additional financial support to vulnerable families, and parents have improved their knowledge of positive parenting practices. ■ Practices and strategies that have been adopted include bi-annual evaluation of RCCs, recruiting and training foster families, case management and the BID process, and the process of reintegration. ■ FCF did not have a clear exit strategy in place. ■ Sustainability of project successes is at risk due to lack of funding within the current political context. Conclusions: Practices and strategies that were adopted by local partners and the Government as a result of the project were evaluation of residential care centers on a bi-annual basis, recruiting and training foster families, case management and the best interest determination process, and the process of reintegrating children with their families. However, many of these activities are unlikely to continue without continued funding and resources allocated to them. There was a missed opportunity to better prepare partners in advance prior to the end of the project. The project did not have a strong exit strategy, which was further exacerbated by the changing political context in the country. This did not allow sufficient time to ensure the institutionalization of its initiatives. Lessons learned had not yet been shared at the time of the Evaluation Team’s data collection. The areas in which the project was successful and could have done this are in raising awareness of the importance of family-based care, case management processes and deinstitutionalization, as well as improved knowledge of positive parenting practices. RECOMMENDATIONS FOR FUTURE PROGRAMMING ACROSS ALL EVALUATION QUESTIONS National Level Systems Strengthening ■ Advocate to MDPHASG to increase the budget allocation and financial commitment towards child welfare services. Advocate to relevant Ministries to cover the costs of physiotherapy and health-related costs for children with disabilities above the age of 5. ■ The MDPHASG could share more widely information on policies and strategies developed at the national level with their counterparts at the sub-national level. ENDLINE PERFORMANCE EVALUATION | 10 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project ■ Senior staff at the Ministry of Solidarity could assist the CDFC Coordinators to develop a system for coordinating the activities of the social assistants and facilitating information sharing. ■ There should be a national policy requiring accessible schools, community services, and even basic construction rules of houses in order to facilitate integration of persons living with disabilities. National Guidelines for the Alternative Care of Children ■ The MDPHASG should continue collaboration with UNICEF and NGOs to lead in the development of the National Guidelines for the Alternative Care of Children. ■ Finalize the policy document on foster care. ■ The MDPHASG should widely share and disseminate the finalized Guidelines with officials at the commune and colline levels. Case Management ■ Establish a case management system, including well-defined assessment procedures3, case planning, follow up and review, and case close out procedures; standardizing forms; and recognizing case management as a service that is part of a wider child protection system. ■ Develop specific protocols and mechanisms for information sharing and referrals that are used in practice. ■ Continue to ensure that confidentiality mechanisms are in place. ■ Design detailed training of trainers’ modules, particularly on case management and deinstitutionalization processes for the CDFC. ■ IRC, together with UNICEF and other child protection actors, should review the operational roles social assistants are expected to play and develop a revised approach to case management. Residential Care Centers ■ Continue to phase out or close down operations of existing residential care centers. ■ Develop and enforce repercussions for RCCs that do not meet the minimum standards, such as closing them or putting into place financial penalties or fines. Commune and Colline Levels Advocacy and Awareness Raising Activities 3 IRC should continue to use the assessment guide for field social workers to assess family and household conditions for reunified children and the full cycle of case management tools that were used in this project. ENDLINE PERFORMANCE EVALUATION | 11 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project ■ Engage religious leaders and faith-based organizations to promote family-based care and include them in awareness raising and advocacy activities. ■ Develop educational materials based on the HCF modules to encourage and illustrate positive parenting practices that CDFC staff members can disseminate at the colline and commune levels. ■ Design and implement new outreach techniques for child protection committees, particularly theater in their own communities highlighting the importance of family￾based care, in order to reach more of the population. Systems Strengthening ■ Strengthen community-based systems to monitor and promote child well-being and protection. ■ Conduct a mapping exercise in each province, not only those within the IRC project provinces where this was already conducted, to identify coordinated services and supports within a referral system at the colline and commune levels. ■ Continue to strengthen coordination between community leaders (Chefs de Colline), CPEs, CDFCs, the CPCPE, service providers, local and international NGOs, and the UN. ■ Continue to design and implement outreach techniques for CPEs and the National Forum for Youth so that they are able to raise awareness of the importance of family-based care. Socio-economic Support ■ Encourage the CDFCs to continue to identify areas with a high number of families at risk of separation and then raise awareness of the benefits of VLSAs and support their establishment on a rolling basis. ■ Facilitate discussions between VSLAs and micro finance institutions operating in the provinces4. ■ Validate standard operating procedures. Services for Children with Disabilities ■ Integrate health care packages and financial support for CWD. ■ Reinforce a financial support system to provide adapted helping aids to meet children’s needs. 4 This was something previously organized by IRC during VSLA ‘open days’, but having the CDFCs take the lead on this could improve sustainability of results and capacity building. ENDLINE PERFORMANCE EVALUATION | 12 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project General Programming and Operations ■ IRC staff should identify and document successful approaches and lessons learned and share those with DCOF, UNICEF, and other child protection agencies5. While recommendations have been provided for each level of government, the recommendations as a whole would lead to a more sustainable system. Of priority is a higher government budget allocation to child welfare services since the lack of this funding greatly risks the sustainability of project activities. 5 NORC collected data prior to the project close. IRC reported to us afterwards that they shared lessons learned in both the project closing workshop and in the final narrative report. ENDLINE PERFORMANCE EVALUATION | 13 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project 1.0 EVALUATION PURPOSE & EVALUATION QUESTIONS 1.1 EVALUATION PURPOSE Since its inception in 1989, USAID’s Displaced Children and Orphans Fund (DCOF) has provided financial and technical assistance to improve the well-being of especially vulnerable children (defined as under 18 years of age) through direct interventions that affect children and interventions that strengthen human and institutional capacities at the family, community, and national levels. DCOF’s overall goal is to measurably improve the safety, well-being, and development of highly vulnerable children, particularly those who are living without adequate family care. However, DCOF also gives priority to projects that promise impact beyond direct services, which strengthen local capacity, and offer models and approaches for expansion, adaptation, and/or replication. DCOF attempts to ensure that all funded activities build upon and contribute to the knowledge base of evidence concerning the most appropriate practices for ensuring appropriate care, protection, and development of children. DCOF has therefore funded the evaluation of two projects (in Moldova and Burundi) that seek to ensure that children are in protective and permanent family care by reducing unnecessary separation of children from their families and by placing children who are outside of family care in nurturing families. The project in Moldova (Children in Moldova are Cared for in Safe and Secure Families) is being implemented by Partnership for Every Child (P4EC), and the project in Burundi (Family Care First: A Project to Ensure Children in Burundi are Placed in Protective and Permanent Family Care) is being implemented by the International Rescue Committee (IRC). Evaluations that collected data at baseline and endline were completed for both these projects. This report contains the performance evaluation results for Burundi; a similar report will also be prepared for Moldova. The Burundi project was implemented from June 2014 to June 2017. The overall aim of the project was to work with the government and community-based stakeholders to ensure that children under age 18 are in protective and permanent family care. IRC and its implementing partners Handicap International and Dutabarane worked with all levels of government in 10 provinces6 with the objective of mainstreaming family-based child protection approaches and contributing to shifts in fundamental skills, social attitudes, and norms regarding child protection and welfare. The purpose of this performance evaluation report is to compare the situation from baseline to endline and report any improvements over time. However, since this is a performance evaluation and not an 6 As explained below, IRC’s implementing partners are working in only 4 of the 10 provinces. ENDLINE PERFORMANCE EVALUATION | 14 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project impact evaluation with a control or comparison group, the report does not attempt to attribute causality between project interventions and results in the country. 1.2 EVALUATION QUESTIONS The evaluation is based on six key questions proposed by USAID/DCOF. Additional sub-questions and indictors proposed by NORC for data collection purposes for this evaluation are shown in Annex A: Evaluation Questions and Indicators. 1. Have reintegration methods employed by the projects resulted in stable and sustained placements for children? 2. Did the program measurably improve the safety, well-being, and development of highly vulnerable children, particularly those who are living without adequate family care? Did the program impact beyond direct services? 3. Have prevention methods employed by the projects reduced risks of child/family separation? 4. Did the program bring out systemic changes at the community, provincial, and national levels that are enabling children to live in family care and preventing inappropriate placements in institutional care? 5. By project end, to what extent have functioning structures been established that can continue to provide on an ongoing basis adequate case-management services for children at risk? 6. Did the project offer models and approaches for expansion, adaptation, and/or replication? 2.0 PROJECT BACKGROUND 2.1 PROGRAM CONTEXT7 Situated in Central Africa, Burundi is a landlocked country bordering the Democratic Republic of Congo (DRC), Tanzania, and Rwanda. With a population of approximately 10.48 million (2014), Burundi is one of the poorest countries in the world, reflected by a poverty rate of 67%. More than three-quarters of the population live on less than two US dollars per day. The economy is highly dependent on the agricultural sector, and the legacy of conflict and lack of economic diversity has stunted small business engagement for many Burundians, one of the only real income generating options in rural areas where close to 90% of the population lives. According to the 2016 Human Development Index, Burundi ranked 184 out of 188 countries. 7 Source: IRC’s technical proposal, with some updated data. ENDLINE PERFORMANCE EVALUATION | 15 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Over a decade has passed since the signing of the Arusha Agreement, marking the official end of ethnic conflict in Burundi, yet socio-political instability and conflict persist, fueled today by political exclusion. Social services, including health and education, suffered widespread disruption and, despite efforts made by the Government of Burundi (GoB) and donors to reinforce and strengthen systems, many problems remain. Residential care centers (RCCs) were established throughout the country following the conflict in the mid-1990s and ensuing economic instability. Since the early 2010s the GoB has engaged a broad range of stakeholders and partners to develop a National Child Protection framework. According to a report completed in May 2011 by IRC and the Ministry of Human Rights, Social Affairs, and Gender (Ministère des Droits de la Personne Humaine, des Affaires Sociales, et du Genre, MDPHASG) and funded by UNICEF, there were 98 residential care centers (RCCs) in Burundi at the beginning of 2011 with a total of 5,520 children (47% girls and 53% boys). The majority of RCCs were located in the capital of Bujumbura and urban center of Gitega. Fifteen of them were specifically designated for children with mental or physical disabilities. The RCCs were largely managed by churches (48%), followed by NGOs and local associations (26%), individual persons (18%), international NGOs (5%), and the Government (3%). One of the recommendations to the donor community in the report was to support a “pilot” deinstitutionalization process, reintegrating children from RCCs that received the poorest ratings. IRC has been working in Burundi for almost 20 years and has provided technical assistance to the GoB in developing policy frameworks and national action plans regarding child protection. At the national level, the MDPHASG was responsible for taking the lead on issues related to child protection and rolling-out national policies and strategies to its counterparts at the colline, commune, and provincial levels. While efforts to strengthen sub-national actors had been made in recent years, the child protection system remained weak due to the lack of government resources. Community￾based Child Protection Committees (Comité de Protection de l’Enfant, CPE), created and supported by the government, with the support of IRC and UNICEF, needed continued capacity reinforcement so that members were able to effectively carry out their roles as community volunteers. Government decentralized services, known as Family and Community Development Centers (Centre de Développement Familial et Communautaire, CDFC) experienced challenges, specifically in relation to financial and human resources, that made it difficult to effectively carry out their wider responsibilities, particularly with regard to case management and follow-up of especially vulnerable populations, such as girls and children with disabilities (CWD). In this stage of early national recovery, without robust social protection services or family and community￾ENDLINE PERFORMANCE EVALUATION | 16 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project based support, children were especially vulnerable to abuse, neglect and placement into residential care. In 2015, Burundi experienced a political crisis around the 2015 presidential elections. This then led to an economic crisis. The conflict impacted government capacity and created additional challenges for vulnerable families and children. As reported by IRC, some of the progress that had been made towards recovery was undone and protection risks increased. It also made it challenging for the project to deliver against the original objectives that were set. 2.2 PROGRAM OBJECTIVES The Family Care First (FCF, Dushigikire Umwana Mu Muryango) project, implemented from June 1, 2014 to June 30, 2017 aimed to prevent unnecessary separation of children from their families and to deinstitutionalize children from RRCs. The project worked within the framework of the national policy on deinstitutionalization, introduced in 2012 by the MDPHASG following a national survey on children in institutional care which IRC supported in 2011. The two main objectives of the project were: 1. Reducing unnecessary separation; and 2. Placing children in nurturing families. The related expected project outcomes were: 1. Households with children at risk of family separation are stabilized and strengthened; 2. Community mechanisms and structures reinforce child protection goals; 3. National child protection system enabled to advance policy frameworks; and 4. Nurturing families and alternative care options are identified and prepared for reunification and placement. The project targeted families (those at risk of separation and those in which deinstitutionalized children were placed) and the national child protection system through a theory of change leading to the outcomes and objectives listed above. For families, the theory of change assumed that project activities (described below in Section 2.3) of Village Savings and Loan Associations (VSLA) and Family Support Groups (FSG) would create the financial capacity for families to provide for all their children in the home; the Healing Families and Communities (HFC) parental training would improve child-caregiver relationships and social capital; parental training of children with disabilities, case management, and training of technicians in rehabilitation centers would improve the capacity of these caregivers to meet the needs of children ENDLINE PERFORMANCE EVALUATION | 17 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project with disabilities; teacher training on inclusive education would allow children with disabilities to attend school with children without disabilities; and raising awareness among the communities would improve support for alternative, family-based care. This would all lead to a decrease in support among caregivers for institutional care. For the national child protection system, the theory of change assumed that building the capacity of the Government of Burundi, Child Protection Committees (Comité de Protection de l’Enfant, CPE), Family and Community Development Centers (Centre de Développement Familial et Communautaire, CDFC), social assistants, and Disabled Persons’ Organizations (DPOs) on family tracing and reunification, case management, referral systems, facilitation of HFC, and building information management systems – coupled with technical support in rolling out a national action plan at the provincial level – would create a national child protection system capable of preventing family separation and supporting the deinstitutionalization process. At the start of the project, FCF aimed to deinstitutionalize 1,800 children living in residential care centers (RCCs) – this number was reduced to 350 children in the last year of the project due to the political and resulting economic crises Burundi experienced around the 2015 presidential elections. IRC also initially aimed to strengthen 11,755 vulnerable caregivers (those whose families were at risk of being separated and caregivers of deinstitutionalized children) through increased access to savings and credit and improved parenting practices – this number was revised down to 9,600 in year two, when IRC set targets in the Performance Monitoring Plan (PMP) for USAID. The 9,600 target was based on the information that each VSLA would include an average 20 members. The rest of the project targets remained the same: to build specialized caregiving skills for 540 caregivers of CWD; strengthen 130 older children aged 15 – 18 through entrepreneurship and life skills education; and strengthen the national child care and protection system to enforce child protection policies, building improved mechanisms and strengthening the capacity of human resources at all government and community layers. 2.3 PROGRAM DESIGN AND MANAGEMENT The project was implemented in 10 of Burundi’s 18 provinces (Bujumbura Mairie, Bujumbura Rural8 , Cankuzo, Gitega, Kayanza, Makamba, Ngozi, Kirundo, Muyinga, and Muramvya). The main project activities consisted of the following: 8 During the course of the project, Bujumbura Rural was split into two provinces, and one part was annexed to a new province called Rumonge. Therefore some of IRC’s activities in Bujumbura Rural were in Rumonge at the end of the project, but IRC continued to report on the 10 provinces as they were at the beginning of the project. ENDLINE PERFORMANCE EVALUATION | 18 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project ■ Village Savings and Loan Associations (VSLA) and Family Support Groups (FSG) that aimed to increase economic resources for vulnerable families and families with deinstitutionalized children; ■ Healing Families and Communities (HFC) parenting modules for households participating in the VSLA and FSG groups that aimed to improve parenting skills; ■ Life skills training for households participating in FSG; ■ Parental training sessions in order to teach parents how to stimulate the development of CWD who are cared for at home; ■ Training of teachers on inclusive education that aimed to improve CWD’s access to education; ■ Family tracing and reunification and alternative care placements that aimed to deinstitutionalize children in RCCs using Best Interest Determination (BID); ■ Support to the national child protection system through building the capacity of the Government of Burundi, Child Protection Committees (Comité de Protection de l’Enfant, CPE), Family and Community Development Centers (Centre de Développement Familial et Communautaire, CDFC), social assistants, and Disabled Persons Organizations (DPOs) on family tracing and reunification, case management, referral systems, facilitation of HFC, and building information management systems; ■ Technical training of technicians working in rehabilitation centers in the development and fabrication of positioning, mobilization and other aids to facilitate the independence and self-care of CWD and facilitate the health care of the child; and ■ Vocational and entrepreneurship training to a limited number of reunified youth to allow them to gain economic independence and start their own businesses. Under the FCF project, IRC worked in partnership with the national NGO, Dutabarane, which implemented VSLA activities in four of the provinces; as well as Handicap International, which trained parents of children with disabilities in four Physical Rehabilitation Centers (PRCs) in four provinces, worked to raise awareness of integration of CWD in schools, and collaborated with Disabled Persons’ Organizations (DPOs) to raise awareness of the importance of deinstitutionalization of CWD. Key stakeholders included the Government of Burundi, namely the MDPHASG and UNICEF at the national level, and the CDFC, CPCPE, and CPE structures and Family Tracing and Reunification (FTR) focal points established by the project at the provincial and community levels, in addition to DPOs. Figure 1 presents an organizational chart of the government Child Protection System per the National Child Protection Policy of 2012-2016. Social assistants are staff of the CDFC. ENDLINE PERFORMANCE EVALUATION | 19 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Figure 1: Organizational Chart of the Child Protection System (National Child Protection Policy 2012-2016) ENDLINE PERFORMANCE EVALUATION | 20 Source: IRC Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Acronyms in the Figure: CPPE: Comité de Pilotage de la Protection de l’Enfance (Steering Committee for Child Protection) MDPHASG: Ministère des Droits de la Personne Humaine, des Affaires Sociales et du Genre (Ministry of Human Rights, Social Affairs, and Gender) MFPTSS: Fonction Publique, du Travail et de la Sécurité Sociale (Public Service, Labor and Social Security) CTCNPE: Comité Technique de Coordination Nationale pour la Protection de l’Enfant (National Coordinating Technical Committee for Child Protection) DG SN: Directeur General de la Solidalite Nationale (General Director of National Solidarity in the MDPHASG) DEF: Directeur du department de l’Enfant et la Famille (Director of the Department of Child and Family in MDPHASG) ONG, SC: Organisations Non Gouvernementales et la Société civile (Non Governmental Organizations and Civil Society, including RCCs and churches) CDFC: Centre de Développement Familial et Communautaire (Family and Community Development Center in MDPHASG) CNLS: Conseil National de Lutte contre le Sida (National Council for the Fight Against Aids) DPE: Directeur Provincial de l’Enseignement Primaire et Secondaire (Provincial Director of Primary and Secondary Education) VSLA groups are based on the widely practiced model, in which a self-selected group of 15-25 people meet on a weekly basis. For the first ten weeks they receive support in running a VSLA from a local trained animator, followed by training on Healing Families and Children and lastly business skills training (micro-enterprise fundamentals). Each week, they place an agreed amount into a savings account and a smaller sum into a solidarity fund. Members can also withdraw credit – up to mutually agreed amounts – for income generation, which has to be returned with an interest rate of usually 10%. The solidarity fund also has agreed uses and individuals can borrow this on a direct support basis, for example for funeral costs. All credit and savings agreements are made by the VSLA group members themselves. VSLA was implemented in 3 different cohorts, one per project year. The VSLA cycle was between 8-10 months before sharing out accumulated funds. IRC also piloted a Family Support Group (FSG) which included 250 households too poor to join a VSLA. This involved case management for each member of the FSG in order to assess the specific difficulties for each household and define and tailor the most appropriate support. The support package aimed at stabilizing the most vulnerable families included elements such as provision of assets, social services, training in parenting skills, cash transfers, basic health training and insurance, and other stabilizing and income generating support. In addition to the tailored support, HFC was also offered to the FSG as well as appropriate trainings (such as life and business skills and ENDLINE PERFORMANCE EVALUATION | 21 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project basic literacy/numeracy). On completion of the FSG, except for a small attrition, most households joined the VSLA cohort 3.9 IRC had previously supported the formation of VSLA groups in Burundi with well￾documented positive results. In the case of FCF, the innovation was that they targeted particularly vulnerable families and, over time, sought to involve families who have a child or children who have returned from residential care and to support family reintegration and prevent further separation. The groups not only carried out economic activities, but also received a ten week parenting support course (HFC) adapted from existing IRC modules but with an added component of awareness raising around institutional and family-based care. After the HFC training, groups were encouraged to set up Parenting Support Groups with members from VSLA groups, to provide individual support and/or community mobilization. Four of the ten project provinces had already been supported by UNICEF with USAID/DCOF funding a deinstitutionalization process from 2014-2017. IRC’s work on decentralization was within the activities conducted by a National Technical Working Group on Minimum Standards, which meets monthly. Twelve RCCs were prioritized for deinstitutionalizing children in the first instance, the RCCs who were judged to have the worst standards in a 2011 survey and who agreed to collaborate with the government on measuring standards. FCF worked with local social assistants who are placed at the commune level by the government and the provincial CDFCs. The social assistants were expected to support and work with commune and colline (village) level CPEs, to raise awareness among communities about the negative effects of residential care, to support families at risk of separation, provide follow up support to children who are or who need to be separated from their families, and provide follow up and referrals for families that participated in parental training sessions on how to care for CWD’s (among a range of other non￾institutional care-related child protection functions). IRC had an extensive team of 39 staff members working on the FCF project. They were led by a FCF Coordinator and split into two main teams, the first larger team of 26 staff focused on the Economic and Family Strengthening components and a second team of 11 staff focused on Child Protection components. The two teams were supported by a Monitoring and Evaluation Manager. 9 Of the 248 FSG households in the baseline, IRC was able to account for the following: 190 households joined VSLA cohort 3, 25 did not begin VSLA and were replaced, 16 were no longer members of the FSG, and 3 did not begin VSLA and their replacement status was unclear. NORC therefore had a sample of 239 FSG households for the endline. ENDLINE PERFORMANCE EVALUATION | 22 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project 3.0 EVALUATION METHODS & LIMITATIONS To gather data required for this evaluation, NORC’s Evaluation Team used several techniques which entailed a mix of mutually reinforcing qualitative and quantitative methods that reflect the program design, research questions being addressed, and indicators. NORC combined the results of each technique to capture the diversity of opinions and perceptions of beneficiaries and stakeholders about key child/family care and protection issues at the start of the project. The qualitative analysis, which includes key informant interviews (KII) and focus group discussions (FGD), provides the local context and represents concrete examples that illustrate in greater detail the quantitative findings. Our approach to selecting the appropriate methodology is based on the USAID Evaluation Policy as well as our experience conducting evaluations in the field. NORC Evaluation’s Team conducted the evaluation in a participatory manner which involved engaging USAID/DCOF, implementing partner IRC, project beneficiaries, and other stakeholders. A complete list of documents the Evaluation Team reviewed is included in Annex V, Sources of Information. 3.1 EVALUATION MANAGEMENT The Evaluation Team for Burundi includes Ritu Nayyar-Stone (Project Director), Mawadda Damon Gartner (Evaluator), Siân Long (Subject Expert - Baseline), Celina Jensen (Subject Expert - Endline), Huyen Le (Research Analyst - Baseline), Russell Owen (Research Analyst - Endline), Letitia Onyango (Research Analyst - Endline) Carlos Fierros (Research Analyst - Endline) and Samantha Downey (Research Assistant - Endline). Graduate research assistants from the Harris School of Public Policy at Chicago, Selena Zhong and Hannah Breslau, provided research support during qualitative analysis. Local data collection was undertaken by CARD Engineering (CARD) who provided in-country logistical support, facilitated the FGDs and took notes10, and administered the evaluation survey. To ensure high data quality, NORC’s Senior Survey Director Kareem Kysia provided targeted training to CARD on undertaking the survey on tablets as well as survey administration/quality control at baseline. A follow-up training was led by Carlos Fierros prior to the endline survey. NORC undertook a data quality review of the received data, and did all the analysis. 10 Due to concerns regarding security and highly politicized upcoming Presidential elections, it was advised that interviewees and focus group discussants would not be comfortable having audio recordings of the conversations and this would affect their willing to give feedback. NORC therefore relied on detailed notes of all interviews and focus groups undertaken. ENDLINE PERFORMANCE EVALUATION | 23 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project 3.2 STUDY DESIGN Due to concerns about directly contacting children and staff in the RCCs, the study design required NORC to depend solely on IRC’s support to obtain information on the well-being of institutionalized children. NORC and IRC worked closely together to develop indicators of child well-being11 for the evaluation, and IRC staff worked with social assistants in each province to obtain these child well-being indicators for case management and monitoring and evaluation purposes. The evaluation therefore comprises of primary data collection led by NORC in the form of key informant interviews (KIIs) and focus group discussions (FGDs) with either the same or similar designation individuals at both baseline and endline. Baseline data was collected from March - April 2015 and endline data was collected in May 2017. Primary data was also collected via a panel survey of direct beneficiaries of the project; caregivers in households who participated in the VSLA (cohort1) and FSG project components at baseline prior to the start of project implementation. For the endline the same group of VSLA (cohort 1) was surveyed who had continued with a second VSLA cycle with some supervision from IRC, as well as a third VLSA cycle on their own initiative without any IRC assistance. The same FSG respondents were also surveyed at endline, most of whom had joined VSLA cohort 3, with some minor attrition. Secondary data used in the evaluation includes data collected by IRC – child well-being indicators for children deinstitutionalized from RCCs. The analysis methodology therefore includes a pre-post analysis of quantitative data as well as content analysis of qualitative data, using NVivo software for detailed FGD summaries. To estimate if the trends between baseline and endline are statistically significant, we ran a series of regressions on key project outcomes for evaluation questions 2, 3, and 4. The regressions test whether or not trends between baseline and endline were statistically significant, while controlling for potentially confounding factors. Specifically, the regressions test whether VSLA or FSG families improved between baseline and endline, over and above any changes influenced by the education level of the primary caregiver, the age of the household head, and the household size (measured in the number of children and adults living in the household. NORC drafted and finalized the data collection tools (key informant interview protocols, focus group protocols, and survey questionnaires) for Burundi. These were shared with DCOF and the implementing partner for their feedback prior to finalization. The tablet￾based survey questionnaire was programmed by TNS East Africa Limited since working 11 A participatory process was used to develop child wellbeing indicators grouped by health and nutrition, education, safety, material and psychosocial. Stakeholders included USAID, DCOF, IRC (headquarters and local staff in Burundi) and NORC. ENDLINE PERFORMANCE EVALUATION | 24 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project with tablets for data collection was new for our data collection subcontractor, CARD, and there was no in-house expertise for programing the survey into tablets. The survey was pre-tested prior to beginning enumerator training and a pilot of survey respondents was conducted in the field. NORC developed all training materials and led FGD and survey trainings prior to the baseline and endline data collection periods. 3.3 TARGET POPULATION Qualitative data collection consisted of a series of 31 focus groups at baseline and 24 focus groups at endline led by our subcontractor, CARD, and key informant interviews conducted by our subject matter experts, who also conducted field visits to the four provinces of Bujumbura Mairie, Gitega, Ngozi, and Muramvya. The number of FGDs was decreased from baseline to endline to improve efficiency in cost and analysis. The Evaluation Team felt that two FGDs per stakeholder group was sufficient to capture all the main ideas expressed. Additionally, the Evaluation Team felt it was enough to have 2 FGDs per age group for FGDs with children. All FGD notes were coded into NVivo by NORC for data analysis (see Annex). Data from KIIs was summarized by the subject matter experts. The baseline quantitative survey – conducted in March 2015 – targeted all 250 households participating in FSG and 179 randomly selected VSLA households from the first cohort of the project in the 10 project provinces for IRC (see Annex). The endline survey – conducted in May 2017 – consisted of all households that responded to the baseline survey, minus those IRC was able to confirm were no longer living in the same location. Additionally, for the VSLA group, the Evaluation Team removed baseline respondents who did not end up participating in the VSLA activities. This resulted in a total of 239 FSG households and 153 VSLA households. The endline survey targeted the same household member that responded to the baseline survey, but replacements were allowed in cases where the original respondent was not available in order to keep the household in the sample. It is important to note that this first cohort of VSLA participants received the project training between the dates of May 2015 and January 201612 and endline data was collected in May 2017. 3.4 SAMPLING The Evaluation Team conducted a Survey of Caregivers in the Village Savings and Loan Association (VSLA) Cohort 1 project participants and Family Support Group (FSG) project participants. Respondents were from households that were verified members of VSLA or FSG, the primary caregivers of the children in the households, and above 18 years of age. 12 Not all of the groups in the first cohort started at the same time because of the political conflict. ENDLINE PERFORMANCE EVALUATION | 25 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project At baseline, a census was conducted of the FSG households. Due to a much larger population in VSLA consisting of 3,918 participant households in 158 VSLAs located in 98 villages, the Evaluation Team, IRC, and USAID agreed upon a sample size of 179 cases following a two-stage sampling methods. First, the team selected 30 collines out of the total of 95 collines with equal probabilities of selection. Second, the team selected six participant households from a randomized list of all VSLA participating households within that colline. In each colline, four extra participant households were included in the list as replacements in case of any refusal, absence, or other issues with the initial sample.13 On learning that each household had on average 7-10 children, NORC limited data collection to getting information on the youngest child in the household in each of the following age groups: 0 – 8, 9 – 14, and 15 – 17.14 The endline sample consisted of all households that responded to the baseline survey, minus those IRC was able to confirm were no longer living in the same location. Additionally, for the VSLA group, the Evaluation Team removed baseline respondents who did not end up participating in the VSLA activities. This generated an endline sample of 153 VSLA households and 239 FSG households. It is important to note that the VSLA data is representative of only the first cohort of VSLA participants. The baseline data was collected in Burundi between 03/09/2015 to 03/20/2015 and endline data was collected between 05/08/2017 to 05/27/2017. NORC completed a comprehensive data quality review of the survey data for both rounds and found it to be of high quality. At baseline, 79% of responding caregivers were female for both VSLA and FSG; at endline, the numbers changed to 80% and 76% for VSLA and FSG respondents respectively. All of the male caregivers were the head of the households. The average age of the respondents was early to mid-40s. At baseline, a majority of the caregivers were not educated (51% in VSLA and 76% in FSG), at endline this dropped to 42% for VSLA and 66% for FSG respondents. 15 Only a few had completed secondary school at either time period. For households surveyed, each age group had some percentage of children with disabilities; with more children in the older age groups having disabilities compared to the youngest age group of 0 – 8 across both time periods. 13 NORC Evaluation Team used a weighting method to ensure the representativeness of the VSLA sample, and make inferences about the participant household population 14 This was done in consultation with DCOF/USAID and IRC and to adhere to the timeline for data collection as well as prevent respondent fatigue. 15 This change in education of the respondent may be due to the change in person who responded to the survey between baseline and endline. As explained in the sampling section, the Evaluation Team attempted to have the same person answer the survey at endline as in baseline but if that person was not available, another knowledgeable caregiver answered the survey. ENDLINE PERFORMANCE EVALUATION | 26 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project 3.5 LIMITATIONS The Evaluation Team encountered some limitations inherent to the design of this evaluation and during its fieldwork in Burundi. Some of the more relevant limitations are listed below: ■ Baseline timeframe. The project started in January 2014, but NORC’s concept note for the evaluation was approved by mid-June 2014; the evaluation design was completed by December 2014; and the data collection was undertaken in March 2015. Thus, some project implementation such as training and capacity building of stakeholders, and some family tracing and reunification had already started prior to the baseline, which may have influenced their feedback and response. ■ Administrative and M&E data from the implementer. Due to concerns on the quality of the M&E data from the implementer, NORC was able to report baseline statistics only for three of nine indicators. Thus, only a subset of the deinstitutionalized child well-being indicators were eligible for baseline-to-endline comparisons. Namely, these indicators were the total numbers of deinstitutionalized children, the percentage of children attending school, the percentage sick with diarrhea or fever in the past month, and the percentage of sick children who received medical attention. Part-way through the project, NORC conducted a capacity building exercise to improve the quality of the M&E data, and was therefore able to report on additional child well-being indicators only at endline. ■ Political instability in Burundi. One significant constraint in relation to the KIIs was the political context in the country at the time of data collection. Burundi has a fragile political history and was in pre-election phase during the baseline data collection. Given the tense environment, particular attention was taken to avoiding any questions that might seem to expect a ‘political’ opinion. While the security context in the country during the time of the endline had stabilized, the political context remains fragile. Consequently, some views on advocacy around policy implementation, for example, may have been self-censored. ■ Inability to record KIIs and FGDs. In discussions with the project implementer, IRC, and USAID, it was decided that it would be best not to attempt to record interviews or focus groups as people in Burundi had major security concerns in the lead up to the elections at baseline and the situation remained fragile at endline. The above limitations, however, did not prevent the Evaluation Team from gathering relevant information and data needed to produce findings, and conclusions for this baseline performance evaluation. ENDLINE PERFORMANCE EVALUATION | 27 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project 4.0 FINDINGS, CONCLUSIONS & RECOMMENDATIONS In this section, we summarize the findings and conclusions from the quantitative, qualitative and administrative endline and baseline data collection. For each evaluation question, NORC developed a series of sub-questions to help with the analysis and obtain targeted feedback to inform the overarching evaluation questions. We have organized the findings under each of these sub-questions. For the qualitative findings, we reference baseline focus groups and key informant interviews where relevant, and otherwise all findings are from endline. Question 1: Have reintegration methods employed by the projects resulted in stable and sustained placements for children? Conclusions: The project was successful in finding stable and sustained placements for 90 percent of the children who were deinstitutionalized (342 out of 378). Of the 342 children who remained in their original placements, the majority were either reunified with their biological families or placed in kinship care. It is important to note that of the 36 cases of re-separation, only 20 of the cases were of children while the rest were youth and young adults above the age of 18. IRC staff explained that the individuals who were above 18 years of age were targeted by the project because they were in need of service given that they were raised in institutions and therefore lacked the maturity for effective autonomy as compared to similar aged children who were raised in a family, that a child does not reach a full level of maturity from a brain development perspective until the early 20s, and that reintegration is more challenging for older children. Overall, while the majority of placements were successful, the project was only able to deinstitutionalize a fraction of the project’s original target number of children to be reintegrated. The target number was reduced due to the length of time that it took to raise awareness amongst the RCC directors and staff of the importance of deinstitutionalization. This process of reintegration took many months, and, ultimately, was a much more arduous process then was originally assumed. In addition, the fragile political situation in the country, as well as a disconnect between the Ministry of Interior and the Ministry of Solidarity relating to the issue of deinstitutionalization, contributed to greater challenges and setbacks for the project. More specifically, the change in Government was accompanied by a perception amongst some officials, as well as the First Lady, that the institutionalization of children living without parental care was a positive response. It also made work at the community level more challenging, because it required repeating awareness building activities around the negative effects of ENDLINE PERFORMANCE EVALUATION | 28 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project institutionalizing children, and further delayed the reintegration process. The lack of a formal policy document on alternative care resulted in the project staff having to bring together government officials from the two ministries at the national level to find some common ground. Finally, a main challenge to reintegration has been the issue of poverty – many families did not have the means to accept their children back home without livelihoods support, and this was intensified by the 2015 crisis and the challenging economic climate that followed. In regard to children with disabilities, there were not as many children with disabilities living in RCCs in need of deinstitutionalization as initially understood by the project – there were 984 children in all 10 RCCs in the target group.16 As a result, the project supported children with disabilities already living with their families by supporting their parents in providing positioning and mobilization helping aids in order to facilitate home care; training their parents in how to stimulate their child’s development and prevent (the augmentation of) physical complications related to their pathologies; and raising awareness among their parents on the importance of their contribution to their child’s welfare with the aim of preventing separation. A continual challenge was that schools were found to lack the preparation and ability to accept and include children with disabilities. The evaluation findings coupled with the current socioeconomic situation in the country make it clear that any further efforts in deinstitutionalization and the prevention of unnecessary separation must be combined with the provision of livelihoods support to families whether in the form of cash transfer, income-generating activities, or VSLAs. Additionally, a policy document on alternative care must be developed so that there is more common ground between the ministries on the issue of alternative care. Findings: a) Are an increased number of children living in residential care facilities being placed in family care? b) During the tracing process, what percentage of children in residential care have been identified a birth family or kinship care option that is safe and appropriate? By the end of the project, IRC exceeded the revised project target of 350 deinstitutionalized children, with the majority of children placed in their families of origin. Once the CDFC and IRC received authorization from the RCC manager to work in their center, IRC entered the center and registered all the children and entered them into their deinstitutionalization database. As of IRC’s July 2017 monitoring and evaluation 16 One RCC had 55 children with disabilities ENDLINE PERFORMANCE EVALUATION | 29 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project data at project end, 476 children had gone through a family tracing process. Not all children registered within an RCC went through a tracing process; IRC prioritized those who accepted the tracing process and for whom they had enough information to start the tracing process, or children under two years of age. Of those 476 children, IRC was able to find families for 470 of them, of which 384 families were willing to be reunified with their child in coordination with the RCC manager. In total 378 (79% of those who had gone through the tracing process) were successfully reunified. Of these 378 deinstitutionalized children, 61% were placed in their families of origin, 31% were placed in kinship care, 7% in foster care, and 1% shifted to living independently (see Table 4.1.1). Only six of the cases that completed family tracing and were willing to be reunified were not able to be reunified before the end of the project because the children would not have been able to continue their specific studies in the geographic area where their family was living due to the nonexistence of their field of study of choice in that area. While the number of deinstitutionalized children exceeds the final project target of 350, IRC had negotiated with USAID a new target from that originally established. The total target number decreased from 1,800 at the start of the project to 500 during the Evaluation Team’s midterm trip and then further to 350 by project end. Endline interviews with stakeholders revealed there were several reasons why the target number was reduced. Firstly, the fragile political situation in 2015 contributed to a delay in implementing the project activities. In addition, IRC initially underestimated the amount of time it would take to raise awareness among RCC directors and staff of the importance of deinstitutionalization and to gain their consent and collaboration in the process. Many RCC directors and staff were reportedly initially unsupportive of the project activities due to fear that their centers would be closed down. In many instances, it took months before the RCC staff agreed to start deinstitutionalization activities, which caused a delay and resulted in the total target needing to be reduced. Table 4.1.1: Placement of Children Deinstitutionalized by the Project, at Endline, by Gender and Age Group Deinstitutionalized Children Placed Into: Boys Girls Total Family of Origin 128 (60%) 102 (61%) 230 (61%) Age 0 – 5 30 21 51 Age 6 – 11 40 24 64 Age 12 – 17 32 42 74 Age 18+ 26 15 41 Kinship Care 70 (33%) 48 (29%) 118 (31%) Age 0 – 5 5 7 12 Age 6 – 11 13 8 21 Age 12 – 17 29 17 46 Age 18+ 23 16 39 ENDLINE PERFORMANCE EVALUATION | 30 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Deinstitutionalized Children Placed Into: Boys Girls Total Alternative Family Based Foster Care 10 (5%) 16 (10%) 26 (7%) Age 0 – 5 1 4 5 Age 6 – 11 1 3 4 Age 12 – 17 2 7 9 Age 18+ 6 2 8 Independent Living 4 (2%) 0 (0%) 4 (1%) Age 18+ 4 0 0 Total 212 (100%) 166 (100%) 378 (100%) Source: IRC M&E data; PMR, July 2017 (Endline) Note: Of the 378 deinstitutionalized at endline, 38 children had disabilities Roughly half of all boys and girls registered during the project period within the project￾targeted residential care centers had been deinstitutionalized and 90% remained in their original placements. Table 4.1.2 shows that both boys and girls were deinstitutionalized at roughly equivalent rates over the project period. Though many more boys had been registered during the project period in RCCs than girls (441 boys compared to 318 girls), for both genders roughly 50% of those admitted were deinstitutionalized and they generally remained in their original placement families. Of those deinstitutionalized, 90% remained in their placements at project end (the reasons for re-separation are explained in Question 2). Table 4.1.2: Rates of Overall Deinstitutionalization, Endline Boys Girls Not reunified 229 (52%) 152 (48%) Deinstitutionalized, in original placement 190 (43%) 152 (48%) Deinstitutionalized, not in original placement 22 (5%) 14 (4%) Total 441 (100%) 318 (100%) Source: IRC’s M&E data, July 2017 (Endline) Reintegrated children were perceived by stakeholders as facing difficulties in adapting to a family environment after leaving the RCC. Focus groups discussed the challenges that they perceived reintegrated children face. Many focus group respondents mentioned that children often had issues with adjusting to a household’s way of eating. A number of focus group respondents also said that reintegrated children struggled because they did not know how to do household work. Additionally, some endline focus group respondents from the child protection sector felt that families were not well enough prepared to receive a child. One focus group respondent specified that host families and neighbors could use more training and that families could use more financial support. Land inheritance rights continue to make reintegration challenging. Problems related to land rights, which can cause community conflict when a child returns to the community ENDLINE PERFORMANCE EVALUATION | 31 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project and attempts to reclaim inherited land, were mentioned at both baseline and endline by several focus groups. c) What factors prevented placement of children in residential care into permanent family care? Reasons for placement into RCCs included: dissolution of the family or of the parents’ marriage, the children being disabled and in need of extra care, and bad parenting practices. With regards to marital and family dissolution, children were said to end up in RCCs if they had been orphaned, abandoned, had divorced parents, had poor families, or were homeless. Being physically vulnerable was also a reason given for children residing in RCCs, as some focus group respondents said that children with disabilities lived there because they needed access to school education or treatment17 such as physiotherapy. Bad parenting is another reason given for why children went to RCCs. Fathers who did not care for the education of their children sent them away. Parents who did not want their children and mistreated them may have intentionally or inadvertently pushed them toward RCCs. A few focus groups blamed the people in their towns/neighbors for not wanting to support the growth of the children and keep them in the community, saying that there was no longer a spirit of solidarity among people. Family poverty is still a strong disincentive for children to leave the RCC. Factors preventing children currently living in residential care centers from being placed in permanent family care arrangements were related to poverty and the seemingly desperate situation of many families. This issue was identified at baseline and continued at endline. At endline, several focus groups mentioned families may not have the financial stability to care for the child. A stakeholder at the Ministry stated: “The first factor is household poverty because parents are reluctant to take their children out of the orphanages because in the household they are dying of hunger.” (Stakeholder at Government Ministry) Another government stakeholder reaffirmed this statement: “The country is in crisis, and there are a lot of orphans. That is one reason. The socioeconomic situation has also deteriorated, as well as the degradation of social values and positive values at the community level. For instance, in the past, the child would be supported by others, but now the value is on individualism.”(Stakeholder at Government Ministry) All of the RCC staff interviewed reiterated this point by mentioning that they have all had cases of children returning to RCCs because the living conditions at home were difficult. 17 Not a treatment that required being permanently admitted to an RCC. ENDLINE PERFORMANCE EVALUATION | 32 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project An IRC staff member gave examples of students who could not continue their studies if they were to return home, including a child studying computing whose parent’s home lacked electricity and another whose educational field of choice was not available in their home location. d) What type of social service follow-up is provided to deinstitutionalized children, by whom and for how long? Social service follow up is described in detail in the findings for evaluation Question 5. Question 2: Did the program measurably improve the safety, well-being, and development of highly vulnerable children, particularly those who are living without adequate family care? Did the program impact beyond direct services? Conclusions: The FCF targeted two of the main drivers of child separation – poverty (through the VSLA and FSG activities) and bad parenting (through the HFC module). For deinstitutionalized children, the project did largely achieve positive and supportive placements as their well-being was found generally to be strong. Therefore, the project did succeed in its aim of supporting households to care for children sustainably. In addition, case management processes improved with the CDFC keeping comprehensive case files for each of the children. However, the project was unable to put in place care and monitoring systems. According to data from IRC, the total number of children in RCCs has decreased due to the support of the project. However, during focus group discussions there was a general perception, amongst both caregivers and those who work in the child care and protection system (CPE and CDFC members, social assistants, and RCC staff), that the numbers of children in RCCs had increased. The project positively affected attitudes as well as it improved the ability of parents to care for their children who had disabilities. The target beneficiaries were better able to care for and respond to their children with disabilities and were reported as not trying to hide their children from other community members. This positive result was attributed by IRC firstly to the role of technical rehabilitation workers who provided the parental training sessions and individual follow up tools; secondly, to the CPE members and social assistants who supported families with children living with disabilities in their communities; and lastly, to the DPOs who worked hard to raise awareness among parents of children with disabilities on the importance of home-based care. However, at the same time, it was apparent that many families are not yet fully equipped to care for children with disabilities, particularly because they do not have wheelchair-friendly and accessible homes, in addition to lacking the material means for their care. Many families ENDLINE PERFORMANCE EVALUATION | 33 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project live in rural locations well away from any paved roads or surfaces and in very hilly terrain, highlighting the challenges of accessibility. Findings: a) Did the project provide a core package of services to help ensure that residential care is prevented when possible and that reunified and deinstitutionalized children and at-risk children remain in family care? Families with deinstitutionalized children were supported by the project according to their needs. Table 4.2.1 below details the percentage of families of deinstitutionalized children that received various services. Every family received a reunification kit and either food aid or psychosocial assistance through the project. Each child then received additional support according to the reintegration plan developed for the child based on an evaluation of their needs. A large number of families also received funding for income-generating activities (41%) and a large number of children were referred to other available services (47%). Table 4.2.1: Services Received by Families with Deinstitutionalized Children Number Received Total Percent of Total The family received food aid through the project18 378 378 100% The child received professional training (outside school) through the project 5 378 1% The family benefited from funding for income-generating activities 155 378 41% The family benefited from housing aid from the project 65 378 17% Housing repairs through project funding 30 378 8% Child received psychosocial assistance through the project 378 378 100% The family received family counseling through project assistance 66 379 17% The child was referred to other available services 179 379 47% Source: IRC M&E Data, July 2017 Death of parents, poverty, marital issues, and parental neglect are the main reasons children are separated from their biological families. Marital and family conflict were given as very common causes for the separation of children from families. Many focus group respondents reported divorces, illegal marriages (including concubinage and polygamy), and parental remarriage as reasons for children to lose family status. The death of one or both parents was also a very prominent reason for the splintering of families. Other reasons given for child separation were unmarried or young mothers 18 This support was designed for children that were not in school, however, the project found that the majority of children were already in school, or not yet old enough to attend school. ENDLINE PERFORMANCE EVALUATION | 34 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project abandoning their children, and the migration to and from Tanzania that leaves the children behind. Poverty was another prominent reason separating children from their families. It was widely felt by a large number of focus group respondents that poverty makes it very difficult for families to care for their children, and some children leave home to try and find a place where their quality of life is better. Bad parenting was also a prominent reason for children to be separated from their families. Child mistreatment and abuse; parental irresponsibility and lack of desire to care for their children; unhappiness of children; lack of a trusting relationship in the household; and mental illness in a parent were said by focus groups to galvanize some children to leave home. Lack of educational opportunities was another reason given by some focus group respondents for children to leave home. Dropping out of high school was said to be common and families were said to lack awareness of good educational practices for children. The lack of financial resources for school attendance was also an issue amongst families and parents were not able to educate their children on their own. Employment opportunities for children were also said to serve as a pull factor away from home, as there was easy access to the town and job searching could begin at an early age, according to multiple focus groups. Sick children were said by several focus groups to be at high risk of separation, especially when no one is able to properly take care of them. It was also said in a large number of focus groups that misbehaving children who engage in bad behaviors, such as vagrancy, theft, and impoliteness, are also sent away from their families. It was also viewed that albino children ought to be placed in residential care, as they are targeted by criminals. The re-separation of deinstitutionalized children was mainly due to the migration of families. According to IRC’s records, in total there were 36 cases of re-separation. Of these cases, only 19 were children under the age of 18. The reasons for re-separation are detailed in Table 4.2.2 below. The most common reason for re-separation was due to families who migrated to Malawi, Tanzania, and Rwanda and reportedly left their children behind. Another common reason for the re-separation of children under 18 was poverty – three children returned to live in the centers because their families reportedly could not provide for their basic needs. Other common reasons for young adult re￾separation included searching for work and living independently. One child and two young adults left to an unknown location because of reported mistreatment of family members in the household. One girl eight years of age left her living situation with her aunt because she was not permitted to attend school. She went to live with her uncle ENDLINE PERFORMANCE EVALUATION | 35 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project instead where she was able to continue school. Two other children aged 16 and 17 left their families and are under the care of other organizations – IRC staff was not clear why these last two had re-separated. Table 4.2.2: Reasons for Re-separation of Deinstitutionalized Children Reason for Re-separation Number of Children Under 18 18 or older Total Migration of families 5 4 9 Children who separated to look for work 2 5 7 Independent living 0 4 4 Poverty of family 3 0 3 Maltreatment 1 2 3 Marriage 1 2 3 Death of caregiver 2 0 2 Death of child 2 0 2 Education 1 0 1 Unknown 2 0 2 Total 19 17 36 Source: IRC monitoring data and key informant interview with IRC staff CPE members and social assistants played an additional role in supporting families of children with disabilities. The project initially planned that physiotherapists from the rehabilitation centers would do follow-up visits with families; however, that proved challenging as many families lived in rural locations and it was not feasible for the physiotherapists to make all of the visits, especially given the constraints with transportation and their time. A staff member from Handicap International explained that it was then decided to try a community-based approach in which some of the CPE members were trained to raise awareness of and refer families to PRCs so that those parents could participate in the parental training sessions. They were also trained to follow up on cases of children with disabilities to ensure the continuation of exercises on a home-based level. In addition, each child had a case file that included instructions on how to provide development stimulation exercises and play activities for the child. Some of the social assistants in Gitega received information and instruction during monthly communication sessions between them and the PRC technicians so that they could support families with children living with disabilities. Follow-up tools were developed in order to follow up, evaluate and report about the child’s development during these monthly sessions. More specifically, according to information provided by Handicap International, many parents regularly followed up with the PRCs and continued stimulation sessions at home (often as a result of the intervention of the CPEs). The CPEs also integrated the follow-up activities of CWD into their activities and informed social workers in several urgent cases of malnutrition where there was a risk of child abandonment, as well as in cases concerning healthcare, such as a child sick with malaria. The approach was successful in Gitega, and shows the potential role that ENDLINE PERFORMANCE EVALUATION | 36 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project community members and social assistants can play in supporting families with children with disabilities. Incorporating social assistants, CPE members, and parents who have seen the benefits of physiotherapy on their child helps to raise awareness and reduce stigma on children with disabilities. Care of children with disabilities has improved and attitudes toward children with disabilities have changed. Both focus groups with caregivers of children with disabilities reported improvements in the care of the children in their custody as a result of FCF (parental training sessions on developmental stimulation and provision of mobilization helping aids). They also reported no longer hiding their children with disabilities. Some caregivers mentioned that they were no longer mocked by their neighbors, while others reported facing challenges in their interactions with children without disabilities and dealing with neighbors who are unsupportive and unsympathetic to the experience of caregivers of children with disabilities. Despite progress in care and follow-up, disability remains another main reason for separation; families are still not equipped to care for children with disabilities and lack support. FCF’s theory of change assumed that parental training of children with disabilities, case management, and training of technicians in rehabilitation centers would improve the capacity of these caregivers to meet the needs of children with disabilities; and teacher training on inclusive education would allow children with disabilities to attend school with children without disabilities. At endline, some similar challenges to reintegration remained for children with disabilities as they were at baseline, indicating that the project design did not address the issue of physical accessibility adequately. At endline, caregivers of children with disabilities and stakeholders interviewed emphasized that homes were not wheelchair-friendly or they did not have helping aids that were appropriate to facilitate health care, (self-) care and developmental stimulation of CWD. Several focus groups stated that there was a lack of material support for caregivers of children with disabilities, including finances, wheelchairs, and other items that may ease the burden of caring for a child with extra needs. Both focus groups with caregivers of children with disabilities felt there was no available school that was adapted to their child’s needs and reported some difficulty in gaining access to schools, but said that more children were receiving education despite the challenges. As Handicap International noted, there are inclusive schools in four provinces in Burundi, but full physical access remains difficult due to situations such as unadapted roads and playgrounds. IRC also reported that there were families who were ready to take their child back, but had no school in the community. Additionally, some ENDLINE PERFORMANCE EVALUATION | 37 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project caregivers of children with disabilities were widely frustrated and felt they had no support from the community, state, and/or administration.19 Both focus groups of caregivers of children with disabilities in Makamba and Muramvya also spoke of the complete lack of local centers with specialized care for children with disabilities, and the focus group in Makamba added that parents were forced to take care of these children full time. Handicap International noted that this lack of local centers also increased the number of children in RCCs – there were ten RCCs of the PRC type with 984 children in 2017.20 Lastly, a group of caregivers of children with disabilities in Muramvya said that materials were being used by children without disabilities, which prevented them from being used for youth with disabilities. These challenges will remain now that the project has ended. However, it should be noted that Burundi recently ratified the UN Convention on Persons with Disabilities, and the Government is in the process of establishing a national policy for children with disabilities as well as legislation on this subject. All of the respondents, apart from one of the CPE Presidents, agreed that family-based care was the best place for children with disabilities. The respondents also confirmed that all of their co-workers held this view within their respective ministries or organizations and that no one held divergent views. A stakeholder at the Ministry of Solidarity stated: “Now everyone understands that we must protect children. All of the ministries understand, and there has been a convergence.” As a result, it is possible that further support to children with disabilities and their families will continue in the future. b) Are there fewer children living in residential care facilities? The common consensus amongst the majority of stakeholders interviewed at the baseline was that the majority of RCCs were set up during ‘the crisis’ period (1993 - 2005). This is validated by the 2011 IRC report that found 51% of RCCs were established during this period, with 15% established between 2005 and 2011.21 For example, the Fondation Stamm Director described the reactive way in which centers were established one-by-one by individuals in response to the emergency of the 1990s. A few institutions have been around for longer, especially church-supported institutions. It was reported by several key stakeholders that since the crisis there has been a shift in mentality from expecting extended family members or community members to support and care for a child whose family was in need to now supporting parents or primary 19 Rehabilitation sessions are not covered for children older than five and the costs of helping aids are not covered at all. 20 According to the last mapping exercise completed for RCCs in Burundi as reported by Handicap International. 21 Armstrong, Miranda “Analysis of the Situation of Children in Residential Centers in Burundi”, IRC May 2011 ENDLINE PERFORMANCE EVALUATION | 38 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project caregivers placing children in RCCs as a solution to wider issues of poverty and other stressors existing within families. This shift in mentality has led to not only a proliferation in the number of RCCs since the crisis, but it has also made the deinstitutionalization process particularly challenging. While data collected by IRC documented a decrease in the number of children in RCCs, the general perception among stakeholders is that the number has increased. The general perception is that the number of children in RCCs is continuing to increase, however, according to IRC, the number of children in RCCs has decreased by approximately 400 cases since 2012 as a result of the FCF project as well as another UNICEF-funded project. While the numbers have decreased, a large number of focus group respondents, including caregivers, RCC staff, social workers, and members of the CDFC and CPEs, felt the that the number of children in RCCs is continuing to increase. Reasons given for this rise were poverty; a lack of solidarity in the community; divorce/polygamy; and war. However, one focus group with province CDFC workers felt that this number was shrinking, due to changes made in the referral process and awareness raising sessions that were conducted with families. c) Is the well-being of deinstitutionalized children assessed as being adequate? The well-being of deinstitutionalized children was generally strong by endline. As mentioned in the findings for Question 1, 378 children were placed by FCF with their biological families or in alternative care arrangements, such as placements with extended family members, foster care arrangements, or independent living arrangements for youth over 18. The results of the well-being indicators used by IRC show there were generally high levels of child well-being by endline. Close to 90% of all children who had been deinstitutionalized over the project period had adequate shelter or nutrition by endline, 77% regularly attended school, and children generally did not show signs of psychological stress during the visits, as evidenced by the low rates of children found crying, appearing isolated, or seeming frightened during the visits. There were also generally improvements between baseline and endline, where applicable. Due to concerns with the IRC M&E data quality at baseline, we were not able to report on baseline to endline trends for all well-being indicators. There were four indicators for which we could report on trends – the total number of children deinstitutionalized, the number of children who regularly attend school, the number of children sick in the past month with diarrhea or fever, and, of the children who were sick in the past month, the number who received medical attention. For each of these indicators, children showed modest improvements between baseline and endline. Rates of school attendance increased, and the percentage of sick children decreased. During both baseline and endline, every child who was sick in the past month with diarrhea or fever received ENDLINE PERFORMANCE EVALUATION | 39 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project medical attention. It is important to note that when the evaluation collected “baseline” data, 61 children had already been deinstitutionalized by the project.22 Table 4.2.3: Well-being Indicators for Child Deinstitutionalized by the Project, Number (percent) Well-being Indicators for Deinstitutionalized Children Baseline Endline Total Number Deinstitutionalized 61 378 Have adequate shelter N/A 345 (91%) Have adequate nutrition N/A 337 (89%) Child found crying during social assistant's visit N/A 4 (1%) Child shows signs of insomnia N/A 6 (2%) Child seemed frightened during social assistant's visit N/A 39 (10%) Child appeared isolated N/A 17 (4%) Regularly attend school (4/5 day school attendance in last week) 40 (66%) 291 (77%) Sick in the last month with diarrhea or fever 16 (26%) 48 (13%) Sick in the last month who received medical attention 16 (100%) 48 (100%) Source: IRC M&E data At endline, signs that the child was not doing well after returning from an RCC included the child being withdrawn and not participating in school or not being well-educated; the child being abused or mistreated by the family or community; the child being unable to find food or clothing; and the child being sickly. The spot check and review of case files showed that each child in the project’s caseload had a case file. In order to ensure that case files for each child were being completed and stored in accordance to confidentiality procedures, a spot check of case files was carried out at the IRC office in Gitega as well as the CDFC office in Bujumbura Rural. All case files in both locations were kept in a locked cabinet with one designated staff member having access to them. The following forms were being used by the IRC and CDFC for case management: ■ Registration form (Enregistrement) ■ Request for Family Tracing (Demande de recherché familliale) ■ Results of the Tracing (Resultats de pre-recherche familliale) ■ Family Evaluation form (Evaluation Familiale) ■ Reunification form (Formulaire de reunification) ■ Close-out form (Formulaire de cloture) ■ Plan of Reintegration (Plan de Reintegration) 22 As noted in the project limitations section of the report, the project started in January 2014, but NORC’s concept note for the evaluation was approved by mid-June 2014; the evaluation design was completed by December 2014; and the data collection was undertaken in March-April 2015. Thus some family tracing and reunification had already started prior to the baseline data collection. ENDLINE PERFORMANCE EVALUATION | 40 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project ■ Temporary Placement in Foster Family form (Fiche de placement en famille d’accueil temporaire) ■ Services form (Fiche de services) For complex cases only, the Best Interest Determination (BID) process was carried out by IRC in coordination with the CDFC, and information was recorded on a specific form – the Determination des Interets Superiors. Both the process and the form are based on the internationally recognized BID form used by UNHCR and its partners during humanitarian crises and was adapted to the Burundi context under the FCF project from the UNHCR Handbook. Copies of children’s files were stored in the IRC office and another in the CDFC office. Each file also included medical, school, and birth certificate documents depending on the child and the service received. All information on the relevant forms kept in each of the randomly selected files appeared to be complete. It was not considered by DCOF to be in the best interests of the children concerned for NORC to confirm details in the files with the families concerned. Sustainability of follow-up of cases is in question after project end. Importantly, while the evaluation found that the CDFC Coordinators and social assistants had been trained on case management and had been following case management procedures, such as the follow up of cases, one CDFC Coordinator and several social assistants emphasized the challenges in continuing to follow up on cases after the end of the project. They mentioned the lack of resources and logistical support to be able to trace families, follow up on cases due to families living in rural communities, and purchase materials such as paper in order to complete the forms. The majority of key informants did not think these activities would continue to the same extent once the project ended, although they were all committed to continuing activities to the best of their ability. d) Have there been other unanticipated positive or negative results of the program? We have no findings to report on this question apart from the unexpected and positive role played by CPE members and social assistants in supporting families with children with disabilities as mentioned above. This finding was an unanticipated positive result because the project initially planned that physiotherapists from the rehabilitation centers would do follow-up visits with families. When this was not possible, the project then trained some CPE members and social assistants (see paragraph above about positive role played by CPEs that explains this in more detail). ENDLINE PERFORMANCE EVALUATION | 41 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Regression Analysis for Question 2: Did the program measurably improve the safety, well-being, and development of highly vulnerable children, particularly those who are living without adequate family care? Did the program impact beyond direct services? Tables R1 and R2 indicate that there were statistically significant positive trends in children having safe and secure relationships with caregivers between baseline and endline, for both the VSLA and FSG cohorts, though these are mostly limited to children under the age of 15. In these tables, having safe and secure relationships with caregivers is defined as children who feel safe when they are alone, are not exposed to physical or verbal punishment, and who are only made to do chores a few times a week or less. The strongest baseline to endline positive trends were present in the children aged 0 – 8 years, with children in families with a VSLA participant being 15% more likely to have safe and secure relationships, and children in families participating in a FSG being 24% more likely. For the VSLA cohort, this seems to have been primarily driven by improvements in children doing fewer chores as punishment, while for FSG it was driven by reductions in exposure to verbal punishment. Children aged 9-14 in households participating in a FSG had statistically significant improvements in feeling safe and reductions in exposure to both physical and verbal punishment. Children aged 9-14 also have positive trends in this category, though generally to a smaller degree than for the younger children. Table R1: Child Safety Regression Results, Pre/Post Comparison, VSLA Child has safe and secure relationship with caregiversii Child feels safe when alone Child not exposed to physical punishment Child not exposed to verbal punishment Child does chores a few times a week or less Children age 0 - 8 Pre-post comparison 0.15*** -0.023 -0.021 -0.016 0.45** (0.053) (0.15) (0.13) (0.12) (0.23) Observations 273 197 273 273 273 Children age 9- 14 Pre-post comparison 0.077* -0.023 0.097 0.0018 0.0071 (0.040) (0.051) (0.064) (0.068) (0.080) Observations 240 179 240 240 240 Children age 15 - 17 Pre-post comparison -0.022 0.024 0.040 -0.0091 -0.024 (0.060) (0.062) (0.051) (0.095) (0.093) Observations 124 99 124 124 124 i. Regressions include controls for: education level of primary caregiver; age of HH head; number of children in HH; number of adults in HH ii. Safe and secure relationships are defined as: the child feels safe when alone, is not exposed to physical or verbal punishment, and is only made to do chores as punishment a few times a week or less iii. Statistical significance levels: * 10%; ** 5%; *** 1% ENDLINE PERFORMANCE EVALUATION | 42 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Table R2: Child Safety Regression Results, Pre/Post Comparison, FSG Child has safe and secure relationship with caregiversii Child feels safe when alone Child not exposed to physical punishment Child not exposed to verbal punishment Child does chores a few times a week or less Children age 0 - 8 Pre-post comparison 0.24*** 0.074 0.12 0.24** 0.29 (0.047) (0.14) (0.100) (0.096) (0.19) Observations 377 301 377 377 377 Children age 9- 14 Pre-post comparison 0.11*** 0.085** 0.16*** 0.29*** 0.016 (0.034) (0.041) (0.051) (0.053) (0.057) Observations 361 291 361 361 361 Children age 15 - 17 Pre-post comparison N/Aiii 0.11** 0.031 0.16** -0.044 N/A (0.053) (0.042) (0.071) (0.063) Observations 184 155 184 184 184 i. Regressions include controls for: education level of primary caregiver; age of HH head; number of children in HH; number of adults in HH ii. Safe and secure relationships are defined as: the child feels safe when alone, is not exposed to physical or verbal punishment, and is only made to do chores as punishment a few times a week or less iii. 100% of observations in both baseline and endline showed safe, secure relationships so pre/post impacts could not be estimated iv. Statistical significance levels: * 10%; ** 5%; *** 1% Question 3: Have prevention methods employed by the projects reduced risks of child/family separation? Conclusions: The project contributed to greatly increasing the number of households that are saving money overall. However, it did not increase the average amount of money saved when compared to savings of households at the baseline. In addition, savings actually decreased for VSLA households over the project period. There are some possible explanations: 1) “new savers” may be saving smaller amounts than those households that were already saving prior to the start-up of the project; or 2) the amount being saved may have decreased due to the current economic and political situation in the country and the ensuing rise in inflation, costs, and subsequent food insecurity23. Despite positive results in the numbers of households that were saving, it was not clear that households were materially better off. There was no clear overall increase in assets from the survey data, although a number of focus groups with children of VSLA and FSG households did report that their families were more financially secure. In addition, households reported feeling less constrained in their ability to provide food and shelter 23 IRC believes that this is the reason for the decrease in savings. ENDLINE PERFORMANCE EVALUATION | 43 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project to their children. Poverty still remains the main challenge to vulnerable families, and coupled with other stressors within the family, such as an ill or deceased parent, or external factors, such as raising inflation and food insecurity, reduces the ability of the family to cope. The project has improved children’s well-being among the most vulnerable (FSG) families and improved material, health, and psychosocial well-being of children in VSLA families (there was not much positive change in education or safety indicators). Handicap International noted that there was very little collaboration between VSLA activities and inclusive education activities, which they believe is why the Evaluation Team was not able to show an impact on education. Households also increased spending on children’s clothing, education, and medicine/medical care, showing that there is a direct link between the livelihoods support and the increase of children’s access to services, such as healthcare and education, an important contribution. Regarding other risk factors, at endline more families were confident in their ability to provide moral and spiritual guidance to their children, the most vulnerable FSG families were more socially engaged in their communities, and caregivers of children with disabilities have improved their care and reported that the community was more supportive. Strategies suggested by the findings to prevent family separation include financial or material support, including initiatives supporting income-generating activities; moral support; raising awareness/educating families about separation prevention strategies; and raising awareness of the national policy for the protection of children and favorable living conditions for children. Regarding local child welfare human resource capacities, support from social assistants has improved over the project period. There are now more social assistants available to vulnerable families and opinions of the social assistants have improved. The number of meetings with social assistants increased for VSLA households but did not increase significantly for FSG households. There was also greater awareness of CPEs at project endline and opinions of the CPEs also improved. a) Are households with children at risk of family separation stabilized and strengthened? Across the board, caregivers felt that income generating activities led to greater financial stability and ability to provide food and shelter; gender issues arose regarding women controlling money. All focus groups with caregivers reported greater financial stability thanks to their FCF project involvement. Several focus groups with children also said that they have noticed that the financial situation of the parents has become more ENDLINE PERFORMANCE EVALUATION | 44 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project secure thanks to the project. Table 4.3.1 shows caregivers’ perceptions regarding their ability to provide support to their children. The majority of caregivers, 52% and 68% of caregivers at baseline in VSLA and FSG respectively reported feeling unable to provide food and shelter to children in the households. Both of these figures dropped at endline to 41% and 64% for VSLA and FSG respondents, respectively. Most importantly, only 5.8% of the VSLA caregivers (22 out of 179 respondents) had answered “yes” they know where to go to get the support to provide better care for their families at baseline. All FSG caregivers answered “no” at baseline. The numbers rose to 12% for VSLA respondents and 24% for FSG respondents by endline. However, several focus groups with caregivers mentioned that there were potential challenges that surrounded allocating money between husband and wife. Specifically, there is stigma around women controlling money and fear that women will turn away from their husbands once they achieve financial autonomy. IRC reported that this was an issue they were aware of, and it was the reason they added an additional session on family financial budgeting into the VSLA cycle for the 2nd and 3rd cohort using the approach developed through the IRC EASE curriculum. Table 4.3.1: Opinions of Caregiver Ability to Provide Food and Shelter (%) Parental Support Indicators Baseline Endline VSLA (%) FSG (%) VSLA (%) FSG (%) How well do you feel you are able to provide food and shelter to the children in your household? (%) Very well 18 10 23 9 Somewhat 30 22 36 26 Not at all 52 68 41 64 Source: NORC evaluation survey Families at risk of separation are much more likely to have savings after the project, but cash incomes did not change – families remain financially constrained. The number of families that save money drastically increased over the project period to over 70% of both VSLA and FSG households reporting they had savings in the past month. This was an especially large increase of over 60 percentage points for FSG households and over 35 percentage points for VSLA households since the baseline. While a much larger number of households are now saving, the average amount saved by these households did not increase. The average amount saved for FSG respondents stayed the same over the project period and VSLA average savings dropped from $8 at baseline to $5 at endline24, even though more households were saving. Average monthly cash income did not change much over the project period. Both VSLA and FSG households 24 It should be noted that the overall economic situation in Burundi was much worse at the end of the project than it was at the project start ENDLINE PERFORMANCE EVALUATION | 45 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project continued to have very low average monthly incomes (Table 4.3.2), with VSLA being slightly better off at baseline and endline compared to FSG, which would be expected. It is possible that decreases to the average savings amount were because “new” savers were saving a smaller amount than those that were already saving, or, as seen in Table 4.3.3, households increased expenditures on some liquid assets, the highest expenditures being on frying pans and kerosene or gas stoves, which likely caused a decrease in savings. Additionally, there was some reduction in savings as a result of the current economic and political situation in the country, which has witnessed an increase in inflation over the project’s duration, although it is important to note that only about 40% of households reported this impact (see Table 4.3.5) Table 4.3.2: Average Monthly Household Cash Income and Household Saving25 Income Indicators VSLA FSG Mean Lower confidence interval Upper confidence interval Mean Min value (USD) Max value (USD) Baseline Average monthly household cash income * 12 9** 15** 9 0 63 Average monthly household saving 8 NA NA 3 0.25 13 Endline Average monthly household cash income* 12 9** 14** 10 0 116 Average monthly household saving (USD) 5 NA NA 3 0.17 35 Qualitative data confirmed that poverty remains one of the main challenges for vulnerable families. A large number of focus group respondents in both baseline and endline periods cited poverty as one of the main challenges for vulnerable families that led to families’ inability to meet basic needs such as food, shelter, clothing, and education for their children (as shown in survey results above and by focus group respondents as documented in findings for Question 2), and that necessitates children to contribute to income generation of the household (documented in findings for Question 2). Stakeholders interviewed also emphasized the issue of poverty and parents not being able to provide for their children’s most basic needs. All respondents mentioned poverty frequently as the main driving force for the institutionalized care of 25 Note: Baseline exchange rate as of March 15, 2015 and endline exchange rate as of May 15, 2017. *Respondents were asked about total household cash income in the last month. This does not include food grown by the family, or items bartered or traded. **Since we have weighted the VSLA data we show the lower and upper confidence interval rather than the minimum and maximum values. ENDLINE PERFORMANCE EVALUATION | 46 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project children in Burundi. While it is one step to identify and register children, it is another to reintegrate them back with their biological families who often live in poverty and without the means available to provide for their children’s most basic needs. Several respondents, including one of the CDFC Coordinators and two of the directors of the centers, explained that there were no RCCs prior to 1993. Three of the RCCs that were visited during the interviewing process asserted that they had opened as a result of the deteriorating situation following the 1993 crisis. However, as a result of the conflict, there was a breakdown in the socioeconomic situation combined with many children who had lost their parents, so various centers were established to address this issue. One director stated that prior to 1993: “Families were solid, even if the mother died. The father would marry a new woman and the relatives would support in raising the child. But now you can see there is a real problem, perhaps because of poverty.” Other respondents upheld this perspective and added that many of the children living in the centers had lost their mothers and were placed in the centers because their fathers did not believe they were capable of caring for the children on their own. According to one IRC staff: “There is a lack of engagement of parents with their children because the parents are not able to meet their basic needs, so the children do not have a good model and they go to live on the streets. They are trying to get by.” Several respondents agreed that many parents believe that placing their children in an RCC is in their child’s best interest because the centers will provide schooling and food. One respondent stated that there was a need to change people’s mentality so that the money put into running institutions was instead invested in placing children back in their communities and supporting families. The Counselor in charge of Child Protection at the Ministry stated: “The first factor is household poverty because parents are reluctant to take their children out of the orphanages because in the household they are dying of hunger.” The Director of Child and Family reaffirmed this statement: “The country is in crisis, and there are a lot of orphans. That is one reason. The socioeconomic situation has also deteriorated, as well as the degradation of social values and positive values at the community level. For instance, in the past, the child would be supported by others, but now the value is on individualism.” Several focus groups stated that poverty affects children with disabilities in particular because more resources and attention are required to meet their needs, including special care: “Yes, the child's disability may make it difficult to support the child by his family because the disabled child requires a very different upbringing…; they do not play with others; we have to stay next to them and bring them into the hands everywhere you go. The work is even paralyzed because of this child.” (Caregiver of a child with disabilities in Muramvya, baseline) ENDLINE PERFORMANCE EVALUATION | 47 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Other challenges faced by vulnerable families reported in Question 2 include death of parents, single parent households, step-parents that do not treat children well, and issues with parental behavior such as negligence, domestic violence, drug abuse, and parental conflict. In other words, poverty is often not the only reason children separate or are at risk of separation, but rather it is poverty combined with other stressors within the family that reduces the ability of the family to cope. More families are spending on children’s clothing, education, and medicine/medical care. As shown in Table 4.3.3, at project end, over half of VSLA and FSG caregivers targeted expenditures on their children’s education and medicine/medical care and the number of VSLA and FSG families that acquired clothing and education-related items for their children rose over the project period (an increase of 12 and 11 percentage points for respectively for VSLA and FSG families for both categories). Additionally, more VSLA families acquired shoes (an increase of 34 percentage points) and medicine/medical care (an increase of 7 percentage points) for their children than at baseline. VSLA and FSG caregivers spent approximately the same amount on children’s clothes, shoes, education, and medicine and these numbers did not change much from the baseline with the exception of a decrease in the value of medicine/medical care acquired by FSG families26 (decrease of 11 percentage points). Table 4.3.3: Household Expenditure on Children27 Did you obtain/purchase any of the following items by trade in the past 12 months? Baseline Endline Difference from Baseline to Endline VSLA FSG VSLA FSG VSLA FSG Percent incurring expenditures on clothes (%) 47 41 59 52 +12 +11 Values of the purchase of clothing (USD) 13 11 13 12 0 +1 Percent incurring expenditures on shoes* (%) 0 25 34 29 +34 +4 Value of the purchase of shoes (USD) 7 7 7 5 0 -2 Percent incurring expenditures on education (books, fees, transportation, etc.) (%) 67 59 79 70 +12 +11 Value of school expenses (USD) 18 16 17 12 -1 -4 Percent incurring expenditures on medicine or medical care for children, including traditional doctor and witch (%) 79 71 86 75 +7 +4 Values of the purchase of medical expenses (USD) 19 23 17 12 -2 -11 Source: NORC evaluation survey 26 The project supported a portion of VSLA group members and all parents/caregivers of reunified children to access health care cards through the VSLA solidarity funds in order to cover the bulk of their family’s medical expenses. 27 Note: * the percent incurring expenditures on shoes is less than 1%. ENDLINE PERFORMANCE EVALUATION | 48 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project The types of assets owned have changed for VSLA and FSG groups from baseline to endline, but there has been no clear overall increase in assets. As seen in Table 4.3.4, asset ownership has changed positively for some assets and negatively for others for both the VSLA and FSG households over the project period. The most striking change is the increase in cooking supplies with an increase in frying pan ownership of 49 percentage points and 30 percentage points for VSLA and FSG households respectively due to the project’s distribution of kitchen materials. Both groups also saw increased ownership of kerosene or gas stoves (10 percentage points for VSLA and 4 percentage points for FSG) and of ownership of their dwelling (7 percentage points for VSLA and 8 percentage points for FSG). Overall, both VSLA and FSG households increased smaller durable good/assets, while lumpy28 assets remained the same over the baseline and endline period, except for obtaining ownership over their dwelling. Table 4.3.4: Household Assets Does your household currently own any of the following items? Baseline Endline Difference from Baseline to Endline VSLA (%) FSG (%) VSLA (%) FSG (%) VSLA (%) FSG (%) Tables 46 21 36 18 -10 -3 Chairs 66 47 72 51 +6 +4 Sofa 6 4 3 3 -3 -1 Beds 76 58 75 60 -1 +2 Mattress 13 2 8 2 -5 +0 Cupboard 4 0 1 1 -3 +1 Towels 24 8 20 8 -4 +0 Frying pans 23 23 72 53 +49 +30 Kerosene or gas stove 7 4 17 8 +10 +4 Radios 27 19 17 8 -10 -11 Watches 11 9 15 8 +4 -1 Mobile phones 39 13 40 16 +1 +3 Charcoal or electric irons 3 1 5 0 +2 -1 Water storage tanks of at least 100 liters 1 1 0 0 -1 -1 Sewing machines 2 0 1 1 -1 +1 Poultry such as chickens and other fowl 31 19 28 23 -3 +4 Livestock such as cows, goats, and/or sheep 42 28 35 34 -7 +6 Mosquito nets 94 79 37 29 -57 -50 Bicycles 8 5 9 6 +1 +1 Refrigerators 0 0 0 0 0 0 28 Lumpy Assets: Assets that cannot be acquired in small increments but must be obtained in large, discrete units. ENDLINE PERFORMANCE EVALUATION | 49 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Does your household currently own any of the following items? Baseline Endline Difference from Baseline to Endline VSLA (%) FSG (%) VSLA (%) FSG (%) VSLA (%) FSG (%) Motorcycles 0 0 0 0 0 0 Your dwelling 88 80 95 88 +7 +8 Dwelling in this colline besides the one where you live 3 1 2 1 -1 -0 Land in this colline other than the land on which your home is located 21 19 26 13 +5 -6 Land outside this colline 11 7 13 12 +2 +5 Source: NORC evaluation survey While the current political climate impacted some VSLA and FSG caregivers, the majority reported that the political circumstances have not affected the operation of their savings groups, nor their ability to earn money. As seen in Table 4.3.5, less than 15% of VSLA and FSG caregivers reported that the current political climate affected their ability to operate savings groups. Of those that reported that their savings groups were impacted, the majority responded that they lost group members due to the political situation. A higher percent of respondents (40% for both VSLA and FSG respondents) reported that the recent political events have affected their, or another household member’s, ability to work or earn money in the past year. The most common ways in which their ability to work or earn money were impacted was through the loss of key markets, volatility of prices, and limited security causing unsafe working conditions. Table 4.3.5: Political Impacts on VSLA Operation and Financial Opportunity Political Impacts VSLA (%) FSG (%) Have the recent political events affected the operation of your VSLA/FSG group, or the assistance you receive from IRC? Yes 13 11 How have the recent events affected your VSLA/FSG group? VSLA meetings were suspended 26 16 VSLA lost members 53 64 VSLA group disbanded 0 0 VSLA money was lost or stolen 25 0 Other 31 48 Have the recent political events affected your or another household member’s ability to work or earn money for your household during the past year? Yes, very much 15 36 Yes, but only a little 25 4 No, not at all 60 60 How have the recent events affected your VSLA/FSG group? We were victims of discrimination from employers 0 10 Increased crime rates affected our work 0 0 ENDLINE PERFORMANCE EVALUATION | 50 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Political Impacts VSLA (%) FSG (%) Our assets were destroyed or seized 29 0 Key markets disappeared 59 30 Prices became more volatile 79 60 Limited security made it unsafe to work 62 30 The recent events caused health problems 0 0 Other 12 30 Source: NORC Evaluation Survey The well-being of children improved in FSG households and on some measures for VSLA households but not in others. NORC collaborated with DCOF and IRC to develop several indicators that would measure child well-being, grouped into five broad categories: material, education, safety, health, and psychosocial well-being. In general, children in both FSG and VSLA households saw improvement over the project period in material, health, and psychosocial well-being. School attendance only improved notably for FSG households. At the project’s end, VSLA households felt less safe leaving their children at home whereas FSG households felt more safe; VSLA households felt their children were better respected and treated by the community whereas there was not much change in FSG households; FSG households physically and verbally punished their children less frequently whereas there was not much consistent change among VSLA households. Material -- shelter, and clothing: Table 4.3.6 provides an overview of the children’s access to some of the most basic materials. The most noticeable changes are the decline in children’s access to bed nets and their increased access to school supplies. Across both groups and all three age groups, children’s access to bed nets dropped by at least 40 points. Conversely, all three age groups in both VSLA and FSG households saw increased ownership in school uniforms and school materials. Another interesting trend is that all three age groups for VSLA households saw a decrease in child ownership of blankets (of between 2 and 8 points), while each age group in FSG households saw an increase in child ownership of blankets of over 15 points. ENDLINE PERFORMANCE EVALUATION | 51 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Table 4.3.6: Overview of Materials for the Children in VSLA and FSG29 Materials the child has: Baseline Endline Difference from Baseline to Endline VSLA (%) FSG (%) VSLA (%) FSG (%) VSLA (%) FSG (%) Children age 0-8 Blanket 28 10 26 35 -2 +25 A pair of shoes 19 9 25 11 +6 +2 At least two sets of clothing 4 2 3 2 -1 -0 A mat or mattress 89 75 88 81 -1 +6 A bed net 89 76 43 31 -46 -45 School uniform 4 6 17 16 +13 +10 School materials 19 10 65 33 +46 +23 Children age 9-14 Blanket 28 12 20 27 -8 +15 A pair of shoes 30 15 42 22 +12 +7 At least two sets of clothing 2 1 2 3 +0 +2 A mat or mattress 85 73 92 82 +7 +9 A bed net 82 64 28 22 -54 -42 School uniform 43 33 51 52 +8 +19 School materials 72 57 95 79 +23 +22 Children age 15-17 Blanket 26 12 18 28 -8 +16 A pair of shoes 51 47 61 39 +10 -8 At least two sets of clothing 4 5 3 1 -1 -4 A mat or mattress 83 79 84 75 +1 -4 A bed net 83 62 16 19 -67 -43 School uniform 44 38 65 48 +21 +10 School materials 63 45 79 65 +16 +20 Source: NORC evaluation survey Education: As seen in Graph 1, the percentage of children going to school rose between baseline and endline for each of the three age groups in FSG households. Meanwhile, for VSLA households, the percentage of children going to school did not increase significantly for any of the age groups. It is likely that school attendance increased because there was an increase in expenditures on education (school fees, books, transportation, etc.) of 12 percentage points for VSLA and 11 percentage points for FSG that may be as a result of the project (see Table 4.3.3). 29 Note: These figures reflect the percentage of caregivers reporting of having at least one item for each material listed. ENDLINE PERFORMANCE EVALUATION | 52 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Graph 1: Percentage of Children Going to School Source: NORC evaluation survey As seen in Table 4.3.7, of those going to school, attendance in school by children, both with and without disabilities rose for all three age groups in FSG households. The percentage of children attending school in VSLA households did not change significantly for all age groups. The same pattern can be seen for the percent of children with and without disabilities who attended 4 out of 5 days of school with the exception of VSLA households for children aged 6-8 whose school attendance rose significantly. Table 4.3.7: Of those who Go to School, Number of Children and CWD Going to School 4 or 5 Days of the Previous Week Baseline Education Children age 6-8 Children age 9-14 Children age 15-17 VSLA FSG VSLA FSG VSLA FSG Number of children 26 46 130 182 63 81 Number of CWD 5 8 23 36 10 27 Percent of children going to school 73% 37% 89% 71% 68% 49% Percent of CWD going to school 60% 13% 83% 67% 70% 48% Percent of children who went to school 4/5 days of the previous week 65% 33% 77% 64% 54% 43% Percent of CWD who went to school 4/5 days of the previous week 60% 13% 70% 58% 40% 41% Endline Education Children age 6-8 Children age 9-14 Children age 15-17 VSLA FSG VSLA FSG VSLA FSG Number of children 24 50 111 179 61 103 Number of CWD 2 5 30 37 11 19 Percent of children going to school 71% 58% 92% 80% 64% 60% Percent of CWD going to school 100% 60% 87% 73% 45% 74% Percent of children who went to school 4/5 days of the previous week 63% 48% 77% 67% 46% 49% Percent of CWD who went to school 4/5 days of the previous week 100% 60% 73% 59% 36% 63% Source: NORC evaluation survey 89% 92% 68% 64% 73% 71% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Endline VSLA Panel Age 9 - 14 Age 15 - 17 Age 6 - 8 71% 80% 49% 60% 37% 58% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Endline FSG Panel Age 9 - 14 Age 15 - 17 Age 6 - 8 ENDLINE PERFORMANCE EVALUATION | 53 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Health and food security: Tables 4.3.8 and 4.3.9 show the health status of children in FSG and VSLA households. By project end, children in both FSG and VSLA households were more likely to see a health care provider when sick. The VSLA results show that the percentage of children who have been sick in the past year and seen a healthcare provider increased for all three age groups, with the children 15-17 seeing the largest increase. Similar to the VSLA results, the FSG results show that the percentage of children who were sick and saw a healthcare provider increased for all three age groups, with both children aged 9-14 and those 15-17 increasing by over 10 percentage points. Table 4.3.8: Health Status of VSLA Children, percent VSLA Health and Nutrition Children age 0-8 Children age 9-14 Children age 15-17 Baseline Endline Change Baseline Endline Change Baseline Endline Change In the past year, how many times the child has been sick with diarrhea or a fever and seen a health provider? Never 12 8 -4 35 32 -3 44 25 -19 Once 13 17 +4 20 9 -11 20 14 -6 2-5 times 59 56 -3 41 49 +8 34 49 +15 More than 5 times 16 20 +4 4 10 +6 2 13 +11 Who provided the health care? Health clinician 98 100 +2 99 100 +1 100 100 0 Traditional healer 1 0 -1 1 0 -1 0 0 0 Other 1 0 -1 0 0 0 0 0 0 Source: NORC evaluation survey Table 4.3.9: Health Status of FSG Children, percent FSG Health and Nutrition Children age 0-8 Children age 9-14 Children age 15-17 Baseline Endline Change Baseline Endline Change Baseline Endline Change In the past year, how many times the child has been sick with diarrhea or a fever and seen a health provider? Never 16 17 +1 44 36 -8 44 37 -7 Once 17 10 -7 20 17 -3 27 21 -6 2-5 times 58 60 +2 32 43 +11 28 40 +12 More than 5 times 8 13 +5 2 3 +1 1 2 +1 Who provided the health care? Health clinician 96 99 +3 94 96 +2 100 100 0 Traditional healer 1 1 +0 1 3 +2 0 0 0 Other 4 1 -3 5 2 -3 0 0 0 Source: NORC evaluation survey ENDLINE PERFORMANCE EVALUATION | 54 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project In terms of food security, children were eating more meals by project end. At baseline, 58% – 62% of children across all age groups had less than two meals per day at baseline (Graph 2). The numbers dropped significantly for both VSLA and FSG groups. At endline the percent of children who went without two meals a day dropped on average by 31 percentage points for children across all age groups in VSLA families and by 24 percentage points on average for children across all age groups in FSG families. The corresponding increase was reflected largely in those eating two meals a day, seven or five days a week (Table 4.3.10). Table 4.3.10: Food Security Status of VSLA and FSG Children, percent Days in past week ate at least 2 meals per day Children age 0-8 Children age 9-14 Children age 15-17 Baseline Endline Change Baseline Endline Change Baseline Endline Change VSLA None 58 22 -36 60 25 -35 58 36 -22 1 11 5 -6 6 3 -3 5 4 -1 2 12 9 -3 11 8 -3 12 10 -2 3 5 7 +2 5 7 +2 7 6 -1 4 3 2 -1 4 2 -2 2 4 +2 5 2 17 +15 4 16 +12 4 5 +1 6 0 2 +2 0 1 +1 1 0 -1 7 5 35 +30 10 38 +28 10 35 +25 NA 4 0 -4 0 0 0 0 0 0 FSG None 60 36 -24 60 37 -23 62 36 -26 1 8 5 -3 9 5 -4 9 7 -2 2 9 15 +6 10 15 +5 12 17 +5 3 5 13 +8 7 13 +6 7 14 +7 4 3 5 +2 4 4 +0 2 6 +4 5 1 13 +12 1 12 +11 1 10 +9 6 1 3 +2 2 2 -0 6 2 -4 7 5 11 +6 6 12 +6 0 10 +10 NA 5 0 -5 0 0 0 0 0 0 Source: NORC evaluation survey ENDLINE PERFORMANCE EVALUATION | 55 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Graph 2: Children Who Do Not Get 2 Meals Per Day at Least Once a Week, percent Psychosocial well-being: As seen in Tables 4.3.11 and 4.3.12, at both baseline and endline, caregivers reported that almost all the children under their care in each age group have a friend outside the home that they play with every day. Across all age groups and both VSLA and FSG households, caregivers were complimenting their children more by project end. At baseline, 19 % to 28% of caregivers said that they “never” compliment the children under their care at baseline. At endline, the number of caregivers who said this had dropped across all age groups for both VSLA and FSG groups to between 8% and 17%. Table 4.3.11: Psychosocial Indicators of VSLA Children, percent VSLA Psychosocial Indicators Children age 0-8 Children age 9-14 Children age 15-17 Baseline Endline Baseline Endline Baseline Endline Child has a friend outside the home that they play with a bit every day? 90% 96% 95% 94% 97% 97% Child has someone they can go to when they have worries? 91% 99% 93% 100% 99% 98% In the past month, the child has appeared to have worries? 63% 69% 47% 52% 48% 42% In the past month, child asked for advice about decisions that they feel are important? 20% 14% 59% 57% 68% 72% In the last year, have you discussed future plans or wishes with them? 20% 37% 62% 87% 80% 93% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Endline VSLA Age 0 - 8 Age 9 - 14 Age 15 - 17 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Endline FSG Age 0 - 8 Age 9 - 14 Age 15 - 17 ENDLINE PERFORMANCE EVALUATION | 56 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project VSLA Psychosocial Indicators Children age 0-8 Children age 9-14 Children age 15-17 Baseline Endline Baseline Endline Baseline Endline In the past week, how many times have you told the child that s/he is a good boy or a good girl? Never 20% 13% 20% 16% 25% 17% Once or twice 40% 27% 40% 29% 32% 27% Once every few days 5% 11% 5% 20% 4% 15% Once every day 7% 15% 7% 9% 14% 12% Several times a day 27% 34% 27% 26% 24% 30% Source: NORC evaluation survey Table 4.3.12: Psychosocial Indicators of FSG Children, percent FSG Psychosocial Indicators Children age 0-8 Children age 9-14 Children age 15-17 Baseline Endline Baseline Endline Baseline Endline Child has a friend outside the home that they play with a bit every day? 90% 90% 99% 95% 96% 94% Child has someone they can go to when they have worries? 86% 96% 53% 94% 96% 95% In the past month, the child has appeared to have worries? 59% 54% 51% 45% 47% 33% In the past month, child asked for advice about decisions that they feel important? 19% 27% 51% 57% 64% 65% In the last year, have you discussed future plans or wishes with them? 26% 45% 69% 85% 80% 88% In the past week, how many times have you told the child that s/he is a good boy or a good girl? Never 28% 15% 27% 12% 28% 8% Once or twice 30% 17% 36% 25% 35% 23% Once every few days 10% 11% 9% 16% 11% 17% Once every day 10% 19% 9% 15% 9% 18% Several times a day 23% 38% 18% 33% 17% 34% Source: NORC evaluation survey Safety: The younger the children, the lower the percentage of caregivers who feel the child is safe at home in the absence of the caregiver (Tables 4.3.13 & 4.3.14). VSLA households generally felt less safe while FSG households generally felt safer leaving their children at home from baseline to endline. Tables 4.3.13 & 4.3.14 show that VSLA respondents’ perception of how the children in their care are treated by community members improved from baseline to endline for all three age groups with more respondents saying the children are “well respected and treated”, while there was not much change in FSG households. The VSLA group responded that children 9 – 14 are more marginalized, while the FSG respondents reported that children 0 – 8 are more marginalized. ENDLINE PERFORMANCE EVALUATION | 57 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project In general, FSG households treated their children better at endline, by less frequently physically punishing them or calling them lazy, idiot, or another insult. VSLA households did not consistently change physical punishment trends, showing an increase with children aged 0-8 and a decrease with children aged 15-17; VSLA households did not show much change in the rate of insulting their children. Table 4.3.13: VSLA Safety Indicators for Children, percent Safety Indicators Children age 0-8 Children age 9-14 Children age 15-17 Baseline Endline Baseline Endline Baseline Endline Do you feel the child is safe at home when you are not there? (%) Always 50% 44% 67% 65% 81% 76% Sometime 30% 32% 24% 21% 9% 14% Never 20% 24% 9% 14% 10% 10% How well is the child treated by others in the community? (%) Well respected and treated 56% 60% 55% 68% 63% 72% Neither well respected nor marginalized 33% 30% 31% 23% 34% 20% Marginalized or badly treated 11% 11% 14% 9% 3% 8% In the past month, have you physically punished the child (smack, hit with a hand or stick) in your care? (%) Frequent 0% 3% 3% 3% 0% 0% Sometime 22% 28% 37% 34% 14% 5% Never 78% 69% 60% 63% 86% 95% In the past month, how often have you called the child lazy, idiot, useless, or a similar word? (%) Frequent 5% 1% 18% 9% 4% 5% Sometime 42% 44% 50% 57% 51% 48% Not at all 53% 55% 32% 34% 45% 47% How often the child has done chores and other household works? (%) Everyday 2% 6% 22% 18% 32% 38% A few times per week 18% 8% 36% 26% 31% 25% Rarely 9% 20% 18% 36% 25% 26% Never 71% 65% 25% 20% 11% 11% Source: NORC evaluation survey Table 4.3.14: FSG Safety Indicators for Children, percent Safety Indicators Children age 0-8 Children age 9-14 Children age 15-17 Baseline Endline Baseline Endline Baseline Endline Do you feel the child is safe at home when you are not there? (%) Always 48% 45% 70% 72% 78% 86% Sometime 23% 32% 16% 21% 10% 10% Never 29% 23% 14% 7% 12% 4% How well is the child treated by others in the community? (%) Well respected and treated 57% 55% 56% 54% 61% 58% Neither well respected nor marginalized 32% 34% 33% 38% 33% 34% ENDLINE PERFORMANCE EVALUATION | 58 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Safety Indicators Children age 0-8 Children age 9-14 Children age 15-17 Baseline Endline Baseline Endline Baseline Endline Marginalized or badly treated 12% 12% 11% 8% 6% 8% In the past month, have you physically punished the child (smack, hit with a hand or stick) in your care? (%) Frequent 2% 1% 3% 1% 1% 0% Sometime 28% 18% 37% 23% 8% 6% Never 70% 81% 60% 77% 91% 94% In the past month, how often have you called the child lazy, idiot, useless, or a similar word? (%) Frequent 5% 4% 14% 3% 3% 2% Sometime 38% 17% 53% 33% 42% 21% Not at all 56% 80% 33% 64% 55% 77% How often the child has done chores and other household work? (%) Everyday 6% 5% 25% 36% 37% 54% A few times per week 15% 10% 36% 26% 34% 26% Rarely 10% 23% 16% 24% 19% 8% Never 69% 62% 22% 13% 10% 12% Caregivers are more positive in their ability to give moral and spiritual guidance to children in their household at the endline. 58% and 50% of the caregivers in VSLA and FSG respectively said that they feel “very well” about their ability to provide moral and spiritual guidance at baseline. Those numbers improved to 85% and 62% for VSLA and FSG respondents, respectively at endline. Table 4.3.15: Opinions of Ability to Provide Moral and Spiritual Guidance (%) Parental Support Indicators Baseline Endline VSLA (%) FSG (%) VSLA (%) FSG (%) How well do you feel you are able to provide moral and spiritual guidance? (%) Very well 58 50 85 62 Somewhat 38 41 12 32 Not at all 5 9 3 5 Source: NORC evaluation survey FSG caregivers were more socially isolated than VSLA caregivers at baseline and they became almost as socially engaged as VSLA caregivers at endline. At baseline, 90% of VSLA caregivers were members of community groups, whereas the figure was only 54% for FSG caregivers. The FSG figure rose substantially by endline, with 92% of FSG respondents reporting that they were members of a community group, similar to VSLA caregivers at 93%. There was not much change in attendance of community meetings among VSLA caregivers although they belonged to more community groups at endline. ENDLINE PERFORMANCE EVALUATION | 59 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Table 4.3.16: Caregivers’ Engagement in Community Meetings Community Group Indicators Baseline Endline VSLA (%) FSG (%) VSLA (%) FSG (%) How often do you attend community meetings? (%) Daily 1 1 1 Weekly 26 10 28 31 Monthly 51 55 55 57 Once a year 13 28 15 7 Less than once a year 8 6 1 3 Are you a member of any community group? (%) Yes 90 54 93 92 How many [community groups are you a member of]? (%) 1 75 83 60 82 2 19 15 31 14 3 3 2 8 4 4 3 - 1 - 5+ 1 - - - Source: NORC evaluation survey b) Were the relevant families chosen for inclusion in the project? Families chosen for inclusion in the project were very much in need. As documented in Question 2 and the first sub-question for Question 3, poverty is one of the main risks of separation. Survey data confirms that households that were selected to be part of the project were in need of financial assistance, as documented in Table 4.3.2. Using the mean monthly income as a base, we can calculate that VSLA households had a mean daily income of $0.56/day at baseline and endline and FSG household had a mean daily income of $0.42/day at baseline and $0.46/day at endline. Although FSG households are considered to be more at risk, both the VSLA and FSG mean incomes at baseline and endline show that these groups fall below the poverty line of $1.90/day. 30 c) Are children at risk of losing family care continuing to live in appropriate, permanent, and protective family care due to improved national policies and local child welfare human resource capabilities and service delivery? Strategies suggested to prevent family separation include financial or material support, including initiatives supporting income-generating activities; moral support; and raising awareness/educating families about separation prevention strategies; and raising 30 The World Bank has an established international poverty rate of $1.90/day in 2011 Purchasing Power Parity (PPP). Using the conversion factor for Burundi for 2015-2016, the poverty line for the time of data collection becomes $0.76/day. While we don’t have consumption data, we can use the reported mean monthly income in Table 4.3.2 to get mean daily income. ENDLINE PERFORMANCE EVALUATION | 60 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project awareness of the national policy for the protection of children and favorable living conditions for children. Focus group respondents proposed a number of strategies that could prevent separating children from families. Raising awareness about favorable living conditions for children and children’s rights and duties was highlighted by a large number of focus groups at both baseline and endline. Several baseline focus group respondents felt that family separation could be reduced by raising awareness about the national policy for the protection of children among families and children and by raising awareness among parents on the merits of a child staying in a family. Endline focus groups suggested raising awareness of separation in communities and training people on how to prevent it on both the community level as well as amongst parents and families. At endline, a number of focus group respondents spoke about external support from the community at large (both morally and materially) as well as support from stronger institutions as factors that could help reduce family separation. The names of supporting projects and organizations were given by three different focus groups and included the Family Care First Project; Terre de Hommes; OIDEB; Red Cross; Government; and Vulnerable Families of World Vision. Besides these organizations, endline examples of this moral and material support were turning to extended family to seek advice and to local leaders or the administration; neighbors and friends; social assistants; and religious leaders to request assistance to care for their children. At baseline and endline, financial/material support was viewed as important for preventing separation of families. In particular, school support, health care support, and income-generating activities were highlighted. Stakeholders interacting with FCF were aware of local actors who support children so they can remain in family-based care. Stakeholders were asked if they were aware of policies or programs that support children to remain in family-based care, with their biological families, extended families or foster families. There were several stakeholders who had not directly participated in the Family Care First project who were not aware of its activities or other programs that focused specifically on the promotion of family￾based care or on deinstitutionalization; however, all other respondents mentioned the FCF project as well as a former UNICEF-funded project that also focused on deinstitutionalization. The FCF Coordinator maintained that: “Here in Burundi, IRC is really the key actor in deinstitutionalization and working with the centers.” Several other respondents mentioned activities implemented by SOS Village as well as a local association, Girayuja, and local NGO, Sojpae, that give support to families in need. It was mentioned that all of these organizations work together in synergy in the protection sector. ENDLINE PERFORMANCE EVALUATION | 61 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project ENDLINE PERFORMANCE EVALUATION | 62 Overall, caregivers had a positive opinion of the services provided by social assistants. As seen in Table 4.3.17, majority of caregivers in both VSLA and FSG households “strongly agreed” with all the statements regarding support services provided by social assistants and these numbers trended upward at endline. Also, nearly a quarter of caregivers in VSLA and FSG households at the baseline “strongly disagreed” that the social assistants provided help with problems caregivers are facing with their children; these numbers decreased at endline, as seen on Table 4.3.17 to 15% for VSLA respondents and 12% for FSG respondents. Table 4.3.17: Opinions of Support Services Provided by Social Assistants (%) Do you strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, or strongly disagree with the following statements: The social assistant: VSLA FSG Strongly agree Somewhat agree Neither agree nor disagree Somewhat disagree Strongly disagree Strongly agree Somewhat agree Neither agree nor disagree Somewhat disagree Strongly disagree Baseline Seemed to understand my problem 84 10 2 2 3 82 12 0 1 5 Empathized with me 78 9 1 0 12 68 22 4 0 6 Came back when promised to 63 14 5 3 15 65 20 2 2 11 Did the things they said they would do 63 17 3 5 12 61 26 2 5 6 Provided me with regular information about my progress relating to the issue 56 23 4 1 17 56 24 4 2 14 Helped me with problems I am facing with my children 56 19 3 1 22 53 15 6 0 25 Endline Seemed to understand my problem 83 10 1 1 4 81 16 0 1 2 Empathized with me 86 10 0 1 3 76 18 0 1 5 Came back when promised to 85 8 0 7 74 15 0 2 9 Did the things they said they would do 82 14 0 1 4 71 21 1 1 7 Provided me with regular information about my progress relating to the issue 74 14 0 0 13 67 20 1 1 11 Helped me with problems I am facing with my children 73 12 0 0 15 71 17 0 1 12 Source: NORC evaluation survey The percent of caregivers who said they have a social assistant in their province and commune rose from baseline to endline from 89% to 97% for VSLA respondents and 86% to 95% for FSG respondents (Table 4.3.18). Despite the increase in the percent of FSG respondents who reported having a social assistant in their commune, the percent of FSG households who said they wanted to meet with a social assistant but were not able to rose by 11 percentage points. Even though FSG caregivers are more in need of Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project support, this disparity suggests that social assistants are not targeting the most vulnerable households. Table 4.3.18: Access to Social Assistants Access to Social Assistants VSLA FSG Baseline Endline Change Baseline Endline Change Is there a social assistant in your province and your commune? Yes 89 97 8 86 95 9 No 6 3 -3 13 5 -8 DK 5 0 -5 2 0 -2 In the past year, how many times have you met with the social assistant? Never 4 0 -4 6 2 -4 Once 14 18 4 29 31 2 From 2 to 6 70 59 -11 51 54 3 From 7 to 12 6 4 -2 8 5 -3 More than 12 5 19 14 6 8 2 Did you want to meet the social assistant but not able to? Yes 45 43 -2 31 42 11 No 55 57 2 69 58 -11 Source: NORC evaluation survey Caregivers had a positive opinion of CPEs in their communities. Table 4.3.19 shows that only 1/3 of VSLA and FSG caregivers had heard of the CPEs working in their commune at baseline; of these a large percent had correct knowledge of the role of the CPEs and believed they were doing a good job in fulfilling their role. These numbers rose to roughly 1/2 of VSLA caregivers and 2/3 of FSG caregivers having heard of CPEs working in their commune at endline, with over 90% of both groups believing that the CPEs are doing a good job in fulfilling their role. The social assistants were officially supposed to work primarily through CPEs at the colline level, but this data suggests that this was not routinely the case in survey sites. The percent of VSLA and FSG households that had to contact the chief of the colline or police dropped from baseline to endline, with VSLA reports dropping by 9 percentage points and FSG reports dropping by 6 points. The service received by FSG households improved, with increased reports of respondents always getting the services they needed, while VSLA services stayed relatively the same between baseline and endline. ENDLINE PERFORMANCE EVALUATION | 63 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Table 4.3.19: Knowledge of Child Protection Committees (%)31 Knowledge of CPEs Baseline Endline VSLA (%) FSG (%) VSLA (%) FSG (%) Have you heard of the Child Protection Committee working in your commune? Yes 31 32 48 66 No 69 68 52 34 What do you believe the role of CPE is?* Mobilize communities around child protection 67 84 87 91 Inform communities about the rights of the child 62 61 76 86 Mobilize communities to prevent cases of abuse, exploitation, violence, neglect 46 52 57 52 Facilitate birth registration 58 59 67 55 Respond to individual cases of abuse, exploitation, violence, neglect 44 37 46 45 Support social assistants working with vulnerable children in the community 41 29 49 51 Don't know 1 3 2 1 Do you think the CPE did a good job fulfilling their role?* Very much 82 77 96 91 Somewhat 16 16 4 8 A little bit 0 3 1 Not at all 1 1 1 Don't know 1 3 In the past year, did someone in your household need to contact the chief of the colline or police? Yes 34 27 25 21 No 66 73 75 79 Were they able to get the service needed? Always 57 52 58 63 Sometimes 7 15 6 10 Never 35 33 35 27 DK 1 What was the main reason why they were not able to get the service needed? Someone in the family blocked contact 0 19 4 6 Too far away 6 Turned away at the door 11 6 Do not trust the chief of colline, a Bashigantahe, or the police 21 19 7 17 Others 64 59 70 67 DK 14 3 7 Source: NORC evaluation survey 31 Note: *Responses only by those saying “yes” they have heard of the CPE working in their community. ENDLINE PERFORMANCE EVALUATION | 64 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Regression Analysis Question 3: Have prevention methods employed by the projects reduced risks of child/family separation? To understand how the risks of child/family separation have evolved between baseline and endline, we ran three sets of regressions. The first set, whose results are presented in Tables R3 and R4, show the trends on child well-being indicators between baseline and endline. Children with higher levels of well-being would, in theory, be at a lower risk of child/family separation. The second set of regressions concerns the caregivers’ perceptions of their knowledge of caring for the children and their perceptions on the quality of their relationships with the children. In these cases, presented in Table R5, children placed in homes where the caregivers have higher levels of confidence in their abilities to care for the child would, in theory, be at a lower risk of separation. Finally, Table R6 presents trends in changes of economic conditions for households in the FSG and VSLA cohorts. In these cases, households that saw improvements in income￾generation, savings, and expenditures on children would, in theory, be at lower risk of child/family separation. According to Table R3, the only statistically significant positive child well-being indicator for children in all age groups in VSLA households was in the number of days per week that the child ate two meals, including a protein source such as fish, meat, eggs, or legumes. The increases were largest for the younger age groups in the VSLA cohort, with children under 15 eating more meals over 2 additional days per week. The increases were not as large in the FSG cohort, as shown in Table R4, though they still showed statistically significant improvements with children in those age groups eating two meals an additional day or so per week. The FSG cohort also had a 14% improvement in the percentage of children aged 9-14 with adequate clothing. Trends on caregiver perceptions on their caregiving capacity are presented in Table R5. In general, there were few statistically significant improvements. The only positive trends were for the VSLA cohort’s self-reported knowledge of moral care techniques32, which showed a 23% improvement between baseline and endline. Also, the FSG cohort showed an improvement of about 8% in caregivers reporting they know moral care techniques very well, or are highly knowledgeable about child development. Finally, trends in the economic conditions of the households are presented in Table R6. In general there were few statistically significant changes between baseline and endline for FSG or VSLA. For VSLA, there was a statistically significant improvement in the number of working adults per household, however any changes in household income, savings, or expenditures on children were not significant. In FSG households, the only statistically significant changes were for the value of household savings. On average, 32 Moral and spiritual guidance ENDLINE PERFORMANCE EVALUATION | 65 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project household savings for FSG households increased by over 3,000 Francs, and this was statistically significant at the 1% level. Table R3: Child Well-being Regression Results, Pre/Post Comparison, VSLA Outcome Pre-post comparison Standard Error Observations Children age 0 - 8 Child has adequate clothing -0.027 (0.055) 273 Child attended at least 3 days of school last week 0.072 (0.10) 43 Days per week child ate two meals including a protein 2.58*** (0.31) 256 Times child was sick with diarrhea last year 0.60 (0.46) 273 Child appeared stressed in the past month 0.11* (0.059) 260 Child has friends outside the home they can play with 0.059 (0.045) 178 Children age 9- 14 Child has adequate clothing 0.077 (0.059) 298 Child attended at least 3 days of school last week -0.027 (0.049) 218 Days per week child ate two meals including a protein 2.18*** (0.35) 240 Times child was sick with diarrhea last year 0.67* (0.35) 240 Child appeared stressed in the past month 0.058 (0.065) 240 Child has friends outside the home they can play with -0.0022 (0.033) 179 Children age 15 - 17 Child has adequate clothing 0.093* (0.052) 287 Child attended at least 3 days of school last week -0.10 (0.097) 82 Days per week child ate two meals including a protein 1.29** (0.51) 124 Times child was sick with diarrhea last year 0.74* (0.37) 124 Child appeared stressed in the past month -0.042 (0.093) 123 Child has friends outside the home they can play with 0.0046 (0.025) 99 i. Regressions include controls for: education level of primary caregiver; age of HH head; number of children in HH; number of adults in HH ii. Statistical significance levels: * 10%; ** 5%; *** 1% Table R4: Child Well-being Regression Results, Pre/Post Comparison, FSG Outcome Pre-post comparison Standard Error Observations Children age 0 - 8 Child has adequate clothing 0.049 (0.039) 377 Child attended at least 3 days of school last week -0.10 (0.11) 50 Days per week child ate two meals including a protein 1.34*** (0.25) 348 Times child was sick with diarrhea last year 0.59 (0.42) 377 Child appeared stressed in the past month -0.053 (0.053) 363 Child has friends outside the home they can play with -0.018 (0.037) 273 Children age 9- 14 Child has adequate clothing 0.14*** (0.048) 418 Child attended at least 3 days of school last week -0.064 (0.041) 274 Days per week child ate two meals including a protein 1.14*** (0.25) 361 Times child was sick with diarrhea last year 0.29 (0.21) 361 ENDLINE PERFORMANCE EVALUATION | 66 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Outcome Pre-post comparison Standard Error Observations Child appeared stressed in the past month -0.086 (0.054) 361 Child has friends outside the home they can play with -0.049** (0.021) 289 Children age 15 - 17 Child has adequate clothing 0.023 (0.043) 412 Child attended at least 3 days of school last week -0.074 (0.073) 102 Days per week child ate two meals including a protein 1.00*** (0.33) 184 Times child was sick with diarrhea last year 0.35 (0.25) 184 Child appeared stressed in the past month -0.13* (0.075) 184 Child has friends outside the home they can play with -0.030 (0.038) 154 i. Regressions include controls for: education level of primary caregiver; age of HH head; number of children in HH; number of adults in HH ii. Statistical significance levels: * 10%; ** 5%; *** 1% Table R5: Caregiver Knowledge Regression Results, Pre/Post Comparison Outcome Pre-post comparison Standard Error Observations VSLA Knows basic care techniques very well 0.053 (0.046) 323 Knows moral care techniques very well 0.22*** (0.050) 322 Knows how to talk to boy children about life and family -0.14 (0.26) 326 Boy children are generally able to talk about life 0.15 (0.27) 326 Knows how to talk to girl children about life and family -0.39* (0.22) 326 Girl children are generally able to talk about life -0.17 (0.23) 326 Highly knowledgeable on child development -0.021 (0.049) 324 Highly confident the caregiver can support the child -0.027 (0.047) 323 FSG Knows basic care techniques very well -0.0036 (0.028) 472 Knows moral care techniques very well 0.083* (0.045) 471 Knows how to talk to boy children about life and family -0.10 (0.21) 480 Boy children are generally able to talk about life -0.11 (0.22) 480 Knows how to talk to girl children about life and family -0.23 (0.20) 480 Girl children are generally able to talk about life -0.24 (0.22) 480 Highly knowledgeable on child development 0.086** (0.035) 474 Highly confident the caregiver can support the child 0.026 (0.036) 474 i. Regressions include controls for: education level of primary caregiver; age of HH head; number of children in HH; number of adults in HH ii. Statistical significance levels: * 10%; ** 5%; *** 1% ENDLINE PERFORMANCE EVALUATION | 67 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Table R6: Household Economic Strengthening Outcomes Outcome Pre-post comparison Standard Error Observations VSLA Number of household assets owned -0.17 (0.33) 326 Household income -1409.7 (4592.5) 321 Number of working adults per HH 0.18** (0.071) 326 Value of household savings 1720.2 (1826.0) 326 Total household expenditures on children 1286.5 (11259.6) 307 Expenditures on clothes for children 1997.9 (5001.7) 179 Expenditures on shoes for children -177.1 (2682.7) 116 Expenditures on education for children -2631.7 (5307.9) 233 Expenditures on medicines and medical care for children -3064.5 (9075.5) 262 FSG Number of household assets owned -0.22 (0.23) 480 Household income 3171.2 (1945.1) 479 Number of working adults per HH -0.017 (0.057) 480 Value of household savings 3335.2*** (454.7) 477 Total household expenditures on children -3936.9 (12059.6) 412 Expenditures on clothes for children 4633.7 (3358.7) 203 Expenditures on shoes for children 292.0 (2218.9) 111 Expenditures on education for children 2853.1 (3020.9) 273 Expenditures on medicines and medical care for children -22156.9 (15455.5) 324 i. Regressions include controls for: education level of primary caregiver; age of HH head; number of children in HH; number of adults in HH ii. Statistical significance levels: * 10%; ** 5%; *** 1% Question 4: Did the program bring out systemic changes at the community, provincial, and national levels that are enabling children to live in family care and preventing inappropriate placements in institutional care? Conclusions: Systemic improvements have resulted due to the project. The project was the first of its kind in Burundi to focus on the deinstitutionalization of children, and subsequently it has started a dialogue around such issues as minimum standards to alternative care and improving the social work workforce. The project has been a positive starting point. Further reinforcement of the social work workforce and a greater share of the national budget to social services needs to be made in order to continue and build upon the successes of this project in the future. The systemic challenges that remain include the fact that there is very little knowledge or awareness of key elements of national policies and strategies, especially at the sub￾national level among those working in the child care and social protection system. The ENDLINE PERFORMANCE EVALUATION | 68 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project high staff turnover rate and low budget available for CPEs makes it challenging to keep everyone informed of key child protection policies and strategies. This means that not all the laws to protect children are enforced. On the other hand, the majority of key informants interviewed were aware of the Minimum Standards for Children in Institutions, even though they were not being fully implemented, and they were working to support their implementation. These individuals for the most part, however, were trained by the project and therefore had greater awareness of the Standards. The key informants who had not been trained by the project and were not involved directly in its activities were not aware of them, indicating that a greater degree of awareness raising is needed on child welfare issues, including appropriate care placements for children, alternative care, and case management, at the national level and sub-national level. Implementation of the Standards has improved conditions in the RCCs that were identified in the mapping exercise in 2011-2012. The project began improving communication and collaboration between the Ministry of Solidarity and the Ministry of Interior to enforce the Standards although challenges remain and there is a need for a more harmonized and consolidated approach between the two ministries.33 The project started discussions to commence the drafting of the Guidelines for the Alternative Care of Children at the national level, and even though the drafting of the Guidelines has not yet taken place, the project supported and contributed to the development of a component on foster families. This initiative, while not yet finalized prior to the end of the project is an important step forward towards the drafting of the wider Guidelines for the Alternative Care of Children. It will be important in the future for the Government and other leading actors in child protection to ensure information about the drafting and contents of the national Guidelines for the Alternative Care of Children is not only widely shared and disseminated to the CDFCs and CPEs at the colline and commune levels, but also that staff members are trained and provided with technical support. Further, well-planned campaigns or awareness-raising initiatives need to be carried out to change public attitudes. Overall, while it was not possible to put into place the wider Guidelines and alternative care framework during the project’s lifespan, the efforts have been made to develop the Guidelines and the drafting of the foster family component are steps in the right direction.34 33 After completion of the draft of this report, IRC specified that since 2016, the two ministries have exchanged correspondence that the Ministry of Interior and Patriotic Training must consult with the Ministry of Human Rights Social Affairs and Gender before agreeing to approve new RCCs. No new RCCs have been approved since. 34 Handicap International, in comments to this report, mentioned that a guide for organizing trainings for parents for readaptation centers was developed and published although there was no mention of this during the Evaluation Team’s data collection. ENDLINE PERFORMANCE EVALUATION | 69 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project A recognized success of the project has been in identifying and registering children in the RCCs and in gaining the support of these RCCs to deinstitutionalize the children in their care. The project has also greatly improved the capacity, knowledge, and awareness of CDFC coordinators, which has strengthened the deinstitutionalization process, though reintegration still faces challenges due to the level of poverty, which is the root cause of many separations, as discussed elsewhere. While the general attitude was to support family-based care, and there was an evident knowledge base of actors involved in deinstitutionalization processes and case management, it is unlikely that many of these activities will continue without either an increased share of the Government’s budget to social services or the start-up of further donor-funded projects related to this topic. Over the project period, there was greater acceptance of family-based care over placement in RCCs. While caregivers (as demonstrated through survey findings) and almost all interviewed practitioners on the ground from the commune and province levels believed that children were better off growing up in a family, the sentiment among stakeholders interviewed and among some focus group respondents was that there was still a role for RCCs under certain circumstances. The few who do think it appropriate to place a child in an RCC largely cited reasons of poverty. Caregivers largely believed children with disabilities were better off living with families, although there was still some support for placement of children with disabilities in RCCs among those working within the child care and protection system. The project has improved parenting through behavioral education, and as a result, relationships between parents and children have also improved. This includes caregivers of children with disabilities who are now better able to care for and communicate with their children. There is still room for caregivers to better engage with their children, especially girls, and to become more confident in finding solutions to a child in need. Findings: a) Do professional and public attitudes show increased knowledge of and increased support of national policies that prevent unnecessary family￾child separation and promote appropriate family care for children without parental care? At endline, there was very little knowledge or awareness of key elements of national policies and strategies, especially at the sub-national level. The small number of individuals who were aware of the key national child protection policies or strategies in Burundi, such as the National Strategy for Child Protection (which was in place until 2016), or other national documents, such as the National Strategy for Street Children or ENDLINE PERFORMANCE EVALUATION | 70 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project the Plan of Action on the worst forms of child labor, were either respondents working for the INGOs or UNICEF or the more senior government officials. In regards to informing child protection actors at all levels on the policies, strategies, and legislation supporting children and their rights, the government staff who were interviewed at the national level maintained that the information was disseminated at all levels across the country. While the Director of the Child and Family Department at the Ministry of Solidarity stated that these documents are, in principle, available to everyone, the lack of awareness of them was apparent during the interview process. For instance, only one social assistant interviewed was aware of the documents. None of the other social assistants, child protection committee members, or directors of the residential care centers interviewed were aware they existed, which highlights a distinct disconnect between the central Government and its counterparts at the colline, commune, and provincial levels. An IRC staff member mentioned that there is not a culture of reading in Burundi, and even if the documents were available at the CDFC, it was unlikely that people would go there to read them. It was noted instead that there needs to be training and awareness raising in regards to the key elements of national policies and strategies concerning child protection in order for others to be made aware of them. One of the CDFC Coordinators underlined the challenge in widespread dissemination of key child protection policies and legislation by stating that with every election, staff change. She explained: “In the province of X alone there are 157 CPEs at the colline level, each with seven members, while at the commune level there are another nine CPEs. Then there are 23 members of the CPCPE. So, think about the numbers. It is a lot of people.” (CDFC Coordinator in one of the provinces) She maintained that not even all of the CPCPE (Provincial Committee for Child Protection) members were informed. While it is an achievement of the Ministry of Solidarity that there is such a large presence of CPEs volunteering to oversee child protection issues in their communities, it is not surprising that, given the logistical challenges and a lack of funding, not all child protection actors are informed and kept up-to-date with the wider child protection policy framework. The lack of awareness among many local authorities of legislation on children resulted in challenges in enforcement of those laws. The majority of stakeholders interviewed maintained that local authorities did not always respect laws in place to protect children. There is a collection of all the rights of the child under the Recueill des textes legislatifs et reglementaires relatifs aux droits des enfants as well as the Family Code, and penal procedures. While there is a national inter-ministerial committee that discusses child protection issues on a quarterly basis, as well as committees in the provinces, the CPCPEs, and the CPEs who operate under the provincial coordination committee, ENDLINE PERFORMANCE EVALUATION | 71 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project several respondents reinforced the point that it was difficult to inform all local authority staff because there are no funds to organize meetings to raise awareness on the laws. One respondent also mentioned that whether or not the laws are respected depends on the context as well as the particular case. There was an evident perception amongst key informants that there was a need to make information regarding legislation protecting the rights of children more widespread. The majority of stakeholders interviewed were aware of the Minimum Standards for Children in Institutions and are working to support their implementation. Through key informant interviews, several questions were posed to determine the overall knowledge of stakeholders involved in working with children living outside of family care at the colline, commune, provincial, and national levels. The questions posed sought to determine whether or not the project increased knowledge and application of national policies related to children living outside of family-based care, particularly the Minimum Standards for Children in Institutions. The majority of respondents were aware of the Standards and were working to support their implementation. There were several respondents who were not aware of them, including a Director of one of the residential care centers, two social assistants in Bujumbura Rural and Bujumbura Marie, and several of the community Child Protection Committee members. The individuals who were not aware of the Standards had, for the most part, not directly participated in the project’s activities. Most caregiver focus groups and focus groups with those working in the child care and protection system were aware of the national policy recommendation that children should not live in RCCs as long as there are other family options available – only one focus group with VSLA caregivers was unaware. Most focus groups felt positively towards the policy, with several focus groups saying that all children, even children with disabilities, were better off in a family, children were better educated in the family, and children must grow up in a family to learn how to work. There are varied levels of implementation of the Standards. Stakeholders interviewed and focus group respondents gave varied responses to whether or not the minimum standards for children in RCCs were being implemented. In general, stakeholders interviewed who were aware of them agreed that they were being implemented, albeit to varying degrees. Most stakeholders interviewed noted the challenges in implementation due to lack of funding to the social service sector in general. One stakeholder interviewed mentioned that he did not see a concrete plan to put the Standards into place by the Government. He noted that the Standards were being implemented solely as a result of the initiative of IRC and the Family Care First project, as well as the initiatives taken by UNICEF. A few focus groups felt that people were not aware of the minimum standards and so were not acting accordingly. ENDLINE PERFORMANCE EVALUATION | 72 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Among the respondents who were aware of the Standards, all acknowledged the positive impact their implementation has had on the conditions in many of the residential care centers that were identified during the mapping exercise in 2011 to 2012. It was agreed that while some centers still did not meet the Standards and a few were still reticent, many have now focused on the deinstitutionalization of children. They have also attempted to improve their overall conditions and policies, for instance, in keeping case files of each child. However, it was also mentioned by some respondents that there remain challenges between the communication and collaboration between the Ministry of Solidarity and the Ministry of Interior (MoI). Particularly, while the Standards exist and the Ministry of Solidarity can encourage the RCCs to respect and apply them, there are no repercussions for those centers that do not. More specifically, it is the Ministry of Interior that gives permission for a center to operate; however, it is the Ministry of Solidarity that has the responsibility for ensuring that the standards are respected. If there is a center that is not respecting them, the Ministry of Solidarity has no authority to shut down its operations. Under the FCF project, IRC has worked with its partners to advocate for the Ministry of Interior to address this issue. While some success has been achieved, a challenge remains that the staff of the MoI are not child protection staff and therefore do not always understand the issues at hand. Additionally, while efforts have been made in regard to advocacy and awareness raising at the ministerial level with staff from the MoI, the political crisis in 2015 resulted in high staff turnover, which meant that efforts had to be duplicated to train and inform new staff. Overall, the need for a more harmonized and consolidated approach between the Ministry of Solidarity and the MoI was apparent. However, a starting point in the coordination between the two ministries has been initiated as a result of the project, and it was mentioned by the FCF Coordinator that there is now a greater understanding amongst staff of the two ministries that there are issues concerning the well-being of children that need to be taken into consideration when opening a new center. A few respondents, including one of the CDFC Coordinators, also mentioned that some directors of the RCCs do not support the deinstitutionalization process because they believe that their center will be closed. She mentioned that gaining acceptance amongst RCC staff of the deinstitutionalization process is a task that requires continual reinforcement so that its benefits for the well-being of children are better understood. As a result, it was indicated by one respondent from IRC that there is a contradiction in the norms. What compounds the situation is that over the last two years, the overall budget of the Government has been reduced as a result of the current political situation. There is a need to continue efforts on development of the Guidelines for the Alternative Care of Children. At baseline, there were no policies or guidelines in place regarding ENDLINE PERFORMANCE EVALUATION | 73 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project alternative care, on foster care, or supported independent living. An IRC staff member interviewed at the baseline spoke of the need for a policy framework to continue the work on child protection policies towards development of an alternative care policy. IRC, under the FCF project, took the initiative to start this process with the drafting of the Guidelines for the Alternative Care of Children currently being discussed at the national level. The plan was to develop and adapt the international Guidelines for the Alternative Care of Children to the Burundian country context. The UNICEF respondents mentioned that a lot of advocacy work has been done at the national level to start the process of developing the Guidelines, but there has been an overall lack of political engagement and gatekeeping. In addition, the political crisis in 2015 caused a further delay and, combined with a weak social workforce across the country, has resulted in a context that has not been altogether favorable in regards to the initiative. While it was not possible to finalize the Guidelines prior to the end of the project, a component on foster families has been finalized and is awaiting validation by the Government. During the project, it became apparent that there was a need to have foster families in place as an alternative care option for children living in the centers or for any child in need of alternative care. Some of the other NGOs working in child protection, for instance with street children or repatriated children, have also placed children with foster families; however, there has been no standardized approach. IRC has been working with UNICEF and the Ministry of Solidarity to harmonize the approach used by organizations so that there are not only standardized procedures in place, but also to ensure that quality of care is monitored and adhered to. This component will eventually be incorporated into the national Guidelines for the Alternative Care of Children once they are drafted. At endline, the only respondents who were aware that the draft is currently being planned were those working at the national level of the Government, the CDFC Coordinators, and the IRC and UNICEF staff interviewed. At the baseline, family members and children themselves were regarded as the main players in deciding child placement. Focus group respondents at the baseline largely stated that family members (parents or extended family) are the ones who decide who will take care of a child and/or children if they are of a certain age. Other main players in decision making reported by baseline focus group respondents were CPEs or local leaders and neighbors. At endline, focus group respondents held differing opinions about whether or not the child should be involved in the placement decision. At the endline, extended family, the community, and the immediate family members were seen as the ones who were in charge of making the placement decision (see Table 4.4.1). There were several focus group respondents who felt that involving the child was the best thing to do. Respondents who believed that involving the child was a good thing said that doing so leads to a positive impact on the life of the child and the family. Other respondents in a ENDLINE PERFORMANCE EVALUATION | 74 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project social assistants focus group said that it is the child’s right to choose his or her foster family. Respondents in nine focus groups said that age is a factor in involving the child in the decision, and three said that maturity is a condition that would be considered before involving the children. However, four groups of children said that the children’s views are by and large not taken into account, with the older children said that only some children are consulted. Table 4.4.1: Placement Decision, NORC Focus Group Discussions Entity Involved in Placement Decision Number of Focus Groups who Mentioned Entity Breakdown of Focus Groups Extended family 10 FSG Caregivers (2); VSLA Caregivers (1); Colline CPE workers (1); Commune CDFC workers (1); Province CDFC workers (1); Children (3); Social assistants (1) Community 9 FSG Caregivers (1); VSLA Caregivers (2); Colline CPE workers (1); Province CDFC workers (1); Children (4) Biological family 8 Children with disabilities caregivers (1); VSLA Caregivers (1); Commune CDFC workers (1); Children (4); Social assistants (1) Foster family 4 Colline CPE workers (2); Children (2) Administration 4 VSLA Caregivers (1); Commune CDFC workers (1); Children (2) CPE or CDFC 4 Colline CPE workers (2); Commune CDFC workers (1); Province CDFC workers (1) Police 1 Province CDFC workers (1) NGOs 1 Province CDFC workers (1) Social assistants 3 VSLA Caregivers (1); Social assistants (2) Person who finds the child 1 Children (1) Source: NORC Endline Focus Group Discussions b) Is there an improvement in caregivers’ parenting skills and practices? According to those working in the child care and protection system, behavioral education has resulted in improved parenting, although there is still room for improvement. The FCF project involved raising awareness amongst parents to convince them that family-based care is in the best interests of the child. Respondents in both focus groups of social assistants noted that the behavioral education parents received resulted in positive changes amongst parents. One focus group with social assistants felt this resulted in a decrease of child separation, while the other focus group with social assistants felt there needed to be more training and educational efforts targeting parents. There was a lot of support for parental behavior education. Several focus groups highlighted that parents must be informed of the importance of child education, even while the child is in an RCC and particularly when the child leaves the RCC and ENDLINE PERFORMANCE EVALUATION | 75 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project returns home. A focus group of CDFC workers felt that awareness raising sessions helped to follow-up with families, while a focus group with colline CPE workers suggested that education and awareness-raising should happen at the national level. Caregivers and children also report improved parental behavior. A number of observations made about the impact of the Family Care First Project (which included a Healing Families and Communities training module for both the VSLA and FSG households) had to do with relationships with the parents of children in the project. Improved communication was a commonly mentioned impact, with many focus groups of children saying that they are now speaking with their parents more frequently and are now joking and playing with their parents, although a few children did report that there was no impact from the project on communication. Many focus groups of children also reported that the parents no longer use corporal punishment. Several focus groups also reported that parents now listen to children. Caregivers themselves said that they felt a stronger affection or were generally closer to their children. On the household financial management side, many focus group respondents felt that parents had learned how to better manage their finances. In addition, several focus group respondents also mentioned that parents had stopped drinking, which helped financially and most likely improved relations in the household. Additionally, some focus group respondents reported that now parents are more careful to maintain hygiene. While results show that positive parenting practices have increased, some respondents mentioned that the parental capacity to engage with children still needs improvement. The evaluation survey asked caregivers questions on household protective factors such as the ability of caregivers and children to engage with each other – i.e. the ability of caregiver to “generally talk to children about important things in your life and in the family” (Table 4.4.2) and caregivers’ perception that “children in your care are generally able to talk to you about important things in their lives” (Table 4.4.3). The results generally show a large positive improvement from the baseline, with a much larger improvement for boys than girls. The percent of caregiver respondents who reported always being able to talk to boys rose more (11 points for both VSLA and FSG households) as compared to girls (an increase of 5 points for VSLA households and 3 points for FSG households). The percent of caregiver respondents who reported that they could never talk to boys dropped by more (21 points for VSLA and 24 points for FSG) as compared to girls (12 points for VSLA and 8 points for FSG). Despite the improvement, the results show more work can be done on engaging with children, especially girls. At endline, only 12 to 30 percent of VSLA and FSG caregivers answered “always” or “most of the time” to questions regarding being able to talk with the boy/girl child. An exception at the endline was the dramatic improvement in the ENDLINE PERFORMANCE EVALUATION | 76 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project perception that boys are generally able to talk to caregivers the about important things in their lives “always” or “most of the time” (60% for VSLA and 50% for FSG caregivers). Table 4.4.2: Caregivers’ Ability to Share with Children in Their Care Response VSLA FSG Baseline Endline Change Baseline Endline Change Do you feel that you are generally able to talk to the boy children in your care about important things in your life and in the family? Always (%) 7 18 +11 9 20 +11 Most of the time (%) 24 30 +6 26 24 -2 Some of the time (%) 36 29 -7 34 38 +4 Rarely (%) 9 5 -4 8 5 -3 Never (%) 25 19 -6 23 13 -10 Do you feel that you are generally able to talk to the girl children in your care about important things in your life and in the family? Always (%) 9 14 +5 9 12 +3 Most of the time (%) 33 17 -16 30 19 -11 Some of the time (%) 23 34 +11 33 39 +6 Rarely (%) 13 11 -2 9 9 +0 Never (%) 22 23 +1 19 21 +2 Source: NORC evaluation survey Table 4.4.3: Caregivers’ Perception of the Children’s Ability to Share Important Things Response VSLA FSG Baseline Endline Change Baseline Endline Change Do you feel that the boy children in your care are generally able to talk to you about important things in their lives? Always (%) 5 20 +15 4 19 +15 Most of the time (%) 27 40 +13 17 31 +14 Some of the time (%) 27 22 -5 34 33 -1 Rarely (%) 13 10 -3 12 8 -4 Never (%) 29 8 -21 34 10 -24 Do you feel that the girl children in your care are generally able to talk to you about important things in their lives? Always (%) 6 18 +12 7 16 +9 Most of the time (%) 30 30 -0 23 22 -1 Some of the time (%) 27 28 +1 29 32 +3 Rarely (%) 10 11 +1 16 13 -3 Never (%) 25 13 -12 25 17 -8 Source: NORC evaluation survey There was an increase in FSG caregivers’ perception of themselves as knowledgeable on child development issues; perception was already high at baseline among both VSLA and FSG caregivers. At baseline, the majority of caregivers in both VSLA and FSG households considered themselves as “somewhat” or “highly knowledgeable” of ENDLINE PERFORMANCE EVALUATION | 77 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project issues important for a child’s development (Graph 3; 83% for VSLA and 75% for FSG). FSG caregivers ‘caught up’ with VSLA caregivers at endline, increasing their self￾perception of being “highly knowledgeable” by 11 percentage points, while VSLA caregiver numbers did not notably change. At endline, 85% of VSLA caregivers and 84% of FSG caregivers considered themselves as “somewhat” or “highly knowledgeable” of issues important for a child’s development. Graph 3: Self-Assessment of Knowledge of Issues Important for a Child’s Development Source: NORC evaluation survey Caregivers expressed low confidence in the ability to find solutions for a child in need at both baseline and endline, but confidence did rise slightly between the two times for VSLA and FSG caregivers. Even though slightly over 60% of caregivers in both VSLA and FSG households considered themselves to be “somewhat knowledgeable” about issues important for a child’s development at both baseline and endline, this did not translate into confidence in their ability to find solutions for a child in need (see Graph 4 below). There was some improvement in caregivers’ confidence over time, with VSLA confidence rising slightly from 51% at baseline to 54% at endline. FSG caregivers remained less confident than VSLA caregivers, but their confidence rose more significantly from 39% at baseline to 48% at endline. Increases in confidence were calculated by combining reports of medium and high confidence in their ability to find solutions for a child’s needs. 22% 23% 11% 22% 61% 62% 64% 62% 17% 15% 24% 16% 0% 10% 20% 30% 40% 50% 60% 70% Baseline Endline Baseline Endline VSLA FSG Highly knowledgeable Somewhat knowledgeable Not at all knowledgeable ENDLINE PERFORMANCE EVALUATION | 78 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Graph 4: Confidence in the Ability to Find Solution for a Child in Need Source: NORC evaluation survey c) Is there any change in attitude towards residential care among parents, extended family and community members? At the end of the project, family-based care was viewed as the most appropriate care option for children; there is now greater acceptance of family-based care over placement in RCCs. While many actors at the sub-national level were not aware of child protection policies at the national level or the Minimum Standards for Children in Institutions, there was an overall perception and agreement amongst actors that family￾based care was the most appropriate care option for children. Many stakeholders interviewed continually emphasized the perceived change in mindsets relating to the institutionalization of children, stating that there is now a greater acceptance to finding solutions for families at risk of separation or who have separated that entail family￾based care as opposed to placement in residential care centers. Some of the IRC staff interviewed noted that the biggest impact that they have witnessed is the overall change in mentality regarding family-based care. One staff member explained that at the beginning of the project, some people were “angry about what we were saying, but after the awareness raising they now understand the ideal place for the child is with the family and not with the centers.” In addition, from discussions with government officials from the national to sub-national levels, there was a willingness to continue promoting awareness of the importance of family-based care amongst parents and community members, as well as to continue strengthening the foster care system, identifying and training foster families for use as temporary care options. At endline, almost all caregivers believe children are better off living with families rather than in RCCs and this increased by about 10 percentage points over the project period. An overwhelming number of caregivers believe that children are better off living in family 20% 21% 15% 18% 31% 33% 24% 30% 50% 46% 61% 52% 0% 10% 20% 30% 40% 50% 60% 70% Baseline Endline Baseline Endline VSLA FSG High Medium Low ENDLINE PERFORMANCE EVALUATION | 79 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project care rather than in RCCs (Graph 5 below). Despite being more financially vulnerable, nearly the same percent of FSG (80%) and VSLA (82%) caregivers felt that children are better off living with families rather than in RCCs at baseline, and at endline a higher percent of FSG (94%) caregivers believe that children are better off living with families rather than in RCCs compared to VSLA (93%) caregivers. Graph 5: Caregivers’ Perception on Where Children Are Better Off Living Source: NORC evaluation survey Focus group respondents at both the baseline and endline also confirmed the majority point of view that children are better off in families rather than RCCs. A number of solutions were proposed by focus group respondents when asked about the best possible alternative care options for children. One of the prevailing attitudes put forth by focus group respondents at endline was that family was the best option for raising a child, for reasons that ranged from the child’s happiness, better education, and the benefits it has for the child’s development. All of the government KII respondents reported that family-based care was the official policy of the Government; however, the Counselor in Charge of Child Protection at the Ministry of Solidarity contradicted this assertion and explained that it was not yet an official policy, but the plan is to make it one under the Guidelines for the Alternative Care of Children. Disadvantages of RCCs were reported as losing family and community connections, not learning certain life skills, and a lack of affection. Some focus groups mentioned that children placed in an RCC risk losing their connection to community and to family property. One group of FSG caregivers even said that children who enter RCCs forget their family members. 82% 93% 80% 94% 18% 7% 20% 6% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Endline Baseline Endline VSLA FSG Families RCC ENDLINE PERFORMANCE EVALUATION | 80 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project A large number of focus groups also felt that children cannot be educated properly in RCCs and that RCCs did not provide children certain key life skills such as responsibility. A specific example was that children in RCCs do not learn manual labor necessary for working hard, which troubles some respondents who think that learning manual labor is important for reintegration. A few baseline focus groups and some endline focus groups highlighted that RCCs do not provide children much affection. There is limited support among caregivers for placing a child in an RCC. The strong belief that children are better off living with their families is also confirmed by the fact that only 1/5 (20 percent) of caregivers surveyed (both VSLA and FSG) stated that they would ever consider placing their children in an RCC at baseline, and the figures dropped to less than 15% for both groups at endline. Of those responding that they would consider putting a child in an RCC, the primary reason cited was poverty for both VSLA and FSG households. It is interesting to note that the percent of respondents who listed poverty as the main reason dropped for both VSLA and FSG respondents, by 2 and 3 percentage points respectively. VSLA households show an increase of respondents who reported that they would put a child in an RCC due to a death of a parent. The percent of VSLA respondents who cited a child’s disability as the main reason to put a child in an RCC also rose, though only by 3 percentage points. FSG responses saw a 10 percentage point decrease, down to 0%, of respondents who cited a child’s behavior as the main reason for placing a child in an RCC. FSG households also saw an increase, from 0% to 7%, in education as the main reason for placing a child in an RCC. Table 4.4.4: Reasons for Placing the Children in RCC (%; Of Those Who Would Consider Placing Child in RCC) Caregiver Consideration and Reason for Placing Children in RCC VSLA (%) FSG (%) Baseline Endline Change Baseline Endline Change Would you ever considered placing your children in RCC? Those saying “yes” 19 13 -6 20 12 -8 What would be the main reason for making this choice (of those saying “yes” they would consider placing children in RCC) Divorce / remarriage of parent 5 0 -5 2 0 -2 Abandonment 0 0 0 2 0 -2 Family separation due to natural disaster 0 0 0 2 0 -2 Death of parent 2 9 +7 10 11 +1 Poverty 65 63 -2 53 50 -3 Family violence 3 3 0 6 4 -2 Family illness 3 3 0 6 7 +1 Child’s disability 19 22 +3 18 21 +3 Child’s behavior 2 0 -2 10 0 -10 Education 0 0 0 0 7 +7 Source: NORC evaluation survey. ENDLINE PERFORMANCE EVALUATION | 81 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project While almost all caregivers believe children with disabilities are better off in families, there is still some support among those working within the child care and protection system for placement of children with disabilities in RCCs. Burundi has recently ratified the UN Convention on Persons with Disabilities, and the Government is in the process of establishing a national policy for children with disabilities as well as legislation on this subject. All focus groups with caregivers were in agreement with the national policy of keeping children with the family. In particular, both focus groups of caregivers of children with disabilities believed that the ideal form of care for the child was living in the family. Some caregivers of children with disabilities and members of the child care and protection system specified that even if children are educated at an RCC (because public schools are not yet equipped to handle children with disabilities), they should live with their families instead of being housed at the RCCs. All of the stakeholders interviewed, apart from one of the CPE Presidents, agreed that family-based care was also the best place for children living with disabilities. The respondents also confirmed that all of their co-workers held this view within their respective ministries or organizations and that no one held divergent views. A stakeholder at the Ministry of Solidarity stated: “Now everyone understands that we must protect children. All of the ministries understand, and there has been a convergence.” (Stakeholder at the Ministry of Solidarity) A focus group of RCC workers reiterated that the national policy requires deinstitutionalization, even for children with disabilities. However, an RCC in Makamba said that while children with disabilities receive shelter and education from RCCs, it is difficult for RCCs to integrate the disability component of child care. On the other hand, several focus groups of those working within the child care and protection system and many focus groups of children said that children with disabilities requiring specialized care should stay in RCCs. Several focus groups with those working within the child care and protection system said that RCCs were suitable and helpful for certain categories of children, including children with disabilities and HIV￾positive children. There also continues to be some very limited support for placement of children who do not have disabilities in RCCs in certain cases of poverty, a mentally ill parent, and children under the age of two. The survey of caregivers showed that while at endline only about 13% of caregivers would ever consider placing their child in an RCC (see Table 4.4.4 above), the main reason they would do so is poverty (63% for VSLA and 50% for FSG households). Two of the Presidents of the CPEs interviewed (one who had participated in the project and one who had not) mentioned that if parents could not provide for a child in terms of education and food, then it was better to place the child in ENDLINE PERFORMANCE EVALUATION | 82 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project a residential care center. RCCs were also regarded as being better for babies under the age of two by a few focus groups of those from the child care and protection system. One focus group with province CDFC workers said that sometimes children are born to a mentally ill parent and that the best place for them is in an RCC. Another praised aspect of RCCs was the educational options they provide to children. According to one focus group of colline CPE workers, RCCs can educate children without difficulties. Several focus groups with children and one focus group with RCC staff also made a point to say that children in RCCS have the chance to be educated, implying that those children in RCCs may not otherwise have the opportunity. Several stakeholders interviewed agreed that many parents believe that placing their children in RCCs is in their child’s best interest because RCCs will provide schooling and food. One stakeholder interviewed stated that there was a need to change people’s mentality so that the money put into running institutions was instead used to place children back in their communities and support their families. Kinship care was reported as the most common alternative care option for children who could not live with their biological families. Extended family was the most common answer for who takes care of children. RCCs were another popular answer. Several focus groups of children and one focus group with FSG caregivers said the community should raise separated children. A few focus groups with caregivers mentioned foster families as the perceived guardians of separated children. Negative stereotypes exist of children who live in RRCs. At the baseline, it was found that some focus group respondents felt that children from RCCs have behavior problems such as not respecting parents or guardians, stealing, drug abuse, and prostitution and unwanted pregnancies. Focus groups with children at endline revealed that those who live in RCCs are still viewed quite negatively. Children brought up words that are commonly associated with separated children, and included terms like “homeless and dirty/beggars”, “uneducated”, “thieves/delinquents”, “worthless”, “prostitutes”, “disabled”, and “abused”. Land ownership issues were said to inhibit the uptake of kinship and foster care at the baseline; this issue remains the same at endline. Several stakeholders interviewed at baseline mentioned that problems with land inheritance were the main barriers to foster and kinship care. The Gitega social assistant interviewed at baseline mentioned that there was no strategy for foster care. The absence of a legal framework around foster care and adoption that addresses inheritance is a barrier. There are people who would do foster care but cannot because there is no security for the child or the fostering family, especially in relation to legal rights such as inheritance: ENDLINE PERFORMANCE EVALUATION | 83 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project “A child growing up without a family is not good. We could look for foster families. But in Burundi the problem is land. I can take a child without the condition that the child will inherit their own land. But I can’t if it means that my own children won’t inherit.” (Gitega social assistant, baseline) Terre des Hommes referred to the potential for commune administrators to take a greater role in identifying foster families. However, according to those interviewed, it is not entrenched into their role and not in their current terms of reference. The Dutabarane representative interviewed at baseline mentioned that their organization is addressing this issue by supporting 2,000 foster families (in another project which has now ended) – identified with local administration and churches who check that the children are treated well in foster care. A CDFC coordinator interviewed at baseline felt there needed to be a change in attitude in Burundi regarding foster care: “…we need to raise awareness among Burundians about child care (fostering). If not, we will have to do inter-country adoption! I have two adopted / fostered children and one of them is about to get married. We [Burundians] don’t have this mentality”’ (CDFC Coordinator, baseline) At endline, the Child Protection Expert’s interviews with stakeholders and field visits revealed that these issues remained. d) Has the government capacity to provide oversight on the standard of care in existing residential care facilities improved? There continues to be little will or mechanisms in place to hold RCCs accountable to standards of a national child protection system or the current child protection policy. Stakeholders interviewed at baseline demonstrated a minimal sense of need for accountability to standards of a child protection system at a range of levels – from national government at the ministry level, to actors working directly with children, and civil society organizations. One of the representatives of the MDPHASG interviewed at baseline noted when asked about plans for accountability around RCCs – and specifically what would happen if RCCs did not adhere to national standards – that: “It is not yet in the real plan. It will be important to have an understanding of the difficulties.” While the official plan is that every RCC is to be inspected every six months and measured against the national Minimum Standards, in practice this appeared to be far less frequent at baseline – the Ministry of Interior interviewed noted that only one RCC ENDLINE PERFORMANCE EVALUATION | 84 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project was inspected every few months.35 Overall, the situation seemed to remain the same at the endline. However, according to IRC, at the beginning of the project only four provinces were regularly participating in monthly meetings to ensure follow up on minimum standards at RCCs, whereas at the end of the project, all ten provinces were participating in these meetings, and meeting frequency had increased from once a month to twice a month. A success of the Minimum Standards is that children in RCCs have been identified and registered. The greatest success of the Standards to date, and one recognized by all stakeholders interviewed who were aware of them, is that children living in RCCs in the areas of operation have been identified and registered, and many of the centers have agreed to support the wider deinstitutionalization of children living under their care. This accomplishment is due to the increased knowledge and awareness of the CDFC Coordinators on the importance of family-based care. Prior to the project, they did not understand the process or importance of deinstitutionalization. It was mentioned by IRC protection staff that the CDFC Coordinators now have the knowledge to work more independently on case management. The two CDFC Coordinators interviewed were both well informed and knowledgeable of the overall process of deinstitutionalization and subsequent case management. Regardless, the challenges that remain are multiple. While it is one step to identify and register children, it is another to support their reintegration back into their biological families who often live in poverty and without the means to provide for their children’s most basic needs. One respondent concluded that there is still a long way to go before the standards are fully implemented throughout the country. Conflicting incentives may work against family-based care. There were mixed views in response to the question of whether or not actors responsible for the protection of children were all acting in the best interests of the children. The majority of stakeholders interviewed agreed that not all of them are because some people, primarily the directors of RCCs, may have an economic interest at stake, and therefore, do not have an interest in promoting family-based care. Project staff agreed that motivating the Government, local partners, and the RCC staff to engage in deinstitutionalization efforts with equal levels of commitment was a challenging task. There was a good deal of discussion amongst focus group respondents about the roles and responsibilities of the RCCs and other related actors in the child protection system. 35 There is an “acte d’engagement,” an accountability sheet that is signed by CPEs at all levels. Those that do not follow this are to be replaced – at least in theory. At baseline it was reported that in reality replacements did not happen much. Some CPEs are active, while others are not. Some only exist on paper, while others have been trained, but don’t know their job responsibilities. At the national level the Ministry asks the members of the CPCPE to submit a report of their work every 6 months. However, submissions are not done by volunteers. ENDLINE PERFORMANCE EVALUATION | 85 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project The objectives of RCCs were said to be to educate children, provide care for children with disabilities, help children gain autonomy, and “help vulnerable children and orphans”. In terms of care, both focus groups with RCC workers saw themselves as responsible for tailoring the care of each child depending on their case. Regression Analysis Question 4: Did the program bring out systemic changes at the community, provincial, and national levels that are enabling children to live in family care and preventing inappropriate placements in institutional care? To understand the improvements in making community-level changes that enable children to live in family care, we ran regressions on the caregivers’ knowledge of where to get support to better care for the family and their opinions on children’s’ well-being in RCCs as compared to family care. These results are presented in Table R7. The regression results show positive improvements for both cohorts, though the improvements were larger for FSG than for VSLA. FSG caregivers showed a 20% improvement in knowing where to get support to better care for their families, as compared to a 7.3% improvement for VSLA families. The FSG cohort also showed larger improvements in their propensity to believe that children are better off living in families over RCCs, with 16% improvements in FSG as compared to 11% in VSLA. Finally, the FSG group had statistically significant reduction of 8.2% in their propensity to consider placing their own children in an RCC. Table R7: Caregiver Knowledge Regression Results, Pre/Post Comparison Knows where to get support to better care for the family Believes children are better off living in families over RCCs Would consider placing their child in an RCC VSLA Pre-post comparison 0.073** 0.11*** -0.030 (0.031) (0.039) (0.042) Observations 326 326 325 FSG Pre-post comparison 0.20*** 0.16*** -0.082** (0.033) (0.032) (0.035) Observations 479 480 475 i. Regressions include controls for: education level of primary caregiver; age of HH head; number of children in HH; number of adults in HH ii. Statistical significance levels: * 10%; ** 5%; *** 1% ENDLINE PERFORMANCE EVALUATION | 86 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Question 5: By project end, to what extent have functioning structures been established that can continue to provide on an ongoing basis adequate case￾management services for children at risk? Conclusions: Overall, the project supported the strengthening of the child protection system. While challenges remain, the project reinforced the capacity of actors responsible for the protection of children at all levels who were directly supported under the project, including the CPCPE members, CDFCs, and the CPEs. The project has reinforced the child protection structures and linkages in place at the colline and commune levels. The mapping of available services and subsequent referral mechanism at the colline level helped actors make referrals, thereby strengthening the referral mechanism at the colline and commune levels. Additionally, community members are now more aware of the CPE structure and are better able to take more advantage of the support it offers. Links were also made between the various ministries responsible for protecting children, and while there were challenges in harmonizing the approach of deinstitutionalization amongst the Ministry of Interior and the Ministry of Solidarity in particular, progress was made. Time will tell if this collaboration will continue given the fragile political situation in the country, but it is hoped that conversations around alternative care will continue. The project was not able to address the child protection funding problem, which was outside of the scope of the project. The limited budget allocation at the national level to the social service sector in general and designated to the CDFCs and CPEs in particular is constraining the ability of the child care and protection system to function well. This is especially evident for CPEs and social assistants who are not able to ensure the needed follow-up of cases, and who generally reported feeling unsupported by officials at the national level. There are too few social assistants, with only one or two per commune, and they lack the materials and resources, such as computers to work with electronic records36. Additionally, the resources are so insufficient that they are not able to print paper forms without external project support, meaning that they will not be able to keep case files for children in the future unless a greater share of the budget is allocated to child welfare activities or another project is able to cover these costs. The project carried out advocacy activities to encourage a budget line specifically for the protection of children, but it had not led to a result by project end. 36 The project did provide two CDFC Coordinators with laptops because they did not have computers. ENDLINE PERFORMANCE EVALUATION | 87 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project The sustainability of case management and follow-up is in question. While the project supported the case management system, it is not at a point where it can continue to function independently. The CDFC was still reliant on IRC to provide the tools and forms for case management and the project was facilitating follow-up visits that are unlikely to be sustained at the same level with the continued lack of funding. There is a need to integrate the CPEs into the system so that they can play a bigger role in following up with families with reunified children or families at risk of separation. Additionally, there is no standard approach to case management at the national level, which will be needed to ensure all child protection actors are using the same operating procedures and tools. While the project did improve the knowledge and awareness of CDFCs, CPEs, and social assistants, additional training and awareness raising is still needed. The knowledge is not being transferred beyond those directly trained by the project and, coupled with high staff turnover, has left CPEs and social assistants feeling untrained to fully address all their responsibilities. Greater communication and scale down from the national to sub-national level between the various actors would help with knowledge transfer. Findings: a) Do government authorities and state and non-state service providers have adequate attitude, knowledge and skills to build family resilience, involve, support, and protect children at the local level? CENTERS FOR FAMILY AND COMMUNITY DEVELOPMENT, CDFC CDFCs were created in December 2012 and are the decentralized provincial representatives of the MDPHASG. Part of their mission is to improve the capacity of child protection personnel through coordination of the CPEs at the provincial, communal, and colline levels and support CPEs to achieve their missions.37 As learned through stakeholder interviews, child protection is a relatively new responsibility – the CDFCs were initially created as a structure that focused on women’s rights, gender equality, and addressing gender-based violence. At baseline, it was clear that CDFCs working with IRC had received only some basic training on their role in child protection. According to interviews with provincial CDFC and CPCPE members at baseline, CDFC priorities appeared closely correlated with local funded NGO initiatives, rather than a ministry-driven process of expanding CDFC action on child protection. In baseline stakeholder interviews, CDFCs were almost universally noted as key to implementation, with the exception of RCC managers, who did not talk about the 37 Information obtained from the terms of reference for CPCPEs, commune CPEs, and Colline CPEs received from IRC. ENDLINE PERFORMANCE EVALUATION | 88 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project government structures. Terre des Hommes, for example, noted that the CDFC presence made a difference, but it depended on CDFC representatives being fully in place, receiving training, and their individual interpretation of their role, given such a wide mandate. According to baseline stakeholders interviewed, there was a need for more coaching at the CDFC level for them to take the lead with CPEs. Where this capacity building did happen, the big investment led to some difference – the CDFC met regularly and child protection was discussed in the collines, child protection was being included in commune and colline-level development budgets, and the VSLA was working well. At endline, stakeholders interviewed reported increased knowledge and awareness of the CDFC Coordinators on the importance of family-based care. Prior to the project, they did not understand the process or importance of deinstitutionalization. It was mentioned by IRC protection staff that the CDFC Coordinators now have the knowledge to work more independently on case management, which is done at the commune level under close supervision. CDFC members need more financial and technical support. Areas that needed improvement according to focus groups with CDFC members were financial supports, training and awareness raising, techniques for identifying vulnerable children, and inadequate staffing. CHILD PROTECTION COMMITTEES, CPE CPEs were created through the same policy as CDFCs, when MDPHASG developed a decentralized structure. The CPE is one of the main structures of implementation for the national policy for child protection. The CPE structure exists at the province, commune, and colline levels. Where present, they are the commune and colline-level entity for the ministry but, unlike the provincial-level CDFC, they have no salaried staff within them. They operate on a non-remunerated volunteer basis and sign a code of conduct. Provincial Committees for Coordination of Child Protection (CPCPEs) consist of province level representatives from the health (doctors and HIV representatives), education, justice (police, tribunal courts), prison, NGOs, media, and religious group sectors, two children elected to the Children’s Forum, presidents of commune CPEs, the CDFC coordinator, and the provincial government socio-cultural counsellor. As one baseline CPCPE focus group explained, they meet once every quarter to review reports from colline and commune CPEs and then send the reports to the provincial governor who then passes them on to the Ministry of the Interior and the CDFCs give the reports to the Ministry of Solidarity (focus group with CPCPE in Bujumbura Rural).38 Commune 38 At baseline, IRC clarified that in reality the CPCPEs rarely meet, unless the meetings are organized and paid for by an NGO. At endline, while the Evaluation Team did not pose a direct question about ENDLINE PERFORMANCE EVALUATION | 89 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project CPEs are composed of the same multi-sectoral mix of actors as the CPCPEs but at the commune-level.39 According to baseline focus group respondents, they help vulnerable children access basic social services such as helping children get issued a certification of their poverty status that helps them in seeking medical services and free education. Colline CPEs are composed of the chief of the colline and men, women, and children from the community.40 According to focus group respondents at the endline, CPEs are responsible for the referral, care, and reintegration of children into families. In terms of referral, the CPEs are reportedly in charge of identifying cases, referring cases to the proper authorities, and collaborating with social assistants and the administration. The care of the children is said to involve protecting the rights of children, listening to children’s grievances and finding solutions, and combating child labor. Reintegration duties involved were said to be bringing the child back to the family of origin and following up with the families after reintegration is complete. Training parents and families was mentioned as another responsibility of the CPEs at endline, with the training focusing on reconciling families at risk of separation, educating families, raising awareness of child welfare, and providing parents of children with disabilities with referrals to PRCs so that they can attend their parental training sessions. According to both of the focus groups with colline CPE workers, everyone is aware of the role of CPEs, and there are high levels of commitment on all fronts. Stakeholders interviewed at both baseline and endline felt that CPEs played an important role in protecting children at the colline level. At baseline, Terre des Hommes highlighted their role in referrals – identifying and providing support to children at risk – and a CDFC at commune level complimented the multi-sectoral nature of their work. Challenges experienced by CPEs include a lack of financial resources available for use, lack of knowledge about the child protection system, issues with management, and a general lack of support from external sources. Several focus groups noted how the lack of financial resources affected the ability of the CPEs to monitor children. Specific examples given were an inability to travel to cases and to generate training materials. The lack of knowledge about the child protection system within the CPEs was seen as a prominent problem in several focus groups. The lack of CPE capacity was documented at baseline and does not seem to have been fully addressed by endline. A number of stakeholders interviewed at the baseline had noted concerns about the capacity of whether they actually meet quarterly, none of the stakeholders interviewed independently mentioned anything regarding the frequency of CPCPE meetings. 39 Terms of reference for commune CPEs and confirmed by baseline focus group respondents. 40 Terms of reference for colline CPEs. ENDLINE PERFORMANCE EVALUATION | 90 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project CPEs to meet the needs of vulnerable children. The Handicap International representative interviewed at this point was critical of their ability to deliver on the ground. His conclusion was that delivery depends on a greater level of ministerial desire to work on the issues and a need to confront the structural antagonism between ministries and between departments within the Ministry of Solidary, all of whom rely on the CDFCs: “In Gitega, the CPE is not so much a group, more a ‘system’ that is flexible, so it depends on the leader. It can work well. The CDFC coordinator and the assistant social are fundamental. They [CPEs] are there because the CDFC believes in them.” (Handicap International representative, baseline) CPEs themselves also felt at baseline that they had not received enough training, particularly in child protection (eight out of eight baseline focus groups with province, commune, and colline CPEs) and that they lacked the capacity to monitor the implementation of the child protection policy. CPE management was also seen as an issue among many focus groups; these issues involved feeling overwhelmed by cases, a lack of formal process for exchanges of information, poor organizational structure within the committees, and a lack of direction for case management. Lastly, there was a general sense of lack of support for CPEs that was mentioned by both colline CPE focus groups and one of the focus groups with social assistants. The colline CPE focus groups explained that they did not have a large amount of support from IRC or from the community. Respondents suggested reinstitution of reports, additional financial and material support, and additional training and technical support for CPEs. There were several solutions posed by focus group respondents that addressed the management issues as well as the need for more resources to assist operations. A focus group with colline CPE workers said that it would be helpful if reports that kept stakeholders informed were reinstituted. That same group, as well as another focus group with colline CPE workers, said that CPEs required additional financial, technical, and material support to improve operations within the committees. Improved communication between agencies was also mentioned as a possible solution to the current challenges faced by CPEs, and better training of the actors involved in the CPEs was also proposed. ASSESSMENT AND REFERRAL The following paragraphs describe the assessment and referral system at project end. The referral system draws on sources such as the community, families who bring their children into RCCs, colline CPEs, and female leaders/socio-cultural advisers. The rate of referrals was viewed differently by each focus group. A focus group with colline CPE workers said they receive one case a week, another focus group of colline CPE workers ENDLINE PERFORMANCE EVALUATION | 91 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project said that they are overwhelmed by the number of cases, and one focus group of province CDFC workers said that the rate of referral has decreased because the CDFC takes time to assess cases.41 Assessment involves reporting, placement, and follow-up. Reporting to local authorities is very common in the assessment process, according to three focus groups. Four other focus groups said that when a child needs care, their case is reported to CDFC. Placement requires the CDFC to locate the family for child placement, according to three focus groups. However, one focus group with Commune CDFC workers says that the CDFC lacks financial ability to advocate for and place children. Also falling under placement is assessing children based on the severity of their disability in order to place them with a family. Lastly, placement was said to involve reunifying children and families after the family is identified. Follow-up services entail providing moral support of the child and family post-placement as well as material support for host families. The follow-up process also involves the services of a socio-cultural advisor, who monitors the cases. According to the Counselor in Charge of Child Protection, when a child is placed in a center, the social assistant in the commune as well as the administrative authority should be involved, particularly if it is a case of child abandonment. A case file will then be opened, and a social assistant will place the child in a center as a last resort. However, the CDFC is only aware of these cases if the child is referred to them. The majority of respondents agreed that it is the CPEs and the Chef de Colline who are most aware of cases of family separation, child abandonment, and if a child is living in kinship care, highlighting the important role that community members play in overseeing the well-being of children in their communities. According to the Counselor in Charge of Child Protection, there is a social protection strategy in place that has criteria for defining what makes a family vulnerable. FCF had its own set of criteria for defining such families, which were validated during the project launch by administrative authorities and other actors in child protection including INGOs and CBOs and UNICEF, which were adapted based on the context in each of the target provinces in collaboration with local authorities and community leaders. Once again, respondents mentioned the important role that the CPEs and the Chef de Colline have played in identifying families at risk of separation. One of the social assistants explained that once a family who is in a difficult situation is identified, the CPE members write a report at the Colline level, which is sent to the CDFC at the commune level. They also try to give some support and guidance to the family. One of the CPE members added that: “helping the family is very difficult, perhaps we can give something small, but it is 41 The colline CPE does not manage a case in the formal process; he/she is a focal point per their terms of reference. ENDLINE PERFORMANCE EVALUATION | 92 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project difficult” to provide material support. It is likely, however, that even providing verbal support and being a supportive presence in the community helps families who might not know where else to turn. CASE MANAGEMENT FCF has supported the creation and management of case files in addition to follow-up. When asked about their case management methods, stakeholder responses varied. All of the RCC directors stated that they kept a case file for the children under their care with basic identification information, a starting point in a case management system. The family tracing and reunification process has been facilitated by the CDFC with logistical and technical support provided by the FCF project and IRC staff. IRC, under the FCF project, has provided the CDFC with supplies and forms to complete so that a case file for each child is kept at the CDFC level. Logistical support has also been provided to social assistants and the CDFC Coordinators to carry out activities on the ground. Social assistants feel spread thin and that they have more responsibilities than are reasonable given the demands of the community. According to focus group respondents, there was one social assistant per commune facilitated by the provincial CDFC42; social assistants were responsible for identifying cases and they collaborated with CPEs to report cases to the CDFC. The chain of referral started with a member reporting a case to the CPE who then reported the case to a social assistant. CPEs felt that social assistants were aware of their role in protecting children, and that everyone was aware of their different roles when a child in need is identified. One focus group with social assistants said that social assistants must behave like parents, teach the child, visit the child’s school, and go to visit their family. Another focus group with social assistants said that social assistants had the responsibility of informing parents about the importance of keeping the child in the family, and a focus group of CPE members said in a similar vein that the social assistants were charged with raising awareness among families. According to stakeholders interviewed, the number of social assistants was not sufficient to address all of the cases in their communities. One respondent declared that the social assistants are “overloaded and understaffed.” This was supported by a focus group with social assistants who mentioned that having only one social assistant in each commune made it difficult to keep track of cases. The understaffing made it difficult for social assistants to address all of their cases and there was also a reported lack of electronic records or machine based registers available for use (both focus 42 Commune sizes vary widely within and across provinces. Population ranges are from 24,000 to 145,000. ENDLINE PERFORMANCE EVALUATION | 93 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project groups with social assistants). According to stakeholders interviewed, the geographic zones were large, and with the limited number of social assistants at the commune level, it was not possible for them to follow up regularly on all of the cases under their responsibility. One focus group with social assistants reported that while there were job vacancies to become a social assistant, the social assistants themselves were not responsible for recruitment and felt unable to fill those vacancies. The FCF Coordinator mentioned that it was therefore not feasible to expect them to work on all cases related to child protection as well as gender issues, repatriation of refugees, or emergency preparedness, all of which fall under their responsibilities. A focus group of social assistants explained that their responsibilities included gender￾based violence, paternity research, “raising awareness of the well-being of families”, and supporting children in court in addition to child protection services. One focus group with social assistants said that most child protection cases were solved at the village level, and the other focus group with social assistants said that most social work cases actually involved rape or paternity research. There is a lack of government budget allocated to social work. According to stakeholder interviews, the challenges the child protection actors encountered were primarily related to lack of funds and the lack of administrative budget allocated to the social service sector. IRC has carried out advocacy activities to encourage a budget line specifically for the protection of children. The majority of stakeholders interviewed acknowledged that weaknesses existed in the system, and highlighted the funding gap. When there was an emergency, often the staff members needed funds for transportation. One social assistant interviewed stated that, in his opinion, the system was not working. He explained that when he received a case that required even a small amount of money to address, such as to cover transportation, he was blocked because he was not able to cover the costs himself and funding was not provided by the administration. In addition, social assistants did not have the funding or budget from the administration to do regular follow up visits. Sustainability of case management and follow-up visits is in question. Overall, all stakeholders interviewed who were involved in the case management process, particularly the CDFC and the social assistants, explained that there was no official follow-up system or close-out system in place, but that each case was handled on a case-by-case basis. IRC staff members facilitated follow up visits, which means that these activities are unlikely to continue once the project finishes. According to a focus group of social assistants, IRC did not provide means for case follow-up and social assistants had to “pick up the slack43”. Respondents in the same focus group of social 43 According to IRC, providing budget to social assistants for follow-up was never part of the project plan. However, IRC did provide support with transportation and office materials necessary to ensure monitoring ENDLINE PERFORMANCE EVALUATION | 94 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project assistants also said they could not adequately support host families because they were not involved in the follow-up process. Another focus group with social assistants said the project helped social assistants follow-up with children. Some stakeholders interviewed maintained that it was the CPEs who most often facilitated follow-up activities, however, none of the CPE members interviewed were aware of what a case management system was. Standardization of the case management system is still needed. Case management forms have not yet been standardized and different organizations, such as Terre des Hommes, use different tools, while World Vision apparently does not use any forms. There is a need for the entire case management approach to be standardized. Additionally, one IRC staff member stated that the CDFC is not yet autonomous in regards to carrying out case management as they relied on IRC to provide the tools and forms. The IRC staff interviewed who weren’t directly responsible for case management still recognized the need to harmonize and standardize a case management approach at the national level to ensure that all child protection actors are using the same operating procedures and tools. In addition, one IRC staff member said that there was a need to reinforce confidentiality, as government actors, despite having facilitated several trainings in this regard during the project, have not respected this policy. While none of the stakeholders interviewed raised the importance of confidentiality during the questioning period, both focus groups with social assistants did state that confidentiality is crucial. The UNICEF respondents also mentioned that the system was not yet in place, however, it was an area that they are working on to ensure that a wider system is standardized and validated by the Government. Additional training is needed. In terms of training in the delivery of alternative care programs, only some of the stakeholders interviewed, such as the CDFC Coordinators and a couple of the social assistants, who had worked specifically in the target locations of the project and as partners of the project, had participated in training relating to alternative care. One focus group with social assistants said that the project had trained them on the standard minima. According to IRC’s final report, the FCF project trained 27 social assistants on HFC module. FCF also trained 26 CPCPE members and 88 CFCD staff on minimum standards for RCCs and the consequences of institutionalization on the development of a child. None of the CPE members or the RCC directors stated that they had received any training from the FCF project44. Due to the high staff turnover at some of the centers and the changeover of government staff in 2015, it is possible that these individuals’ predecessors had received training. All stakeholders interviewed, for cases that were within the project scope. In addition, the project also provided support for specific cases according to the child’s needs, including school materials, etc. 44 Per IRC, RCC staff were trained on minimum standards. ENDLINE PERFORMANCE EVALUATION | 95 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project however, had been informed of the importance of deinstitutionalization by FCF project staff. One CDFC Coordinator explained that there is a knowledge gap, as many social assistants do not have knowledge of case management, and mentioned that there was a need for continual refresher training. This was supported by a focus group of social assistants who said when attempting to prioritize cases, they often feel untrained and unprepared to manage them. Despite the challenges the project staff and its partners have experienced, achievements have been gained. While the FCF project has not been able to address the issue of governmental funding for child protection actors at the sub-national level, it has reinforced the child protection structures and linkages in place at the colline and commune levels. An IRC staff member stated that the success was primarily at the colline level where there had been a greater engagement of actors. He noted that now people “feel very much responsible for the protection of children”, more than they did prior to the start of the project. He explained that at the beginning of the project in 2014, the CPE members could identify a case of abuse, for instance, but referrals were often not made because members did not know who to make referrals to. Several stakeholders interviewed saw the mapping of available services and subsequent referral mechanism at the colline level as a great success of the project, while others acknowledged that service provision itself needs to be improved. In addition, there is now a structure in place at the communal level that has been decentralized to allow for greater autonomy of the child protection actors at the colline level, particularly the CPEs. It was mentioned that the structure of the CPEs and the CPCPEs have more visibility within their communities than they did prior to the project, and more community members come to them for support. One focus group with social assistants supported this idea, saying the project helped nurture closer relationships with the community and with children and that the project allowed social assistants to be more involved in the supervision and education of children. A couple of the stakeholders interviewed, however, recognized that not all of the CPE members played their roles as they should in every location. It was pointed out that some individuals were more motivated than others, which is understandable given the voluntary nature of the role. CPEs at the colline level did not see an improvement in the child protection system and felt unsupported. While a focus group with commune CDFC workers said that they have seen an improvement in the child protection system thanks to FCF, CPEs at the colline level had a more negative view. Colline CPE members reported in focus groups that CPEs were in better shape at their initial establishment and that the situation has degraded over time. CPE members were angry, felt they had been placed in a system of child protection with no support, and said there was insufficient financial support and knowledge of the child protection system to continue the project (2 focus groups with colline CPE members). ENDLINE PERFORMANCE EVALUATION | 96 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Both of the CDFC Coordinators interviewed were extremely knowledgeable of the deinstitutionalization and case management processes, although the sustainability of activities is at risk without sufficient funding. Although they recognized the challenges in implementation due to lack of funding, they mentioned that they would continue to raise awareness of the importance of family-based care amongst not only parents and community members, but also new staff. It was continually mentioned that the capacity of CPE members had also been reinforced as a result of the project. However, it should be noted that social assistants and CPE members who were not directly trained by project had much less knowledge of deinstitutionalization and case management processes. There is an evident need for greater information sharing amongst the CDFCs to ensure that staff members that are trained on issues related to their positions share that information with other colleagues. Information sharing would inevitably enhance institutional knowledge and would prevent knowledge being lost with staff turnover. Without sufficient funding, the continuation of activities is unlikely. However, based on the responses of key informants, there did appear to be a will to continue awareness raising and promoting good parenting practices at the colline, commune, and provincial levels. As the Counselor of Child Protection to the Ministry stated at endline: “We are just in the middle and there is a long way still to go.” b) Have government authorities and state/non-state service providers adopted a joint approach to build family resilience, involve, support, and protect children at the local level? There is good collaboration between entities at the national and provincial levels, although there is room for improvement. Stakeholders interviewed agreed that the most important actors working in child protection at the sub-national level are the CPE members, Chefs de Colline, social assistants, the CDFC Coordinators, and the CPCPE. It was mentioned that all of these actors collaborate when a child encounters a problem. Overall, stakeholders interviewed agreed that there is good collaboration between the various ministries. This includes the Ministry of Solidarity, Ministry of Education, Ministry of Public Security, Ministry of Justice, Ministry of Interior, and the Ministry of Health. There is a committee that meets at the national level as well as a CPCPE in each province that meet on a quarterly basis to discuss child protection issues, and it is an area that has been reinforced by the project. Several CDFC and CPE focus groups respondents felt that the provincial response to the project allowed for good collaboration to take place, while others felt that there was a lack of structure for communication and too many stakeholders were involved in the child protection system. Some focus group respondents felt that there was good collaboration between agencies at the provincial level and that CDFCs collaborate with CPEs at the communal level. ENDLINE PERFORMANCE EVALUATION | 97 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Question 6: Did the project offer models and approaches for expansion, adaptation, and/or replication? Conclusions: Practices and strategies that were adopted by local partners and the Government as a result of the project were evaluation of residential care centers on a bi-annual basis, recruiting and training foster families, case management and the BID process, and the process of reintegrating children with their families. However, many of these activities are unlikely to continue without continued funding and resources allocated to them. There was a missed opportunity to better prepare partners in advance and help them identify funding opportunities prior to the end of the project. The project did not have a strong exit strategy, which was further exacerbated by the changing political context in the country. This did not allow sufficient time to ensure the institutionalization of its initiatives. Lessons learned had not yet been shared with local stakeholders at the time of the Evaluation Team’s data collection, but there were some plans to do so. The areas in which the project was successful and could have done this are in raising awareness of the importance of family-based care, case management processes and deinstitutionalization, as well as improved knowledge of positive parenting practices. Findings: a) Were lessons learned widely discussed and disseminated during project implementation//n? Lessons learned had not yet been shared at the time of the Evaluation Team’s data collection but there were some plans to do so. Key informants who were directly implicated in the project all agreed that lessons learned would be shared with colleagues and counterparts. UNICEF respondents mentioned in particular that they had already started sharing lessons, and were in the process of developing an action plan integrating the notion of community monitoring. IRC’s final project report includes some mentions of lessons learned. Additionally, IRC later informed the Evaluation Team that lessons learned were discussed at a closing workshop in June 2017. As reported by IRC, all provincial CDFCs, representatives of local administrations, RCCs, UNICEF, the Ministry of Social Affairs, the Ministry of the Interior, and implementing partners were present and participated in an exchange on lessons learned. Overall, the project did not directly offer a model for expansion, adaptation, or replication. In addition, based on responses from the key informants at the time of the ENDLINE PERFORMANCE EVALUATION | 98 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Evaluation Team’s data collection, lessons learned were not widely discussed45 or disseminated, nor were best practices or techniques institutionalized. This was a missed opportunity of the project, and it is something that could have been achieved if there had been a clear exit strategy in place. There were clear achievements made by the project, especially in regards to awareness raising and bringing together various ministries and partners involved in overseeing the protection of children. While it perhaps started a wider discussion around alternative care, and has supported the drafting of the foster family component, it has not specifically shared lessons or offered advice on how best practices could be institutionalized. Time will tell if other partners and the Government take the project as a model for expansion. The awareness raising campaign was a success, CDFCs have increased their capacity and knowledge in case management, and a much larger percentage of RCCs now meet the minimum standards. All of the stakeholder respondents who were aware of the project agreed that it had had a positive impact; however, not all agreed that the project had been successful in meeting its objectives. The majority of respondents mentioned that the awareness raising component amongst families, residential care centers, government officials, and community members has been greatly beneficial. The Counselor of Child Protection to the Ministry stated that the success of the awareness raising campaign not only resulted in families being more willing to receive a child back, but it also prevented new children from entering into the centers. However, he equally acknowledged: “We are just in the middle and there is a long way still to go.” The CDFC Coordinators have also increased their capacity and knowledge in case management. In addition, in 2012 only 20 percent of the residential care centers met the minimum standards. With the last evaluation, a total of 68 percent of the centers met the minimum standards, while some had been closed entirely. Evidently, there has been more engagement from the Government to monitor existing centers. The project was able to deinstitutionalize a number of children and the VSLA/FSG approach was able to provide additional financial support to vulnerable families. In addition, 378 children have been withdrawn from the centers as a result of the project, and foster families have been trained to act as an alternative care option for children. Several respondents, including CPE members and one of the social assistants interviewed, referred to the VSLA approach and emphasized the success of the VSLAs in providing additional financial support to vulnerable families. They equally mentioned the HFC parenting modules and explained that they witnessed a positive impact of the modules in the communities where they work. Interestingly, none of the respondents, 45 IRC later informed the Evaluation Team that lessons learned were discussed at the closing workshop in June 2017. All provincial CDFCs, representatives of local administration, and RCCs, UNICEF, the Ministry of Social Affairs, the Ministry of the Interior, and implementing partners were present and participated in exchanging on lessons learned. ENDLINE PERFORMANCE EVALUATION | 99 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project however, mentioned the impact the cash transfers have had on increasing overall family income46. The respondents from UNICEF also highlighted that the project has brought the Ministry of Solidarity, Ministry of Justice, and Ministry of Interior together to validate and formalize foster families in Burundi, which is a key step in putting into place temporary, alternative care arrangements for children. However, while it is important to move children towards deinstitutionalization, several respondents asserted that it is crucial to maintain a high level of care for the children who are currently in or moving out of the system. VSLA caregivers were included in the learning process and believe that improvements were made from earlier VSLA cycles to later cycles. The majority of VSLA caregivers from cohort 1 reported having provided feedback to IRC regarding the training and/or support they received (81%). Similarly, the majority of respondents reported that, to their knowledge, lessons learned from the first VSLA cohort were incorporated into new cohorts, with 73% of VSLA caregivers agreeing. When asked what kind of additional support from IRC would have been useful for those who participated in the FCF VSLA cohort 2 and cohort 3, respondents reported that more trainings on VSLA and money management would have been useful, with more reporting additional VSLA management training for cohort 2 and money management for cohort 3 (Table 4.6.1). There were also many diverse “other” types of support mentioned that would have been useful. 47 Table 4.6.1: Desired Support During Second and Third VSLA Cycles What kind of additional VSLA support from IRC would have been useful? Cycle 2 Cycle 3 VSLA (%) VSLA (%) Trainings on VSLA management 38 17 Trainings on money management 22 37 Organizing meetings 2 1 Networking events with other VSLA groups 2 2 Other 35 44 b) Were any best practices or successful techniques institutionalized? Practices and strategies that have been adopted include bi-annual evaluation of RCCs, recruiting and training foster families, case management and the BID process, and the process of reintegration. Specific practices or strategies that had been adopted by local partners and the Government as a result of the project include plans to continue evaluating residential care centers on a bi-annual basis to ensure that they are meeting 46 IRC clarified that cash transfers were only given to one group of 25 people in each province, whereas approximately 400 people per province for each cohort participated in VSLAs. 47 IRC worked with 3 VSLA cohorts and 1 FSG cohort of 250 households which was piloted for the first time in Burundi. VSLA cohort 1 was given some limited support and supervision during their self-initiated second cycle and no support during their self-initiated third cycle. ENDLINE PERFORMANCE EVALUATION | 100 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project minimum standards, recruiting and training foster families, case management and the BID process, as well as the process of reintegrating children with their families, which as one social assistant noted, was not something that existed prior to the FCF project. In addition, a CPE member mentioned that the project has resulted in a greater engagement of her CPE with more regular meetings and discussions amongst the members. The overall coordination amongst child protection actors was also mentioned, as were the monthly meetings of the CPCPE. FCF instituted the importance of family-based care. One of the CDFC Coordinators concluded: “The success is that the child is reunified in their community, and that there is an understanding that the child should stay in the community and that a child should be in the family. This is what IRC has helped us to understand. IRC has reactivated the values that Burundi has – that the child should be near his mother, father, grandparents, and relatives. In Kirundi there is a saying: ‘Umwana ni uwabose’ or ‘A child is a child for all’. As a result of the crisis [in 1993] and poverty, we have lost this value.”(CDFC Coordinator) The majority of stakeholders interviewed emphasized that the awareness raising of the importance of family-based care will continue, as will the sharing of good parenting practices. In addition, the VSLAs will continue, and a couple of respondents mentioned that new VSLAs might start, although this was not the same for all locations included in the KII visits. Sustainability of project successes is at risk due to lack of funding. Stakeholders interviewed noted that it would be difficult to continue most of the project’s activities without further funding, particularly in relation to the deinstitutionalization activities and the necessary family tracing, reunifications, and follow up. A residential care center director stated that: “We will continue but we will be much more limited and we will be blocked. We aren’t being left with a good feeling. Also we don’t have the material support. We will have difficulty doing the follow up. We will continue because there are many children who are very young, but we will have to reduce the visits…we will continue with the [deinstitutionalization] activities but they cost a lot of money so it relies on a supplementary expense.” (RCC Director) In addition, several stakeholders interviewed mentioned that despite the progress made in the overall coordination, it was the project funding that enabled the coordination meetings at the national level to discuss the development of the Alternative Care ENDLINE PERFORMANCE EVALUATION | 101 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Guidelines. Unless another organization or donor is able to fund these meetings, respondents mentioned that the meetings were unlikely to continue. The CDFC of one province stated: “I’m scared that if the project ends here, the activities will stop apart from the awareness raising. We can raise awareness amongst parents, that the place of the children is with the family; and evaluate the centers. The family tracing will stop, and as a result the reintegration will stop. The kits will stop also.” (CDFC worker) Lastly, the project provided guidelines and capacity building to PRCs on how to provide and continue the parental training sessions; a package of developmental stimulation materials; and installed adapted training session spaces in the PRCs. However, due to the vulnerability of the families, almost all of them lack the means to participate in these activities (costs of transport and implementing the training sessions that are not currently provided by the government). FCF did not have a coherent exit strategy. In addition, IRC staff were asked about the exit strategy of the FCF project, and from their responses, there did not appear to be a structured exit strategy in place. IRC staff explained that the strategy was to coach and train government staff at the various levels, and to identify interventions at the community level that could take over activities, for instance, identifying projects implemented by other organizations that can continue activities to some degree, such as in the identification of vulnerable families and reintegration of children. The objective was that the Government would continue the deinstitutionalization component; however, it has limited resources to do so, and it is therefore not realistic to expect the Ministry to work independently following the project closeout. An exit strategy requires project staff to effectively communicate with and work alongside government counterparts to ensure that the capacity is in place to carry on activities that could have led to the further institutionalization of some of the activities and practices implemented by the project. A stakeholder at the Ministry mentioned that: “The exit strategy should have been done a lot longer ago so that we could have had more of a plan in place. We were only informed three months ago so the time is not sufficient to ensure that we have what we need in place to continue activities.” (Stakeholder at the Ministry) UNICEF staff added that “IRC did not have a coherent exit strategy” in place. It is likely that workshops or discussions including all partners on how activities could continue without being supported by the project funding or staff most-likely would have been useful to better prepare the Government and partners. ENDLINE PERFORMANCE EVALUATION | 102 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project SUMMARY CONCLUSIONS The project sought to place children in nurturing families and to reduce unnecessary separation by achieving the following: strengthening households at risk of family separation, reinforcing community child protection mechanisms, advancing the national child protection system and policy frameworks, and identifying and preparing reunification and placement of institutionalized children into families or alternative care options. Below we summarize how the project performed in relation to the outcomes expected in the theory of change. The below tables show the achievement of key project objectives and outcomes using the following color coding (blue = success; gray = partial success; red = failure). Families have the financial capacity to provide for all children in the home PARTIAL SUCCESS While results among households participating in the VSLA and FSG programs have shown increases in spending on children, an increase in the number of families that are saving, and a feeling among families of less financial strain, families had not yet increased the amount they were saving or their cash incomes. Additionally, the sustainability of these families continuing with VSLA in the absence of IRC is not certain. Improved child-caregiver relationships; improved sense of parenting efficacy SUCCESS The project did largely achieve its parental training objectives, and parents have improved their relationships with their children. Caregivers have improved capacity to meet the needs of disabled children PARTIAL SUCCESS Parental skills and attitudes in caring for children with disabilities were improved, but homes and schools were not fully accessible to these children. Inclusive education PARTIAL SUCCESS There were inclusive schools in four of the 10 project targeted provinces, but there was a lack of local centers with specialized care for children with disabilities. The national child protection system is capable of preventing family separation and supporting the deinstitutionalization process PARTIAL SUCCESS The project was successful in identifying and registering children in the targeted RCCs; gaining the support of these RCCs to deinstitutionalize the children in their care; improving the conditions in RCCs; ensuring the well-being of children that were deinstitutionalized; and improving the capacity, knowledge, and awareness of CDFC coordinators. The project reinforced the child protection structures and linkages in place at the colline and commune levels and strengthened the referral mechanism. There is not enough government funding to ensure the sustainability of project activities. Households with children at risk of family separation are stabilized and strengthened SUCCESS Children’s well-being improved across all indicators among the most vulnerable (FSG) families. Children’s material, health, and psychosocial well-being improved in VSLA families. ENDLINE PERFORMANCE EVALUATION | 103 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Strengthening Families IRC’s theory of change assumed the VSLA and FSG programs would create the financial capacity for families to provide for all their children in the home and the HFC parental training would improve child-caregiver relationships and social capital. The project did largely achieve its parental training objectives, and parents have improved their relationships with their children. On the other hand, while results have shown increases in spending on children, an increase in the number of families that are saving, and a feeling among families of less financial strain, families had not yet increased the amount they were saving or their cash incomes.48 Another indication that vulnerable families were strengthened is that children’s well-being improved across all indicators among the most vulnerable (FSG) families and children’s material, health, and psychosocial well-being improved in VSLA families. For children with disabilities, the project was successful in improving the capacity of caregivers for children with disabilities but was not able to ensure families and schools were fully equipped for their care. IRC’s theory of change assumed that parental training of children with disabilities, case management, and training of technicians in rehabilitation centers would improve the capacity of these caregivers to meet the needs of children with disabilities; and teacher training on inclusive education would allow children with disabilities to attend school with children without disabilities. While the project did improve parental skills and attitudes in caring for children with disabilities, it did not ensure that homes and schools were fully accessible to these children. Additionally, while there were inclusive schools in four of the 10 project provinces, there was a lack of local centers with specialized care for children with disabilities. Creating a National Child Protection System Capable of Preventing Family Separation and Supporting the Deinstitutionalization Process IRC’s theory of change neglected to take into account the lack of government funding for the child care and protection sector. This affected the success of the project’s objectives regarding the national child protection system and greatly jeopardized the sustainability of their achievements. The project assumed that capacity building and technical support at the provincial level to implement the government’s action plan would be sufficient to reach the goal of a national system that is capable of preventing family separation and supporting the deinstitutionalization process. It neglected to focus on advocating for sufficient government funding to support the system. 48 The project was operating within a difficult economic and political context with a rise in inflation, costs, and subsequent food insecurity so it is unclear whether stable economic and political circumstances would have led to a greater effect on financial capacities. ENDLINE PERFORMANCE EVALUATION | 104 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project Despite this weakness of the theory of change, the project was a positive starting point to achieving the goal. The project was successful in identifying and registering children in the targeted RCCs; gaining the support of these RCCs to deinstitutionalize the children in their care; improving the conditions in RCCs; ensuring the well-being of children that were deinstitutionalized; and improving the capacity, knowledge, and awareness of CDFC coordinators, which has strengthened the deinstitutionalization process. The project also began improving communication and collaboration between the Ministry of Solidarity and the Ministry of Interior to enforce the Minimum Standards for Children in Institutions. On the other hand, the high staff turnover rate and low budget available for CPEs made it challenging to keep everyone informed of key child protection policies and strategies. As a result, there was very little knowledge or awareness of key elements of national policies and strategies, especially at the sub￾national level among those working in the child care and social protection system. The project reinforced the child protection structures and linkages in place at the colline and commune levels and strengthened the referral mechanism. Additionally, community members are now more aware of the CPE structure and are better able to take more advantage of the support it offers. The project supported the case management system but it is not yet at a point where it can continue to function independently – the government did not yet have a nationally standardized case management system and was not yet providing the tools and forms necessary nor funding follow-up visits. The project was successful in shifting most, but not all, attitudes toward supporting family-based care over placement in RCCs. The project’s theory of change assumed the activities to strengthen families discussed in the above two paragraphs coupled with communication campaigns and raising awareness would decrease support among caregivers for institutional care. Over the project period, there was greater acceptance of family-based care over placement in RCCs, although there was still some support among those working within the child care and protection system for placement of children with disabilities in RCCs and some limited support for placement of children in RCCs in cases of poverty. RECOMMENDATIONS FOR FUTURE PROGRAMMING ACROSS ALL EVALUATION QUESTIONS National Level Systems Strengthening ■ Advocate to the MDPHASG at the national level to increase the budget allocation and financial commitment towards child welfare services, particularly to activities related to the CDFCs and CPEs, such as case management. ENDLINE PERFORMANCE EVALUATION | 105 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project ■ Advocate to relevant Ministries to cover the costs of physiotherapy and other health￾related costs for children with disabilities above the age of 5, including provision and fabrication of helping aids. ■ The MDPHASG could share more widely information on policies and strategies developed at the national level with their counterparts at the sub-national level. ■ Senior staff at the MDPHASG could assist the CDFC Coordinators to develop a system for coordinating the activities of the social assistants and facilitating information sharing among them. ■ There should be a national policy, under the National Plan for Education or Inclusive Education (or similar), that requires accessible schools, community services, and even basic construction rules of houses in order to facilitate integration of persons living with disabilities. It should include regulations / guidelines on accessibility norms of houses, schools and other community based facilities. The overall infrastructure for this is poor or lacking but the Ministry of Education could allocate some funding for accessibility to make schools inclusive. National Guidelines for the Alternative Care of Children ■ The MDPHASG should continue collaboration with UNICEF and NGOs to lead in the development of the National Guidelines for the Alternative Care of Children. ■ Finalize the policy document on Foster Care. ■ The MDPHASG should widely share and disseminate the finalized National Guidelines for the Alternative Care of Children with officials at the commune and colline levels. Case Management ■ Establish a case management system, including well defined assessment procedures49, case planning, follow up and review, and case close out procedures, standardizing forms, and recognizing case management as a service that is part of a wider child protection system. It is particularly important to support harmonization of tools and procedures for case management at the national level in Burundi; this is an area where the IRC can continue to provide additional support to the Ministry of Social Affairs. ■ Develop specific protocols and mechanisms for information sharing and referrals that are used in practice to ensure quality case management, confidentiality procedures and the best interests of the child are upheld. 49 IRC should continue to use the assessment guide for field social workers to assess family and household conditions for reunified children, and the full cycle of case management tools that were used in this project. ENDLINE PERFORMANCE EVALUATION | 106 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project ■ Continue to ensure that confidentiality mechanisms are in place that are linked to sharing information on a need-to-know basis only, and are shared only with those individuals who require information in order to protect the child. ■ Design detailed training of trainers’ modules, particularly on case management and deinstitutionalization processes for the CDFC so that they can continue sharing information and knowledge amongst new staff members as well as supporting the CPEs. ■ IRC, together with UNICEF and other child protection actors, should review the operational roles social assistants working for the Ministry of Solidarity are expected to play and develop a revised approach to case management that is realistic and effective. Residential Care Centers ■ Government, the Ministry of Solidarity and Ministry of Interior working together and with UN/NGOs should continue to phase out or close down operations of existing residential care centers. ■ Government should develop and enforce repercussions for residential care centers that do not meet the minimum standards, such as closing the RCC or putting into place financial penalties like fines. Commune and Colline Levels Advocacy and Awareness-Raising Activities ■ Engage religious leaders and faith-based organizations to promote family-based care and include them in awareness raising and advocacy activities, for instance include them in the child protection committees. ■ Develop educational materials based on the HCF modules to encourage and illustrate positive parenting practices that the CDFC staff members can disseminate and share more widely at the colline and commune levels. ■ Design and implement new outreach techniques for child protection committees, particularly theater in their own communities highlighting the importance of family￾based care in order to reach more of the population. Systems Strengthening ■ Strengthen community-based systems to monitor and promote child well-being and protection. More specifically, include CPE members and social assistants in efforts to follow up with and support families with children with disabilities. ■ Conduct a mapping exercise in each province, not only those within the IRC project provinces where this was already conducted, to identify coordinated services and ENDLINE PERFORMANCE EVALUATION | 107 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project supports within a referral system at the colline and commune levels ensuring that all child protection actors are aware of services available and referral mechanisms so that children have greater access to services when they are in need of them. ■ Continue to strengthen coordination between community leaders (Chefs de Colline), CPEs, CDFCs, and the CPCPE with service providers, and local and international NGOs and the UN. ■ Continue to design and implement outreach techniques for CPEs, and the National Forum for Youth so that they are able to raise awareness of the importance of family-based care and promote the care of children within communities as opposed to in residential care centers. Socio-economic Support ■ Encourage the CDFCs to continue to identify areas with a high number of families at risk of separation to raise awareness on the benefits of the VLSAs and support the establishment of VSLAs in those communities on a rolling basis. ■ In order to enable VSLA members to have better access to more financial services and economic opportunities and consequently increase children’s access to services, the CDFCs could facilitate discussions between VSLAs and micro finance institutions (MFI) operating in the provinces. 50 ■ It is important that standard operating procedures are in place, and in fact, this was a recommendation that was repeated during the KIIs by government staff as well as IRC staff. Services for Children with Disabilities ■ Integrate health care packages and financial support for CWD’s (for instance, 30 days of care in PRCs in order to allow parents to follow the parental training sessions). ■ Reinforce a financial support system to provide adapted helping aids to meet children’s needs. General Programming and Operations ■ Recognizing that there is significant ongoing interagency collaboration and exchange to identify good practice approaches to deinstitutionalization and children’s reintegration, IRC staff should identify and document successful 50 This was something previously organized by IRC during VSLA ‘open days’, but having the CDFCs take the lead on this could improve sustainability of results and capacity building. ENDLINE PERFORMANCE EVALUATION | 108 Burundi: Endline Performance Evaluation of USAID/DCOF’s Family Care First Project approaches and lessons learned51. It should share those with DCOF, UNICEF, and other child protection agencies. While recommendations have been provided for each level of government, the recommendations as a whole would lead to a more sustainable system. Of priority is a higher government budget allocation to child welfare services since the lack of this funding greatly risks the sustainability of project activities. 51 NORC collected data prior to the project close. IRC reported to us afterwards that they shared lessons learned in both the project closing workshop, and in the final narrative report. ENDLINE PERFORMANCE EVALUATION | 109