This publication was produced for review by the United States Agency for International Development. It was prepared independently by Vaiddehi Bansal, Brooke Jardine, and Samantha Austin from NORC at the University of Chicago. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION Final Evaluation Report Prepared under Contract No.: GS-10F-0033M / 7200AA18M00016, Tasking N054 STAFF DEFINITIONS OF ‘RESILIENCE’ PRE- AND POST-TRAINING CREDIT: WOMEN AGAINST RAPE (WAR) DRG LEARNING, EVALUATION, AND RESEARCH ACTIVITY II GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION Final Evaluation Report (JUNE 28, 2023) Prepared under Contract No.: GS-10F-0033M /7200AA18M00016, Tasking N054 Submitted to: Matthew Baker, USAID COR Submitted by: Vaiddehi Bansal, CARE-GBV Team Lead Brooke Jardine, Qualitative Specialist Samantha Austin, Quantitative Specialist Contractor: NORC at the University of Chicago 4350 East West Highway, 8th Floor Bethesda, MD 20814 Attention: Matthew Parry Tel: 301-634-5444; E-mail: Parry-Matthew@norc.org DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ABSTRACT NORC at the University of Chicago (NORC) was contracted by the United States Agency for International Development (USAID) to complete a Portfolio Performance Evaluation of gender-based violence (GBV) activities, which comprises four activity clusters. This evaluation report focuses on the Collective Action to Reduce Gender-Based Violence (CARE-GBV) Activity. The Small Grants Program, which is the focus of this evaluation, awarded small grants ($50,000–$125,000) through an open call to new, local, and under￾utilized partners to improve staff wellness and resiliency in GBV programming, fill global data gaps related to self- and collective care and wellness for staff of GBV organizations, and promote learning. The evaluation addressed the following three main questions: (1) Are the activity clusters based on context-specific and international evidence? (2) To what extent are each of the activity clusters achieving the targeted GBV results? and (3) To what extent are the activity clusters sustainable? NORC applied a mixed-methods approach to answer the research questions, using a combination of desk review, key informant interviews (KIIs), and a web-based survey. The evaluation found that the CARE-GBV cluster was successful in raising awareness on the harmful effects of vicarious trauma among GBV responders and implementing programs that assisted staff with improving their self-care, wellness, and resilience. All five grantees conducted some form of needs assessment to ground their intervention in empirical evidence about participant needs and relevant contextual factors. Several approaches were seen as successful and likely to be sustained, and are recommended for continuation; however, smaller organizations noted they were challenged by the bureaucratic reporting process and funding constraints. ii | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ACKNOWLEDGEMENTS We would like to thank the following colleagues at NORC: Ritu Nayyar-Stone (Team Lead for the overall Gender-based Violence Portfolio Performance Evaluation), Cathy Zimmerman (Gender/Gender￾based Violence Expert), and both Ridhi Sahai and Ingrid Rojas Arellano (Evaluation Specialists) for their valuable direction and contributions. Alma Omeragic provided local translator services in Bosnia and Herzegovina. iii | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV TABLE OF CONTENTS ABSTRACT I ACKNOWLEDGEMENTS II ACRONYMS VI EXECUTIVE SUMMARY VII EVALUATION DESIGN VII MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS VII EVALUATION PURPOSE AND QUESTIONS XIII EVALUATION QUESTIONS XIV 1. PROJECT BACKGROUND 1 2. METHODS AND LIMITATIONS 3 DESK REVIEW 4 KEY INFORMANT INTERVIEWS 4 SURVEY 5 LIMITATIONS 6 3. FINDINGS, CONCLUSIONS & RECOMMENDATIONS 7 A. FINDINGS ACROSS GRANTEES FOR THE ACTIVITY CLUSTER QUESTIONS 7 EVALUATION QUESTION 1: ARE THE AC’S BASED ON CONTEXT-SPECIFIC AND INTERNATIONAL EVIDENCE? 8 EVALUATION QUESTION 2: TO WHAT EXTENT ARE EACH OF THE ACS ACHIEVING THE TARGETED GBV RESULTS? 15 EVALUATION QUESTION 3: TO WHAT EXTENT ARE THE ACS SUSTAINABLE? 25 4. CONCLUSIONS FOR THE CARE GBV ACTIVITY CLUSTER 29 EVALUATION QUESTION 1: ARE THE ACS BASED ON CONTEXT-SPECIFIC AND INTERNATIONAL EVIDENCE? 29 EVALUATION QUESTION 2: TO WHAT EXTENT ARE EACH OF THE ACTIVITY CLUSTERS ACHIEVING THE TARGETED GBV RESULTS? 29 EVALUATION QUESTION 3: TO WHAT EXTENT ARE THE ACTIVITY CLUSTERS SUSTAINABLE? 30 5. RECOMMENDATIONS FOR THE CARE GBV ACTIVITY CLUSTER 30 EVALUATION QUESTION 1: ARE THE ACS BASED ON CONTEXT-SPECIFIC AND INTERNATIONAL EVIDENCE? 30 EVALUATION QUESTION 2: TO WHAT EXTENT ARE EACH OF THE ACTIVITY CLUSTERS ACHIEVING THE TARGETED GBV RESULTS? 31 EVALUATION QUESTION 3: TO WHAT EXTENT ARE THE ACTIVITY CLUSTERS SUSTAINABLE? 32 6. IMPLEMENTATION EVALUATION FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 32 A. FINDINGS FOR THE IE 32 iv | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV EVALUATION QUESTION 1: IS THE ACTIVITY DESIGN BASED ON LOCAL CONTEXT AND FLEXIBLE TO ACHIEVE RESULTS ON THE GROUND? 32 EVALUATION QUESTION 2: IS THE ACTIVITY REACHING PARTICIPANTS THEY ARE MEANT TO TARGET? 35 EVALUATION QUESTION 3: IS THE ACTIVITY ACHIEVING SUSTAINABILITY? 36 B. CONCLUSIONS FOR THE IE 37 C. RECOMMENDATIONS FOR THE IE 38 ANNEX A, EVALUATION STATEMENT OF WORK 40 BACKGROUND 40 PPE OBJECTIVES 40 ACTIVITIES INCLUDED IN THE EVALUATION 41 EVALUATION QUESTIONS 41 POSSIBLE DATA COLLECTION METHODS 43 EVALUATION TIMELINE. 43 REPORTING AND DELIVERABLES. 44 CLEARANCE PAGE 52 ANNEX B, GBV ACTIVITIES EVALUATED BY THE PPE 53 ANNEX C, DATA COLLECTION INSTRUMENTS 58 WEB SURVEY INSTRUMENT 58 KII GUIDE – IP SENIOR STAFF (CCH) 73 KII GUIDE – IP SENIOR STAFF (SOAR) 77 KII GUIDE – IP SENIOR STAFF (SVRI) 82 KII GUIDE – IP SENIOR STAFF (WAR) 86 KII GUIDE – IP SENIOR STAFF (ZSU) 90 KII GUIDE – MAKING CENTS INTERNATIONAL 94 KII GUIDE – USAID 97 ANNEX D, SOURCES OF INFORMATION 100 DOCUMENTS REVIEWED 101 KEY INFORMANTS 102 ANNEX E, DISCLOSURE CONFLICTS OF INTEREST 103 v | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV TABLES Table 1. Evaluation Findings and Conclusions.......................................................................................................... viii Table 2. Evaluation Recommendations........................................................................................................................xi Table 3. Evaluation Questions.....................................................................................................................................xiv Table 4. Activities Included in CARE-GBV Evaluation..............................................................................................1 Table 5. Web Survey Response Rates..........................................................................................................................5 Table 6. Activity Cluster Findings..................................................................................................................................7 Table 7. SVRI Course Completion Rates in Pilot Period.......................................................................................16 Table 8. Sustainability of SOAR Activities .................................................................................................................36 FIGURES Figure 1. Evaluation Research Design ...........................................................................................................................4 Figure 2. Survey Respondent Demographics..............................................................................................................6 Figure 3. Integration of Participant Needs.................................................................................................................10 Figure 4. To what extent is the WCT effective in monitoring the emotional health and well-being of WAR staff? (n=17)...........................................................................................................................................................15 Figure 5. Effectiveness of Approaches Used by SOAR...........................................................................................16 Figure 6. SVRI Training Key Skill Development (n=68)..........................................................................................17 Figure 7. CCH Skill Development (n=18) ................................................................................................................18 Figure 8. ZSU Training Effects......................................................................................................................................18 Figure 9. Since the implementation of this activity, to what extent has the culture of your organization changed to be understanding of and responsive to vicarious trauma? ...............................................................19 Figure 10. Which of the SOAR activities were most successful in communicating an understanding of vicarious trauma and best practices to strengthen self-care, wellness, and resilience? (n=23)....................22 Figure 11. To what extent has the new Human Resource and Wellness Officer role helped in monitoring and supporting the well-being of staff and promoting a culture of self- and collective care? (n=18)..........23 Figure 12. SVRI Dare to Care Course Accessibility and User Friendliness ......................................................24 Figure 13. To what extent will mechanisms applied by each activity be sustained?........................................25 Figure 14. To what extent do you think the new policies and practices on managing vicarious trauma will be implemented and maintained after the project period?....................................................................................26 Figure 15. To what extent do you think the implementation and adoption of these policies is sustainable for SOAR and WAR?......................................................................................................................................................27 Figure 16. Implementation and Maintenance of SOAR Policies*..........................................................................36 vi | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ACRONYMS AC Activity Cluster BTC Better Together Challenge CARE-GBV Collective Action to Reduce Gender-Based Violence CCH Crisis Center Hope CSO Civil Society Organization DDI Democracy, Development, and Innovation DPI Development Partners International DRG Democracy, Rights, and Governance GBV Gender-Based Violence IE Implementation Evaluation IP Implementing Partner KII Key Informant Interview MCI Making Cents International NGO Non-governmental Organization NORC NORC at the University of Chicago PPE Portfolio Performance Evaluation RISE Resilient, Inclusive & Sustainable Environments SGBV Sexual and Gender Based Violence SOAR Sexual Offences Awareness and Response Initiative STS Sensory Traumatic Stress SVRI Sexual Violence Research Initiative TOC Theory of Change USAID United States Agency for International Development WAR Women Against Rape WCT Wellness Check-In Tool WEE Women’s Economic Empowerment ZSU Žene sa Une vii | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV EXECUTIVE SUMMARY As part of the Democracy, Human Rights, and Governance Learning, Evaluation, and Research (DRG￾LER) II Activity, NORC was contracted to complete a portfolio performance evaluation of USAID’s gender-based violence (GBV) activities. The evaluation’s purpose was to identify facilitators and barriers to activity effectiveness, where knowledge still needs to be developed, and what can be improved upon in the GBV portfolio of the United States Agency for International Development’s (USAID’s) Gender Equality and Women’s Empowerment Hub (USAID/GenDev). This evaluation report focuses on the Collective Action to Reduce Gender-Based Violence (CARE-GBV) Activity. Although it is widely acknowledged that GBV services providers can experience vicarious trauma, there has been limited programming to understand its various manifestations and potential mitigation strategies. The CARE-GBV activity was implemented by Development Professionals, Inc.-Making Cents International (DPI-MCI) with the aim of supporting USAID/GenDev in developing guidelines, strategic plans, training, and professional networking support for its GBV programming. To support GBV organizations’ capacity-building, CARE-GBV awarded grants from $50,000 to $125,000 over a one-year period (July 2021–July 2022) to five organizations through an open call. The Small Grants Program, which is the focus of this evaluation, awarded small grants to new, local, and under-utilized partners to improve staff wellness and resiliency in GBV programming, fill global data gaps related to self- and collective care and wellness for staff of GBV organizations, and promote learning. Each of the organizations selected for the CARE-GBV Small Grants Program was led by women, including women who identify as survivors of GBV. Activities implemented by the following organizations were evaluated under the CARE-GBV cluster: 1. Crisis Center Hope (CCH), North Macedonia 2. Sexual Offenses Awareness and Response Initiative (SOAR), Nigeria 3. Sexual Violence Research Initiative (SVRI), Global 4. Women Against Rape (WAR), Botswana 5. Žene sa Une (ZSU), Bosnia and Herzegovina NORC answered the following evaluation questions: 1. Are the activity clusters based on context-specific and international evidence? 2. To what extent are each of the activity clusters achieving the targeted GBV results? 3. To what extent are the activity clusters sustainable? EVALUATION DESIGN Primary and secondary data were collected from March 2022 to February 2023. NORC’s evaluation drew on a combination of quantitative and qualitative methods, starting with a desk review of 81 program documents, followed by 8 key informant interviews (KIIs) with USAID, MCI, and grantee senior staff and partners. Additionally, NORC conducted a web-based survey of grantee staff and external program participants, which was completed by 142 respondents via Qualtrics. NORC also conducted an implementation evaluation of one activity. MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS A summary of main findings, conclusions, and recommendations from the evaluation are presented in viii | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV the table below. Table 1. Evaluation Findings and Conclusions FINDINGS CONCLUSIONS ACTIVITY CLUSTER EQ1. Are the activity clusters based on context-specific and international evidence? • All five grantees conducted some form of needs assessment to inform their proposal submission and/or intervention design. • Results of needs assessments indicated that staff were experiencing feelings of burnout and stress, unable to regulate emotions or separate work from home, and had limited understanding of self- and collective care. • Grantees experienced a significant increase in GBV caseload during the COVID-19 pandemic which prompted them to prioritize and address staff well-being and resilience. • Majority of participants across all five activities reported that their needs were taken into account by grantees. • The cluster was grounded in the understanding that GBV responders experience vicarious trauma as a result of the job and that many of them are survivors and bystanders themselves, which increases the risk of re-traumatization. • Although the cluster followed an ethical imperative to deliver safe and effective programming through a do-no-harm approach, there was no overall theory of change (TOC). • Implementation during the COVID-19 pandemic required unexpected adaptations to planned programming, such as increasing staff capacity and transitioning to virtual or hybrid training and dissemination events. • Needs assessments were critical for understanding participant needs, contextual factors, and existing evidence and programming on vicarious trauma. • Grantees’ adaptations in response to the COVID-19 pandemic were reportedly effective in responding to evolving needs of GBV responders (increased stressors and burnout) and survivors (increasing safe house services). • Grantees integrated mechanisms for collecting feedback on program activities. However, the collection of post-training feedback and monitoring of training indicators was somewhat restricted by funding and the implementation timeframe. EQ2. To what extent are each of the activity clusters achieving the targeted GBV results? • Some grantees noted they were challenged by USAID’s bureaucratic reporting process and funding constraints. Being grassroots organizations with limited staff capacity, and receiving USAID funding for the first time, they were unfamiliar with the reporting process. • Grantees had adequate flexibility to adapt their programming and outcomes to meet participant needs which was appreciated. ix | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV FINDINGS CONCLUSIONS • Overall, the cluster was able to raise awareness on vicarious trauma and implement programs that assisted staff with improving their self-care and personal wellness. • Tools that required consistent independent use were less successful, as workers acknowledged that they had a difficult time integrating self-care tools into their already busy schedules and did not want the additional work. Further, self-paced courses were also viewed as additional work by some respondents. A diversity of learning aides (videos, group work, etc.) helped engage course participants and break up lectures or reading-focused content. • Despite administrative challenges in aligning their processes to meet USAID’s reporting requirements, the CARE-GBV cluster was able to raise awareness on vicarious trauma and promote a work culture that prioritizes staff well-being, care, and resilience. Grantees increased participants’ understanding of vicarious trauma and burnout and improved self- and collective care practices. • Across all five activities, course content was reported as user-friendly and easy to understand. Program participants appreciated practical, low-effort guidance (such as breathing techniques), which helped them manage the day-to-day stress of their work. EQ3. To what extent are the activity clusters sustainable? • Survey respondents were likely to believe that mechanisms implemented throughout the program would be continued after the project period. • All grantees reported that one or more aspects of their project could be scaled, replicated, or transferred for GBV responders’ self-care and wellness needs. • Lack of staff capacity, political will, and access to funding were the most commonly cited barriers to creating sustainable mechanisms in the CARE-GBV cluster. • Lessons learned on vicarious trauma were likely to be sustained after the project ended, and new policies and practices for managing vicarious trauma were likely to be implemented and maintained after the project period. However, challenges include retaining knowledge from trainings and workshops, the need for repeated instruction and refreshers and the need for a longer time to apply the lessons learned from training programs. Further, grantees noted lack of continued funding sources and small organization size as limiting factors for maintaining outcomes of the trainings. IMPLEMENTATION EVALUATION EQ1. Is the activity design based on the local context and flexible to achieve results on the ground • SOAR utilized several methods to reach their objectives. Staff felt that the self-care and wellness meetings were the most successful mechanisms to understand the impacts of vicarious trauma on their work and personal lives. Additionally, SOAR management noted the wellness policy was effective to institutionalize self-care and wellness within their organization. • SOAR’s activity was designed to meet staff needs, and they indicated that overall, they had the information and support needed to design an effective intervention. However, they felt there were knowledge gaps in self￾care and wellness for GBV responders to survivors of childhood sexual abuse both within their organization and their wider network. x | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV FINDINGS CONCLUSIONS • There was adequate flexibility to adjust the program as needed to meet objectives; however, it was mentioned that staff were burnt out and stretched thin to address issues within their local contexts, and there was inadequate staffing and funding to effectively address these problems. • SOAR’s TOC reflected these knowledge gaps and aimed to empower their staff with information and sufficient resources to support their own wellness process. They noted that they had the flexibility to change their approach to meet the needs of their staff and were able to effectively monitor the intervention and its impact on staff. • The COVID-19 pandemic increased overall staff burnout and impacted SOAR’s ability to effectively implement and digest various aspects of the program. EQ2. Is the activity reaching participants they are meant to target? • SOAR engaged both their own staff as well as staff from 12 other GBV organizations in their region. They were able to do this by creating a network of organizations to participate in their activities. • Verbal check-ins with staff throughout the program gave insights into what aspects were or were not working. Additionally, SOAR committed to annually reviewing their wellness policy. • SOAR engaged four additional staff from their GBV network outside their initial target beneficiaries, for a total of 36 participants. Throughout the implementation period, SOAR conducted check-ins during meetings to determine what program aspects were or were not working for staff. Throughout the grant period, these activities influenced organizational changes in staff wellness. • Monitoring throughout the activity period was challenging due to tight deliverable timelines, which made it difficult to track change, use, or institutionalize some mechanisms over time. EQ3. I the activity achieving sustainability? • Staff and management alike felt that various aspects of the program could be maintained after the program period ended. However, staff expressed concerns about whether certain self-care and wellness policies would be maintained. • Staff indicated that the monthly self-care and wellness meetings were the most successful mechanisms utilized, and the management team believed the wellness policy would institutionalize self-care in their organization, leading to more sustainable practices to support staff well-being. • Based on current data, it is difficult to determine the sustainability of these behaviors. xi | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Table 2. Evaluation Recommendations ACTIVITY CLUSTER EQ1. Are the activity clusters based on context-specific and international evidence? • Needs assessments should be considered an essential first step of intervention design. This is also necessary to align with participant needs and contextual factors and understand how different genders may experience vicarious trauma including the needs of male GBV responders and how they prefer to receive training and support. Resources to inform the intervention can include review of international practices to identify relevant elements, review of common or important stressors in the local context, and consultations with GBV responders to understand their needs. • Stakeholder engagement efforts should be expanded to include a diverse group of partners to better address vicarious trauma within an organization and in the GBV field. Grantees could consider engaging an established network of GBV service providers, local and international subject matter experts, traditional healers and leaders, government agencies, and civil society organizations/non-governmental organizations (CSOs/NGOs). While grantees conducted needs assessments, there was little evidence of establishing greater linkages to other sources of community-based continuous support, which may help to scale and amplify the achievement of outcomes. • Future programs that seek to deliver trainings to GBV service providers should consider the preferred learning styles of participants and how to best deliver the content. Self-paced, asynchronous styles may be slightly less suited to adapting course content to participant needs and may be less effective in delivering sensitive content. • Future funders and grantees should build in adequate flexibility to be able to respond to unexpected events (e.g., COVID-19, elections, etc.) and adapt programming accordingly. The CARE-GBV cluster was designed to allow flexibility at the organizational level to enable grantees to customize their intervention based on staff needs and contextual factors. EQ2. To what extent are each of the activity clusters achieving the targeted GBV results? • USAID should continue funding programs that address vicarious trauma among GBV responders. Financial support should be designed to ensure GBV responders’ overall self-care and wellness needs are met in ways that enable them to maintain support for survivors and do not jeopardize their well-being. • Encourage grantees to work with their staff to co-determine ways to protect staff well-being and maintain self- and collective care activities. While it is often difficult for resource-challenged groups to avoid over-working, especially when responding to urgent needs of GBV clients, it is important to determine how work hours and responsibilities can be structured to accommodate staff self-care needs and prevent burnout, while maintaining the essential client services. • Future grantees should be encouraged to leverage the needs assessment phase to identify locally relevant support services. By mapping and contacting local services, it might be possible to identify a network of partners to help meet the needs of staff, which may go beyond psychological support. Grantees may want to consider joining forces with other GBV organizations that may be grappling with staff burnout or trying to deliver vicarious trauma interventions. It may be possible to identify larger organizations with more self-care practices or resources, such as international organizations, that can share resources with smaller groups. USAID may be able to broker relationships with larger organizations that have a capacity to support smaller, local groups. xii | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ACTIVITY CLUSTER • Consider additional research and programming to understand how self-care trainings can be customized to support male GBV responders. Organizations should be encouraged to recognize that male responders may require different approaches to self-care than women. GBV subject matter experts, donors, and implementing organizations should explore how the perception of self-care differs by gender and identify different coping techniques to mitigate vicarious trauma. • Consider adopting alternate contract mechanisms for small grants to local organizations. USAID should design funding mechanisms that ensure funding does not force groups to delay their activities or self-fund project work prior to receiving USAID funds. Smaller grassroots organizations struggled to operate under current funding mechanisms, which tied payment disbursements to deliverables/milestones. • Consider increasing funding to support a longer duration of future projects. Funding strategies should be designed to enable grantees to complete all deliverables and activity components during the span of the project. An 18-month or longer contract is likely to give small organizations more time to implement self-care activities and entrench organizational norms, beliefs, and behaviors related to vicarious trauma EQ3. To what extent are the activity clusters sustainable? • Future interventions should continue to emphasize low-effort self-care practices, as participants are more likely to implement practices that are easier to integrate into their busy work schedules in the long term. • A “training of trainers” approach could be used to ensure that training programs can be delivered to new staff and existing workers can be reminded of lessons learned during the initial round of trainings. This could also aid in the scalability of programs as other organizations could be trained to deliver similar programs to their workers. Regular reminders and follow-ups are required for continued application of self-care practices and for sustained knowledge of vicarious trauma, due to staff turnover. • The integration of one or more staff members (e.g., Wellness Officers) to promote collective care practices might foster longer-term application of lessons learned. Organizations could explore appointing one or two individuals to promote lessons learned from training programs. These staff might help coordinate new and refresher trainings to adopt self￾care practices into organizational cultures. • Staff wellness and care should be a core component of these organizations. Senior management could integrate trainings on vicarious trauma, self-care, and wellness into their onboarding process as well as conduct wellness check-ins with staff on a regular basis. IMPLEMENTATION EVALUATION EQ1. Is the activity based on local context and flexible to achieve results on the ground? • Invest in further self-care and wellness interventions for professionals supporting violence survivors. Caring for trauma-affected populations is challenging and stress-filled, often creating burnout, which risks both harm to these care professionals and subsequent losses for vulnerable individuals. Findings clearly indicated that attention to self-care was used, highly valued, and beneficial to care providers.1 • Expand training topics to help GBV responders better serve survivors of child sexual abuse. Survey respondents mentioned further topics could focus on the reintegration processes and other support for victims/survivors after receiving services; prevention of Sexual and Gender￾based Violence (SGBV); and accessibility of psychosocial support and its sustainability for the survivors of SGBV. 1 This is an important recommendation and is therefore indicated for all the grantees in the cluster, including the implementation evaluation grantee. xiii | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ACTIVITY CLUSTER EQ2. Is the activity reaching participants they are meant to target? • Encourage greater activities that bring GBV workers together. Because group activities, such as workshops, meetings, and focus groups, appeared to be highly valued by so many participants, implementing agencies should consider trying to include greater numbers of collective, sharing, and mutual support activities. • Embed components to support monitoring and adaptations to self-care interventions, making them responsive to staff health and wellness needs. By including sufficient funds and time for grantees to track the influence of the interventions, identify gaps in current needs, and respond to emerging stressors, funders can help groups maintain effective activities beyond the funding cycle. Respondents suggested biannual refresher and feedback sessions with staff to reinforce learning outcomes. EQ3. Is the activity achieving sustainability? • Encourage knowledge and resource sharing across different agencies likely to experience vicarious trauma and burnout. Based on the lessons learned about implementing wellness programs and about self-care by recipients, it seems that a next funding round could offer the opportunity for learning to be shared across more care sectors, which may make these resources more cost-effective and sustainable. Donors should consider funding a central Wellness Contact Point that can serve multiple agencies simultaneously, to maximize funding value. EVALUATION PURPOSE AND QUESTIONS Under the Democracy, Human Rights, and Governance Learning, Evaluation, and Research (DRG-LER) II Activity, the United States Agency for International Development’s (USAID) Gender Equality and Women’s Empowerment Hub (USAID/GenDev) in the Bureau for Development, Democracy, and Innovation (DDI) contracted NORC at the University of Chicago (NORC) to carry out a portfolio performance evaluation (PPE) of its gender-based violence (GBV) activities. The assignment included co￾creation of the scope of work (SOW)2 with GenDev, an evaluability assessment, 3 an evaluation design report, implementation evaluation, performance evaluation, and dissemination. The purpose of the evaluation was to identify facilitators and barriers to activity effectiveness, where knowledge still needs to be developed, and what can be improved upon in USAID/GenDev’s GBV portfolio. Additionally, one activity was selected from each activity cluster for an implementation evaluation. The four activity clusters (ACs) in the portfolio performance evaluation include:4 1. Better Together Challenge (BTC) with GBV prevention and response interventions; 2. Collective Action to Reduce Gender-Based Violence (CARE-GBV) small grants activities; 3. The Resilient, Inclusive & Sustainable Environments (RISE): A Challenge to Address Gender-Based Violence in the Environment; and 4. The Women’s Economic Empowerment (WEE) activities directly funded by USAID/GenDev integrating GBV prevention and response activities. This evaluation report focuses on the CARE-GBV activity cluster, which was implemented by Development Professionals, Inc. (DPI) and Making Cents International (MCI) from July 2021 to July 2022. The CARE-GBV Small Grants Program, which is the focus of this evaluation, comprises five grantees 2 See Annex A for the scope of work. 3 The evaluability assessment examined all activities suggested by GenDev to see if they were evaluable. Based on those findings and in consultation with GenDev, all five were selected for the performance and one for the implementation evaluation. 4 See Annex B for a detailed summary of all 20 activities that were evaluated across the four clusters. xiv | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV who received funding to promote capacity-building and learning focused on GBV staff and organizational wellness and resiliency. Primary and secondary data collection for the evaluation was carried out between March 2022 and February 2023. The evaluation team included Vaiddehi Bansal, Brooke Jardine, and Samantha Austin from NORC. There are separate evaluation reports for each of the other three ACs and an overall portfolio performance evaluation report that has its own evaluation questions and compares findings across all the ACs. EVALUATION QUESTIONS NORC addressed the following evaluation questions, which were co-created with GenDev: 5 Table 3. Evaluation Questions ACTIVITY CLUSTER QUESTIONS 1. Are the activity clusters based on context￾specific and international evidence? • Needs Assessment and Intervention Evidence: How well were needs assessments conducted and interventions evidence collected to inform the cluster activities? o In what ways were the courses, trainings, and overall activity components designed to meet the unique needs of grantee staff and other GBV responders based on the survivor groups (e.g., domestic violence, child sexual abuse, refugees, trafficking victims, etc.) and the local context? o In addition to organization staff, grantees collaborated with external stakeholders, including GBV experts, counselors/psychologists, and partner organizations working on GBV prevention, among others. What were the contributions of these additional stakeholders and was their participation valuable? • Assumptions: What assumptions were made to design and implement the activity clusters? How accurate were any assumptions? • Causal Pathways: What causal pathways or theories of change were articulated for the ACs? • Monitoring and Adaptation: How well are interventions monitored and are emerging findings contributing to intervention adaptations or improvements? 5 Similar activity cluster and implementation evaluation questions were asked across all four activity clusters. Additional CARE-GBV–specific questions are shown in italics. xv | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ACTIVITY CLUSTER QUESTIONS 2. To what extent are each of the activity clusters achieving the targeted GBV results? • Outcomes: Are the stated outcomes realistic and achievable within the timeframe of the AC? What progress is being made toward achieving the outcomes? • Planning and Activity Design: How and how well were activity plans and designs developed to achieve different GBV outcomes? • Intervention Implementation: How well are interventions implemented to reach their target groups and influence change? o CARE-GBV focuses on building staff wellness and resilience among GBV responders. Which interventions had an effect? Which interventions have not performed as anticipated? Why? o What do you think is the biggest obstacle in minimizing vicarious trauma? How did your program address this? • Mechanism: What are the most effective aspects of the interventions? How do these “active ingredients” operate in each AC? 3. To what extent are the activity clusters sustainable? • Sustainability: What aspects of the ACs contributed to their sustainability? What components are needed for greater sustainability? o What are the primary ways in which this activity has changed the way that grantees address vicarious trauma? o How sustainable are online courses, self-reported wellness tools, workshops, and other such mechanisms developed and implemented under this activity? o Do you envision any barriers to continued implementation and uptake of these resources? • Replicability, Transferability, and Adaptability: In what ways are the ACs replicable in the same contexts? Adaptable for other contexts? • Scalability: What aspects of the ACs are most amenable to be scaled up? IMPLEMENTATION EVALUATION QUESTIONS6 1. Is the activity design based on the local context and flexible to achieve results on the ground? • Design: What factors contributed to the design of the activity? How were priority GBV problems identified? o There is no common theory of change for the CARE-GBV cluster. What prompted SOAR to develop an independent theory of change for the activity? How is this grounded in the local context? • Implementation: What are the key interventions’ methods to achieve objectives? • Flexibility: Is there sufficient staffing to respond to local priorities? Is there flexibility to change approaches to respond to lessons and changing challenges in the local environment? 6 These questions were posed to one of the CARE-GBV grantees selected for the implementation evaluation: Sexual Offences Awareness and Response Initiative (SOAR) in Nigeria. xvi | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ACTIVITY CLUSTER QUESTIONS 2. Is the activity reaching participants they are meant to target? • Target participants: What are the barriers to reaching participants? o SOAR provides counseling services to survivors of child sexual abuse. Can you tell me if and how this unique focus was incorporated into the study design? What factors were considered so they can better support this group of GBV survivors? What are the specific self-care and wellness needs of GBV responders working with child survivors of sexual violence? • Monitoring: Is the activity collecting evidence on what is working, not working, and what could be done differently to achieve results? 3. Is the activity achieving sustainability? • Sustainability: What plans are in place for sustainability? What is the evidence of potential sustainability? 1 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV 1. PROJECT BACKGROUND To support GBV organizations in building their capacity and ability to manage vicarious trauma from working with survivors, the CARE-GBV Small Grants Program awarded grants from $50,000 to $125,000 over a one-year period (July 2021–July 2022) to five organizations through an open call.7 The grants were awarded to new, local, and under-utilized8 partners to improve staff wellness and resiliency in GBV programming, fill global evidence gaps related to self- and collective care and wellness for staff of GBV organizations, and promote learning. All five grantees selected for the CARE-GBV Small Grants Program were led by women, including women who identify as survivors of GBV. At the onset of the evaluation, NORC reached out to all five grantees who confirmed interest and availability to participate in the portfolio and AC evaluation. The table below presents the five activities that were included in the CARE-GBV evaluation. Table 4. Activities Included in CARE-GBV Evaluation No. Activity Organization Country Funding (USD) Activity Components Target Beneficiaries9 1. Supporting Innovative Practices in Self￾Care, Wellness, and Resiliency among GBV Workers in North Macedonia Crisis Center Hope (CCH) North Macedoni a $69,581 Working in partnership with Pleiades Organization, CCH planned to: • Develop a training curriculum on self-care, wellness, and resiliency of GBV workers at CCH and partner organizations. • Conduct two training workshops comprising nine modules. • Organize a national conference for dissemination of best practices in policies and work protocol. • Develop and disseminate a guide for GBV workers as a key tool for support in self￾care. • Provide mentoring and psychosocial support to GBV workers and organizations. 7 CCH staff and 23 external GBV professionals 7 The CARE-GBV Small Grants Evaluation Committee reviewed 518 applications from 68 countries. 8 Grassroots organizations that have potential but have been neglected/not adequately included in GBV programming. 9 The number of survey respondents differs from the number of target beneficiaries for some activities. This is because some grantees (CCH and SOAR) were able to reach more beneficiaries than anticipated who were also interviewed as part of the evaluation. In the case of WAR, 6 emails bounced back so we were only able to reach 33 beneficiaries. 2 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV No. Activity Organization Country Funding (USD) Activity Components Target Beneficiaries9 2. Promoting Staff Wellness and Resilience for Effective Response to Sexual and Gender￾Based Violence Programmi ng Sexual Offences Awareness and Response Initiative (SOAR)* Nigeria $112,691 SOAR’s intervention aimed to strengthen its institutional capacity and equip other CSOs in Nigeria’s Federal Capital Territory region to promote staff wellness and resilience and undertake effective GBV prevention and response. At the onset of the project, SOAR also planned to conduct a Stress Risk Assessment Audit to identify and control potential causes and areas of work-related stress conditions of staff. Additionally, SOAR planned to conduct: • Focus group discussions • Training on trauma counselling and psychosocial support for child survivors of sexual abuse • On-site learning visit to the Domestic and Sexual Violence Response Agency of Lagos State • Self-care and wellness meetings with relevant stakeholders • Stakeholder consultation to review existing sexual abuse and exploitation policies. • Consultation to develop training manuals 8 SOAR staff and 24 external GBV professionals 3. We Care— Institutiona lizing Accessible Staff Wellness and Resilience Policies, Tools, and Practices for the GBV Field Sexual Violence Research Initiative (SVRI) Global $124,000 SVRI’s intervention aimed to strengthen and advance work on wellness, resilience, and care, both internally and globally, by: • Developing an online course comprising four modules focused on self- and collective care, wellness, and resilience, including a focus on institutionalizing policies and practices that support staff well-being and resilience. • Hosting a knowledge-exchange series focused on self-, staff-, and collective care, wellness, and resilience, including live events and knowledge products. • Institutionalizing staff care policies and practices within SVRI. SVRI developed the online course and knowledge products as resources for the GBV field. There was no target number of beneficiaries as the course is self￾paced, flexible, and available when people want to participate. 3 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV No. Activity Organization Country Funding (USD) Activity Components Target Beneficiaries9 4. Thuso Ya Bathusi (Enhancing Staff Resilience and Wellness) Women Against Rape (WAR) Botswana $93,728 WAR’s project goals were to: • Build its institutional capacity to prevent, recognize, and respond to the presence of vicarious trauma and promote emotional resiliency. • Build staff capacity to better support and respond to the needs of survivors of GBV. • To achieve these objectives, WAR planned to: • Establish a permanent Human Resources position to serve as the health and wellness officer who would be responsible for monitoring and supporting staff well-being, managing training, and evaluating the effectiveness of interventions. • Develop a locally relevant training curriculum of six 2-hour modules. • Develop a smart phone-based Wellness Check-In Tool (WCT) to enable counsellors, particularly those working in remote locations, to share feelings, experiences, and challenges. 39 WAR staff 5. Udruzenje Žene sa Une (Žene sa Une Staff Wellness Program) Žene sa Une (ZSU) Bosnia and Herzegov ina $100,000 ZSU’s intervention aimed to help internal staff at other GBV prevention and response organizations move from a sense of threat to a sense of safety. The planned to undertake the following activities: • Use somatic techniques to renew bonding among staff and help them examine the sociopolitical context and systemic interdependencies, while being cognizant of the overlap between one’s personal and professional life. • Facilitate training sessions about staff wellness, care, and resilience, as well as demonstrate and model approaches to embed these principles into the organizational culture. • Disseminate findings externally to promote awareness among other GBV prevention and response actors and stakeholders in the sector. 11 ZSU staff Note: * Activity selected for the implementation evaluation. 2. METHODS AND LIMITATIONS As the first step, NORC conducted a preliminary review of program documents and an evaluability assessment to ensure that all grantees were evaluable: all five grantees were. Review of program documents also guided conversations with grantees and informed instrument development. NORC employed a mixed-methods approach, including a desk review and qualitative and quantitative data collection to generate credible evidence to answer each evaluation question. The methods included key informant interviews (KIIs) with various stakeholders (USAID, MCI, and grantees) and a web survey administered to grantee staff and other program participants (as relevant). The implementation 4 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV evaluation was also conducted in parallel. As part of this process, NORC collected mixed-methods implementation-specific data from SOAR staff, partner organizations, and program participants. Figure 1. Evaluation Research Design Desk Review Review of 48 key program documents, including progress and final reports, training manuals and curricula, policy documents, and MEL data gathered by MCI. Quantitative Approach One web survey completed via Qualtrics by 142 respondents (including grantee staff and staff from other organizations that received training. Questions were customized for each activity. Qualitative Approach 15 key informants interviewed: 1 USAID GenDev staff 2 MCI staff 12 grantee senior staff (across all five activities) DESK REVIEW NORC conducted a thorough desk review of program documentation for all five activities. A total of 81 documents were obtained from MCI and reviewed for information across all evaluation questions. Documents included progress reports, training manuals, attendance records and reports, theory of change, policy documents, final reports, monitoring data, public outreach and communication documents, workplans, Monitoring, Evaluation, and Learning (MEL) plans, and other program documents. Program documents were reviewed and analyzed using a detailed Excel matrix that was organized by research questions and sub-questions. KEY INFORMANT INTERVIEWS Following the desk review, NORC conductedvirtual KIIs with 16 respondents including USAID/GenDev staff, MCI staff, grantee senior staff, and local partners via Zoom. We interviewed three SOAR staff for the implementation evaluation. Interviews ranged between 60–90 minutes and were audio-recorded (with respondents’ consent) to enable transcription and in-depth analysis. Seven interviews were conducted in English, and one was conducted via simultaneous translation between Bosnian and English. After administering the informed consent protocol, NORC used a semi structured interview guide that was organized by portfolio and AC-level topics. Interview guides included questions on needs assessments, causal pathways, monitoring and adaptations, planning and design, intervention implementation, and sustainability, among other evaluation themes. See Annex C for all KII guides. Transcripts were uploaded to MAXQDA (version 2022), a qualitative analysis software for coding and analysis. A detailed codebook was iteratively developed and tested to include portfolio level, AC level, and implementation evaluation codes across activity clusters in this PPE, including codes that were specific to CARE-GBV. KIIs were analyzed using the codebook, and coded segments were exported to conduct in-depth analysis and prepare code summaries. 5 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV SURVEY In addition to the KIIs, NORC administered five web-based surveys via Qualtrics, accessible through desktop and mobile devices. The survey was administered to a total of 1,121 respondents comprising grantee staff, partner organizations, and GBV responders who participated in program activities (Table 5). Once grantee senior staff confirmed that all participants had internet access, NORC shared a unique link with each participant via email using the Qualtrics server—except for SVRI, who opted to coordinate the process and distribute an anonymous survey link to ensure confidentiality. For ZSU staff, the survey was translated into Bosnian, and responses were translated back into English. The surveys covered respondent demographics and participation in activities, effectiveness of the activities, most helpful/useful components, and sustainability. Each survey included a mix of multiple￾choice questions, Likert scales, and open-ended questions. Using survey logic, certain modules or questions were shown to respondents based on previous answers and whether they were internal or external to the organization. This way, respondents were only asked about the activities they actually participated in and were not overburdened with irrelevant questions. The survey introduction included a detailed consent script describing the purpose of the survey and an informed consent statement. Respondents were required to provide their consent before beginning, or the survey would automatically end. The surveys were each open for a month, during which the research team regularly monitored survey responses and sent three email reminders to increase response rates. See Annex C for a paper version of the survey. Table 5. Web Survey Response Rates No. Organization Sample Size # of Responses Women Men Other Age Range Response Rate 1. Crisis Center Hope (CCH) 35 19 19 0 0 27–74 54.29% 2. Sexual Offences Awareness and Response Initiative (SOAR) 36 24 22 1 1 24–49 66.67% 3. Sexual Violence Research Initiative (SVRI) 1,006 7410 56 15 3 25–60 7.36%11 4. Women Against Rape (WAR) 33 18 10 7 1 25–60 54.55% 5. Žene sa Une (ZSU) 11 7 6 1 0 26–56 63.63% Total 1,121 142 113 24 5 In total, 142 respondents completed the surveys across all five activities. The survey reached a greater percent of women compared to men, and 3 percent identified as gender nonconforming or nonbinary (Figure 2).Survey data were cleaned to exclude unfinished responses, after which each survey was analyzed separately and then compared by activity. The evaluation team generated frequency tables for each question using Stata (version 16.1) and then created charts and figures. 10 55 percent of respondents completed at least 1 module of the SVRI training. 36 percent could not remember how many modules they had completed, and 8 percent did not respond. 11 The CARE-GBV small grants program supported the pilot launch of the Dare to Care course before it was made public. During the grant period, 83 pilot participants signed up for course, 14 of whom completed the course. Since making the course public and during the time of this evaluation, 1,006 participants had enrolled in the course. Course completion rates were not tracked after the pilot period. 6 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Results were disaggregated by gender where the surveys had a sufficient sample size of multiple genders. This was applicable to surveys administered to WAR and SVRI, but not SOAR, CCH, and ZSU, which had primarily women respondents. NORC has reported survey results based on gender where there are variations in how different genders perceive the effectiveness of approaches. In cases where the team found no distinguishable differences, aggregated results are included. LIMITATIONS Evaluation data were limited in scope for several reasons. First, given the timeline of the CARE-GBV Small Grants Program and subsequent evaluation, many grantees did not have data on the activity’s effectiveness beyond the grant period. Second, in terms of qualitative and quantitative data, the evaluation was limited by recall bias, where respondents cited lack of remembrance of specific aspects of the activity. Third, the data is also limited by social desirability bias—a response bias where respondents tend to answer questions in a manner that will be viewed favorably which can lead to underreporting of bad, unintended, or negative outcomes. Fourth, the quantitative component of the evaluation was limited by survey response rates due to remote data collection. NORC directly distributed survey links to organization staff and external program participants for four grantees. However, some emails bounced back because respondents had changed jobs, and the email on file was no longer active. In some cases, grantees were able to provide alternative email addresses, but for some respondents they did not have the most updated contact information. In total, six emails from WAR and one email from SOAR were not valid, with no available alternative contact information. In cases where the email was delivered, some respondents were unresponsive to the survey or did not respond to all questions, despite the soft nudge programmed into the survey to remind respondents about unanswered questions. Fifth, given that some respondents did not participate in the web survey, there is a possibility of selection bias. It is possible that respondents who chose to complete the survey might differ from those who did not in terms of their attitudes, perceptions, socio-demographic characteristics, and experiences. Fifth, there is also a known tendency among respondents to under-report socially undesirable answers and alter their responses to approximate what they perceive as the social norm, called halo bias. This manifests in responses from training recipients who may respond favorably as beneficiaries of CARE-GBV support who are reflecting on the benefits of a program they have already completed. Finally, it is important to note that SVRI’s activity is different from those implemented by the other four grantees, which makes it difficult to compare findings. Unlike the other grantees, SVRI does not directly provide services to GBV survivors. Instead, SVRI is the world’s largest network of researchers, practitioners, policymakers, and other stakeholders that address GBV. The organization contributes to ending GBV by building evidence, particularly in low- and/or middle-income countries (LMICs), through Figure 2. Survey Respondent Demographics 7 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV research and practice-based knowledge, strengthening capacities, promoting partnerships, and influencing change. Due to the low response rates, it is difficult to make generalizations about activity performance and effectiveness solely based on survey responses. To account for this challenge, NORC’s methodology included qualitative data collection and a desk review to triangulate findings from the survey across the three data sources. 3. FINDINGS, CONCLUSIONS & RECOMMENDATIONS A. FINDINGS ACROSS GRANTEES FOR THE ACTIVITY CLUSTER QUESTIONS Table 6. Activity Cluster Findings CCH SOAR SVRI WAR ZSU EQ 1: Are the activity clusters based on context-specific and international evidence? • Assessed existing formal and informal policies and practices in the country. Drew on actual cases to understand staff needs. • Partnered with Pleiades Organization, subject matter experts, state institutions working with GBV survivors, Ministry of Internal Affairs, and inspectors for domestic violence. • Adapted programming to virtual/ hybrid format during COVID￾19. • Modified stakeholder outreach strategy to include informal modes of contact and accommodate local election schedule. • Conducted needs assessment consultations with its staff and other GBV service providers. • Developed a theory of change (TOC) to guide its intervention. • Programming was grounded in the humanitarian principle of “Do No Harm.” • Assumed that organizations working in GBV prevention would be interested in participating in the project. • Conducted a needs assessment survey and a desk review. • Developed a TOC to guide its intervention. • Embodied principles of feminist learning, decolonizing knowledge approaches, and collective care. • Assumed that the course would be effective in driving change and completion rates would be high. • Partnered with subject matter experts from Rutgers University and University of Pennsylvania. • Adapted programming to virtual/hybrid format during COVID-19. • Implemented a smartphone based WCT monitor emotional health and well-being. • Conducted individual assessment meetings with each staff member. • Adopted the Common Threads Project (CTP) methodology, which was initially developed to support women survivors of war, displacement, and SGBV. • Adapted programming to accommodate a 30–40 percent increase in need for GBV services during COVID￾19. 8 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV CCH SOAR SVRI WAR ZSU EQ 2: To what extent are each of the activity clusters achieving the targeted GBV results? • Staff survey responses were generally positive about workshops provided by CCH. • Management cited a lack of literature and resources around designing programs that focus on building self-care and wellness practices of GBV responders. • According to survey respondents the focus group discussions (FGD), monthly wellness meetings, and three-day onsite training were the most effective mechanism utilized. • Management staff cited their wellness policy as the most effective element to institutionalize wellness within the organization. • Management cited struggles with developing a self￾paced, online program that would be contextually relevant for a global audience. • Survey respondents said learning aides such as videos or reflective activities were the most impactful. • Partnered with researcher to help fill knowledge and practice gaps. • Course curriculum and the vicarious trauma trainings were most useful to staff. • Mixed reactions about the effectiveness of the WCT; both in utilization by management and technological issues with web links. • Focused on the body in their work and calming the nervous system. • Staff surveys showed they believed these trainings helped minimize their residual trauma. • Management noted that their wellness policy was an effective tool to institutionalize wellness within their organization. EQ 3: To what extent are the activity clusters sustainable? • Knowledge gained from the workshops is being shared in their organization. • A “training of trainers” approach was cited as beneficial for scaling up these practices. • Policies laid out in internal training manuals and are being implemented by other organizations in their network. • Funding is a barrier to sustainability and scalability of the program. • Online course has been maintained. • Currently scaling up by having the course translated into Spanish. • WCT has been maintained for staff check-ins. • Course curriculum is in the process of being accredited. • Self-care plan continues to be implemented by the organization. • Elements can be replicated with contextualiz￾ation. EVALUATION QUESTION 1: ARE THE AC’S BASED ON CONTEXT-SPECIFIC AND INTERNATIONAL EVIDENCE? NEEDS ASSESSMENTS AND INTERVENTION EVIDENCE: How well were needs assessments conducted and intervention evidence collected to inform the cluster activities? All five grantees conducted some form of needs assessment to inform their proposal submission and/or intervention design. Recognizing that each grantee differed in its capacity, resources, and programming priorities, there was no mandate for organizations to conduct a specific type of needs assessment. Instead, grantees were encouraged to tailor the needs assessment to the local context and intervention design. Qualitative findings suggest that grantees confirmed the need for interventions to address vicarious trauma, 12 protect staff well-being, and build their resilience. Grantees 12 Vicarious trauma is the result of being exposed and empathically listening to stories of trauma, suffering, and violence, which may worsen with repeated exposure to traumatic material. 9 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV adopted various techniques for their needs assessment, as summarized below: • Program documents indicate that CCH assessed existing formal and informal policies and practices related to staff’s self-care, wellness, and resiliency in the country. They reviewed legal and policy frameworks, reports, analyses, web pages, media sources, and other relevant data. Further, through KIIs, NORC learned that CCH also drew on actual cases they were dealing with to understand staff needs. • SOAR conducted focus group discussions with its own staff and other GBV service providers to understand health (mental and physical) impacts of the job and the state of self-care needs and practices in Abuja, the Federal Capital Territory of Nigeria. Through these conversations, they learned that there was poor understanding and implementation of self-care practices among SGBV responders, which had led to severe burnout and emotional breakdowns. As shared during the KII, SOAR also learned that many organizations providing GBV services, particularly psychosocial services, did not have trained professionals. Instead, GBV responders “were working out of passion and lacked appropriate training.” [SOAR KII] • SVRI conducted a needs assessment survey that was open to all members of the organization to understand what they would like to see covered in program activities, their understanding and practices of self- and collective care, and any challenges they faced during COVID-19. Program documents revealed that survey respondents highlighted increased workloads, an inability to separate work and personal lives, having to undertake more unpaid care work, and feelings of isolation. Although respondents articulated for more collective care, they had little to no understanding of collective care but had higher perceived knowledge of self-care. As explained by SVRI, this can be attributed to social media narratives that tend to individualize notions of self-care, neglecting to acknowledge the ways in which stress and trauma can be experienced collectively. In parallel, SVRI also conducted a desk review to explore the current state of evidence on how well￾being, resilience, and self- and collective care can be institutionalized in GBV prevention across LMICs. Academic and gray literature published between 2011–2021 were extracted from online databases. Findings from the different types of needs assessment informed the development of a self￾paced online course, learning events, knowledge exchange series, and case studies. • WAR conducted a survey through Qualtrics to guide its grant application. The purpose of this needs assessment was to understand how staff felt about the job, the effect it had on them, and their needs to address vicarious trauma. Through this exercise, WAR learned that staff were experiencing immense stress and burnout, which subsequently informed their proposal submission and intervention design. One staff member poignantly summarized their feedback in the survey as follows: “It’s very draining to my soul. Sometimes I struggle to sleep. When people tell me their stories, I get really shocked and then this helplessness thing it doesn’t matter what we do, nothing makes a difference.” [WAR KII] • ZSU conducted individual assessment meetings with each staff member to understand “the roles that they played in the organization, the particular stressors that they were challenged with, and what they might already be doing to resource themselves.” [ZSU Progress Report] Through these interviews, ZSU learned that GBV responders found it difficult to regulate emotions, establish a healthy separation between work and personal life, and manage bodily symptoms experienced as a result of vicarious trauma. These findings subsequently informed their intervention design to promote staff wellness, care, and resilience. Further, grantees recognized the need to address gaps in promoting staff self-care, well-being, and resilience, especially in light of the COVID-19 pandemic that exacerbated stress and accelerated 10 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV burnout. Through KIIs, NORC learned that many grantees experienced a significant increase in GBV caseload during the COVID-19 pandemic, which put added strain on resource and capacity constrained grantees serving GBV survivors. Increased demand for GBV services and pandemic-induced stressors prompted grantees to prioritize staff well-being and resilience to better cope with vicarious trauma. The following quotes capture these perspectives: “The Corona period opened our eyes about our mental health…. Everyone was really scared because of everything that was going on. Before starting the implementation of this project, we can say that the idea was created based on our opinions and needs in those moments. We wanted to create something that could help other professionals who were struggling in this period.” [CCH KII] “Issues of vicarious trauma came around due to COVID-19. Staff were reporting this in meetings because we had so few staff working. [COVID-19] cases were astronomical, and we had situations where staff were breaking down. Staff really did not know how to report or handle it. Almost no organizations had self-care practices [in place] to support their staff. So, this information largely determined what we decided to do on the project.” [SOAR KII] “Working with vulnerable people and providing support in various crises situations, we concluded that they need additional support during the pandemic because the level of violence had increased. We had to take steps to mitigate risks for our clients, but also for our staff.” [ZSU KII] Grantees consulted staff, GBV organizations, subject-matter experts, and other key stakeholders in the needs assessment process. Stressing the need to involve beneficiaries in intervention design, one grantee said: “It’s important to address the needs of the beneficiaries. And you can show from the survey that this is what people are feeling, this is what they want, and this is the project you are going to design so that we address this need.” [WAR KII] This finding is validated by survey respondents who mostly felt that their needs were considered when grantees designed training programs, curriculums, and workshops (Figure 3). Figure 3. Integration of Participant Needs Source: NORC web-based survey 1 1 1 1 6 2 3 14 11 14 3 0 2 4 6 8 10 12 14 16 18 20 SOAR: To what extent do you think the training manuals meet the needs of GBV responders (n=15) WAR: To what extent do you think the specific needs of WAR staff was considered while developing the curriculum? (n=18) CCH: To what extent do you think the content covered in the workshop met the needs of GBV responders? (n=18) ZSU: To what extent do you think the needs of ZSU staff was considered while developing the training program? (n=7) Not at all Neutral Somewhat To a great extent 11 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV In terms of partner selection, CCH noted that they selected Pleiades Organization based on their previous collaborations and the organization’s focus on work supporting women from minority groups. Given the multi-ethnic composition of Macedonian society, CCH wanted to include more ethnic groups in the project. In addition to Pleiades Organization and subject matter experts, CCH also collaborated with state institutions working with GBV survivors, the Ministry of Internal Affairs, and inspectors for domestic violence, which CCH believes strengthened partnerships. These perspectives are captured in the following quotes: “Pleiades Organization works specifically with Albanian women....This cooperation actually had an added value to the project because we were able to include other organizations that are working in the same area, but have different background, speak a different language.” [CCH KII] “Of course, we have to have cooperation with the institutions all the time when we deal with victims of GBV. But this project, maybe, brought us a little bit closer and strengthened our cooperation so after the sessions we could call them [inspectors] personally, not through the institution.” [CCH KII] WAR partnered with subject matter experts from Rutgers University and University of Pennsylvania, both of whom WAR had previously worked with and contributed to program design. WAR also engaged academics from University of Botswana to understand and integrate local needs and perspectives. ZSU formally partnered with Common Threads project, but informally, collaborated with local organizations that support survivors of trauma. Common Threads had extensive experience in conducting workshops on mind, body, and stress management, especially in the context of war and conflict. This informed the design of workshops conducted as part of the ZSU’s intervention in Bosnia, where people have lingering war-related trauma. ASSUMPTIONS: What assumptions were made to design and implement the activity clusters? How accurate were any assumptions? Program documents and KIIs indicate that USAID, MCI, and grantees made certain assumptions related to intervention design, some of which held true while others did not. An important consideration during the design of the small grants program was that GBV prevention work takes a toll on staff care and resiliency. Further, many GBV responders are survivors or bystanders, which increases their risk of re-traumatization. As a result, staff may experience burnout that could manifest in the form of headaches, fatigue, lowered immune function, irritability, and loss of interest in the work. Recognizing the high rates of staff turnover, burnout, and mental health impacts, USAID wanted to invest in staff who are dedicated to work in GBV prevention but leave the job because of recurring trauma (vicarious trauma and re-traumatization) that they experience at work. MCI further noted that grantees were encouraged to contextualize assumptions and use a participatory approach, by engaging survivors and experts in their discussions to ensure that assumptions are informed by the community. Some examples are summarized below:13 • According to program documents, SOAR assumed that organizations working in GBV prevention who understand the importance of self-care and wellness would be interested in participating in the project. They also assumed that facilitating conversations through an established network of GBV organizations would help them get support from other member organizations working in the GBV field. Key informants mentioned that by leveraging the Sexual & Gender-Based Violence Response Team (SGBV-RT) network in Nigeria’s Federal Capital Territory (FCT) region, they were able to reach out to other GBV organizations and engage them in program activities. 13 NORC did not learn of any assumptions related to WAR and CCH activities. 12 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV • SVRI’s intervention was rooted in two main assumptions: 1) people would be willing to participate in and complete the online self-paced course, and 2) the course would be effective in driving change to make the GBV field kind, caring, and nurturing. SVRI had received overwhelming positive feedback on the course and had the highest number of enrollments compared to other online courses they offered. Although this feedback underscored the need for this type of intervention, few people actually completed the course. This suggests that both assumptions did not hold true. • ZSU initially attributed high staff turnover to their desire for a higher salary but later realized that staff were quitting due to work-related stress and vicarious trauma. This learning faulted their assumption and subsequently informed the design of their organizational policy that includes staff care as a key component. CAUSAL PATHWAYS: What causal pathways or theories of change were articulated for the activity clusters? Grantees were encouraged to integrate do-no-harm, 14 survivor-centered, 15 and participatory16 approaches into their programming. Although the grant was rooted in this ethical imperative to deliver safe, inclusive, and effective programming, there was no theory of change (TOC) at the AC level. Similar to the needs assessment component, there was no predetermined format or requirement for grantees to have a TOC. Therefore, only two grantees (SOAR and SVRI) developed independent TOCs to ground their intervention. SVRI developed a TOC because they believed that programming must be grounded in some sort of evidence-informed strategy, which is often a TOC for them and helps them think through the different aspects of the activity, contextual factors, and intended outcomes. According to the TOC, “Successful completion of the course and implementation of contextualized care strategies and systems would result in caring and supportive teams as well as caring and nurturing organizations.” [SVRI Theory of Change Narrative] It was posited that these caring institutions at the micro level would then contribute to a culture shift in the whole field where GBV providers are resilient and able to cope well with work-related stress and trauma. Consequently, it was expected that this kind of workforce would lead to improved sustainability and effectiveness of GBV prevention programs and build a vibrant community of practice where innovation thrives, and equitable partnership accelerate learning and progress. In addition to the TOC, SVRI’s online course also embodied principles of feminist learning, decolonizing knowledge approaches, and collective care17 to reflect values that are central to GBV prevention. SVRI embraces collective care as “an ongoing process and journey, to continuously reflect on what is working, what is not working, and how we can grow or improve.” [SVRI Policy Document] Qualitative data and program documents indicate that SOAR considered issues that were important for their staff and local GBV organizations, gaps and challenges in programming, and activities that would address these barriers and help achieve the desired outcomes. According to SOAR’s causal pathway, improved counseling and psychosocial services for child survivors of sexual abuse, adoption of self-care practices by GBV responders as part of their lifestyle, and institutionalization of stress management and 14 The safety of GBV survivors, their dependents, and service providers was considered top priority. [MCI MEL Report] 15 A survivor-centered approach to GBV response that prioritizes survivors’ rights, needs, and agency should be integral to organizational practice. This means ensuring that survivors (including staff, who may be GBV survivors) have access to appropriate, accessible, and high-quality services. [MCI MEL Report] 16 Engagement of staff, proposed participants, and the community where activities will take place, and participation of diverse voices are considered essential. [MCI MEL Report] 17 Collective care encourages us to see well-being as a shared responsibility of the wider group rather than the sole burden of an individual. It encourages us to work collectively to change the oppressive systems that cause us stress, trauma, and harm. [SVRI Blog Post] 13 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV wellness practices by the grantee and other GBV organizations would result in improved support against vicarious trauma and burnout in Nigeria’s FCT region. “We wanted staff to be empowered with knowledge, information, and the skill to support their self-care and wellness so that, ultimately, they would be able to better support the children. So that basically informed the theory of change that we developed.” [SOAR KII] Although other grantees did not develop TOCs, their programming was grounded in certain theories, conceptual frameworks, or methodologies. For example, drawing on existing literature on resilience, ZSU emphasized the concept that resilience does not solely depend on one’s personal emotional state, but is also a factor of how we are influenced by social ecology. Given this, the grantee’s programming was grounded in the casual pathway that optimal work environments can contribute to building resilience and adaptive capacities to grow through adversities. ZSU also adopted a non-pathologizing approach where sensory traumatic stress (STS) was considered “as not something that people suffer from because they are weak or less than, but rather something they are challenged by because they took risks to engage empathically with people in vulnerable situations and allowed themselves to be deeply affected.” [ZSU Program Curriculum] Consequently, trainings were designed to foster an STS-informed culture and help staff strike a balance between their desire to help survivors and their need to prioritize self-care. ZSU also adopted the Common Threads Project (CTP) methodology, which was initially developed to support women survivors of war, displacement, and SGBV. In their view, “any implementation of the CTP program benefits from local partnerships where the partner organization considers staff care and wellness core to its mission and work. CTP facilitators are critical to the delivery of the program in their communities as they guide and support women through a delicate and nuanced process.” -- [ZSU Final Report] Similarly, review of SOAR’s safeguarding policy and training materials indicated that the grantee’s programming was grounded in the humanitarian principle of “Do No Harm.” With the intention of building protective environments, programs were implemented through careful risk management to create a safer environment for vulnerable populations. MONITORING AND ADAPTATIONS: How well were activity plans and designs developed to achieve different GBV outcomes? As mentioned earlier, the CARE-GBV Small Grants Program was introduced and implemented during COVID-19. The pandemic not only informed program design but also prompted grantees to modify intervention design and delivery due to COVID-19 impacts and restrictions. One KII respondent mentioned that witnessing an increase in GBV case load, their organization had to adapt programming to accommodate a 30–40 percent increase in need for safe house services (mostly related to domestic violence cases) and day care center service for children and families at risk. “The Covid-19 impact has stretched the organization and staff in their efforts to respond effectively and compassionately to the social crises taking place in their community.” [ZSU Program Curriculum] Similarly, many trainings and convenings that were designed to be conducted in-person had to be organized virtually. Although two grantees had international partners who were not able to travel, they made the best use of the collaboration through virtual engagements. For example, WAR trainers delivered face-to-face trainings in the mornings, including recorded presentations from U.S.-based trainers who connected virtually in the afternoons to debrief and answer questions. Grantee staff believe “this model worked well and ensured the best quality training in the face of challenging and unexpected circumstances.” [WAR Final Report] In addition to the pandemic, CCH’s intervention was also delivered at a time when local elections were underway, which posed added implementation challenges. Through program document review, NORC 14 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV found that CCH found it difficult to engage with GBV professionals because many were involved in the campaigning process. Consequently, they modified their outreach approach to contact them after local elections had wrapped up. As part of their outreach efforts, CCH also found that GBV professionals are more responsive when they are contacted informally via phone or text as opposed to formal communication channels such as email. When contacted informally, they are more open to sharing their opinions and willing to engage in program activities. Through their programming, they further learned that male identifying GBV responders did not want to engage in group discussions about self- and collective care, given the stigma around mental health outcomes. This prompted CCH to explore alternative ways of reaching out to and engaging male participants in the discussions. After COVID-19 measures were lifted, many events were scheduled at overlapping times which made it challenging for the IP to find a suitable time to conduct group mentoring and psychosocial sessions. Despite this unexpected challenge, they maximized their outreach efforts and recruited the desired number of participants. Some of these accommodations are detailed below: “We decided to hold both workshops online via Zoom instead of in-person as we had previously planned. We want to give equal opportunity to our colleagues from different [gender-based violence organizations] GBVO to attend the workshops regardless of their COVID status. Since the workshops will take place on ZOOM, we have the opportunity to invite more participants and at the same time to protect our health. Also, there are many different technical opportunities with ZOOM platform, that enable work group, polls, quizzes and using different application, to make the workshop interactive and interesting.” [CCH Progress Report] SOAR planned to conduct ongoing annual reviews of the Stress Management and Wellness policy developed from the grant. This monitoring exercise will be carried out by Human Resources in collaboration with the Wellness Committee to assess the effectiveness of the policy. They also developed a self-audit tool to help organizations measure whether they are meeting the standards for making children and vulnerable adults safe. In addition, SOAR recognized that they have a responsibility to ensure that partner organizations have safeguarding policies and procedures in place that protect vulnerable women and children. Given this mandate, they “would consider amending partnership agreements to reflect a commitment to and to putting protection measures in place in line with these standards and review the partner’s own safeguarding policy.” [SOAR Policy Document] As part of SVRI’s intervention, pilot participants were given a month to review course material and provide their feedback on course design and content via a survey and focus group discussions. The pilot aimed to explore general satisfaction, accessibility, contextual relevance, trauma-sensitive design, and gaps and recommendations, among other considerations. The grantee also conducted two case studies with GBV organizations (Raising Voices18 and HaRT19) to understand how they strive to institutionalize self- and collective care, both of which revealed that that many staff members and their families were directly impacted by the different waves of COVID-19. Recognizing the impact of the pandemic on staff’s personal lives, Raising Voices reduced staff workload, increased work flexibility by transitioning to monthly workplans (which allowed more time for staff to review and adapt regularly), offered individual and group counseling sessions, and made accommodations to provide extended sick leave. Internal 18 Raising Voices is a feminist nonprofit organization based in Kampala, Uganda, working toward preventing violence against women and children through three inter-related program areas: practice, learning, and influencing. The organization understands violence as an abuse of power—not just at individual levels, but also at the level of norms, systems, and structures. 19 HaRT is a feminist organization dedicated to holistic healing among women and girls who have experienced human trafficking and GBV. Their main program, Move with HaRT, is a 12-week intervention that uses a variety of contemplative practices, including yoga and mindfulness, as well as theme-based discussions, as a way to support collective healing and community among participants. 15 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV communication was also increased, with the leadership team conducting regular check-ins through WhatsApp groups. Survey respondents from WAR found the weekly smartphone-based staff Wellness Check-In Tool (WCT) at least somewhat effective to monitor emotional health and well-being. However, a majority of men responded that it was only “somewhat effective” whereas women responded that it was effective “to a great extent” (Figure 4). Figure 4. To what extent is the WCT effective in monitoring the emotional health and well-being of WAR staff? (n=17) Source: NORC web-based survey As the IP, MCI played an important role in collecting monitoring data to track activity progress. Program documents indicate that MCI developed a detailed MEL plan, including custom indicators and data collection templates that were reviewed and approved by USAID/GenDev staff. The CARE-GBV indicators were drawn from USAID’s gender indicator list and grantees were given indicator tracking templates for each of the key performance indicators. MCI conducted remote monitoring, support check-ins, and follow-up trainings with grantees to strengthen their MEL capacity and assist with completion of tasks. During the KII, MCI noted that grantees found creative solutions to implement the intervention despite pandemic-related restrictions and challenges. Further, as described below, grantees had built in their own monitoring and feedback mechanisms which informed program adaptations. “I think the organizations didn’t leave learning for the end. They really built in different mechanisms to get feedback and to capture lessons learned throughout and make changes.” [MCI KII] EVALUATION QUESTION 2: TO WHAT EXTENT ARE EACH OF THE ACS ACHIEVING THE TARGETED GBV RESULTS? OUTCOMES: Are the stated outcomes realistic and achievable within the timeframe of the AC? What progress is being made towards achieving the outcomes? Based on data gathered, all five grantees reported their proposed outcomes were realistic and achieved. When discussed in the KIIs, each grantee had generally positive attitudes about their 1 1 1 1 2 4 6 1 0 1 2 3 4 5 6 7 8 9 10 Women Men Don't Know Not at all Neutral Somewhat To a great extent 16 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ability to complete these outcomes This was corroborated by the web surveys. Throughout the surveys, survey participants were asked various questions about the effectiveness and learning outcomes associated with approaches utilized by their organizations under the CARE-GBV activity. All survey respondents from SOAR, who support child survivors of sexual assault, felt that the activities at least somewhat improved their understanding of vicarious trauma. Twenty responded that activities improved their understanding to a great extent, while three responded that the activities somewhat improved their understanding. Approaches such as the Group Discussion, three-day online learning visits, and Self-Care and Wellness Meetings were also received positively by respondents (Figure 5). Figure 5. Effectiveness of Approaches Used by SOAR20 Source: NORC web-based survey SVRI piloted its course with 83 persons prior to launching the online modules globally. Completion rates are summarized in Table 6. In their final report, SVRI discussed challenges associated with achieving high completion rates of self-paced, online courses. They noted that with these types of offerings, on average only about 5–15 percent of learners will complete the entire course. 21 Based on the pilot data, the SVRI course surpassed this estimate by two percentage points. Table 7. SVRI Course Completion Rates in Pilot Period Module Number Completion Rate (% of participants who fully completed the module as well as the preceding modules) Module 1 31% (n=31) Module 2 27% (n=22) Module 3 19% (n=16) Module 4 17% (n=14) [These pilot participants completed the full course] Source: SVRI Progress Report 20 The number of respondents varies in Figure 5 as these questions were only asked to those who had actually participated in the activity. Twenty-three respondents participated in the Self-care and Wellness Meetings, four participated in the 3-day online learning visits, and 10 participated in the Focus Group Discussions. 21 Melchor, E. (2021, October 15). “How to Boost Online Course Completion Rates in 4 Easy Steps.” Thinkific. https://www.thinkific.com/blog/improve-course-completion-rates/ 1 1 2 2 1 6 7 3 14 0 2 4 6 8 10 12 14 16 18 20 To what extent did the Focus Group Discussion help you understand self-care requirements and practices to prevent burnout and secondary trauma? (n=10) To what extent did the 3-day online learning visit help you identify good methods that may be emulated from existing GBV responsive structure with track record of excellent delivery? (n=4) At the end of the Self-Care and Wellness Meetings, to what extent did you observe an improvement in the self￾care practices of staff who are on the frontline of addressing sexual and gender-based violence in the… Very little Neutral Somewhat To a great extent 17 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Participants who completed the training provided by SVRI felt that the training at least somewhat helped them accomplish the tasks described in Figure 6. However, 27 of 68 respondents who answered the question on module completion reported that they did not remember the modules they had completed. Twenty-four respondents reported completing all four modules, and seven respondents only completed the first module. Figure 6. SVRI Training Key Skill Development (n=68) Source: NORC web-based survey In short answer responses, SVRI training participants appreciated the emphasis on self-care and self￾reflection throughout the courses as well as the methods described for dealing with burnout and vicarious trauma. One survey respondent described their enhanced ability to identify feelings of burnout after participating in training: “I realized I had burnout and [was] overstretched. So, I now have better understanding of stress, burn-out, and vicarious trauma, and recognize the signs in myself and others especially child protection network members in Lagos State who are my colleagues.” [SVRI Survey Respondent]. Similarly, the majority of CCH workshop participants described that the workshop at least somewhat helped them develop skills related to stress management and wellness (Figure 7). 7 6 7 1 1 1 1 4 8 7 6 20 24 25 31 29 26 0 10 20 30 40 50 60 70 Develop a shared understanding of stress, burn-out and vicarious trauma and recognize the signs in yourself and your colleagues. Develop your own definitions, practices, and systems of self and collective care, including understanding how they are interlinked and interdependent Identify the factors undermining and enabling collective care in your organization or team and know what works to institutionalize collective care No response Not at all Very little Neutral Somewhat To a great extent 18 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Figure 7. CCH Skill Development (n=18) Source: NORC web-based survey Additionally, ZSU staff surveyed responded that the Staff Wellness and Resiliency training helped strengthen their understanding in critical areas of minimizing residual trauma; however, they also felt that the training was slightly more successful in strengthening their understanding of boundaries and how to manage their boundaries (Figure 8). Figure 8. ZSU Training Effects Source: NORC web-based survey The majority of the survey respondents from SOAR, WAR, and ZSU responded that the approaches instituted by the projects helped to shift the culture of their organizations “somewhat” or “to a great extent.” (Figure 9). 1 1 1 1 2 4 6 3 13 10 12 0 2 4 6 8 10 12 14 16 18 Acquire important information and new knowledge about different types of stress triggers, their recognition and naming Develop abilities and habits for maintaining well-being, calmness, and self-esteem in stressful situations Develop an understanding of and apply practices to preserve your health (physical, mental, emotional, and sensory) Not at all Neutral Somewhat To a great extent 1 1 3 3 4 4 3 2 0 1 2 3 4 5 6 7 To what extent did the training strengthen your understanding of boundaries and how to manage and protect your boundaries when providing services to GBV survivors? (n=7) To what extent did the training strengthen your understanding of residual impact for GBV responders? (n=7) To what extent did the training strengthen your understanding of ways to minimize negative effects and maintain an empathic position? (n=7) No response Neutral Somewhat To a great extent 19 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Figure 9. Since the implementation of this activity, to what extent has the culture of your organization changed to be understanding of and responsive to vicarious trauma?22 Source: NORC web-based survey Short program timelines were commonly reported by grantees when asked about challenges related to achieving project outcomes. Four of the five grantees mentioned having a limited ability to reach project outcomes or monitor their work due to the timeframe. “We struggled a little, but we achieved everything on time. But for a project of this type I believe that it should be a little bit longer than 12 months…and at the end we have some rush to do everything in time. But for project of this kind with all of these activities that should be a little bit longer.” [CCH KII] This sentiment was echoed by other grantees that while they were able to get all deliverables in on time, delays in project start dates in particular (mostly due to fund disbursement as explained below), made reaching these goals a challenge. SVRI and WAR noted that their projects were delayed at the beginning by two to three months, leaving them with about nine months to implement their projects. Both grantees explained the delays were due to delays in fund disbursement—without which they were unable to begin work. SVRI mentioned that these delays meant they only had four weeks to pilot their online course, which led to low completion rates of the later modules as users were unable to complete the entire course in such a short timeframe. This same grantee mentioned these delays also meant they were unable to market their course to a wider audience geographically, which they had initially intended to do. Related to project delays, two grantees explained the main factor for this was a delay in fund disbursement. This posed a problem for these grantees since they are small, local organizations and often do not have the capital on hand to begin implementing projects until funding is received. “The start date for the project was held up due to the delayed disbursement of funds, and it took two months off the project time. So that was two months where we could have actually been starting and getting things going. But we are a small organization. We can’t do anything until we actually get the money, and you know it’s not that we have capital that we can draw on to implement projects: we actually have to wait until the money is in the bank.” [WAR KII] Conversely, three of the five grantees (SOAR, WAR, and ZSU) discussed unintended outcomes from their projects. They spoke of them positively and were generally pleased with those results. For example, WAR had initially felt frustration at USAID pulling them into several projects, most notably taping a podcast, that fell outside the scope of work and felt that USAID was 22 No data from SVRI due to only one respondent being a current SVRI staff member. Additionally, questions about cultural changes within the organization were not asked to CCH respondents. 3 1 2 3 4 3 6 9 2 0 2 4 6 8 10 12 14 16 18 20 SOAR (n=9) WAR (n=17) ZSU (n=7) Very little Neutral Somewhat To a great extent 20 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV asking too much of them, but toward the end of the project they realized they had learned quite a lot from the experiences that USAID had asked them participate in: “[USAID] pulled us into a lot of things, and in the beginning…we were constantly being bombarded with requests to participate in this activity, do a podcast, do this, engage in that meeting. But then, about two-thirds of the way through, we suddenly realized actually we’ve learned a lot from these kinds of interactions…Reflecting back, we definitely did gain a lot. It was fantastic.” [WAR KII] The other two grantees both mentioned unintended outcomes related to their staff sharing knowledge they had learned with others in their community and having impacts on others outside their immediate staff. While there were no mentions of negative unintended outcomes, we understand negative outcomes can likely be underreported by grantees (social desirability bias) or not remembered (recall bias). PLANNING AND ACTIVITY DESIGN: How and how well were activity plans and designs developed to achieve different GBV outcomes? As mentioned in the section on Needs Assessments, all five grantees conducted a needs assessment that was utilized to inform the project design and ensure the projects met participants’ needs. At a high level, the CARE-GBV activity was intentionally designed to be flexible and accommodating to the needs of local organizations. As mentioned in the KII with MCI, “These [CARE-GBV activities] were really flexible, and meant to be tailored to the context, to the organization, and where they were at. So, from my vantage point, I think that local priorities [and] organizational priorities are at the heart of what these grantees were doing.” [MCI KII] Additionally, it was mentioned that the thought process behind the RFP was to understand what types of local solutions work best for these first responders to trauma and how to support their own well-being. While the mechanisms utilized by the grantees were vastly different, their main goal was to center self￾and collective care in their program design. This was corroborated through KIIs and the document review process, where several grantee projects incorporated staff assessment points (e.g., surveys or interviews) at various intervals to allow for feedback and ensure the program content addressed the specific needs of staff. As one grantee explained, “We did some interviews…with each participant individually before we began the training, at the midpoint of the training, and at the end of the training. And so we just collected their words on what they were looking for, what they hoped to get out of it, what were they getting out of it, and then what did they get out of it. We [also] did a PANAS (positive affect negative affect scale) survey on a weekly basis during the training, and then we did one at the very end of the project. That sort of showed some benefits to the program.” [CCH KII] Lack of information and evidence related to self-care, wellness, and vicarious trauma of GBV responders was one of the most reported challenges during the planning and design stage of the activity. Throughout the KIIs, nearly every stakeholder group (MCI, GenDev, and grantees) mentioned a lack of evidence as well as little to no programs globally that were using similar methods. "When we were planning this project, we thought that we have to have some more specific information if there is any literature or anything that is used, whether within the civil society organization or an institution, and through the research we found out actually that there are no programs that [do this].” [CCH KII] To fill the above-mentioned gaps, these small, local organizations would often lean on their partners to assist in providing knowledge or guidance. One grantee explained, “Yeah, we had [a researcher] as part of our project, and she gave us a whole bunch of reading material prior to putting together the training, and the course, and the wellness project. So yeah, we have some very good, very kind of specific literature.” [WAR KII] Some organizations would look to other grantees within CARE-GBV for insight and guidance on aspects that were or were not working for those organizations and best practices. This engagement between grantees was informal in nature (virtual meetings were most common) but was viewed as beneficial by grantees. In addition, three of the grantees mentioned that the bimonthly meetings and presentations 21 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV between all CARE-GBV organizations and GenDev were useful for sharing best practices or lessons learned from other activities. One challenge mentioned by a grantee who created an online opensource course was that they found it difficult to tailor these programs to meet very specific needs worldwide. Especially once they considered the cost to develop and maintain such features that would allow for contextualizing their content. They highlighted “difficulties around how do you make referrals on a global level for a self-based course. We really struggled and grappled over that as a team. There are no ideal solutions I can say. Things that we saw that were online were way too expensive or just inaccessible to people outside of the global north, really, so that was tricky.” WAR said they focused on both people as well as systems when designing their activity (i.e., institutionalizing self-care and wellness efforts/training within the organization). “And I think we put a lot of focus on the body. On, you know, regulation of people’s nervous systems, because, you know, some of the work people end up in quite [a] charged up [state]. I think the second thing was, you know, our strong belief that it is not an individual responsibility to build resilience, and we really looked very carefully at the distribution of resilience across systems rather than making it some sort of just personal choice.” [ZSU KII] INTERVENTION AND IMPLEMENTATION: How well are the interventions implemented to reach their target groups and influence change? Every grantee agreed that they were able to reach their target audience. This is likely due to the fact that the audiences targeted were largely their organization’s staff members. Only SVRI mentioned that they wished they had more time to advertise their courses in wider geographical areas. Throughout implementation of the projects, grantees mentioned various strategies or guiding principles they followed. The most common throughout was incorporating a survivor-centered approach for staff. SOAR stressed the importance of this lens for their work: “A survivor-focused approach creates a supportive environment in which survivors’ rights and wishes are respected, their safety is ensured, and they are treated with dignity and respect. Recognizing that experiences of GBV often affect survivors’ sense of control, the survivor-centered approach aims to acknowledge and respect the survivor’s agency and autonomy by ensuring that she is the primary actor and decision-maker throughout the helping process.” [SOAR Document Review] Additionally, the implementation of each of these projects took place during the height of the COVID￾19 pandemic, which offered unique opportunities and challenges for the grantees. In general, most grantees reflected positively on their response to the pandemic and their ability to pivot projects accordingly. The largest challenge noted was the utilization of technology to hold meetings. Several grantees noted they needed to upgrade their technology capabilities or learn how to utilize existing systems (such as Zoom, Google Meet, etc.). However, CCH said they felt the COVID-19 pandemic actually improved their ability to implement this project as they were able to leverage online meeting platforms to engage GBV professionals in other cities around their country in a way they never would have considered or been able to previously. SOAR described challenges related to their partner organizations during implementation. They noted that participating organizations in their network would often send different staff to partake in meetings, which made it difficult to have consistent and effective engagement from these partners. Additionally, CCH mentioned challenges during implementation related to the political environment locally. At the time of implementation, local elections were taking place, which greatly reduced the involvement and political will of state institutions to partake in GBV-related activities. MECHANISMS: What are the most effective aspects of the intervention? How do these 22 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV “active ingredients” operate in each AC? Based on survey data, the most effective mechanisms were trainings or courses provided on psychosocial support, self-care, and wellness. This was common across responses from all five grantees, where respondents felt the knowledge obtained in these sessions helped them the most in their day-to-day work and personal lives. SOAR survey respondents highlighted the six monthly self-care and wellness meetings as the most successful mechanism used in communicating an understanding of vicarious trauma and best practices to strengthen self-care, wellness, and resilience. Every respondent participated in this activity. As shown in Figure 10, development of policies (one-day stakeholder consultative meeting), development of training materials, and three-day on-site learning visit to the Domestic and Sexual Violence Response Agency of Lagos State were not selected by respondents. For context, based on survey responses, only four respondents participated in the 3-day on-site learning visit, 12 respondents participated in the development of policies, and 15 participated in the development of training materials. Figure 10. Which of the SOAR activities were most successful in communicating an understanding of vicarious trauma and best practices to strengthen self-care, wellness, and resilience? (n=23) Source: ET web-based survey SOAR survey respondents also highlighted several components they found to be the most helpful through open-ended, short-answer responses. These included: • The Selfcare and Wellness Training: “It helped provide a clearer understanding of how I can understand (self-awareness) and manage myself, my workload, my health, understand the self-care practices that are most suitable for me to engage with.” • Psychological First Aid (PFA) (specifically for children): “Understanding the do’s and don'ts of PFA when working with child survivors. This is a critical aspect when providing intervention for survivors of GBV hence service providers must be culturally sensitive, respectful, and considerate when communicating with the survivors.” 0 0 0 1 5 17 0 5 10 15 20 25 3-day on-site learning visit to the Domestic and Sexual Violence Response Agency of Lagos State Development of training manuals (two–day validation meeting of training manuals) Development of policies (one-day stakeholders consultative meeting to review policies developed) Focus group discussion 5-day training on trauma counseling and psychosocial support for child survivors of sexual abuse 6-monthly self-care and wellness meetings for sexual and GBV responders 23 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV • 5-day workshops, six monthly wellness meetings, and focus groups were all highlighted. This respondent was particularly impacted by the 6-month wellness meetings: “6 monthly self-care and wellness meetings for SGBV Responders because it was one of its kind. I have worked for 15yrs as a social worker and have never had any meeting of sorts. It made me understand how to balance my work as well as take care of myself.” Although the development of policies was not mentioned by SOAR survey respondents as one of the most successful overall, the policies were found to be successful in identifying major safeguarding issues faced by SGBV-focused organizations. Ten of the 12 respondents who participated in this activity responded that the policies helped identify these issues to a great extent, while two responded that it somewhat achieved this. Seventeen of 18 WAR staff surveyed agreed that the curriculum content was at least somewhat easy to understand (12 responded that they agreed to a great extent, while five responded that they somewhat agreed). There was some discrepancy among WAR respondents as to whether the WCT was being fully monitored by WAR’s Health and Wellness Officer, who would then offer assistance to staff on an as￾needed basis. A majority of men surveyed felt this tool was being ‘“somewhat” monitored, whereas most women surveyed agreed to a great extent that this tool was being monitored. Three respondents agreed to a great extent that the technological aspect of the intervention (the use of Google Forms) poses a hindrance or challenge, while eight respondents felt that it somewhat poses a hindrance or challenge. This was further confirmed in WAR’s Evaluation Report of the WCT. They mentioned that staff often utilized their personal phones when accessing the tool and often would have technical issues. The new Human Resource and Wellness offer was found to be at least somewhat helpful to monitor and support the well-being of staff and promote a culture of self- and collective care by men, women, and gender non-conforming individuals (Figure 11). Figure 11. To what extent has the new Human Resource and Wellness Officer role helped in monitoring and supporting the well-being of staff and promoting a culture of self- and collective care? (n=18) Source: NORC web-based survey WAR respondents overwhelmingly mentioned that the training component focused on vicarious trauma was the most helpful. Out of 18 responses, nine mentioned learning about the concept of vicarious trauma and coping with trauma experienced in their daily lives was most helpful. Three respondents also 1 1 1 4 4 1 6 1 0 2 4 6 8 10 12 Women Men Not listed Very little Neutral Somewhat To a great extent 24 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV mentioned that components focused on resilience were helpful: “Resilience part. Where we were taught to withstand difficult moments in the process of helping clients.” [WAR Survey Respondent] SVRI course participants generally found the course to be easy to understand and accessible, as shown in Figure 12. Figure 12. SVRI Dare to Care Course Accessibility and User Friendliness Source: NORC web-based survey Dare to Care course participants appreciated the learning aides—including videos and reflective activities for those completing the course individually and adapted activities for those working through the course collectively with colleagues. In an open-ended survey response, 29 respondents had a positive impression of the learning aides, appreciating the content diversity that the aides added. Respondents found that they were “helpful and added variety to the course materials and made it feel more interactive” [SVRI Survey Respondent] and that they “helped to break up the reading portion of the modules and helped contextualize the work.” [SVRI Survey Respondent] Twelve survey respondents mentioned that they had experienced challenges with the course. These respondents mentioned that they struggled with online accessibility and the limited time to complete the modules. Some struggled with motivation and working the training into their already-busy schedules. This was common across activities, where several grantees mentioned in their reporting that staff viewed these programs as additional “work” rather than opportunities for personal development. CCH survey respondents appreciated the training’s components focused on coping related to working with GBV survivors. ZSU survey respondents appreciated their training’s focus on boundaries and self￾care techniques. Throughout the KIIs a focus on the body and teaching people about stress, its impact on the nervous system, and how to utilize different self-care techniques were also mentioned as an effective mechanism for staff to use. The following quotes provide examples of training components that survey respondents found to be the most helpful: “Breathing exercises, stretching, setting up a wall (boundaries) around you with the aim of protecting your personality.” [Survey Respondent] 7 6 2 4 12 8 15 15 33 36 0 10 20 30 40 50 60 70 80 To what extent would you say that the course content was easy to understand? (n=69) To what extent would you say that the online course was easy to access and user friendly? (n=69) No response Very little Neutral Somewhat To a great extent 25 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV “As a lawyer who works on gender-based and domestic violence cases on a daily basis, it was extremely important for me to gain knowledge about how I can help myself using the mechanisms for mental and emotional relief and stress.” [Survey Respondent] While survey respondents were less likely to report the wellness policies as being effective to their work, the KIIs indicated that the grantees themselves found these policies to be effective at institutionalizing self-care and wellness within their organizations. They mentioned that these policies allow for them to both put these ideas into practice within their organization, but also to share this knowledge with other organizations in their networks. “So, [the wellness policy] is not only on paper, but we have also developed the tools for the application of this policy in practice. It proved both useful and meaningful because that put us in a position where we can share what we know with other organizations.” [ZSU KII] EVALUATION QUESTION 3: TO WHAT EXTENT ARE THE ACS SUSTAINABLE? SUSTAINABILITY: What aspects of the activity clusters contributed to their sustainability? What components are needed for greater sustainability? Survey respondents across grantees generally agreed that the mechanisms used by each activity were likely be to be maintained after the project ended. Respondents were asked about the likelihood of sustainability of the following mechanisms: • Learning from training manuals (SOAR) • WCT maintained to help staff share feelings, experiences, and challenges (WAR) • Continued use on knowledge acquired from workshops (CCH) • Continued implementation of behaviors included in self-care plans (ZSU) See Figure 13 for a summary of the likelihood that each of these will be sustained. Figure 13. To what extent will mechanisms applied by each activity be sustained?23 Source: NORC web-based survey Respondents mentioned the following facilitating factors and challenges in maintaining these activities. For WAR, respondents described challenges completing the WCT on a weekly basis, which led to 23 Only one staff member from SVRI participated in the survey and therefore was not able to singularly speak to the sustainability of SVRI policies and practices. 1 1 1 1 1 2 3 4 8 4 11 10 8 0 2 4 6 8 10 12 14 16 18 20 Self-care behaviors (ZSU, n=7) WCT use (WAR, n=17) Training manual learnings (SOAR, n=14) Knowledge acquired from 2-day workshop (CCH, n=18) Not at all Very little Neutral Somewhat To a great extent 26 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV fatigue. One respondent mentioned that it may need to be conducted less frequently, on a monthly basis rather than weekly. “Weekly completion fatigue and loss of interest and just ticking without thinking about the responses. So the responses might not reflect what the reality was.” [WAR Survey Respondent] CCH survey respondents highlighted that continued use of knowledge acquired during workshops would be facilitated by more continuous training: “We must be regularly reminded to practice what we learned, these practices of self-care and wellness to become part of our organizational and personal culture.” [CCH Survey Respondent] Several respondents mentioned time limitations as a barrier to prioritizing self￾care “because they [workers] have not much time at work to practice the learned skills to relieve them from stress” [CCH Survey Respondent] indicating their skepticism about the sustainability of these changes. ZSU survey respondents noted certain behaviors were more likely to be sustainable, including paying more attention to themselves, not taking work home, implementing deep breathing in high-stress situations, and paying more attention to themselves in daily life. Survey respondents agreed that policies and practice on managing vicarious trauma would at least somewhat be implemented and maintained after the project period across activities (Figure 14). Figure 14. To what extent do you think the new policies and practices on managing vicarious trauma will be implemented and maintained after the project period? Source: NORC web-based survey WAR developed a Staff Wellness Policy, and SOAR developed a Safeguarding Policy. Both sets of respondents mostly agreed that the implementation and adoption of both of these policies was sustainable (Figure 15). 3 1 3 1 2 3 4 6 8 2 0 2 4 6 8 10 12 14 16 18 20 SOAR (n=9) WAR (n=17) ZSU (n=7) Very little Neutral Somewhat To a great extent 27 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Figure 15. To what extent do you think the implementation and adoption of these policies is sustainable for SOAR and WAR? Source: NORC web-based survey Both grantees and MCI mentioned these policies would assist with sustainability of self-care and wellness practices within the organization. “What I will say is that I appreciated activities that included embedding some of these practices into policies and procedures for the organization. I think that that is helpful for sustainability. It’s not like a one-off curriculum, but it’s like, okay, maybe there is a curriculum that staff can go through, but how do we really embed self and collective care practices into the way we do business from recruitment, to time off, to all of these different things. So yeah, so I would just say that that I appreciate that as an important component of sustainability.” [MCI KII] This reference includes specific training materials used for onboarding new staff. Online components were considered more easily sustained and maintained. One key informant noted that “obviously, anything in the technology world is more sustainable than the human world just like you have an online module on wellness that’s going to be [more] sustainable." This is consistent with SVRI’s approach where they recorded webinars and meditation sessions and maintain them on their website. CCH discussed how the connections they made throughout the project with other CARE-GBV grantees, local GBV organizations, or local government officials was a giant boon for their organization and how they are still utilizing those connections to continue their work. “I can say that the meetings we had with every stakeholder included in this project was very important to us, and we have those connections even after the project ended, and we are still using them in order to prevent our stress and to make everything that is best for the victim.” [CCH KII]. Conversely, there were several barriers to sustainability that were brought up through the KIIs that were conducted. These barriers include size of the organization, political will, and access to funding. In KIIs, grantees mentioned that due to their size, they were unable to implement changes or maintain their programs after the activity ended. ZSU mentioned political will and government funding as a driving factor for their determining sustainability. Uncertainty of their ability to operate is “the biggest obstacle.” Access to consistent funding was the largest barrier to sustainability of activities mentioned by all five grantees. For example, CCH mentioned that without financial support, they were unable to keep the therapist they hired to do psychosocial support groups on staff and now consult that therapist in an informal manner. 2 6 6 5 8 0 2 4 6 8 10 12 14 16 18 20 Safeguarding Policy (SOAR, n=11) Staff Wellness Policy (WAR n=16) Neutral Somewhat To a great extent 28 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV “I spoke about opportunities, but [a] major challenge I would say would be regarding funding. We are on the ground, we have the capacity, we have the network, but we require the funding to do what needs to be done.” [SOAR KII] REPLICABILITY, TRANSFERABILITY AND ADAPTABILITY: In what ways are the AC’s replicable in the same contexts? Adaptable for other contexts? Every grantee mentioned they believed at least one , if not all, aspects of their activity could be replicated or transferred to other areas in their country, region, or globally. These activity components included online course materials, policy documents, web-based surveys, and other mechanisms utilized throughout the activity. They noted that even though materials and programs were made to fit their very specific needs, and would be highly beneficial for other organizations, interventions and products should be tailored to the needs of different organizations in different contexts. They also were keen to share their thoughts and experiences with other organizations looking to develop these kinds of programs. One common challenge for replication was contextualizing materials and translating materials into local languages, which can often be quite costly and time intensive. Aspects of the programs that don’t require much speech/language usage were cited by ZSU as something easily transferred to different contexts. Their example focused on grounding, breathwork, and other ways to calm the nervous system that were highly impactful and don’t require materials or have communication challenges. “I think the components of our program could be applied in a lot of different contexts. I mean, obviously, you'd have to adapt it to that particular context, and you would really need to buy in from the stakeholders of whatever place you're in. But caring for people who care for others I think it’s a pretty universal need.” [ZSU KII] When discussing sustainability of their projects, two grantees mentioned that they have already begun (or continued) assisting other organizations in their communities that want to incorporate various self￾care and wellness measures into their work with GBV responders. Such activities mentioned include advocating for self-care and wellness procedures to be implemented, training other organizational staff on these measures, and sharing knowledge or materials. SCALABILITY: What aspects of the AC’s are most amendable to be scaled up? When asked about scaling their projects, most grantees agreed that they would like to scale up the work they are doing and would want to continue with all aspects of their projects. This was echoed by both USAID and MCI. Additionally, both grantees that offered online courses or other learning modules said they believe their courses could easily be scaled up and offered more widely, and they were actively scaling these products up at the time of the interviews. For example, SVRI was working to translate their course into Spanish, and WAR was in the process of getting their course accredited. Training of trainers was mentioned by two grantees as useful to assist with scaling up the projects and build capacity. Additionally, another grantee that conducted trainings on counseling and psychosocial support said that if they tried to scale up their program, they would focus most on these types of trainings in other organizations: “We want this to be more sustainable and to make it last because we have trained trainers within an organization who provide help [to] gender-based violence victims. Then this idea will last, and they can help [an] even more, bigger number of professionals within their organizations.” [CCH KII] According to two grantees, funding was also the main barrier to scaling up their existing projects, even in response to requests to provide support. 29 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV 4. CONCLUSIONS FOR THE CARE GBV ACTIVITY CLUSTER EVALUATION QUESTION 1: ARE THE ACS BASED ON CONTEXT-SPECIFIC AND INTERNATIONAL EVIDENCE? What worked: All five grantees integrated context-specific and international evidence by conducting various forms of needs assessments activities. Most interventions (four out of five) had a country-specific focus, which enabled grantees to design interventions grounded in context-specific perspectives and needs. A majority (42 out of 57 or 74 percent) of participants across four activities (ZSU, CCH, WAR, and SOAR) agreed to a great extent that their needs were taken into account (see Figure 3). Grantees were encouraged to integrate do-no-harm, survivor-centered, and participatory approaches to promote staff self-care and well-being. Two grantees (SOAR and SVRI) independently created TOCs to guide and ground their interventions. The other grantees grounded their programming in certain theories, conceptual frameworks, or methodologies. CARE-GBV program activities were implemented during the COVID-19 pandemic, which required unexpected adaptations to programming, such as increasing staff capacity and safe house services to support greater GBV caseload. Grantees were able to make swift adaptations to the intervention design, both in terms of program activities and mode of delivery. Grantees also built in mechanisms to gather feedback on program activities. Findings also indicate that one grantee ultimately associated high staff turnover with vicarious trauma, disproving their previous assumption that turnover rates were driven by staff’s dissatisfaction with salaries. Challenges: Related to assumptions, grantees assumed that their staff and external GBV responders would be willing to participate in program activities. Although these assumptions were generally accurate, in the case of one grantee, intended beneficiaries were less likely to participate in asynchronous online, self-paced courses and tended to prefer synchronous training opportunities. With respect to monitoring, grantees reported that collecting post-training feedback and monitoring of training indicators was somewhat restricted by funding. For example, one grantee only received funding to support pilot launch of their training program, limiting their ability to track completion rates and feedback on the utility of the course after it was made publicly accessible. Since the end of the grant, the grantee has gathered some core funding to keep the online course running, but there is no mechanism in place to track completion rates. EVALUATION QUESTION 2: TO WHAT EXTENT ARE EACH OF THE ACTIVITY CLUSTERS ACHIEVING THE TARGETED GBV RESULTS? What worked: Based on the findings, the CARE-GBV cluster was able to raise awareness on vicarious trauma and promote a work culture that prioritizes staff well-being, care, and resilience. Grantees achieved results in increasing participants’ understanding of vicarious trauma, burnout, and self and collective care practices. Many GBV service providers had not received this kind of training before and expressed gratitude for the new tools they learned to process day-to-day trauma and preserve their mental and physical health. Course content was reported as user friendly and easy to understand. Framing vicarious trauma as both a mental and physiological problem helped GBV service providers conceptualize it as an important and critical issue. Program participants appreciated practical, low-effort guidance (such as breathing techniques) that helped them manage the day-to-day impacts of their work. Additionally, diverse learning aides (videos, group work, etc.) helped course participants engage more and break up lectures or reading content. 30 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Challenges: Conversely, findings show that tools that required independent engagement were less successful. Service providers acknowledged that they had a difficult time integrating self-care tools into their already busy schedules and did not want the additional work. Further, self-paced courses were perceived as additional work among some respondents. However, some grantees noted that the bureaucratic reporting process and funding constraints were challenging, particularly among those for whom it was their first time to receive USAID funding. EVALUATION QUESTION 3: TO WHAT EXTENT ARE THE ACTIVITY CLUSTERS SUSTAINABLE? What worked: Broadly, new policies and practices on managing vicarious trauma were likely to be implemented and maintained after the project period, due to suggestions that such programs are a worthwhile investment to protect staff well-being. Findings suggested that interventions are likely to be both replicable and sustainable because they can be delivered in different ways. While contextualization was noted as being highly necessary, in-person sessions appeared to foster more personalized engagement and community-building. Additionally, all grantees mentioned their projects could be scaled up with proper funding. Incorporating a training of trainers model was most cited as an avenue for future work in this field. Challenges: While staff appreciated and benefitted from the variety of self- and collective care activities, how these interventions are delivered is important. For example, it was often difficult to maintain engagement in the self-directed, asynchronous learning and resulted in high drop-off rates. Additionally, many of these extra check-ins were viewed as additional time and work for staff. Based on the findings, knowledge retention from trainings and workshops may be hindered by the need for repeated instruction and refreshers as well as the need for longer project periods to incorporate lessons learned into training programs. Further, grantees noted difficulties with maintaining training benefits because there was not additional funding to continue implementing their projects. This was especially limiting for these small, local organizations. 5. RECOMMENDATIONS FOR THE CARE GBV ACTIVITY CLUSTER EVALUATION QUESTION 1: ARE THE ACS BASED ON CONTEXT-SPECIFIC AND INTERNATIONAL EVIDENCE? • Needs assessments should be considered an essential first step of intervention design. As confirmed by all five grantees, this was an effective tool for designing the intervention to align with participant needs and contextual factors. Needs assessments should also disaggregate findings by gender to understand how different genders may experience vicarious trauma, with a specific focus on understanding the needs of male GBV responders and how they prefer to receive training and support. Resources to inform the intervention can include review of international practices to identify relevant elements, review of common or important stressors in the local context, and consultations with GBV responders to understand their needs. • Stakeholder engagement efforts should be expanded to include a diverse group of partners to better address vicarious trauma within an organization and in the GBV field. Grantees could consider engaging an established network of GBV service providers, local and international subject matter experts, traditional healers and leaders, government agencies, 31 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV and CSOs/NGOs. While grantees conducted needs assessments, there was little evidence of establishing greater linkages to other sources of community-based continuous support, which may help to scale and amplify the achievement of outcomes. • Future programs that seek to deliver trainings to GBV service providers should consider the preferred learning styles of participants and how to best deliver the content. Self-paced, asynchronous styles may be slightly less suited to adapting course content to participant needs and may be less effective in delivering sensitive content. • Future funders and grantees should build in adequate flexibility to be able to respond to unexpected events (e.g., COVID-19, elections, etc.) and adapt programming accordingly. The CARE-GBV cluster was designed to allow flexibility at the organizational level to enable grantees to customize their intervention based on staff needs and contextual factors. As a result, grantees were able to effectively modify activities during COVID-19. EVALUATION QUESTION 2: TO WHAT EXTENT ARE EACH OF THE ACTIVITY CLUSTERS ACHIEVING THE TARGETED GBV RESULTS? • USAID should continue funding programs that address vicarious trauma among GBV responders. Financial support should be designed to ensure GBV responders’ overall self-care and wellness needs are met in ways that enable them to maintain support for survivors and do not jeopardize their well-being. USAID should consider continuing with funding projects that address this need and combine them with other kinds of GBV programming. • Encourage grantees to work with their staff to co-determine ways to protect staff well-being and maintain self- and collective care activities. While it is often challenging for resource-challenged groups to avoid over-working, especially when responding to urgent needs of GBV clients, it may be useful for organization leaders to work together with staff to determine how work hours and responsibilities can be structured to accommodate staff self￾care needs and prevent burnout, while maintaining the essential client services • Future grantees should be encouraged to leverage the needs assessment phase to identify locally relevant support services. By mapping and contacting local services, it might be possible to identify a network of partners to help meet the needs of staff, which may go beyond psychological support. Grantees may want to consider joining forces with other GBV organizations who might be grappling with staff burnout or trying to deliver vicarious trauma interventions. It may be possible to identify larger organizations with more self-care practices or resources, such as international organizations, who can share resources with smaller groups. USAID may be able to broker relationships with larger organizations that have a capacity to support smaller, local groups. • Consider additional research and programming to understand how self-care trainings can be customized to support male GBV responders. Organizations should be encouraged to recognize that male responders may require different approaches to self-care than women. GBV subject matter experts, donors, and implementing organizations should explore how the perception of self-care differs by gender and identify different coping techniques to mitigate vicarious trauma. • Consider adopting alternate contract mechanisms for small grants to local organizations. USAID should recognize the limitations of organizations and design funding mechanisms that ensure funding is received in ways that do not force groups to delay their activities or self-fund project work prior to receiving USAID funds. Smaller grassroots 32 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV organizations operating in the GBV space struggled to operate under current funding mechanisms, which tied payment disbursements to deliverables or milestones. • Consider increasing funding to support a longer duration of future projects. Funding strategies should be designed to enable grantees to complete all deliverables and activity components during the span of the project. An 18-month or longer contract is likely to give small organizations more time to implement self-care activities and entrench organizational norms, beliefs, and behaviors related to vicarious trauma. EVALUATION QUESTION 3: TO WHAT EXTENT ARE THE ACTIVITY CLUSTERS SUSTAINABLE? • Future interventions should continue to emphasize low-effort self-care practices as participants are more likely to implement practices that are easier to integrate into their busy work schedules in the long term. • A training of trainers approach could be used to ensure that training programs can be delivered to new staff, and existing workers can be reminded of lessons learned during the initial round of trainings. This could also aid in the scalability of programs as other organizations could be trained to deliver similar programs to their workers. Regular reminders and follow-ups are required for continued application of self-care practices and for sustained knowledge of vicarious trauma, especially since staff turnover is a known issue. • The integration of one or more staff members (e.g., Wellness Officers) to promote collective care practices might foster to ensure longer-term application of lessons learned. Although organizations may have resource challenges, it might be possible to appoint one or two individuals to promote lessons learned from training programs. These staff might help coordinate new and refresher trainings to adopt self-care practices into organizational cultures. • Staff wellness and care should be a core component of these organizations. Senior management could integrate trainings on vicarious trauma, self-care, and wellness into their onboarding process as well as conduct wellness check-ins with staff on a regular basis. • Combine local communities with a network for sustainability. Grantees should consider utilizing the expertise of a network of GBV service providers to further institutionalize and sustain program results. 6. IMPLEMENTATION EVALUATION FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS A. FINDINGS FOR THE IE As noted previously, SOAR was selected among the five CARE-GBV grantees for the Implementation Evaluation (IE). IE questions were explored via key informant interview and a web survey, focusing primarily on sustainability of the project and specific mechanisms utilized. EVALUATION QUESTION 1: IS THE ACTIVITY DESIGN BASED ON LOCAL CONTEXT AND FLEXIBLE TO ACHIEVE RESULTS ON THE GROUND? DESIGN: What factors contributed to the design of the activity? How were priority GBV 33 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV problems identified? Prior to the call for proposals, SOAR had identified gaps in its response to GBV for survivors of childhood sexual abuse, particularly with cases of GBV and child abuse increasing during the COVID-19 pandemic. The management team at SOAR held several meetings with their staff where they learned how burnt-out staff were while handling the increase of cases. Additionally, they realized that many of the people working with survivors of childhood sexual abuse do so out of “passion” and oftentimes do not have adequate training. To combat this, SOAR developed trainings to build staff capacity to provide psychosocial support and improve resiliency of children survivors of GBV. SOAR had initially planned to reach 32 staff, both internally and in their network, but were ultimately able to reach 36 staff members with their program. “We were able to look inwards at the response to GBV here in the Federal Capitol Territory and identify the gaps, but there was never really any funding to address these gaps, especially during the COVID-19 period where cases of GBV, particularly of children, you know, was on the increase. It was in several meetings that we attended [where] organization’s staff members were talking about how they were really burnt out, had to work.” [SOAR KII] “We identified that a lot of the organizations that were providing SGBV services, particularly psychosocial services, were not trained, were not professionals. A lot of people working in this sector are working out of passion and did not have the appropriate training. So, we included…trainings for counselors…to strengthen their capacity to provide counselling and psychosocial and improve [the] resilience of children.” [SOAR KII] The implementation team for SOAR based the intervention on assumptions that a network of GBV professionals would benefit from additional training and support. Additionally, the grantee assumed that participating organizations would understand the importance of the professional support SOAR was proving and want to be involved in their program. From the SOAR KII, the team felt these assumptions underpinned their theory of change and project design. Additionally, the team mentioned having several calls with USAID and MCI about their project plans to ensure that they were in line with GenDev’s policies and to discuss their ideas for the activity. SOAR was the one of two grantees to develop their own theory of change under CARE-GBV. Their TOC was informed by looking at issues faced by their organization as well as by the organizations in their network. The first step in their plan was to identify gaps in professional support and address barriers with the goal of empowering their staff with knowledge and the skills for self-care and wellness approaches. “For the Theory of Change, we looked at what the issues were for us, and for the response team, and more broadly what the issues were regarding the organizations that respond to these issues. We looked at the gaps, at the variance, and then, having collected information we looked at what it was that will help address these barriers and these gaps…We wanted staff to be empowered with knowledge, with information, and the skill to support their self-care and wellness so that, ultimately, at the end of the day, they would be able to support the children that they provide support for are better supported.” [SOAR KII] KEY IMPLEMENTATION METHODS: What are the key intervention methods to achieve objectives? The SOAR activity utilized several implementation methods: a focus group discussion, a five-day training on trauma and psychosocial support, a three-day onsite learning visit, six monthly self-care and wellness meetings, the development of training manuals, and the development of organizational policies. 34 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV As noted, SOAR survey respondents highlighted the six monthly self-care and wellness meetings were among the most successful mechanisms used because they communicated an understanding of vicarious trauma and best practices to strengthen self-care, wellness, and resilience. Each survey respondent reported participating in this activity. Conversely, respondents felt that the new wellness policies and the three-day on-site learning visit to the Domestic and Sexual Violence Response Agency of Lagos State were less effective. However, through KIIs with the SOAR implementing team, they mentioned the focus group discussion and wellness policy were the most effective mechanisms for institutionalizing self￾care and wellness within their organization. This variance is likely due to the difference in target groups for the surveys and KIIs. KIIs were conducted with the management team while survey respondents were SOAR staff. Initially, SOAR had intended to only conduct a stress risk assessment questionnaire to gauge baseline effectiveness of the implementation; however, they realized they were not getting an accurate depiction of organizational wellness. To address this, SOAR decided to host an FGD instead and noted, “The questionnaire was a completely different picture to when we were assessing and analyzing the results compared to the focus groups where people were speaking and sharing…. The FGD gave us a better outcome as to how people really felt compared to the questionnaire.” [SOAR KII] Related to challenges, SOAR often cited in their KII that while the project was able to achieve all intended outcomes of the project, they were unable to implement procedures to maintain the sustainability of the project as well as determine the effectiveness of their policy across network organizations. This was largely due to overall project timelines and access to continued funding. Additionally, SOAR mentioned that prior to the project, they struggled with maintaining adequate levels of staff, which was further exacerbated by the COVID-19 pandemic: “In meetings, staff were reporting [that] because we had so few staff working, [COVID-19] cases were astronomical, we had situations where staff were breaking down, the vicarious trauma, staff really did not know how to report or handle it. There was really nothing that was helping the caregivers to support their own mental health.” [SOAR KII] Through the grant, they were able to address some of these challenges by hiring a licensed psychosocial therapist to conduct the monthly training sessions. FLEXIBILITY: IS THERE SUFFICIENT STAFFING TO RESPOND TO LOCAL PRIORITIES? IS THERE FLEXIBILITY TO CHANGE APPROACHES TO RESPOND TO LESSONS AND CHANGING CHALLENGES IN THE LOCAL ENVIRONMENT? Prior to the call for proposals, SOAR had noted that its staff of eight were burnt out and stretched too thin to address local issues identified by their needs assessment. They did not have adequate staffing or funding to effectively address these problems. This was further exacerbated by the COVID-19 pandemic when staff fell sick adding to the burden of those who were well and still working. This led to many staff “breaking down” from the burnout they were feeling. Based on qualitative KII data, the SOAR implementing team indicated they had the flexibility to adjust programs to meet the specific needs of their staff. While they did not need to change or pivot the program much, they shared one example of a shift. They planned to conduct a stress management risk assessment at the start of their program, but they realized the data they gathered from this method may not have the information they needed to develop an effective program. They chose instead to move forward with a focus group discussion. “We were going to conduct a stress management risk assessment which we designed in a questionnaire format and shared with all of the participants…But upon returning those questionnaires we saw…the results were not really speaking to the actual realities on the ground. So fortunately, just after we got those results, we had our bi￾monthly check-in with the grant managements team…[They said] that probably that stress risk assessment was 35 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV not going to give us what we wanted, and from experience they recommended that we should make it more of a discussion which we ended up calling a ‘Focus Group Discussion.” [SOAR KII] EVALUATION QUESTION 2: IS THE ACTIVITY REACHING PARTICIPANTS THEY ARE MEANT TO TARGET? TARGET BENEFICIARIES: What are the barriers to reaching participants? SOAR aimed to increase staff capacity to address issues of sexual violence against children in the Federal Capital Territory of Nigeria. They did this by engaging eight members of their own staff as well as 24 staff from 12 other GBV organizations in the area. During the KII, SOAR’s management team mentioned there had previously been an effort to provide self-care and support for caregivers and practitioners in their region, but it had failed. They noted, “[W]e had identified [their failure] was because there wasn’t enough coordination, and many did not understand what that organization was trying to do. So that informed our decision to work with a network that already had mailing lists and memberships of organizations working in this field.” [SOAR KII] This was a determining factor for SOAR defining their target group of participants in the proposal. Ultimately, this was successful, and SOAR was able to engage 12 organizations in its network throughout the implementation period. As found in the document review, “The curriculum is designed to expose responders to child and other survivors of sexual and GBV to the different tools and activities needed in order to have a healthy and balanced relationship with their work, become more productive in their job, and avoid burnout as they focus on helping children affected by sexual violence.” [SOAR Document Review] They were able to achieve these goals and engage staff by conducting workshops on psychosocial support and resiliency and hiring additional staff to assist with service delivery. Additionally, they sought to strengthen the capacity of the response team. To accomplish this, SOAR conducted on-site visits to the Domestic and Sexual Violence Response agency of Lagos State, as previously mentioned. Through the KIIs, SOAR mentioned the onsite visits were particularly insightful for allowing staff to identify areas where they could strengthen their own operations and support the development of their organizational strategies. Finally, SOAR developed two manuals: one documented the psychosocial support trainings and the other discussed self-care practices for LGBTQ+ care providers. During the KII with SOAR, they mentioned an instance where they attended a meeting with their network organizations where these organizations said they had adopted many of the policies laid out in SOAR’s manuals. “[We also] attended meeting with other organizations, and the feedback we kept getting was thanking us that they did not have policies in place and are now using it, and staff are appreciating the response team, and they have gone on to put a plan in place now to implement what has been stated. It is slow, but at least there is a document that is helping with practices and priorities.” [SOAR KII] MONITORING: Is the activity collecting evidence on what is working, not working and what could be done differently to achieve results? SOAR completed verbal check-ins with staff during their meetings to inquire how they were doing and what they were doing differently—no other data was collected. For example, they explained: “At different points of the meetings, we would have general experience sharing on how they were using the information they were learning, and that really helped inform us that, yes, the information was helping. Helping them with their health, getting better with managing their stress, managing workspace environments.” [SOAR KII] SOAR also committed to reviewing its wellness policy annually through their HR department and 36 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV the creation of a Wellness Committee to assist with this review. The policy will be reviewed to ensure that all objectives are still being met as well as incorporating feedback from employees, management, and their Wellness Committee. Additionally, in an interview MCI indicated that since the grant period ended so recently (July 2022), they have not yet reengaged the grantees to discuss their project. However, MCI is currently having internal discussion about what future engagement would look like and how that might be beneficial while not taxing the grantees too much. EVALUATION QUESTION 3: IS THE ACTIVITY ACHIEVING SUSTAINABILITY? SUSTAINABILITY: What plans are in place for sustainability? What is the evidence of potential sustainability? Twelve SOAR respondents surveyed participated in the development of policies. These respondents mostly felt that new policies and practices would be implemented and maintained after the project ends, including the Safeguarding Policy (Figure 16). Figure 16. Implementation and Maintenance of SOAR Policies* Source: NORC web-based survey *Note that one respondent selected “no response” to the question “To what extent do you think the Safeguarding Policy will be implemented and maintained?” The majority of respondents also observed that there were measures in place to revise policies and conduct orientation sessions for new staff, as seen in Table 8 below. Table 8. Sustainability of SOAR Activities Yes No Are there any measures in place to revise these policies as needed? (n=12) 9 3 Are there provisions in place to revise the training manuals as needed? (n=14) 9 5 Are there provisions in place to conduct refresher trainings for staff? (n=15) 12 3 Are there any measures in place to conduct orientation sessions on these policies 10 1 for new staff? (n=11) Source: NORC web-based survey 1 6 2 5 6 0 2 4 6 8 10 12 To what extent do you think the Safeguarding Policy will be implemented and maintained? (n=12) To what extent do you think the new policies and practices on managing vicarious trauma will be implemented and maintained after the project period? (n=9) Neutral Somewhat To a great extent 37 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Survey respondents mentioned several challenges regarding the sustainable implementation of the Safeguarding Policy, including staff departures and the necessary human resources to implement this policy: “Safeguarding requires staff, volunteers, board of trustees & vendors to be trained to fully understand the implementation of the policy. There is still a gap in this regard in many organizations which can affect the sustainable implementation of the safeguarding policy in those organizations.” [SOAR Survey Respondent] All survey respondents felt that participants would at least somewhat recall and use learnings from training manuals. Four responded “somewhat” while 10 responded “to a great extent.” Similar challenges were echoed in the KII with the SOAR team. They reported the project timeline as a barrier to project sustainability, particularly because there was insufficient time to monitor the effectiveness of the new wellness tools. As noted by the grantee, “If we had more time, we would have been able to work around sustainability…It would have helped us work more around tracking change, tracking use, tracking the institutionalization of some of these policies so we can be sure this is not just a document that will not be developed because of the project and then dumped on a shelf.” [SOAR KII] They said more time would have allowed them to space out activities and analyze findings as the project progressed to understand the effectiveness of different components and implement changes in real time. B. CONCLUSIONS FOR THE IE Evaluation Question 1: Is the activity based on local context and flexible to achieve results on the ground? What Worked: SOAR’s activity was designed to meet staff needs, and they indicated that overall, they had the information and support needed to design an effective intervention. However, they felt there were knowledge gaps in self-care and wellness for GBV responders to survivors of childhood sexual abuse both within their organization and their wider network. Their TOC reflected these knowledge gaps and aimed to empower their staff with information and sufficient resources to support their own wellness process. They noted that they had the flexibility to change their approach to meet the needs of their staff and were able to effectively monitor the intervention and its impact on staff. Challenges: Throughout the project, SOAR struggled to maintain necessary staffing to cover their operational needs. This was further exacerbated by the COVID-19 pandemic which increased overall staff burn out and impacted SOAR’s ability to effectively implement and digest various aspects of the program. Evaluation Question 2: Is the activity reaching participants they are meant to target? What Worked: SOAR was able to engage four additional staff from its GBV network outside the initial number of target beneficiaries, for a total of 36 participants. Throughout the implementation period, SOAR conducted check-ins during meetings to determine what program aspects were or were not working for staff. While throughout the grant period, these activities influenced organizational changes in staff wellness. Challenges: Monitoring throughout the activity period was challenging due to tight deliverable timelines which made it difficult to track change, use, or institutionalize some mechanisms over time. If monitoring could have continued, it might have indicated that group sessions may foster longer-term communal support between co-workers. Evaluation Question 3: Is the activity achieving sustainability? 38 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV What Worked: Staff indicated that the monthly self-care and wellness meetings were the most successful mechanisms utilized, while the management team believed the wellness policy would institutionalize self-care in their organization, leading to more sustainable practices to support staff well￾being. Challenges: Based on current data, it is clear that some components such as monitoring are no longer being done by SOAR. C. RECOMMENDATIONS FOR THE IE Evaluation Question 1: Is the activity based on local context and flexible to achieve results on the ground? • Invest in further self-care and wellness interventions for professionals supporting violence survivors. Caring for trauma-affected populations is a very challenging and stress￾filled profession. Providing this type of care often creates burnout, which risks both harm to these care professionals and subsequent losses for vulnerable individuals. Findings clearly indicated that attention to self-care was used, highly valued, and beneficial to care providers.24 • Consider increasing funding to support a longer duration of future projects. Funding strategies should be designed to enable grantees to complete all deliverables and activity components during the span of the project. An 18-month or longer contract is likely to give small organizations more time to implement self-care activities and entrench organizational norms, beliefs, and behaviors related to vicarious trauma. • Expand training topics to help GBV responders better serve survivors of child sexual abuse. Survey respondents mentioned further topics could focus on the reintegration processes and other support for victims/survivors after receiving services; prevention of SGBV; and accessibility of psychosocial support and its sustainability for the survivors of SGBV. Evaluation Question 2: Is the activity reaching participants they are meant to target? • Encourage greater activities that bring GBV workers together. Because group activities, such as workshops, meetings, and focus groups, appeared to be highly valued by so many participants, implementing agencies should consider trying to include greater numbers of collective, sharing, and mutual support activities. • Embed components to support monitoring and adaptations to self-care interventions, making them responsive to staff health and wellness needs. By including sufficient funds and time for grantees to track the influence of the interventions, identify gaps in current needs, and respond to emerging stressors, funders can help groups maintain effective activities beyond the funding cycle. Respondents specifically suggested biannual refresher and feedback sessions with project participants to reinforce learning outcomes. Evaluation Question 3: Is the activity achieving sustainability? • Encourage knowledge and resource sharing across different agencies likely to experience vicarious trauma and burnout. These psychological phenomena are not uncommon among workers in caring professions. Based on the lessons learned about 24 This is an important recommendation and is therefore indicated for all the grantees in the cluster, including the implementation evaluation grantee. 39 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV implementing wellness programs and about self-care by recipients, it seems that a next funding round could offer the opportunity for learning to be shared across more care sectors, which might make these resources more cost-effective and sustainable. Grantees should consult resources developed by other international donors (such as UN, UNHCR, and UNICEF) on vicarious trauma before planning their own activities. Donors should also consider funding a central Wellness Contact Point that can serve multiple agencies simultaneously, to maximize funding value. 40 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ANNEX A, EVALUATION STATEMENT OF WORK Gender Based Violence: Portfolio Performance Evaluation Scope of Work Version 2 BACKGROUND USAID’s Gender Equality and Women’s Empowerment Hub (GenDev) in the Bureau for Development, Democracy, and Innovation (DDI), advances gender equality and women’s empowerment (GEWE) as fundamental for the realization of human rights, and key to effective and sustainable development outcomes. To achieve Gender Equality and Women’s Empowerment globally, GenDev collaborates with Operating Units (OU) across the Agency supporting USAID’s programming in all sectors. Preventing and responding to gender-based violence (GBV) is a U.S. government (USG) priority. GenDev supports USAID’s efforts to prevent and respond to GBV in more than 60 countries through its thought leadership, training and technical assistance, and programming initiatives. GenDev has contracted NORC at the University of Chicago (NORC) to carry out a performance evaluation of its GBV portfolio comprising four activity clusters: (a) women’s economic empowerment (WEE) activities directly funded by GenDev integrating GBV prevention and response activities; (b) Collective Action to Reduce Gender-Based Violence (CARE-GBV) small grants activities; (c) Resilient, Inclusive & Sustainable Environments (RISE): A Challenge to Address Gender-Based Violence in the Environment; and (d) Better Together Challenge (BTC) activities funded by GenDev integrating GBV prevention and response interventions. This Scope of Work (SOW) 2 document specifies the objectives of the performance evaluation, the activities that will be included in the evaluation, the evaluation questions, possible data collection methods, the timeline/period of the performance and implementation evaluation from Phase 3b25 onwards, reporting, and deliverables. Definitions: Since GenDev included the four activity clusters (ACs) based on a need for further monitoring and evaluation (M&E), the term portfolio is used only to discuss the four ACs together. Activities funded under each AC are referred to as activities to align with the Agency definition. PPE OBJECTIVES This Portfolio Performance Evaluation (PPE) will examine the effectiveness of the portfolio/ACs in achieving their objectives and outcomes, the lessons learned and gaps that are currently not being addressed. Within each AC, NORC will assess if the goal for each AC is being met and how specific projects are being implemented, their quality and challenges. In addition, NORC will conduct an implementation evaluation for a limited set of activities (perhaps one from each AC if feasible), examining how the specific activity is working (if it is on the right pathway to achieving end outcomes), for whom, and in what context. This work will consider the programmatic assumptions, identify intervention challenges and facilitators, and explore engagement with beneficiaries and partners. Findings 25 The Portfolio Performance Evaluation includes several Phases: (1). Scope of Work 1, (2). Evaluability Assessment, (3a). Scope of Work 2, (3b). Evaluation Design Report, (4). Portfolio and Activity Cluster Evaluation and Reporting, (5). Implementation Evaluation and Reporting, (6) Overall PPE Report, Evaluation Debriefing and Dissemination. 41 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV will inform recommendations for USAID’s future programming and guide future monitoring and evaluation approaches to strengthen USAID’s evidence for decision-making. ACTIVITIES INCLUDED IN THE EVALUATION The following activities will be included in the portfolio and activity cluster level evaluation. NORC will also determine one activity within each cluster that will be the target of the implementation evaluation, if appropriate. Table A. 1. Activities under each Activity Cluster ACTIVITY CLUSTER LIST OF EVALUABLE ACTIVITIES COUNTRY Better Together Challenge 1. Democracy International’s (DI) Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME) 2. HIAS’s Shifting Power Dynamics: Engaging Men in Gender-Based Violence Reduction 3. NCC’s Bridging the Gap for Venezuelan Migrants (BTG4VM) 1. Guyana 2. Panama 3. Trinidad & Tobago CARE-GBV 1. Žene sa Une (ZSU) 2. Women Against Rape (WAR) 3. Sexual Offences Awareness and Response Initiative (SOAR) 4. Crisis Center Hope (CCH) 5. Sexual Violence Research Initiative (SVRI) 1. Bosnia & Herzegovina 2. Botswana 3. Nigeria 4. North Macedonia 5. Global RISE Challenge 1. Creative Capacity Building to Address Gender Based Violence in the Artisanal and Small-Scale Mining Sector in Colombia 2. Resource-ful Empowerment: Elevating Women’s Voices for Human and Environmental Protection in Congolese Small-Scale Mining. 3. Conservation of the Alto Mayo Landscape without Gender Violence 4. Tz’unun: Ending Environmental Violence Against Indigenous Women in Guatemala through Empowerment in Community Forestry, Agroecology and Collective Healing Spaces 5. Combatting Gender-based Violence in Vietnamese Conservation 6. Advancing Equitable Gender, Social and Power Norms in Community Conservancies in Kenya. 7. Gender Empowerment and Transformation: Tackling Resource-Based Conflict and Gender-based Violence in Fiji 8. Rising Up!: Promoting Congolese Women’s Land Access and Preventing GBV in eastern DRC 9. Securing Land Rights & Ending Gender Exclusion 1. Colombia 2. Democratic Republic of Congo 3. Peru 4. Guatemala 5. Vietnam 6. Kenya 7. Fiji 8. Democratic Republic of Congo 9. Uganda WEE 1. Global Labor Program: Levi-Strauss Partnership 2. Engendering Utilities (WAGE) 3. A Micro-Journey to Self-Reliance 4. Enabling Environment for Economic Empowerment of Women 5. New Partnerships Initiative (NPI): Latin America 1. Lesotho 2. Global 3. Benin 4. Burundi 5. Guatemala, Honduras, Mexico EVALUATION QUESTIONS Table 2 includes the evaluation questions and sub-questions at the portfolio, activity cluster and individual activity level. 42 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Table A. 2. Evaluation Questions Evaluation Question EQ-Sub-questions PORTFOLIO QUESTIONS 1. How are the USG’s guiding principles and priorities to end GBV being incorporated into the four activity clusters (AC)? Prevention: In what ways are the USG activity portfolio contributing to reduced risks? Protection: How does the portfolio contribute to accessible, effective services for violence survivors? Accountability: How does the portfolio contribute to ending impunity? 2. To what extent are the USG objectives being achieved across the 4 ACs? Coordination: How are the GBV prevention and response efforts being coordinated and managed at the Agency, Activity Cluster and Activity levels?? Integration: How are GBV prevention and response efforts being integrated into current and future GenDev work and informing related programs? Data. How is GenDev’s GBV portfolio collecting, analyzing, and using data and research to enhance prevention and response efforts? Expansion: How is GenDev’s GBV portfolio helping to expand and improve GBV programming? 3. What lessons are being learned and to what extent is there sharing of best practices, lessons, and information across the 4 ACs? Foundations: Are lessons regarding foundations of GBV being shared with AC implementing partners? Populations: What types of populations are being engaged in the AC? Which vulnerable and underserved populations are been included? Stakeholders: Which stakeholders are being engaged to achieve results? 4. What pervasive gaps still exist in understanding GBV and addressing specific types of GBV? Intervention planning and design: What are important knowledge and practice gaps in planning and designing GBV interventions? Forms of violence: What are important knowledge and practice gaps in addressing specific forms of GBV? Reach and effectiveness: How is the GBV portfolio influencing the reach and effectiveness of interventions? ACTIVITY CLUSTER QUESTIONS 1. Are the activity clusters based on context-specific and international evidence? Needs assessment and intervention evidence: How well were needs assessments conducted and intervention evidence collected to inform the cluster activities? Assumptions: What assumptions were made to design and implement the activity clusters? How accurate were any assumptions? Causal pathways: What causal pathways or theories of change were articulated for the activity clusters? Monitoring and adaptations: How well are interventions monitored and emerging findings contributing to intervention adaptations or improvements? 2. To what extent are each of the activity clusters achieving the targeted GBV results? Outcomes: Are the stated outcomes realistic and achievable within the timeframe of the AC? What progress is being made towards achieving the outcomes? Planning and activity designs: How and how well were activity plans and designs developed to achieve different GBV outcomes? Intervention implementation: How well are interventions implemented to reach their target groups and influence change? Mechanisms: What are the most effective aspects of the intervention? How do these ‘active ingredients’ operate in each AC? 43 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Evaluation Question EQ-Sub-questions 3. To what extent are the ACs sustainable? Sustainability: What aspects of the ACs contributed to their sustainability? What components are needed for greater sustainability? Replicability, transferability and adaptability: In what ways are the ACs replicable in the same contexts? Adaptable for other contexts? Scalability: What aspects of the ACs are most amenable to be scaled up? IMPLEMENTATION EVALUATION QUESTIONS 1. Is the activity design based on the local context and flexible to achieve results on the ground? Design: What factors contributed to the design of the activity? How were priority GBV problems identified? Implementation: What are the key intervention methods to achieve objectives? Flexibility: Is there sufficient staffing to respond to local priorities? Is there flexibility to change approaches to respond to lessons and changing challenges in the local environment? 2. Is the activity reaching beneficiaries they are meant to target? Target beneficiaries: What are the barriers to reaching beneficiaries? Monitoring of results: Is the activity collecting evidence on what is working, not working and what could be done differently to achieve results? 3. Is the activity achieving sustainability? Sustainability: What plans are in place for sustainability? What is the evidence of potential sustainability? POSSIBLE DATA COLLECTION METHODS The evaluation will comply with USAID Evaluation requirements as stated in the ADS and the USAID Evaluation Policy. The expected evaluation type is a Performance Evaluation. The evaluation team will use a comprehensive evaluation design and methodology, using a mixed method approach (e.g., desk review, interviews, focus group discussions, key informant interviews, monitoring indicators, web-based survey, etc.) as indicated in Table 3 below, that will generate the highest quality and most credible evidence on each evaluation question, subject to budget constraints across the full portfolio evaluation. Other data collection methods such as outcome harvesting, and most significant change may also be considered and will be explored by NORC. Note: Considering the ongoing COVID-19 pandemic situation worldwide, the evaluation team must consider an alternative plan for fieldwork, including employment of local consultants and usage of IT tools and approaches to remote evaluation. EVALUATION TIMELINE. Task July - Sept Oct-Dec Jan-March April-June July-Sept 2022 2022 2023 2023 2023 Phase 3B – Evaluation Design Report Phase 4 – Portfolio and Activity Cluster Performance Evaluations & Reporting Project Document Review Finalizing instruments for KIIs, FGDs, and Surveys Data Collection Transcription, Coding and Data Analysis 44 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Task July - Sept Oct-Dec Jan-March April-June July-Sept 2022 2022 2023 2023 2023 Phase 5 – Implementation Research Reporting Project Document Review Finalizing instruments for KIIs, FGDs, and Surveys Data Collection Transcription, Coding and Data Analysis Phase 6 – PPE Report, Evaluation Debriefing & Dissemination Report Writing Dissemination REPORTING AND DELIVERABLES. Evaluation Design: The report will indicate the three levels of evaluation and a detailed approach and methodology to answer the evaluation questions. Implementation Evaluation Report: This report will include an overview chapter as well as 3-4 separate chapters/sections for each of the individual activity implementation evaluations. Performance Evaluation Report: This report will include an overall synthesis report and 4 separate chapters corresponding to each GBV AC. Post evaluation action plan: This report will include various agreed-upon product(s) to debrief the evaluation activities, disseminate findings, discuss recommendations, and follow-up programming actions responding to recommendations. Knowledge sharing and dissemination: The team will present findings to key stakeholders, including policy briefs, webinars and re-usable slide deck. 45 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Table A. 3. Evaluation Design with Data Collection Methods for each AC EVALUATION QUESTION SUB-QUESTIONS BETTER TOGETHER CARE-GBV RISE WEE PORTFOLIO QUESTIONS 1. How are the USG’s guiding principles and priorities to end GBV being incorporated into the four ACs? ● Prevention: In what ways is the USG activity portfolio contributing to reduced risks? ● Protection: How does the portfolio contribute to accessible, effective services for violence survivors? ● Accountability: How does the portfolio contribute to ending impunity? ● KIIs or web surveys with a few open-ended questions posed to stakeholders (not IPs) such as local government officials or local advocacy groups to assess how much the activities impacted these three principles in their geographic areas ● FGDs with program participants ● Journals of survivor participants (for referral services) Not Applicable ● KIIs or web surveys with a few open-ended questions posed to stakeholders (not implementing partners) such as local government officials or local advocacy groups to assess how much the activities impacted these three principles in their geographic zones ● Systematic project document review ● KIIs with IP senior leadership, reps from partner organizations and other key stakeholders ● FGDs with beneficiaries ● Outcome Harvesting to explore outcomes of capacity building interventions for the Vietnam and Kenya activities ● Project document analysis ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● FGDs with beneficiaries 2. To what extent are the USG objectives being achieved across the 4 ACs? ● Coordination: How are the GBV prevention and response efforts being coordinated and managed at the Agency, Activity Cluster and Activity levels? ● Integration: How are GBV prevention and response ● Group KII with Resonance ● Group interview with GenDev activity managers on data use ● Confirmation on design and implementation details with GenDev activity manager and leads ● KIIs or group interviews with CARE-GBV IP to assess efforts to connect grantees together for communities of practice. ● KIIs or group interview with GenDev activity managers on data use ● KIIs with GenDev activity managers and leads ● KIIs or group interviews with Resonance to assess efforts to connect grantees together for ● KIIs with GenDev activity managers and leads ● Group interview with GenDev activity managers on data use ● KIIs with IP senior leadership, reps from 46 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV EVALUATION QUESTION SUB-QUESTIONS BETTER TOGETHER CARE-GBV RISE WEE efforts being integrated into current and future GenDev work and informing related programs? ● Data. How is GenDev’s GBV portfolio collecting, analyzing, and using data and research to enhance prevention, response, and learning efforts? ● Expansion: How is GenDev’s GBV portfolio helping to expand and improve GBV programming? ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders communities of practice. ● Group interview with GenDev activity managers on data use ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders partner organizations, other key stakeholders 3. What lessons are being learned and to what extent is there sharing of best practices, lessons, and information across the 4 ACs? ● Foundations: Are lessons regarding foundations of GBV being shared with AC implementing partners? ● Populations: What types of populations are being engaged in the AC? Which vulnerable and underserved populations are been included? ● Stakeholders: Which stakeholders are being engaged to achieve results? ● KIIs with other funders in this space that have offered additional funding to IP orgs ● KIIs with regional/local GBV experts to discuss activity models/approaches and their appropriateness (e.g., Ladysmith for BTG4VM) ● KIIs with IP senior leadership and reps from partner organizations ● Group interview with Resonance ● KIIs or group interviews with Making Cents International to assess efforts to share lessons learned among grantees. ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● KIIs with other funders in this space that have offered additional funding to IP orgs ● KIIs with local GBV experts to discuss activity models/approaches and their appropriateness ● KIIs with IP senior leadership and reps from partner organizations, particularly those working on business development ● KIIs with GenDev activity managers ● Group interview with GenDev AC leads ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● FGDs with local organizations 4. What pervasive gaps still exist in understanding GBV and addressing ● Intervention planning and design: What are important knowledge and practice gaps in planning ● KIIs with USAID Mission staff ● KIIs with local GBV experts to discuss activity models/approaches and ● KIIs with USAID Mission staff ● KIIs or group interviews with Making Cents International to determine ● KIIs with USAID Mission staff ● KIIs with GenDev activity managers and leads ● KIIs with GenDev activity managers and leads ● KIIs with USAID Mission staff 47 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV EVALUATION QUESTION SUB-QUESTIONS BETTER TOGETHER CARE-GBV RISE WEE specific types of GBV? and designing GBV interventions? ● Forms of violence: What are important knowledge and practice gaps in addressing specific forms of GBV? ● Reach and effectiveness: How is the GBV portfolio influencing the reach and effectiveness of interventions? where there are gaps in programming ● KIIs with GenDev activity managers and leads ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● Project document analysis, especially final reporting/MEL data programming and knowledge gaps. ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● Secondary data analysis (project and MEL data) ● Key informant or group interviews with representatives from Resonance, because they have a big picture view of the full range of intervention types being implemented. ● KIIs with local GBV experts to discuss activity models/approaches and where there are gaps in programming ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● Systematic project document review, especially final reporting/MEL data ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● FGDs with local GBV experts ● Secondary data analysis (indicator tracking and other MEL data) ACTIVITY CLUSTER QUESTIONS 1. Are the activity clusters based on context-specific and international evidence? ● Needs assessment and intervention evidence: How well were needs assessments conducted and intervention evidence collected to inform the cluster activities? ● Assumptions: What assumptions were made to design and implement the activity clusters? How accurate were any assumptions? ● Causal pathways: What causal pathways or theories of change were ● Project document analysis ● Group interview with GenDev AC leads and Resonance, separately ● KIIs with non-GenDev USAID staff/advisors that might have played a role in formation of clusters ● Project document analysis ● KII/group interview with GenDev AC leads ● KIIs or group interviews with Making Cents International to assess their input in activity design ● KIIs or group interviews with grantees to determine their collection and use of MEL data across cluster activities ● Project document analysis ● KIIs with local GBV experts to discuss activity models/approaches and their appropriateness ● KIIs or group interviews with Resonance to assess their use of MEL data across cluster activities ● KIIs with GenDev activity managers and leads ● Group interview with GenDev AC leads ● FGDs with local organizations 48 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV EVALUATION QUESTION SUB-QUESTIONS BETTER TOGETHER CARE-GBV RISE WEE articulated for the activity clusters? ● Monitoring and adaptations: How well are interventions monitored and emerging findings contributing to intervention adaptations or improvements? ● KIIs with GenDev activity managers and leads ● Key informant interviews or maybe web surveys with a few open-ended questions posed to stakeholders (not implementing partners) such as local government officials or local advocacy groups to assess the extent of their involvement in activity design ● Group interview with GenDev AC leads ● KIIs with non-GenDev USAID staff/advisors that might have played a role in formation of clusters 2. To what extent are each of the activity clusters achieving the targeted GBV results? ● Outcomes: Are the stated outcomes realistic and achievable within the timeframe of the AC? What progress is being made towards achieving the outcomes? ● Planning and activity designs: How and how well were activity plans and designs developed to achieve different GBV outcomes? ● Intervention implementation: How well are interventions implemented to reach ● Web survey to IPs with targeted (open ended) questions about their model and theory of change, such as: were your assumptions underlying your theory of change correct in practice? Did the impact pathways you envisioned pan out how you thought they would? Is there anything that you would change about your intervention model? ● KIIs/Web Surveys with reps from partner organizations, other key stakeholders ● Project document analysis ● KIIs with GenDev activity managers ● Group interview with GenDev AC leads ● KIIs/web surveys with IP senior leadership, reps from partner organizations, other key stakeholders ● Secondary data analysis (project and MEL data) ● Beneficiary web surveys, where possible ● Web survey to IPs with targeted (open ended) questions about their model and theory of change, such as: were your assumptions underlying your theory of change correct in practice? Did the impact pathways you envisioned pan out how you thought they would? ● KIIs/Web Surveys with reps from partner ● Project document analysis ● KIIs with GenDev activity managers ● Group interview with GenDev AC leads ● KIIs/web surveys with IP senior leadership, reps from partner organizations, other key stakeholders ● Secondary data analysis (indicator tracking and other MEL data) ● Beneficiary web surveys, where possible 49 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV EVALUATION QUESTION SUB-QUESTIONS BETTER TOGETHER CARE-GBV RISE WEE their target groups and influence change? ● Mechanisms: What are the most effective aspects of the intervention? How do these ‘active ingredients’ operate in each AC? ● Secondary data analysis (indicator tracking and other MEL data) organizations, other key stakeholders ● Secondary data analysis (indicator tracking and other MEL data) ● Beneficiary web surveys, where possible ● KIIs with local GBV experts to discuss activity models/approaches and their appropriateness 3. To what extent are the ACs sustainable? ● Sustainability: What aspects of the ACs contributed to their sustainability? What components are needed for greater sustainability? ● Sustainability: What aspects of the ACs contributed to their sustainability? What components are needed for greater sustainability? ● Sustainability: What aspects of the ACs contributed to their sustainability? What components are needed for greater sustainability? ● Replicability, transferability and adaptability: In what ways are the ACs replicable in the same contexts? Adaptable for other contexts? ● KIIs with GenDev AC lead and BTC Director (Resonance) ● Interviews with people involved with managing the funding mechanisms that GenDev tapped into for CARE ● KIIs with GenDev AC leads ● KIIs or group interviews with CARE-GBV contractor to assess to discuss scale up potential, sustainability and challenges between contexts ● Interviews with people involved with managing the funding mechanisms that GenDev tapped into for RISE ● KIIs with GenDev activity managers ● Group interview with GenDev AC leads ● KIIs or group interviews with Resonance to assess to discuss scale up potential, sustainability and challenges between contexts ● Systematic document review ● KIIs with GenDev activity managers ● Group interview with GenDev AC leads ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● Secondary data analysis (indicator tracking and other MEL data) 50 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV EVALUATION QUESTION SUB-QUESTIONS BETTER TOGETHER CARE-GBV RISE WEE ● Scalability: What aspects of the ACs are most amenable to be scaled up? INDIVIDUAL ACTIVITY QUESTIONS 1. Is the activity design based on the local context and flexible to achieve results on the ground? ● Design: What factors contributed to the design of the activity? How were priority GBV problems identified? ● Implementation: What are the key intervention methods to achieve objectives? ● Flexibility: Is there sufficient staffing to respond to local priorities? Is there flexibility to change approaches to respond to lessons and changing challenges in the local environment? ● Project document analysis ● KIIs with GenDev activity managers ● Web surveys with a few open-ended questions posed to Resonance and IPs. ● Project document analysis ● KIIs or group interviews with Making Cents International to assess design, implementation, and flexibility in the activity ● KII/group interview with GenDev AC leads ● Project document analysis ● KIIs with local GBV experts to discuss activity models/approaches and their appropriateness ● Key informant interviews or maybe web surveys with a few open-ended questions posed to stakeholders (not implementing partners) such as local government officials or local advocacy groups to assess the extent of their involvement in activity design ● KIIs with IP senior leadership, reps from partner organizations, local GBV experts, and other key stakeholders ● Project document analysis ● Web surveys for IP staff and other partner organization staff, where feasible ● KIIs with GenDev activity managers 1. Is the activity reaching beneficiaries they are meant to target? ● Target beneficiaries: What are the barriers to reaching beneficiaries? ● Monitoring of results: Is the activity collecting evidence on what is working, not working and what could be done differently to achieve results? ● Project document analysis ● Secondary data analysis (indicator tracking and other MEL data) ● Survey / KII data from project ● Web survey with some open-ended items with Resonance ● KIIs/Web Surveys with reps from partner organizations, other key stakeholders ● Secondary data analysis (indicator tracking and other MEL data) ● Beneficiary web surveys, where possible ● KIIs with local GBV experts to discuss activity models/approaches and their appropriateness ● KIIs/web surveys with IP senior leadership, reps from partner organizations, other key stakeholders ● Secondary data analysis (project and MEL data) ● Beneficiary web surveys, where possible ● Project document analysis ● KIIs with GenDev activity managers ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● Secondary data analysis (indicator tracking and other MEL data) 51 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV EVALUATION QUESTION SUB-QUESTIONS BETTER TOGETHER CARE-GBV RISE WEE ● Beneficiary surveys ● FGDs with beneficiaries ● Employee web surveys, where applicable 2. Is the activity achieving sustainability? ● Sustainability: What plans are in place for sustainability? What is the evidence of potential sustainability? ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● Web survey with some open-ended items with Resonance and other funders and local organizations ● KIIs/web surveys with IP senior leadership, reps from partner organizations, other key stakeholders ● Secondary data analysis (project and MEL data) ● Beneficiary web surveys, where possible ● Key informant interviews or maybe web surveys with a few open-ended questions posed to stakeholders (not implementing partners) such as local government officials or local advocacy groups to assess how project activities will be sustained ● KIIs/web surveys with IP senior leadership, reps from partner organizations, other key stakeholders ● Secondary data analysis (project and MEL data) ● Beneficiary web surveys, where possible ● Project document analysis ● KIIs with GenDev activity managers ● Group interview with GenDev AC leads ● KIIs with IP senior leadership, reps from partner organizations, other key stakeholders ● FGDs with community members 52 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV CLEARANCE PAGE DDI/GenDev: CShenoy cleared with edits/comments 7/15/22 53 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ANNEX B, GBV ACTIVITIES EVALUATED BY THE PPE ACTIVITY CLUSTER INDIVIDUAL ACTIVITIES IMPLEMENTING PARTNER ACTIVITY COMPONENTS BTC: Launched in September 2019, is a global initiative implemented by Resonance Global to crowdsource, fund, and scale forward-thinking solutions from anywhere in the world to improve the lives of Venezuelans and communities hosting them across Latin America and the Caribbean affected by the regional crisis. Guyana: Building the Gap for Venezuelan Migrants (BTG4VM) National Coordinating Coalition Inc. (NCC) • Map GBV service providers • One-stop shop for GBV services • Awareness campaign • Data collection, communication, and use Panama: Shifting Power Dynamic: Engaging Men in Gender-based Violence Reduction (SDP) Hebrew Immigrant Aid Society (HIAS) • Engage men in GBV reduction workshops • Gender dialogues with men and women • Gender inclusion training for national police • National dialogue table on continuing to involve men in preventing GBV Trinidad & Tobago: Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME)* Democracy International (DI) • Recruit and train advocates • Establish a trusted referral network • Match advocates with survivors • Train advocates to support survivors • Social media • Support scalability and sustainability CARE-GBV: Implemented by Development Professionals, Inc.- Making Cents International (DPI￾MCI), CARE-GBV awarded grants from $50,000 to $125,000 over a one-year period (July 2021 - July 2022) to five organizations, each lead by women, including women who identify as survivors of GBV. The grants were given to new, local, and under-utilized partners to improve staff wellness and resiliency in GBV North Macedonia: Supporting Innovative Practices in Self￾Care, Wellness, and Resiliency among GBV Workers Crisis Center Hope (CCH) • Develop a training curriculum on self-care, wellness, and resiliency of GBV workers • Conduct two training workshops • Organize a national conference for dissemination of best practices in policies and work protocol. • Develop and disseminate a guide for GBV workers as a key tool for support in self￾care. • Provide mentoring and psychosocial support to GBV workers and GBV organizations. Nigeria: Promoting Staff Wellness and Resilience for Effective Response to Sexual and Gender-Based Violence Programming* Sexual Offences Awareness and Response Initiative (SOAR) • Strengthen institutional capacity and equip others CSOs in Nigeria’s Federal Capital Territory region to promote staff wellness and resilience and undertake effective GBV prevention and response. • Conduct a Stress Risk Assessment Audit to identify and control potential causes and areas of work-related stress conditions of staff. • Meet with relevant stakeholders and an online review of existing sexual abuse and exploitation policies to inform SOAR’s policies. 54 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ACTIVITY CLUSTER INDIVIDUAL ACTIVITIES IMPLEMENTING PARTNER ACTIVITY COMPONENTS programming, fill global data gaps related to self- and collective care and wellness for staff of GBV organizations, and promote learning. Global: We Care - Institutionalizing Accessible Staff Wellness and Resilience Policies, Tools, and Practices for the GBV Field Sexual Violence Research Initiative (SVRI) • Develop an online course module focused on self- and collective care, wellness, and resilience. • Institutionalize policies and practices that support staff well-being and resilience. • Host a knowledge-exchange series focused on self-, staff-, and collective care, wellness, and resilience, including live events, and knowledge products. Botswana: Thuso Ya Bathusi (Enhancing Staff Resilience and Wellness) Women Against Rape (WAR) • Establish a permanent Human Resources position to serve as the health and wellness officer. • Develop a locally relevant training curriculum of seven two-hour modules. • Develop a smart phone-based Wellness￾Check tool to enable counsellors, particularly those working in remote locations, to share feelings, experiences, and challenges. Bosnia and Herzegovina: Žene sa Une Staff Wellness Program Žene sa Une (ZSU) • Use somatic techniques to renew bonding among staff while being cognizant of the overlap between one’s personal and professional life. • Facilitate learning about staff wellness, care, and resilience, as well as demonstrate and model approaches to embed these principles into the organizational culture. • Disseminate findings externally to promote awareness among other GBV prevention and response actors and stakeholders in the sector RISE Challenge: Implemented by Resonance Global this activity supports organizations to adapt and implement approaches addressing gender￾based violence in environmental programming. The Challenge aims to: (a) raise awareness of linkages between GBV and environmental programming; (b) Colombia: Creative Capacity Building to Address Gender Based Violence in the Artisanal and Small-Scale Mining Sector Massachusetts Institute of Technology Development Lab (MIT D-Lab), Advocacy Capacity Building (ARM) Three movement building methodologies: • Public narrative: use personal and collective stories to build solidarity and mobilize groups into action around joint objectives. • Creative Capacity Building (CCB), use co￾design to harness local creativity and knowledge to design solutions to identified challenges. • Advocacy Capacity Building, empower miners with advocacy skills to influence changes in governance at local and national levels Peru: Conservation of the Alto Mayo Landscape without Gender Violence Conservation International (CI); PROMSEX • Training of women • Development of informal support system for GBV survivors • Capacity building of CI and partners • Study of social tolerance od GBV 55 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ACTIVITY CLUSTER INDIVIDUAL ACTIVITIES IMPLEMENTING PARTNER ACTIVITY COMPONENTS test new programming approaches; (c) share learning on interventions and policies; and (d) increase attention to this issue with other organizations, implementing partners, and donors for collaboration and co-investment. The activity fosters partnerships across sectors to reduce rates of gender￾based violence related to access to and control over natural resources in contexts that are affected by environmental degradation and climate change. Vietnam: Combatting Gender-based Violence in Vietnamese Conservation WildAct Vietnam; CARE International, Vietnam Association of National Parks and Protected Areas, Women in Conservation Canterbury Network • Workshops and training with employees of wildlife conservation organizations and agencies • Meetings with organization and agency managers to exchange knowledge • Development of safeguarding materials and network for female conservation staff Kenya: Advancing Equitable Gender, Social and Power Norms in Community Conservancies in Kenya Kenya Wildlife Conservancies Association (KWCA); Fauna & Flora International (FFI), Sera Wildlife Conservancy (SWC), CARE International in Kenya (CARE), Centre for Rights Education and Awareness (CREAW) • Awareness raising • Development of organizational policies and programming • Capacity building of organizations Fiji: Gender Empowerment and Transformation: Tackling Resource-Based Conflict and Gender-based Violence (GBV/FGRM+) Marstel Day; Wi-Her, University of the South Pacific, the Fiji Environmental Law Association, Live & Learn Environmental Education, Fiji’s Reducing Emissions from Deforestation, and forest Degradation (REDD+) Program • Application of GBV lens onto existing grievance mechanism Democratic Republic of Congo: Rising Up!: Promoting Congolese Women’s Land Access and Preventing GBV in Eastern DRC* Women for Women International (WfWI); Innovation et Formation pour le Développement et la Paix (IFDP) • Training of existing land management structures • Training of male community leaders and women Change Agents • Integration of GBV prevention into Innovation and Training for Development and Peace (IFDP)’s model Uganda: Securing Land Rights & Ending Gender Exclusion (SLEDGE) Trócaire, SOCADIDO, and LEMU • Training of community leaders and partner staff • Norms change and awareness raising in community • Establishing referral systems • Documenting land rights • Alternative dispute resolution mechanism 56 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ACTIVITY CLUSTER INDIVIDUAL ACTIVITIES IMPLEMENTING PARTNER ACTIVITY COMPONENTS WEE: These activities encompass a broad range of interventions to overcome barriers and foster women’s economic participation through directly working with grassroots participants or enabling systems￾level and environmental change related to gender equality. Benin: A Micro￾Journey to Self￾Reliance: Economic Reintegration for Victims of GBV Management Sciences for Health, Inc. (MSH), Ministry of Health, Ministry of Justice, Association Pour l’Education, la Sexualité et la Santé en Afrique (APESSA). Intimate partner violence (IPV) and other forms of GBV through entrepreneur trainings, mentoring and support. • Physical and virtual centers for treatment of GBV victims • Access to income-generating activity (IGA) women’s groups, mentoring and coaching • Access to networks of organizations/businesses for GBV survivors • Business and entrepreneurship training for victims of GBV and their spouses Burundi: Enabling Environment for Economic Empowerment of Women (E4W) Freedom House, Search for Common Ground (SFCG), Catholic Relief Services (CRS), Kahawatu Foundation IPV and other forms of GBV through entrepreneur trainings, community awareness raising and media professionals’ capacity building. • Economic support for women-led coffee farms, including coaching and business support • Entrepreneur training activities and learning tours •Awareness-raising sessions for men and women in coffee cooperatives and other key male community stakeholders • Gender sensitivity trainings for media professionals •Media broadcasts, radio shows and town hall forums for awareness raising and promoting positive masculinities Nigeria: Engendering Industries Tetra Tech, Inc., Ibadan Electricity Distribution Company (IBEDC) Strengthening company policies around GBV and training staff and engaging men in GBV topics: •Women mentorship activities • DISCO 4 Women conference and platform •Anti-GBVH training and sensitization •Male engagement training Lesotho: Global Labor Program (GLP): Levi-Strauss Partnership Solidarity Center, Federation of Women Lawyers in Lesotho (FIDA), Women and Law in Southern Africa – Lesotho (WLSA), Workers’ Rights Watch (WRW), Independent Democratic Union of Lesotho (IDUL), United Textiles Employees Employment-related gender-based violence and harassment (GBVH) and domestic violence for female apparel workers in 2 Nien Hsing owned garment factories: • Anti-GBVH workshops • Information line • Investigations of GBVH by Workers’ Rights Watch 57 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ACTIVITY CLUSTER INDIVIDUAL ACTIVITIES IMPLEMENTING PARTNER ACTIVITY COMPONENTS Guatemala: Women’s Economic Empowerment in Mesoamerica (WEEM)* Rainforest Alliance (RA), Association of Organizations of the Cuchumatanes (ASOCUCH), Women’s Justice Initiative (WJI) IPV and other forms of GBV through entrepreneur trainings, mentoring and support: • Value-chain, leadership and entrepreneurship training for poultry and café business owners • Gender sensitization campaigns, community forums, and awareness sessions on sexual and reproductive health • IGA livelihood activities • Access to women’s groups and other networks Note: * Activity selected for implementation evaluation within the cluster. 58 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ANNEX C, DATA COLLECTION INSTRUMENTS WEB SURVEY INSTRUMENT ACTIVITY SELECTION Please select the organization that you are affiliated with or the implementing organization whose intervention you participated in from the list below: 1. Crisis Center Hope (CCH) 2. Sexual Offenses Awareness and Response Initiative (SOAR) 3. Sexual Violence Research Initiative (SVRI) 4. Women Against Rape (WAR) 5. Zene Sa Une (ZSU) CONSENT PURPOSE OF THE STUDY We are inviting you to participate in this evaluation because of your participation in the activities implemented by [INSERT NAME OF 1 OF THE 5 GRANTEES BASED ON THEIR SELECTION]. Your answers to this brief survey will help improve programs to support first responders for GBV, which will in turn, help survivors. NORC at the University of Chicago is a non-partisan research institution based in the United States and has been commissioned to carry out a performance evaluation of the portfolio of the gender-based violence (GBV) activities. This work is funded by the United States Agency for International Development (USAID). One of the activity clusters is the Collective Action to Reduce Gender-Based Violence (CARE-GBV) small grants program, an initiative to fund interventions to build wellness, resilience and self-care among GBV responders. The purpose of this study is to understand how the activities worked, what might not have worked so well and what can be improved in the USAID’s GBV portfolio. DESCRIPTION OF THE STUDY PROCEDURES If you agree to be in this study, you will be asked questions about your views on the activity design, implementation, uptake, and sustainability. The online survey will take approximately 15 minutes to complete. RISKS/DISCOMFORTS OF PARTICIPATING IN THIS STUDY Your participation in this study does not involve any risks other than what you would encounter in a normal workday at your workplace. If there are any questions you don’t want to answer, you are free to skip to the next question. However, we will always ask if you are sure you want to continue without providing an answer. Your responses are very valuable to help improve future opportunities to address GBV. 59 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV BENEFITS OF PARTICIPATING IN THE STUDY Your participation is important to help this research and USAID learn more about the implementation of the GBV activities funded, including lessons learned and areas for improvement. You will receive no economic or material incentive for participating. CONFIDENTIALITY Your responses will be kept strictly confidential. We will report all results as aggregated data or averages. We will never share any information that could be used to identify you outside of the research team. At the end of the study, we may share the anonymized data with USAID or others outside the study team. Before sharing the data, we will remove all details that could be used to identify you, such as names, employer, or IP used to answer the survey. As such, no one will know whether you participated in the survey or which answers are yours. Since no one will know which answers are yours, we ask that you answer all questions honestly. RIGHT TO REFUSE OR WITHDRAW The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any time. You have the right not to answer any single question, as well as to withdraw completely from the study at any point during the process; additionally, you have the right to request that I delete your answers. There are no penalties for refusing or withdrawing during the survey or afterwards. RIGHT TO ASK QUESTIONS AND REPORT CONCERNS You have the right to ask questions about this research study and to have those questions answered by me before, during or after the research. If you have any further questions about the study, feel free to contact Vaiddehi Bansal by email at bansal-vaiddehi@norc.orc. Consent. Do you agree to participate in this survey? 1. Yes 2. No Questions for all Respondents 1. Age a. ___________years [Note if <18, terminate the survey] b. Don’t know c. No response 2. Gender Identity a. Man b. Woman c. Transgender Man d. Transgender Woman e. Non-binary/non-conforming 60 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV f. Not listed: _________ g. Prefer not to answer 3. Name of organization where you currently work _____________ 4. No. of years you have worked with this [program above response] __________ 5. No. of years you have worked in the GBV sector ____________ Questions for CCH 1. Are you a staff member of CCH? a. Yes b. No [IF NO TO THE PREVIOUS QUESTION] 2. If you are not a CCH staff member, how best would you describe your primary role as a GBV responder? Select all that apply. a. Researcher/academic b. Practitioner/service provider c. Activist d. Policy maker e. Other: _________ 3. CCH organized a two-workshop for GBV responders on self-care, wellness, and resilience. Did you participate in this training program? a. Yes b. No 4. The 2-day workshop organized by CCH comprised nine modules. Please select all the modules that you completed. a. Module 1 – Stress b. Module 2 – Cognitive (mental) relief strategies c. Module 3 – Physical relief strategies d. Module 4 – Sensory relief strategies e. Module 5 – Strategies for emotional relief f. Module 6 – Rainbow for health, well-being, and resistance to stress g. Module 7 – Mindfulness h. Module 8 – Wheel of wellness and well-being i. Module 9 – Establishing a balance between private and professional life j. Don’t remember 5. To what extent has the workshop helped you acquire important information and new knowledge about different types of stress triggers, their recognition and naming? (Likert scale) a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 6. To what extent has the workshop helped you develop abilities and habits for maintaining well￾being, calmness, and self-esteem in stressful situations? (Likert scale) a. To a great extent b. Somewhat c. Neutral 61 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV d. Very little e. Not at all f. Don’t know g. No response 7. To what extent has the workshop helped you develop an understanding of and apply practices to preserve your health (physical, mental, emotional, and sensory)? (Likert scale) a. To a Great Extent b. Somewhat c. Neutral d. Very Little e. Not at All f. Don’t know g. No response 8. To what extent has the workshop helped you establish a balance between private and professional life? (Likert scale) a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 9. To what extent do you think the content covered in the workshop met the needs of GBV responders? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 10. Which component(s) of the training did you find the most helpful? Please Explain: ___________ 11. Do you have any suggestions for other topics that should have been covered in the workshop? Please explain: ______________________ 12. To what extent do you think participants will continue to draw on or use the knowledge acquired form the workshop? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response Please explain: ___________________ 13. For what reasons might it be difficult for participants to maintain the practices or lessons they learned at the workshop? Please explain: ___________________________ 62 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV 14. Do you have any suggestions to improve the way the workshops were conducted? Please explain: ______________ 15. Are there any other comments on the project that you would like to make? Please explain: ________________ Questions for SOAR 1. Please indicate if you are a staff member of SOAR or member organization of the Sexual & Gender-Based Violence Response Team (SGBV-RT). a. SOAR b. SGBV-RT 2. Which of the following activities did you participate in? Select all that apply. a. Focus group discussion b. 5-day training on trauma counseling and psychosocial support for child survivors of sexual abuse c. 3-day on-site learning visit to the Domestic and Sexual Violence Response Agency of Lagos State d. 6-monthly self-care and wellness meetings for sexual and GBV responders e. Development of policies (one-day stakeholders consultative meeting to review policies developed) f. Development of training manuals (two–day validation meeting of training manuals) 3. To what extent have these activities improved your understanding of vicarious trauma among GBV responders who support child survivors of sexual assault? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 4. Which of the above activities were most successful in communicating an understanding of vicarious trauma and best practices to strengthen self-care, wellness, and resilience? a. Focus group discussion b. 5-day training on trauma counseling and psychosocial support for child survivors of sexual abuse c. 3-day on-site learning visit to the Domestic and Sexual Violence Response Agency of Lagos State d. 6-monthly self-care and wellness meetings for sexual and GBV responders e. Development of policies (one-day stakeholders consultative meeting to review policies developed) f. Development of training manuals (two–day validation meeting of training manuals) [FOR THOSE WHO PARTICIPATED IN THE FOCUS GROUP DISCUSSION] 5. To what extent did the Focus Group Discussion help you gain a deeper understanding of the problems and difficulties experienced by GBV responders? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all 63 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV f. Don’t know g. No response 6. To what extent did the Focus Group Discussion help you understand self-care requirements and practices to prevent burnout and secondary trauma? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response [FOR THOSE WHO PARTICIPATED IN 5-DAY TRAINING ON TRAUMA COUNSELING AND PSYCHOSOCIAL SUPPORT FOR CHILD SURVIVORS OF SEXUAL ABUSE] 7. To what extent did the 5-day training on trauma counseling and psychosocial support help you build organizational capacity to improve on survivor-centered GBV response? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response [FOR THOSE WHO PARTICIPATED IN 3-DAY ON-SITE LEARNING VISIT TO THE DOMESTIC AND SEXUAL VIOLENCE RESPONSE AGENCY OF LAGOS STATE] 8. To what extent did the 3-day online learning visit enhance your learning about best practices in operations, processes, and procedures? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 9. To what extent did the 3-day online learning visit help you identify good methods that may be emulated from existing GBV responsive structure with track record of excellent delivery? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response [FOR THOSE WHO PARTICIPATED IN 6-MONTHLY SELF-CARE AND WELLNESS MEETINGS] 10. At the end of the Self-Care and Wellness Meetings, to what extent did you observe an improvement in the self-care practices of staff who are on the frontline of addressing sexual and GBV in the Federal Capital Territory? a. To a great extent b. Somewhat 64 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV c. Neutral d. Very little e. Not at all f. Don’t know g. No response [FOR THOSE WHO PARTICIPATED IN DEVELOPMENT OF POLICIES] 11. To what extent did this initiative identify major safeguarding issues faced by SGBV-focused organizations, and facilitate the development and adoption of Safeguarding Policy by the FCT SGBV Response Team? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 12. To what extent do you think the Safeguarding Policy will be implementation and maintained? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 13. Do you envision any challenges in the sustainable implementation the Safeguarding Policy? Please explain: ___________________________ 14. Are there any measures in place to revise these policies as needed? a. Yes b. No 15. Are there any measures in place to conduct orientation sessions on these policies for new staff? a. Yes b. No [FOR THOSE WHO PARTICIPATED IN DEVELOPMENT OF TRAINING MANUALS] 16. To what extent do you think the training manuals meet the needs of GBV responders? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 17. Which component(s) of the training did you find the most helpful? Please Explain: ___________ 18. Do you have any suggestions for any other topics that should have been covered in the training manuals? Please explain: ______________________ 19. To what extent do you think participants will recall and use learnings from the training manuals? a. To a great extent 65 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 20. Do you envision any challenges in the sustainable implementation of learnings from the training manual? Please explain: ___________________________ 21. Are there provisions in place to revise the training manuals as needed? a. Yes b. No 22. Are there provisions in place to conduct refresher trainings for staff? a. Yes b. No [FOR SOAR STAFF ONLY] 23. Since the implementation of this activity by SOAR, to what extent has the culture of your organization changed to be understanding of and responsive to vicarious trauma? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 24. To what extent do you think the new policies and practices on managing vicarious trauma will be implemented and maintained after the project period? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response [FOR EVERYONE] 25. Do you have any suggestions on how the project can improve? Please explain: ______________ 26. Are there any other comments on the project that you would like to make? Please explain: ________________ Questions for SVRI 1. Are you a staff member of SVRI? a. Yes b. No [IF NO TO THE PREVIOUS QUESTION] 66 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV 2. If you are not a SVRI staff member, how best would you describe your primary role as a GBV responder? Select all that apply. a. Researcher/academic b. Practitioner/service provider c. Activist d. Policy maker e. Other: ________ 3. The Dare to Care online course comprised four modules. Please check all the modules that you completed. a. Module 1 – Settling into the forest floor b. Module 2 – Exploring the understory c. Module 3 – Tending to the canopy d. Module 4 – Flourishing at the emergent layer e. Don’t remember 4. To what extent has the course helped you develop a shared understanding of stress, burn-out and vicarious trauma, and recognize the signs in yourself and your colleagues? (Likert scale) a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 5. To what extent has the course helped you develop your own definitions, practices, and systems of self and collective care, including understanding how they are interlinked and interdependent? (Likert scale) a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 6. To what extent has the course helped you identify the factors undermining and enabling collective care in your organization or team and know what works to institutionalize collective care? (Likert scale) a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 7. To what extent would you say that the online course was easy to access and user friendly? a. To a great extent b. Somewhat c. Neutral 67 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV d. Very little e. Not at all f. Don’t know g. No response 8. To what extent would you say that the course content was easy to understand? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 9. The course includes a variety of learning aides – including videos and reflective activities for those completing the course individually, as well as adapted activities for those working through the course collectively with colleagues. How did these learning aides work for you and your group? Please explain: ________________________ 10. Did you experience any challenges with the course in terms of content, structure, online accessibility, time to complete, and so on? a. Yes [IF YES] Please explain: _________________________ b. No 11. Which component(s) of the training did you find the most helpful? Please Explain: ___________ 12. Do you have any suggestions for other topics that you would have liked covered in the course? Please explain: ______________________ [FOR SVRI STAFF ONLY] 13. Since the implementation of this activity, to what extent has the culture of your organization changed to be understanding of and responsive to vicarious trauma? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 14. To what extent do you think the new policies and practices on managing vicarious trauma will be implemented and maintained after the project period? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response [FOR EVERYONE] 15. Do you have any suggestions on how the project can improve? Please explain: _________ 68 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV 16. Are there any other comments on the project that you would like to make? Please explain: _____________ Questions for WAR 1. Are you a staff member of WAR? a. Yes b. No 2. WAR developed and implemented a Vicarious Trauma and Self-Care curriculum. Did you participate in this training program? a. Yes b. No 3. The Vicarious Trauma and Self-Care curriculum comprised six modules. Please check all the modules that you completed. a. Module 1 – Vicarious trauma Part I b. Module 2 – Vicarious trauma Part 2 c. Module 3 – Crisis intervention and trauma d. Module 4 – Online and telephone counselling e. Module 5 – Trauma informed supervision f. Module 6 – Managing referrals g. Don’t remember 4. To what extent would you say that the curriculum content was easy to understand? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 5. To what extent do you think the specific needs of WAR staff was considered while developing the curriculum? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 6. Which component(s) of the training did you find the most helpful? Please Explain: ___________ 7. Do you have any suggestions for other topics that you would have liked covered in the curriculum? Please explain: _____________ 8. As part of the intervention, WAR employed a new Human Resource and Wellness Officer. To what extent has this new role helped in monitoring and supporting the wellbeing of staff and promoting a culture of self and collective care? a. To a great extent b. Somewhat c. Neutral 69 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV d. Very little e. Not at all f. Don’t know g. No response 9. As part of the intervention, WAR developed and implemented a weekly smartphone-based staff Wellness Check-In Tool (WCT). To what extent is this tool effective in monitoring the emotional health and well-being of WAR staff? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 10. Do you have any suggestions for additional questions that can be added to the WCT? Please explain: ___________________ 11. The WCT is available on Google Forms and can be completed via cellphones. To what extent does the technological aspect of this intervention pose a hindrance or challenge? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 12. All WAR staff are asked to complete the WCT as part of their regular weekly activities. Have you experienced (or do you envision) any challenges with completing the form on a weekly basis? Please explain: ___________________ 13. The WCT is monitored by WAR’s Health and Wellness Officer, who is responsible for reaching out to staff (as needed) to offer assistance. To what extent do you think this is being implemented? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 14. To what extent do you think the WCT will be maintained to help staff to share feelings, experiences, and challenges? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 70 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV 15. Do you envision any challenges with continued implementation and completion of the WCT? Please explain: _________________________ 16. As part of the intervention, WAR also developed a Staff Wellness Policy. To what extent do you think the implementation and adoption of these policies is sustainable? h. To a great extent i. Somewhat j. Neutral k. Very little l. Not at all m. Don’t know n. No response 17. Since the implementation of this activity, to what extent has the culture of your organization changed to be understanding of and responsive to vicarious trauma? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 18. To what extent do you think the new policies and practices on managing vicarious trauma will be implemented and maintained after the project period? a. To a Great Extent b. Somewhat c. Neutral d. Very Little e. Not at All f. Don’t know g. Refused 19. Do you have any suggestions on how the project can improve? Please explain: ____________ 20. Are there any other comments on the project that you would like to make? Please explain: _____________ Questions for ZSU 1. Are you a staff member of ZSU? a. Yes b. No 2. ZSU developed and implemented a bespoke Staff Wellness and Resiliency Building program to frontline organization staff? Did you participate in this training program? a. Yes b. No 3. Did you complete all 6 training sessions? a. Yes b. No (If no then indicate the no. of sessions completed) 4. To what extent did the training strengthen your understanding of residual impact for GBV responders? 71 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 5. To what extent did the training strengthen your understanding of ways to minimize negative effects and maintain an empathic position? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 6. To what extent did the training strengthen your understanding of boundaries and how to manage and protect your boundaries when providing services to GBV survivors? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 7. The final session of the training was geared towards developing a sustainable self-care plan. What are some intentional behaviors/actions that you will implement as part of your self-care plan? Please explain: ___________ 8. To what extent do you think these behaviors/actions that you described above are sustainable? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 9. To what extent do you think the needs of ZSU staff was considered while developing the training program? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 10. Which component(s) of the training did you find the most helpful? Please Explain: ___________ 72 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV 11. Do you have any suggestions for other topics that you would have liked covered in the training? Please explain: _____________ 12. Since the implementation of this activity, to what extent has the culture of your organization changed to be understanding of and responsive to vicarious trauma? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 13. To what extent do you think the new policies and practices on managing vicarious trauma will be implemented and maintained after the project period? a. To a great extent b. Somewhat c. Neutral d. Very little e. Not at all f. Don’t know g. No response 14. Do you have any suggestions on how the project can improve? Please explain: ____________ 15. Are there any other comments on the project that you would like to make? Please explain: _________ 73 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV KII GUIDE – IP SENIOR STAFF (CCH) Respondent Name, Institution Date: Start Time: Category Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of USAID’s CARE￾GBV cluster. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on GBV prevention and response. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are completely neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming focused on GBV prevention and response. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Ritu Nayyar-Stone, the project director for this study at nayyarstone-ritu@norc.org. [PUT EMAIL IN CHAT] With your permission, I’d like to record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. Before we jump in, I’d like to get to know you bit. Could you please give a briefly introduction and your area of focus within CCH? 74 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question PORTFOLIO QUESTIONS Section Introduction Today we are going to discuss your activity under the CARE-GBV portfolio. We will start with some portfolio￾level questions. Coordination 1a. Have you or your organization participated in meetings with other grantees via USAID’s USAID/GenDev? If so, what was the purpose of these meetings? 1b. IF YES: What information was important or what ideas or other benefits did you obtain from this experience(s)? 1c. Would you recommend more exchanges between grantees and USAID? Why? Or why not? Foundation 2a. Have you been briefed on USAID strategies and priorities around GBV? Stakeholders 3a. CCH worked with other stakeholders including a partner organization – Pleiades Organization, National Conference Stakeholders and a Curriculum Development Expert. Can you describe how you selected these partners? 3b. What were the contributions of these partners? In what ways was the collaboration valuable? 3c. What other stakeholders were you able to engage to accomplish activity goals and what were their contributions? 3d. Do you have any lessons that you could share? Intervention Planning and Design 4a. Thinking about the planning process of these GBV interventions, what were key pieces of information that guided the intervention design? 4b. What were important knowledge or practice gaps? 4c. The main goal of this activity was to introduce policies and practices of self-care, wellness, and resiliency among staff of GBV organizations in North Macedonia. Can you walk me through the conceptualization and design of this activity? Vicarious Trauma 5a. What are key areas of evidence or information that informed your work on vicarious trauma? 5b. What were important knowledge and practice gaps about programming to address vicarious trauma vicarious trauma? Section Introduction Thank you for your valuable insights on this set of questions. We will now transition to activity cluster questions. ACTIVITY CLUSTER QUESTIONS Needs Assessment and Intervention Evidence 6a. What evidence was available on the specific self-care and wellness needs of GBV responders? 6b. What pre-implementation assessments did you do for this project? Did you draw on other similar interventions or intervention evaluations? Were they useful? 75 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question 6c. How did those help your team plan and implement the activity? 6d. Would you recommend any other kinds of research before implementing similar activities? Monitoring and Adaptations 7a. Did you have any measures in place to monitor the activities and effectiveness? If yes, how were these findings used? If not, why not? 7b. Could you share some examples of aspects you might have changed based on emerging evidence? 7c. Were the monitoring tools/templates accessible and user friendly? How was monitoring conducted? 7d. How did you integrate feedback from participants? What changes did you make to the policies and content? Outcomes 8a. What were the primary outcomes of the activity? 8b. Reflecting back on the activities, do you think the outcomes stated in activity design were realistic and achievable? Why or why not? 8c. Was the length of the grant sufficient to achieve these outcomes? Please describe one or two of the main activity outcomes so far. 8d. Did the activities produce any outcomes that were unexpected? If yes, can you describe these, please? 8e. Are there outcomes that you would have liked to see but were not feasible to accomplish? Why? Intervention Implementation 9a. Do you think that your project was able to reach the beneficiaries that it was designed to reach? 9b. How would you describe the reach of the program across North Macedonia? 9c. Were there any challenges in reaching the target groups to influence change? 9d. Who else should be engaged and was not in the activity? Mechanisms 10a. What do you think are the most effective components of your project? Why? Sustainability 11a. What operational challenges did your organization have to implement the activity? 11b. How will CCH monitor the sustainability of self-care policies and practice wellness, and resiliency among staff of GBV organizations beyond the grant period? 11c. What components or approaches in the CARE-GBV activity have remained active even after the USAID funding ended? Can you describe how these were maintained? 11d. What strategies could have enhanced sustainability of activity components that have ended? 76 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question 11e. What have been the primary facilitators and barriers to the sustainability of the activity? Replicability, Transferability and Adaptability 12a. What components or approaches do you think could be replicated in other communities or countries? 12b. Where else would you recommend implementing this activity? 12c. What approaches, components, or tools of the activity would need to be adapted for a different context? Scalability 13a. If you were to scale up your activity, which components of your intervention would you focus on? 13b. Are there any that you would drop? What changes would you make? 13c. What are the main challenges for scaling the activity up in your country/region? 13d. In thinking about the costs of these activities, how cost-effective do you think they’d be at scale, and why? 77 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV KII GUIDE – IP SENIOR STAFF (SOAR) Respondent Name, Institution Date: Start Time: Category Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of USAID’s CARE￾GBV cluster. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on GBV prevention and response. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are completely neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming focused on GBV prevention and response. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Ritu Nayyar-Stone, the project director for this study at nayyarstone-ritu@norc.org. [PUT RITU EMAIL IN CHAT] With your permission, I’d like to record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. Before we jump in, I’d like to get to know you bit. Could you please give a briefly introduction and your area of focus within SOAR? PORTFOLIO QUESTIONS Section Introduction Today we are going to discuss your activity under the CARE-GBV portfolio. We will start with some portfolio-level questions. 78 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question Coordination 1a. Have you or your organization participated in meetings with other grantees via USAID’s USAID/GenDev? If so, what was the purpose of these meetings? 1b. IF YES: What information was important or what ideas or other benefits did you obtain from this experience(s)? 1c. Would you recommend more exchanges between grantees and USAID? Why? Or why not? Foundation 2a. Have you been briefed on USAID strategies and priorities around GBV? Stakeholders 3a. SOAR collaborated with Youth Net and Counseling (YONECO); Coalition of Women Living with HIV and AIDS (COWLHA) to implement the intervention. Can you describe how you selected these partners? 3b. What were the contributions of these partners? a. (probe) In what ways was the collaboration valuable? 3c. What other stakeholders were you able to engage to accomplish activity goals? What were their contributions? 3d. Do you have any lessons that you could share? Intervention Planning and Design 4a. Thinking about the planning process of these GBV interventions, what were key pieces of information that guided the intervention design? 4b. What were important knowledge or practice gaps? Vicarious Trauma 5a. What are key areas of evidence or information that informed your work on vicarious trauma? 5b. What were important knowledge and practice gaps about programming to address vicarious trauma? Section Introduction Thank you for your valuable insights on this set of questions. We will now transition to activity cluster questions. ACTIVITY CLUSTER QUESTIONS Needs Assessment and Intervention Evidence 6a. What evidence was available on the specific self-care and wellness needs of GBV responders? 6b. What pre-implementation assessments did you do for this project? Did you draw on other similar interventions or intervention evaluations? Were they useful? 6c. How did those help your team plan and implement the activity? 6d. Would you recommend any other kinds of research before implementing similar activities? 79 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question Assumptions 7a. There is no common theory of change for the CARE-GBV cluster. What prompted you to develop your independent theory of change for SOAR’s activity? 7b. When formulating the TOC of the activity, what were the main assumptions about: ● The ability of the organization to deliver the activities; ● Of the potential participant s to access the services; ● Of how the services would result in the desired outcomes 7c. How relevant were the assumptions? Monitoring and Adaptations 8a. Did you have any measures in place to monitor the activities and effectiveness? If yes, how were these findings used? If no, why not? 8b. Could you share some examples of aspects you might have changed based on emerging evidence? 8c. Were the monitoring tools/templates accessible and user friendly? How was monitoring conducted? Outcomes 8a. What were the primary outcomes of the activity? 8b. Reflecting back on the activities, do you think the outcomes stated in activity design were realistic and achievable? Why or why not? 8c. Was the length of the grant sufficient to achieve these outcomes? Please describe one or two of the main activity outcomes so far. 8d. Did the activities produce any outcomes that were unexpected? If yes, can you describe these, please? 8e. Are there outcomes that you would have liked to see but were not feasible to accomplish? Why? Intervention Implementation 9a. Do you think that your project was able to reach the beneficiaries that it was designed to reach? 9b. How would you describe the reach of the program across Nigeria? 9c. Were there any challenges in reaching the target groups to influence change? 9d. Who else should be engaged and was not in the activity? Mechanisms 10a. What do you think are the most effective components of your project? Why? Sustainability 11a. What operational challenges did your organization have to implement the activity? 80 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question 11b. What components or approaches of the activity have remained active even after the USAID funding ended? Can you describe how that was maintained? 11c. What strategies could have enhanced sustainability of activity components that have ended? 11d. What have been the primary facilitators and barriers to the sustainability of the activity? Are there any challenges of online learning? Replicability, Transferability and Adaptability 12a. What components or approaches do you think could be replicated in other communities or countries? 12b. Where else would you recommend implementing this activity? 12c. What approaches, components, or tools of the activity would need to be adapted for a different context? Scalability 13a. If you were to scale up your activity, which components of your intervention would you focus on? 13b. Are there any that you would drop? What changes would you make? 13c. What are the main challenges for scaling the activity up in your country/region? Section Introduction Thank you for your valuable insights. For the last part of the interview, we will ask you some implementation evaluation related questions. IMPLEMENTATION EVALUATION QUESTIONS Design 14a. Could you speak to how this activity was designed? What factors influenced this design? 14b. Who was involved in these design decisions? 14c. SOAR provides counseling services to survivors of child sexual abuse. Can you tell me if and how this unique focus was incorporated into the study design? What factors were considered so they can better support this group of GBV survivors? 14d. What findings from the baseline study were considered when developing learning materials throughout the implementation period? 14e. Would you change anything about the design? Implementation 15a. In your opinion which approaches/tools were most effective in achieving the objective of this activity? 15b. Were there specific challenges or enabling factors in implementing this activity? 15c. If there were challenges, were the challenges overcome and how? Flexibility 16a. Is there sufficient staffing to respond to local priorities? 81 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question 16b. Is there flexibility to change approaches to respond to lessons and changing challenges in the local environment? Monitoring of Results 17a. Is the activity collecting evidence on what is working, not working and what could be done differently to achieve results? Sustainability 18a. Do you think that this activity is sustainable moving forward? Do you have any evidence support this? 82 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV KII GUIDE – IP SENIOR STAFF (SVRI) Respondent Name, Institution Date: Start Time Category Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of USAID’s CARE￾GBV cluster. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on GBV prevention and response. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are completely neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming focused on GBV prevention and response. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Ritu Nayyar-Stone, the project director for this study at nayyarstone-ritu@norc.org. [PUT RITU EMAIL IN CHAT] With your permission, I’d like to record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. Before we jump in, I’d like to get to know you bit. Could you please give a briefly introduction and your area of focus within SVRI? PORTFOLIO QUESTIONS Section Introduction Today we are going to discuss your activity under the CARE-GBV portfolio. We will start with some portfolio level questions. 83 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question Coordination 1a. Have you or your organization participated in meetings with other grantees via USAID’s USAID/GenDev? If so, what was the purpose of these meetings? 1b. IF YES: What information was important or what ideas or other benefits did you obtain from this experience(s)? 1c. Would you recommend more exchanges between grantees and USAID? Why? Or why not? Foundation 2a. Have you been briefed on USAID strategies and priorities around GBV? Stakeholders 3a. SVRI collaborated with HaRT and Raising Voices to implement the intervention. Can you describe how you selected these partners? 3b. What were the contributions of these partners? In what ways was the collaboration valuable? 3c. What other stakeholders were you able to engage to accomplish activity goals and what were their contributions? 3d. Do you have any lessons that you could share? Intervention Planning and Design 4a. Thinking about the planning process of these GBV interventions, what were key pieces of information that guided the intervention design? 4b. What were important knowledge or practice gaps? 4c. SVRI developed an online Dare to Care Course to support other GBV stakeholders. What was the process for designing this course? Vicarious Trauma 5a. What are key areas of evidence or information that informed your work on vicarious trauma? 5b. What were important knowledge and practice gaps about programming to address vicarious trauma? Section Introduction Thank you for your valuable insights on this set of questions. We will now transition to activity cluster questions. ACTIVITY CLUSTER QUESTIONS Needs Assessment and Intervention Evidence 6a. What evidence was available on the specific self-care and wellness needs of GBV responders? 6b. What pre-implementation assessments did you do for this project? Did you draw on other similar interventions or intervention evaluations? Were they useful? 6c. How did those help your team plan and implement the activity? 6d. Would you recommend any other kinds of research before implementing similar activities? Assumptions 7a. There is no common theory of change for the CARE￾GBV cluster. What prompted you to develop your independent theory of change for SVRI’s activity? 7b. When formulating the TOC of the activity, what were the main assumptions about: 84 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question • The ability of the organization to deliver the activities; • Of the potential participant s to access the services; • Of how the services would result in the desired outcomes 7b. How relevant were the assumptions? Monitoring and Adaptations 8a. Did you have any measures in place to monitor the activities and effectiveness? If yes, how were these findings used? If not, why not? 8b. Could you share some examples of aspects you might have changed based on emerging evidence? 8c. Were the monitoring tools/templates accessible and user friendly? How was monitoring conducted? 8d. How did you integrate feedback from participants? What changes did you make to the policies and content? Outcomes 9a. What were the primary outcomes of the activity? 9b. Reflecting back on the activities, do you think the outcomes stated in activity design were realistic and achievable? Why or why not? 9c. Was the length of the grant sufficient to achieve these outcomes? Please describe one or two of the main activity outcomes so far. 9d. Did the activities produce any outcomes that were unexpected? If yes, can you describe these, please? 9e. Are there outcomes that you would have liked to see but were not feasible to accomplish? Why? Intervention Implementation 10a. Do you think that your project was able to reach the beneficiaries that it was designed to reach? 10b. How would you describe the reach of the program globally? 10c. Were there any challenges in reaching the target groups to influence change? 10d. Who else should be engaged and was not in the activity? Mechanisms 11a. What do you think are the most effective components of your project? Why? Sustainability 12a. What operational challenges did your organization have to implement the activity? 12b. What components or approaches of the activity have remained active even after the USAID funding ended? Can you describe how these were maintained? 12c. What strategies could have enhanced sustainability of activity components that have ended? 85 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question 12d. What have been the primary facilitators and barriers to the sustainability of the activity? Are there any challenges of online learning? Replicability, Transferability and Adaptability 13a. What components or approaches do you think could be replicated in other communities or countries? 13b. Where else would you recommend implementing this activity? 13c. What approaches, components, or tools of the activity would need to be adapted for a different context? Scalability 14a. If you were to scale up your activity, which components of your intervention would you focus on? 14b. Are there any that you would drop? What changes would you make? 14c. What are the main challenges for scaling the activity up in your country/region? 14d. Thinking about the planning process of these GBV interventions, what were key pieces of information that guided the intervention design? 86 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV KII GUIDE – IP SENIOR STAFF (WAR) Respondent Name, Institution Date: Start Time: Category Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of USAID’s CARE-GBV cluster. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on GBV prevention and response. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are completely neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming focused on GBV prevention and response. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Ritu Nayyar-Stone, the project director for this study at nayyarstone￾ritu@norc.org. [PUT RITU EMAIL IN CHAT] With your permission, I’d like to record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. Before we jump in, I’d like to get to know you bit. Could you please give a briefly introduction and your area of focus within WAR? PORTFOLIO QUESTIONS Section Introduction Today we are going to discuss your activity under the CARE-GBV portfolio. We will start with some portfolio-level questions. Coordination 1a. Have you or your organization participated in meetings with other grantees via USAID’s USAID/GenDev? If so, what was the purpose of these meetings? 87 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question 1b. IF YES: What information was important or what ideas or other benefits did you obtain from this experience(s)? 1c. Would you recommend more exchanges between grantees and USAID? Why? Or why not? Foundation 2a. Have you been briefed on USAID strategies and priorities around GBV? Stakeholders 3a. WAR worked with a range of stakeholders including experts from University of Pennsylvania, Rutgers University and University of Botswana to implement the intervention. Can you describe the how you selected these partners? 3b. What were the contributions of these partners? In what ways was the collaboration valuable? 3c. What other stakeholders were you able to engage to accomplish activity goals and what were their contributions? 3d. Do you have any lessons that you could share? Intervention Planning and Design 4a. Thinking about the planning process of these GBV interventions, what were key pieces of information that guided the intervention design? 4b. What were important knowledge or practice gaps? Vicarious Trauma 5a. What are key areas of evidence or information that informed your work on vicarious trauma? 5b. What were important knowledge and practice gaps about programming to address vicarious trauma? Section Introduction Thank you for your valuable insights on this set of questions. We will now transition to activity cluster questions. ACTIVITY CLUSTER QUESTIONS Needs Assessment and Intervention Evidence 6a. What evidence was available on the specific self-care and wellness needs of GBV responders? 6b. What pre-implementation assessments did you do for this project? Did you draw on other similar interventions or intervention evaluations? Were they useful? 6c. How did those help your team plan and implement the activity? 6d. Would you recommend any other kinds of research before implementing similar activities? Monitoring and Adaptations 7a. What evidence was available on the specific self-care and wellness needs of GBV responders? 7b. What pre-implementation assessments did you do for this project? Did you draw on other similar interventions or intervention evaluations? Were they useful? 7c. How did those help your team plan and implement the activity? 7d. Would you recommend any other kinds of research before implementing similar activities? Outcomes 8a. What were the primary outcomes of the activity? 88 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question 8b. Reflecting back on the activities, do you think the outcomes stated in activity design were realistic and achievable? Why or why not? 8c. Was the length of the grant sufficient to achieve these outcomes? Please describe one or two of the main activity outcomes so far. 8d. Did the activities produce any outcomes that were unexpected? If yes, can you describe these, please? 8e. Are there outcomes that you would have liked to see but were not feasible to accomplish? Why? Planning and Activity Design 9a. WAR designed and implemented a weekly staff Wellness Check Tool (WCT) as part of the activity. How did you use the data collected from this tool? 9b. Did you (or do you plan to) make any changes based on the responses? 9c. How have the various scales utilized (Vicarious Trauma Scale and Burnout Scale) been used by WAR to improve resources given to staff? Intervention Implementation 10a. Do you think that your project was able to reach the beneficiaries that it was designed to reach? 10b. How would you describe the reach of the program across Bostwana? 10c. Were there any challenges in reaching the target groups to influence change? 10d. Who else should be engaged and was not in the activity? Mechanisms 11a. What do you think are the most effective components of your project? Why? Sustainability 12a. What operational challenges did your organization have to implement the activity? 12b. How will you ensure sustainability of WCT? Is there a mechanism in place to send reminders to WAR staff to complete the WCT? Do you envision that staff might lose interest over time? 12c. WAR has submitted the training curriculum modules to the Botswana Human Resources Development Council for accreditation. Can you share the decision-making process behind this and if it will contribute to the sustainability of the intervention? 12d. What components or approaches in the CARE-GBV activity have remained active even after the USAID funding ended? Can you describe how these were maintained? 12e. What strategies could have enhanced sustainability of activity components that have ended? 12f. What have been the primary facilitators and barriers to the sustainability of the activity? Replicability, Transferability 13a. What components or approaches do you think could be replicated in other communities or countries? 13b. Where else would you recommend implementing this activity? 89 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question and Adaptability 13c. What approaches, components, or tools of the activity would need to be adapted for a different context? Scalability 14a. If you were to scale up your activity, which components of your intervention would you focus on? 14b. Are there any that you would drop? What changes would you make? 14c. What are the main challenges for scaling the activity up in your country/region? 14d. In thinking about the costs of these activities, how cost-effective do you think they’d be at scale, and why? 90 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV KII GUIDE – IP SENIOR STAFF (ZSU) Respondent Name, Institution Date: Start Time: Category Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of USAID’s CARE￾GBV cluster. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on GBV prevention and response. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are completely neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming focused on GBV prevention and response. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Ritu Nayyar-Stone, the project director for this study at nayyarstone-ritu@norc.org. [PUT RITU EMAIL IN CHAT] With your permission, I’d like to record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. Before we jump in, I’d like to get to know you bit. Could you please give a briefly introduction and your area of focus within ZSU? PORTFOLIO QUESTIONS Section Introduction Today we are going to discuss your activity under the CARE-GBV portfolio. We will start with some portfolio￾level questions. 91 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question Coordination 1a. Have you or your organization participated in meetings with other grantees via USAID’s USAID/GenDev? If so, what was the purpose of these meetings? 1b. IF YES: What information was important or what ideas or other benefits did you obtain from this experience(s)? 1c. Would you recommend more exchanges between grantees and USAID? Why? Or why not? Foundation 2a. Have you been briefed on USAID strategies and priorities around GBV? Stakeholders 3a. ZSU partnered with Common Threads Partnership to implement the intervention. Can you describe how you selected this partner? 3b. What were the contributions of this partner? In what ways was the collaboration valuable? 3c. What other stakeholders were you able to engage to accomplish activity goals? What were their contributions? 3d. Do you have any lessons that you could share? Intervention Planning and Design 4a. Thinking about the planning process of these GBV interventions, what were key pieces of information that guided the intervention design? 4b. What were important knowledge or practice gaps? 4c. ZSU provides counseling services to domestic violence survivors, trafficking victims, and refugee populations. Can you tell me if and how this unique focus was incorporated into the study design? Vicarious Trauma 5a. What are key areas of evidence or information that informed your work on vicarious trauma? 5b. What were important knowledge and practice gaps about programming to address vicarious trauma? Section Introduction Thank you for your valuable insights on this set of questions. We will now transition to activity cluster questions. ACTIVITY CLUSTER QUESTIONS Needs Assessment and Intervention Evidence 6a. What evidence was available on the specific self-care and wellness needs of GBV responders? 6b. What pre-implementation assessments did you do for this project? Did you draw on other similar interventions or intervention evaluations? Were they useful? 6c. How did those help your team plan and implement the activity? 6d. Would you recommend any other kinds of research before implementing similar activities? 92 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question Monitoring and Adaptations 7a. Did you have any measures in place to monitor the activities and effectiveness? If yes, how were these findings used? If no, why not? 7b. Could you share some examples of aspects you might have changed based on emerging evidence? 7c. Were the monitoring tools/templates accessible and user friendly? How was monitoring conducted? 7d. ZSU developed and implemented a bespoke Staff Wellness and Resiliency Building program to frontline organization staff. How did you plan to integrate feedback from participants? What changes (if any) did you make to the course design and content? Outcomes 8a. What were the primary outcomes of the activity? 8b. Reflecting back on the activities, do you think the outcomes stated in activity design were realistic and achievable? Why or why not? 8c. Was the length of the grant sufficient to achieve these outcomes? Please describe one or two of the main activity outcomes so far. 8d. Did the activities produce any outcomes that were unexpected? If yes, can you describe these, please? 8e. Are there outcomes that you would have liked to see but were not feasible to accomplish? Why? Intervention Implementation 10a. Do you think that your project was able to reach the beneficiaries that it was designed to reach? 10b. Were there any challenges reaching the target groups to influence change? 10c. Who else should be engaged and was not in the activity? Mechanisms 11a. What do you think are the most effective components of your project? Why? Sustainability 12a. What operational challenges did your organization have to implement the activity? 12b. ZSU’s training course included a module on sustainable self-care plan. How will ZSU monitor the sustainability of this activity beyond the grant period? 12c. What components or approaches in the CARE-GBV activity have remained active even after the USAID funding ended? Can you describe how these were maintained? 12d. What strategies could have enhanced sustainability of activity components that have ended? 12e. What have been the primary facilitators and barriers to the sustainability of the activity? 93 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question Replicability, Transferability and Adaptability 13a. What components or approaches do you think could be replicated in other communities or countries? 13b. Where else would you recommend implementing this activity? 13c. What approaches, components, or tools of the activity would need to be adapted for a different context? Scalability 14a. If you were to scale up your activity, which components of your intervention would you focus on? 14b. Are there any that you would drop? What changes would you make? 14c. What are the main challenges for scaling the activity up in your country/region? 14d. In thinking about the costs of these activities, how cost-effective do you think they’d be at scale, and why? 94 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV KII GUIDE – MAKING CENTS INTERNATIONAL Respondent Name, DRG Area(s) of Expertise, Institution Date: Start Time: Category Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of USAID’s CARE-GBV cluster. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on GBV prevention and response. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are completely neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming focused on GBV prevention and response. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Ritu Nayyar-Stone, the project director for this study at nayyarstone￾ritu@norc.org. [PUT RITU EMAIL IN CHAT] With your permission, I’d like to record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. Before we jump in, I’d like to get to know you bit. Could you please give a briefly introduction and your area of focus within CARE-GBV? PORTFOLIO QUESTIONS Section Introduction Today we are going to discuss the CARE-GBV portfolio. Coordination 1a. Could you please tell me about the management structure for the CARE-GBV cluster? 1b. How does information flow from grantees up to GenDev? 1c. How is this information used for decision making? 95 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question Foundation 2a. Were grantees briefed on USAID strategies and priorities around GBV? If yes, what was the procedure and intended learning outcomes? Protection 3a. What do you see as the most influential activities or strategies to improve access to effective services for survivors? Which grantee approaches were effective for this? 3b. Which grantee approaches were effective for this? Stakeholders 4a. Was there a strategy for engaging a broad range of stakeholders for the CARE-GBV cluster? 4b. Could you please give some examples of activities that were able to effectively engage relevant stakeholders versus those who were less able to do so? Intervention Planning and Design 5a. What are key pieces of evidence to inform this type of programming? Which ones were most influential in your planning? 5b. Were there important knowledge and practice gaps when planning and designing GBV interventions? Did you draw on any intervention evaluations from similar programs? Vicarious Trauma 6a. What are key areas of evidence or information that informed this grant on vicarious trauma? 6b. What were important knowledge and practice gaps about programming to address vicarious trauma? Reach and Effectiveness 7a. Does the CARE-GBV cluster produce knowledge that can influence the reach and effectiveness of other GBV interventions, even those not funded by USAID? How so? ACTIVITY CLUSTER QUESTIONS Needs Assessment and Intervention Evidence 8a. Were there requirements for all grantees in the activity cluster to conduct pre-implementation assessments? Why or why not? Assumptions 9a. Were there any key assumptions made when designing and implementing the activity clusters? Can you describe the most important assumptions related to, for example: • organizational capacity • needs of beneficiaries • effective ways to reach the beneficiaries • causal pathways from the activity to the desired outcome • theory of change 9b. Reflecting back, how accurate were any assumptions? Monitoring and Adaptations 10a. How well do you think projects were able to monitor implementation and adapt their implementation using emerging findings? Could you please share some examples of this? 96 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question Outcomes 11a. Upon reflection, how realistic do you think that the outcomes stated in project designs for this activity cluster were realistic? 11b. Do you think that the length of the grants were sufficient to achieve these outcomes? Why or why not? 11c. Which activities were most successful in achieving the outcomes in their project design, and which were less successful? Why? Planning and Activity Design 12a. What do you think about the project designs across the cluster? Could you please share examples of project designs that you think were particularly effective? Intervention Implementation 13a. How well do you think the activities in this cluster were able to reach the intended beneficiaries? 13b. What were the most effective ways participants were reached? What were the main challenges in reaching the right people? Mechanisms 14a. What do you think are the most effective interventions or approaches used by the grantees? Why were these effective? 14b. Were there any approaches that were particularly ineffective or unsuccessful? Why? Sustainability 15a. Could you please tell me which activities or approaches in the CARE-GBV cluster are more sustainable and which are less sustainable? Why? 15b. What would be needed to enhance sustainability for those that are less sustainable? Replicability, Transferability and Adaptability 16a. Are there any interventions or approaches in the CARE-GBV cluster that you think could be replicated in or adapted to other contexts? Are there any that you think could not be replicated or adapted? Scalability 17a. Which interventions in the CARE-GBV cluster do you see as having potential for scale up? Are there any that you think would be very difficult to scale up? Why? IMPLEMENTATION EVALUATION QUESTIONS Design 18a. How was the activity designed? What factors influenced the design? Who led the design? Implementation 19a. What intervention methods were used for the activity? Flexibility 20a. Is there sufficient staffing to respond to local priorities? Is there flexibility to change approaches to respond to lessons and changing challenges in the local environment? Monitoring of Results 21a. Is the activity collecting evidence on what is working, not working and what could be done differently to achieve results? Sustainability 22a. Are there any plans to maintain the sustainability of this activity? 22b. Has any evidence been gathered to support the potential sustainability of this activity? 97 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV KII GUIDE – USAID Respondent Name, DRG Area(s) of Expertise, Institution Date: Start Time: Category Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of USAID’s CARE-GBV cluster. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on GBV prevention and response. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are completely neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming focused on GBV prevention and response. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Ritu Nayyar-Stone, the project director for this study at nayyarstone￾ritu@norc.org. [PUT RITU EMAIL IN CHAT] With your permission, I’d like to record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. Before we jump in, I’d like to get to know you bit. Could you please give a briefly introduction and your area of focus within CARE-GBV? PORTFOLIO QUESTIONS Section Introduction Today we are going to discuss the CARE-GBV portfolio. Expansion 1a. How is GenDev’s GBV portfolio helping to expand and improve GBV programming? Protection 2a. What do you see as the most influential activities or strategies to improve access to effective services for survivors? Which grantee approaches were effective for this? 2b. Which grantee approaches were effective for this? 98 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question Coordination 3a. Could you please tell me about the management structure for the CARE-GBV cluster? 3b. How does information flow from grantees up to GenDev? 3c. How is this information used for decision making? Integration 4a. Have you facilitated or seen coordination between this cluster and any other GBV efforts from USAID? How does this cluster fit into the full range of USAID's GBV programs? Data 5a. Could you please tell me about how GenDev uses data from the CARE-GBV cluster? 5b. Are learnings from the CARE-GBV cluster ever used to inform higher level decision making across GenDev programs? Foundation 6a. Were grantees briefed on USAID strategies and priorities around GBV? If yes, what was the procedure and intended learning outcomes? Stakeholders 7a. Was there a strategy for engaging a broad range of stakeholders for the CARE-GBV cluster? 7b. Could you please give some examples of activities that were able to effectively engage relevant stakeholders versus those who were less able to do so? Intervention Planning and Design 8a. What are key pieces of evidence to inform this type of programming? Which ones were most influential in your planning? 8b. Were there important knowledge and practice gaps when planning and designing GBV interventions? Did you draw on any intervention evaluations from similar programs? Vicarious Trauma 9a. What are key areas of evidence or information that informed this grant on vicarious trauma? 9b. What were important knowledge and practice gaps about programming to address vicarious trauma? Reach and Effectiveness 10a. Does the CARE-GBV cluster produce knowledge that can influence the reach and effectiveness of other GBV interventions, even those not funded by USAID? How so? ACTIVITY CLUSTER QUESTIONS Outcomes 11a. Upon reflection, how realistic do you think that the outcomes stated in project designs for this activity cluster were? 11b. Do you think that the length of the grants were sufficient to achieve these outcomes? Why or why not? 11c. Which activities were most successful in achieving the outcomes in their project design, and which were less successful? Why? Intervention Implementation 12a. How well do you think the activities in this cluster were able to reach the intended beneficiaries? 12b. What were the most effective ways participants were reached? 12c. What were the main challenges in reaching the right people? Mechanisms 13a. What do you think are the most effective interventions or approaches used by the grantees? Why were these effective? 13b. Were there any approaches that were particularly ineffective or unsuccessful? Why? Sustainability 14a. Could you please tell me which activities or approaches in the CARE-GBV cluster are more sustainable and which are less sustainable? Why?14b. What would be needed to enhance sustainability for those that are less sustainable? 99 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Category Topic Question Replicability, Transferability and Adaptability 15a. Are there any interventions or approaches in the CARE-GBV cluster that you think could be replicated in or adapted to other contexts? Are there any that you think could not be replicated or adapted? Scalability 16a. Which interventions in the CARE-GBV cluster do you see as having potential for scale up? Are there any that you think would be very difficult to scale up? Why? 100 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ANNEX D, SOURCES OF INFORMATION [Will be filled in Revision 1 of the report] 101 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV DOCUMENTS REVIEWED NO. DOCUMENT TYPE TITLE YEAR PUBLISHED AUTHORING ENTITY/ INDIVIDUAL 1 CARE-GBV Program Document CARE-GBV Small Grants Program MEL Plan for Grantees—Final 2021 MCI 2 CARE-GBV Program Document Crisis Center Hope & Pleiades Organization Social media Policy and Guidelines 2021 CCH 3 CARE-GBV Program Document Progress Report—September 2021 2021 CCH 4 CARE-GBV Program Document Progress Report—October 2021 2021 CCH 5 CARE-GBV Program Document Progress Report—November 2021 2021 CCH 6 CARE-GBV Program Document Progress Report—December 2021 2021 CCH 7 CARE-GBV Program Document Five Day Training Workshop Report 2021 SOAR 8 CARE-GBV Program Document FGD Wellness Report—December 2021 2021 SOAR 9 CARE-GBV Program Document Three Day Learning Visit Final Report 2021 SOAR 10 CARE-GBV Program Document Project Implementation Updates Report 2021 SOAR 11 CARE-GBV Program Document Theory of Change and Workplan 2021 SOAR 12 CARE-GBV Program Document SVRI We Care Blog Post: “From Me to We: Reclaiming the Need for Collective Care” 2021 SVRI 13 CARE-GBV Program Document Theory of Change Narrative 2021 SVRI 14 CARE-GBV Program Document We Care Evidence Review 2021 SVRI 15 CARE-GBV Program Document Staff Wellness Program Curriculum, Handbook, and Appendices 2021 ZSU 16 CARE-GBV Program Document Progress Report—January 2022 2022 CCH 17 CARE-GBV Program Document Report from National Conference 2022 CCH 18 CARE-GBV Program Document February CARE-GBV Check-in Meeting Progress Report 2022 SOAR 19 CARE-GBV Program Document February 2022 CARE-GBV Progress Report 2022 ZSU 20 CARE-GBV Program Document Progress Report—April 2022 2022 CCH 21 CARE-GBV Program Document Progress Report—May 2022 2022 CCH 22 CARE-GBV Program Document Progress Report—June 2022 2022 CCH 23 CARE-GBV Program Document Short Summary Report of 2-Day Workshops (Objectives and Modules) 2022 CCH 24 CARE-GBV Program Document Participant Evaluation Forms from Fourth Wellness Meeting 2022 SOAR 25 CARE-GBV Program Document FCT SGBV Response Team Strategic Plan 2022-2026 2022 SOAR 26 CARE-GBV Program Document Stress Management and Wellness Policy 2022 SOAR 27 CARE-GBV Program Document SOAR Staff Code of Conduct 2022 SOAR 28 CARE-GBV Program Document SOAR Safeguarding Policy 2022 SOAR 29 CARE-GBV Program Document 3-Day Strategic Workshop of the FCT SGBVRT 2022 SOAR 30 CARE-GBV Program Document Training Manual on Psychosocial Support and Promoting Resilience of Child Survivors 2022 SOAR 31 CARE-GBV Program Document Wellness and Self-care Training Manual for Sexual and Gender-Based Violence Responders 2022 SOAR 32 CARE-GBV Program Document SVRI Final Report 2022 SVRI 33 CARE-GBV Program Document CARE-GBV Mission Presentation and Progress Report 2022 WAR 34 CARE-GBV Program Document Wellness App Evaluation Report 2022 WAR 35 CARE-GBV Program Document Pre- and Post-Evaluation Data: Vicarious Trauma Training Program Summary Report 2022 WAR 102 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV NO. DOCUMENT TYPE TITLE YEAR PUBLISHED AUTHORING ENTITY/ INDIVIDUAL 36 CARE-GBV Program Document CARE-GBV Progress Report—April 2022 2022 ZSU 37 CARE-GBV Program Document CARE-GBV Progress Report—July 2022 2022 ZSU 38 CARE-GBV Program Document ZSU Essential Principles of Staff Care 2022 ZSU 39 CARE-GBV Program Document SOAR CARE-GBV Final Project Report 2022 SOAR 40 CARE-GBV Program Document SVRI Final Report—July 2022 2022 SVRI 41 CARE-GBV Program Document Commitment to Collective Care at SVRI— Policy Document 2022 SVRI 42 CARE-GBV Program Document Raising Voices Case Study 2022 SVRI 43 CARE-GBV Program Document Research with HaRT Case Study 2022 SVRI 44 CARE-GBV Program Document CARE-GBV Final Report 2022 WAR 45 CARE-GBV Program Document Summary Evaluation of the Wellness Check Tool 2022 WAR 46 CARE-GBV Program Document Final Baseline—Post Training Evaluation Summary 2022 WAR 47 CARE-GBV Program Document Thuso Ya Bathusi Virtual Training Manual 2022 WAR 48 CARE-GBV Program Document Final CARE-GBV Progress Report 2022 ZSU KEY INFORMANTS ORGANIZATION NAME TITLE CCH Maja Acevska President CCH Hristina Doneva Licensed Psychologist CCH Mitana Stevkovska Head Lawyer SOAR Annette Pwajok Operations and Project Manager SOAR Chinyere Eyoh Executive Director and Founder SOAR Ndidiamaka Ani Licensed Therapist SVRI Anik Gevers Technical Specialist WAR Helen Apps Research Director WAR Peggie Ramaphane Executive Director ZSU Aida Behrem Executive Director ZSU Alma Omeragic Professional Translator ZSU Cynthia Uccello Counselor and Clinical Supervisor at Common Threads Project ZSU Catherine Butterly Senior Advisor for Trauma Theory and Training at Common Threads Project MCI Chelsea Pallatino Senior Research and M&E Advisory for CARE-GBV MCI Jennifer Davis Senior GBV Advisor for CARE-GBV USAID Chai Shenoy Former COR of CARE-GBV 103 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV ANNEX E, DISCLOSURE CONFLICTS OF INTEREST 104 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Name Vaiddehi Bansal Title Research Director II Organization NORC at the University of Chicago Evaluation Position? ☒ Team Leader☐Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / 7200AA18M00016, Tasking N054 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Collective Action to Reduce Gender-Based Violence I have real or potential conflicts of interest to disclose. ☐ Yes☒No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4.Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5.Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6.Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date April 25, 2023 105 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Signature Date April 25, 2023 Name Brooke Jardine Title Senior Research Associate I Organization NORC at the University of Chicago Evaluation Position? ☐ Team Leader☒Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M /7200AA18M00016, Tasking N054 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Collective Action to Reduce Gender-Based Violence I have real or potential conflicts of interest to disclose. ☐ Yes☒No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4.Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5.Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6.Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. 106 | GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: CARE GBV EVALUATION USAID.GOV Name Samantha Austin Title Research Associate II Organization NORC at the University of Chicago Evaluation Position? ☐ Team Leader ☒ Team member Evaluation Award Number (contract or other GS-10F-0033M /7200AA18M00016, Tasking instrument) N054 USAID Project(s) Evaluated (Include project Collective Action to Reduce Gender-Based name(s), implementer name(s) and award number(s), Violence if applicable) I have real or potential conflicts of interest ☐ Yes ☒ No to disclose. If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 7.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 8.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 9.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 10. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 11. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 12. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date April 25, 2023 U.S. AGENCY FOR INTERNATIONAL DEVELOPMENT 1300 Pennsylvania Avenue, NW Washington, DC 20523