GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: THE BETTER TOGETHER CHALLENGE Final Report (July 11, 2023) Prepared under Contract No.: GS-10F-0033M / 7200AA18M00016, Tasking N054 CREDIT: GUYANESE, ISHAKA JACKMAN, INFORMATION COMMUNICATIONS TECHNOLOGY CONSULTANT. DRG LEARNING, EVALUATION, AND RESEARCH ACTIVITY II GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: BETTER TOGETHER CHALLENGE EVALUATION Final Report (JULY 11, 2023) Prepared under Contract No.: GS-10F-0033M /7200AA18M00016, Tasking N054 Submitted to: Matthew Baker, USAID COR Submitted by: Carlos Echeverria-Estrada, BTC Evaluation Lead Camille Smith, Qualitative Specialist Paige Pepitone, Quantitative Specialist Mithila Iyer, Quantitative Specialist Christel Bamfield, Local Technical Consultant-Guyana Andrea Bolaños, Local Technical Consultant-Panama Kerry Burris, Local Technical Consultant-Trinidad & Tobago Contractor: NORC at the University of Chicago 4350 East West Highway, 8th Floor Bethesda, MD 20814 Attention: Matthew Parry Tel: 301-634-5444; E-mail: Parry-Matthew@norc.org DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i ACKNOWLEDGEMENTS The evaluation team would like to thank the following colleagues at NORC: Ritu Nayyar-Stone (Team Lead for the overall Gender-based Violence Portfolio Performance Evaluation), Cathy Zimmerman (Gender/Gender-based Violence Expert), and both Ridhi Sahai and Ingrid Rojas Arellano (Evaluation Specialist) for their valuable direction and contributions. The evaluation team also recognizes the interpretation support of Arifa Satnarine during interviews with GBV survivors and group discussions with activity staff in Trinidad and Tobago. We also thank Laura Ortiz Salazar (Research Analyst) for her contributions to the data quality review and the qualitative analysis in this report. ii TABLE OF CONTENTS ACKNOWLEDGEMENTS I ABSTRACT IV ACRONYMS VI EXECUTIVE SUMMARY VII EVALUATION DESIGN VII MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS VIII 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS 1 2. ACTIVITY CLUSTER BACKGROUND 3 BRIDGING THE GAP FOR VENEZUELAN MIGRANTS (NCC GUYANA) 5 SHIFTING POWER DYNAMICS: ENGAGING MEN IN GBV REDUCTION (HIAS PANAMA)6 WOMEN EXERCISING LEADERSHIP FOR COHESION & MEANINGFUL EMPOWERMENT (DI T&T) 6 3. METHODS AND LIMITATIONS 7 DESK REVIEW 8 QUALITATIVE DATA COLLECTION: INTERVIEWS 8 QUALITATIVE DATA COLLECTION: FOCUS GROUP DISCUSSIONS 9 QUANTITATIVE DATA COLLECTION: SURVEY 10 LIMITATIONS 2 4. FINDINGS, CONCLUSIONS & RECOMMENDATIONS 3 4.1 FINDINGS FOR THE ACTIVITY CLUSTER QUESTIONS 3 EVALUATION QUESTION 1: ARE THE ACS BASED ON CONTEXT-SPECIFIC AND INTERNATIONAL EVIDENCE? 4 EVALUATION QUESTION 2: TO WHAT EXTENT ARE EACH OF THE ACTIVITY CLUSTERS ACHIEVING THE TARGETED GBV RESULTS? 15 EVALUATION QUESTION 3: TO WHAT EXTENT ARE THE ACS SUSTAINABLE? 24 4.2 CONCLUSIONS FOR THE BTC ACTIVITY CLUSTER 33 4.3 RECOMMENDATIONS FOR THE BTC ACTIVITY CLUSTER 36 5. IMPLEMENTATION EVALUATION FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 37 5.1 FINDINGS FOR THE IE OF THE WELCOME ACTIVITY 37 EQ1: IS THE ACTIVITY DESIGN BASED ON THE LOCAL CONTEXT AND FLEXIBLE TO ACHIEVE RESULTS ON THE GROUND? 37 EQ 2: IS THE ACTIVITY REACHING PARTICIPANTS THEY ARE MEANT TO TARGET?41 EQ 3: IS THE ACTIVITY ACHIEVING SUSTAINABILITY? 46 5.2 CONCLUSIONS FOR THE IE 47 iii 5.3 RECOMMENDATIONS FOR THE IE 48 EQ1: IS THE ACTIVITY DESIGN BASED ON THE LOCAL CONTEXT AND FLEXIBLE TO ACHIEVE RESULTS ON THE GROUND? 48 EQ2: IS THE ACTIVITY REACHING BENEFICIARIES THEY ARE MEANT TO TARGET?49 EQ3: IS THE ACTIVITY ACHIEVING SUSTAINABILITY? 50 ANNEX A. EVALUATION STATEMENT OF WORK 52 ANNEX B. EVALUATION METHODS AND LIMITATIONS 58 ANNEX C. DATA COLLECTION INSTRUMENTS 64 ANNEX D. SOURCES OF INFORMATION 128 DOCUMENTS REVIEWED 129 SITES VISITED 130 KEY INFORMANTS 130 ANNEX E. ADDITIONAL TABLES 132 ANNEX F. DISCLOSURE CONFLICTS OF INTERES 136 iv ABSTRACT NORC at the University of Chicago (NORC) conducted the performance evaluation of three activities of the Better Together Challenge (Reto Juntos es Mejor, in Spanish), funded by USAID and the Inter-American Development Bank to find innovative ideas and solutions to empower Venezuelans affected by the regional migration crisis. The activities include (1) Building the Gap for Venezuelan Migrants (BTG4VM), in Guyana, (2) Shifting Power Dynamics: Engaging Men in Gender￾Based Violence Reduction (SPD), in Panama, and (3) Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME), in Trinidad and Tobago. The evaluation addressed the following three main questions: 1) Was BTC funding based on context-specific and international evidence?; 2) To what extent is BTC achieving the targeted GBV results?; and 3) To what extent is BTC sustainable? Additionally, the report includes the results of an implementation evaluation of the WELCOME activity in Trinidad and Tobago. NORC applied a mixed-methods approach to answer the research questions, using a combination of desk review, key informant interviews (KIIs), site observations, and a web-based survey. The evaluation found that the BTC cluster was successful in targeting the groups of Venezuelan migrants and in adapting its strategies to challenging contexts in which migrants’ intersectionality of migration status, poverty, and small range of connections increase their precarity and ability to engage in GBV protection and prevention programs. All three grantees conducted some form of needs assessment to ground their intervention in empirical evidence about migrant needs and relevant contextual factors. Two of them also incorporated host community members in their activities. While all approaches accomplished their immediate goals, the design of their monitoring strategies and the short-term design of the Challenge limited the activity cluster’s ability to learn about its impact in the medium term, and to establish conditions for sustainability. v LIST OF TABLES Table 1. Evaluation Findings and Conclusions................................................................................................... viii Table 2. Evaluation Recommendations.................................................................................................................xi Table 3. Evaluation Questions.................................................................................................................................2 Table 4. Summary of Evaluated Activities.............................................................................................................5 Table 5. Summary of KII and FGD Respondents................................................................................................9 Table 6. Web Survey Results...................................................................................................................................2 Table 7. Summary of Activity Cluster Findings....................................................................................................3 Table 8. What are the top three resources, services, or programs that the migrant community in your city or neighborhood need most to prevent or combat GBV?.............................................................5 Table 9. Main Implementation Challenges (Service Providers’ Perspective)..............................................21 Table 10. Top three migrant community needs to prevent or combat GBV (Count)............................39 Table 11. Services available for Venezuelan migrants who are GBV survivors.........................................40 Table 12. Status of Service Provision (WELCOME Referral Network), Count........................................47 Table 13: Activities under each Activity Cluster...............................................................................................54 Table 14: Evaluation Questions (SOW Final Version).....................................................................................55 Table 15. Qualitative Fieldwork ............................................................................................................................60 Table 16. Web Survey Sample - WELCOME.....................................................................................................61 Table 17. Web Survey Sample - BTG4VM .........................................................................................................61 Table 18. Demographic Information of Web Survey Respondents..............................................................62 Table 19. Key Informants..................................................................................................................................... 131 Table 20. Affiliated Organizations...................................................................................................................... 131 Table 21. Service Providers’ Immigrant Target Groups, Needs, and Approaches and Resources Used to Satisfy Needs..................................................................................................................................................... 133 LIST OF EXHIBITS Exhibit 1. Organization of the BTC Activities Evaluated...................................................................................4 Exhibit 2. Data Sources and Analysis.....................................................................................................................8 Exhibit 3. Web Survey demographics (BTG4VM) ............................................................................................10 Exhibit 4. Web Survey demographics (WELCOME)........................................................................................10 Exhibit 5. Rate the effectiveness of BTG4VM program components in preventing GBV ......................17 Exhibit 6. Rate the effectiveness of WELCOME network services providers in preventing GBV ......18 Exhibit 7. Which activity components do you think could be implemented in other communities in the country?...............................................................................................................................................................29 Exhibit 8. WELCOME’s Advocates Model .........................................................................................................38 Exhibit 9. Tailor-made Approaches and Allocated Resources by Group of Service Users (Service Providers)...................................................................................................................................................................44 vi ACRONYMS AC Activity cluster BTC Better Together Challenge BTG4VM Bridging the Gap for Venezuelan Migrants (NCC Guyana) CSO Civil Society Organization DI Democracy International FGD Focus Group Discussion HIAS Hebrew Immigrant Aid Society IDB Inter-American Development Bank IE Implementation Evaluation IP BTC Implementation Partner KII Key Informant Interview KPI Key Performance Indicators MEL Monitoring, Evaluation, and Learning MOH Ministry of Health MHSSS Ministry of Human Services and Social Security NCC National Coordinating Coalition NORC NORC at the University of Chicago SFTP Secure File Transfer Protocol SPD Shifting Power Dynamics: Engaging Men in Gender-Based Violence Reduction (HIAS Panama) T&T Trinidad and Tobago UNHCR United Nations High Commissioner for Refugees USAID United States Agency for International Development WELCOME Women Exercising Leadership for Cohesion & Meaningful Empowerment (DI/La Casita Trinidad and Tobago) YWCA Young Women’s Christian Association vii EXECUTIVE SUMMARY NORC at the University of Chicago was contracted to complete a portfolio performance evaluation of USAID’s gender-based violence (GBV) activity clusters (AC), as part of the Democracy, Human Rights, and Governance Learning, Evaluation, and Research (DRG-LER) II Activity. The evaluation’s purpose is to identify facilitators and barriers to effectiveness, where knowledge still needs to be developed, and what can be improved upon in the GBV portfolio of the United States Agency for International Development’s (USAID’s) Gender Equality and Women’s Empowerment Hub (USAID/GenDev). This evaluation report focuses on three activities funded by the Better Together Challenge (BTC) Activity Cluster or Reto Juntos es Mejor in Spanish. Through this crowdsourcing activity, the Inter-American Development Bank (IDB) and USAID aimed to fund innovative ideas and solutions from local Venezuelans and other service providers, test and scale up their solutions, to create conditions of empowerment among Venezuelans affected by the regional migration crisis. Resonance, a global consulting and implementing firm specialized in social impact and development, implemented the BTC under The Catalyst Project, an effort by USAID’s Innovation, Technology and Research (ITR) Hub to spread and enhance open innovation across the Agency’s global portfolio. as . BTC awarded grants from $150,000 to $499,000 to support and accelerate GBV organizations’ ideas, capacity-building, and learning. The funded activities were implemented between July 2020 and December 2021. One activity received its grant in July 2020 and the other two in early 2021. NORC evaluated three activities under the BTC activity cluster: a) Building the Gap for Venezuelan Migrants (BTG4VM), implemented by the National Coordinating Coalition (NCC) in Guyana; Shifting Power Dynamics: Engaging Men in Gender-Based Violence Reduction (SPD), implemented by the Hebrew Immigrant Aid Society (HIAS) in Panama; and Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME), implemented by Democracy International (DI) and La Casita Hispanic Cultural Centre, in Trinidad and Tobago. NORC answered the following evaluation questions: 1. Are the activity clusters, including BTC, based on context-specific and international evidence? 2. To what extent are each of the activity clusters, including BTC, achieving the targeted GBV results? 3. To what extent are the activity clusters, including BTC, sustainable? For WELCOME specifically, NORC conducted an implementation evaluation to answer these questions: 4. Is the activity design based on the local context and flexible to achieve results on the ground? 5. Is the activity reaching participants they are meant to target? 6. Is the activity achieving sustainability? EVALUATION DESIGN Secondary and primary data, both quantitative and qualitative, were collected from March 2022 to April 2023. This included a desk review of 26 program documents; 16 key informant interviews (KIIs) with USAID, IDB, and grantee and subgrantee senior staff and partners; 34 program interviews with GBV survivors and program users; two focus group discussion with activity staff; and one focus group with female partners of male SPD participants. Additionally, NORC conducted a web-based viii survey of service providers that partnered with BTG4VM and WELCOME, which was completed by 20 respondents via Qualtrics. NORC also conducted a site observation of the WELCOME activity, as part of the implementation evaluation. MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS We present a summary of evaluation main findings and conclusions in Table 1. Table 1. Evaluation Findings and Conclusions FINDINGS CONCLUSIONS ACTIVITY CLUSTER (AC) EQ1. Are the activity clusters based on context-specific and international evidence? ●The Catalyst Project prioritized the funding of needs assessments in intervention areas where activity design had no previous experience, namely in Guyana (BTG4VM) and Trinidad (WELCOME). SPD did not conduct a needs assessment in Panama, but the grantee had strong evidence of local needs based on its long experience working with refugees and migrants in the country, and previous experience in Colombia and Kenya. ●All three grantees used evidence from the local context, as well as from other countries and contexts to inform their activity design and outcomes. ●In Guyana, The Catalyst Project involved Ladysmith, an international research organization, to help the grantee develop skills to conduct a needs assessment, which would inform design. ●Initial needs assessments were crucial to guide BTG4VM and WELCOME targets and content. ●The COVID-19 pandemic-imposed challenges to program design and implementation. However, all grantees were reportedly effective in responding to these challenges by providing remote connection to GBV survivors and offering alternative meeting times to accommodate the male participants' busy schedules. ●WELCOME services were relevant to users and adapted to the local context. The AC’s flexibility to address GBV survivors’ required collaboration with local partners, GBV specialists, and representatives of the migrant community in all three countries. EQ2. To what extent are each of the activity clusters achieving the targeted GBV results? ●All evaluated grants met their target user outputs (30 survivors for WELCOME and 165 men and 153 women for SPD), except for BTG4VM, which served 48 Guyanese and Venezuelan GBV survivors without predetermined targets. ●Overall, participants and GBV survivors expressed satisfaction and praised the services received with some caveats regarding following up on individual cases. Please see page 29 for more detail. ●WELCOME staff adapted the workshop calendar and increased advocates’ and staff members’ availability to accommodate migrants’ busy schedules. This, in several instances, burdened advocates’ schedule. ●Customized approaches and direct services, including one-stop-shop, where survivors of GBV could access various services offered by the Ministry of Health (MoH) and Ministry of Human Services and Social Security (MHSS) and advocate-centered assistance proved to be effective support approaches for GBV survivors, particularly when programs aimed to include all genders. Please see EQ2 Outcomes on page 23 for more detail. ●Financial concerns were a priority and hindered migrants’ participation. Migrants prioritized activities that help them generate income over psychosocial or counseling support. ix FINDINGS CONCLUSIONS ●BTG4VM spearheaded an innovative national pathway referral system in Guyana and organized a NGOs Resource Directory to improve the coordination of GBV services. Coordinating with local partners and finding adequate service providers were its main challenges. SPD adapted its targeting strategy by expanding collaboration with local community-based organizations to increase recruitment. It provided participants with transportation and offered them hybrid access to workshop sessions. ●Through WELCOME, the local subgrantee La Casita became an institutional counterpart to local authorities providing support to GBV survivors; they adapted to the need for providing advocacy services to male migrants. ●Among older male adults, attitudes and behaviors towards traditional gender roles resulted in more rigid and overall long-term attitudes to GBV that promote its perpetuation. ●Post-intervention data collection on outcomes was included neither in funding nor in the implementation timeframe. EQ3. To what extent are the activity clusters sustainable? ●While BTG4VM secured two-year funding from IDB, HIAS did not finalize any plans to continue SPD. When WELCOME ended, DI supported La Casita in its transition to secure new funding as the prime to continue providing services. ●Only BTG4VM expanded coverage, from two to five Guyanese regions. It faced important challenges, such as finding reliable service providers to partner with, and adapting staff roles to the local context. ●All three activities are relevant in their context and could be replicated. Both BTG4VM and WELCOME realized that job skills and training are important adaptations that would make participation more attractive. ●Lack of funding, limited organizational capacity of grantees and local service providers, and scarce political will from local authorities were the most common barriers to sustainability. ●The short-term funding and approved scope hindered the grantees’ ability, especially the small ones, to root their activities as sustainable practices. ●The use of volunteer advocates was intended to be a financially sustainable strategy, however without pay, advocates were limited in their ability to commit to their roles. ●Most grantees and subgrantees built relationships with other service providers and established referral processes. Some of these relationships will persist beyond the grant period. ●USAID and IDB explained that the local capacity in the Caribbean is still relatively low and reliable partners are scarce. These conditions limit funders’ ability to scale up interventions to meet the needs of large and growing numbers of Venezuelan migrants in the region. IMPLEMENTATION EVALUATION OF WELCOME EQ1. Is the activity design based on the local context and flexible to achieve results on the ground ●Evidence from the Trinidadian context was used to formulate WELCOME’s theory of change and results framework. The activity considerably revised both based on the local context due to COVID-19 restrictions. ●DI conducted a robust process of learning and a review of its own assumptions and local networks as a consequence of dealing with the new conditions imposed by the COVID￾19 pandemic. x FINDINGS CONCLUSIONS ●WELCOME pivoted the activity’s theory of change and results framework from being focused on GBV harassment to reaching women experiencing GBV at home (intimate partner violence and domestic GBV) using an advocate-centered model to support GBV survivors in accessing social services. This was done in partnership with La Casita Hispanic Cultural Center. ●WELCOME provided safe spaces for survivors and a variety of support options, including food assistance, psychosocial support, and help accessing employment opportunities. The most common services to which La Casita made referrals were legal assistance and food assistance. ●WELCOME’s different versions of the ToC reflected the local knowledge DI gathered in the baseline study of attitudes toward Venezuelan migrants in T&T and the reports obtained of domestic GBV. ●La Casita, DI’s subgrantee, was receiving between four and 15 requests for support from victims of GBV, most of them Venezuelans, in Arima, T&T. This behavior supported the assumption about relevance of these services. ●The assumptions about advocates’ time and La Casita’s organizational capacity to be sufficient to follow up and complete all cases were not correct in several cases. While WELCOME adjusted its approach during implementation, several GBV survivors reported follow up shortcomings in their cases. EQ2. Is the activity reaching participants they are meant to target? ●WELCOME successfully reached and worked with two main populations: five female Venezuelan advocates; and female Venezuelan GBV survivors that the activity set out to target. While WELCOME trained and provided a small stipend to advocates, most volunteered their time and had other job obligations aside from their advocacy work. ●DI/La Casita created the advocate-centered model and formalized its protocols. WELCOME engaged with a network of local service providers with which advocates helped GBVs survivors navigate the services. ●Over time, advocates tended to reduce their involvement in cases and some did not follow up with service providers and GBV survivors. These cases lacked satisfactory conclusions. ●WELCOME created an ad-hoc monitoring mechanism to respond to the local context instead of using traditional KPIs to learn and adapt. During implementation, WELCOME faced challenges in collecting monitoring data from GBV survivors and opted for collecting case data from advocates instead. This increased the advocates’ burden. ●Shortly after beginning implementation, La Casita started receiving cases of male GBV survivors, including individuals self-identified as LGBTQIA+. ●The selection of advocate candidates seemed correct for the planned tasks. The main criteria included: being Venezuelan women over the age of 35 who have been living in T&T for an extended period of time, holding cultural sensitivity and English and Spanish language skills, having experience working with migrants, and able to navigate support systems in T&T. WELCOME trained them to become advocates. ●WELCOME relied on volunteering time from advocates, which does not offer incentives to address and follow up with all the details involved in the GBV survivor cases. Such dynamics over time can lead to staff turnover and loss of institutional knowledge. ●The activity recruited and trained a male advocate for male GBV survivors, after receiving requests from this group of migrants. ●WELCOME reached a diverse population of survivors and migrants including multiple genders, sexual orientations, ages, and ethnic backgrounds. ●The holistic advocate approach relied on advocates understanding each of their cases and helping GBV survivors navigate the network of service providers. In cases of emotional distress of GBV survivors, the advocates’ short available time limited their ability to follow up and work with psychosocial and legal service providers to find better solutions for the GBV survivors. xi FINDINGS CONCLUSIONS EQ3. Is the activity achieving sustainability? ●When funding from USAID/GenDev ended, La Casita reduced the scale of its advocate services. It relied on small donations and sales of services and Venezuelan dishes before receiving funding to continue the work as prime. ●La Casita became listed as a UNHCR potential grantee, but no funds had been assigned to the organization by May 2023. ●La Casita gained organizational capacity in internal case management protocols and strengthened its institutional relations with the GBV Unit of the T&T Police Service as a counterpart that advocates for GBV survivors. ●La Casita’s reach, sustainability, and potential scale-up are limited by funding challenges. ●WELCOME tested the advocate position as a volunteer role in hopes for financial sustainability, however the role was too time intensive for advocates to work without pay. ●Funding opportunities have been scarce in Trinidad and Tobago and La Casita’s staff does not have sufficient resources for an aggressive fundraising campaign. ●WELCOME’s future is uncertain, and its existence relies heavily on the willingness of La Casita’s Director to continue helping Venezuelan women in her community. The evaluation team formulated a series of recommendations based upon its understanding of the Better Together Challenge (BTC) and the evaluation findings. We include them in Table 2. Table 2. Evaluation Recommendations ACTIVITY CLUSTER EQ1. Are the activity clusters based on context-specific and international evidence? ● Strengthening future GBV programming. Several network-building activities could be integrated into proposed activities, which may both improve services for migrant GBV survivors and perhaps achieve more sustainable activities. In future programs, USAID could encourage strategies that strengthen local alliances between service providers or promote a network of services for GBV survivors, which would include two additional components. First, providing options for medium- to long-term housing to support migrants to become homeowners rather than tenants. Second, offering support to promote income-generating activities, including occupational training, entrepreneurship skills, and livelihoods assistance. ● Expand psychosocial support and segment target group (specific to SPD). An ideal path of psychosocial support for male migrants and their female partners should include referrals to support groups to improve self-esteem, work on trauma healing, provide therapy for couples—and also to prevent and heal sexual child abuse. Future interventions on reflecting on masculinity and gender roles should segment target participants by age and focus on the youngest cohorts. EQ2. To what extent are each of the activity clusters achieving the targeted GBV results? ● Address Migrants’ Transportation Needs. Due to migrants’ financial and logistical difficulties related to commuting, if groups want migrants to be able to join the activities reliably (e.g., attend all workshops and check ins), they will need to consider ways of reaching them without costs or hardships for the migrants. For instance, scheduling activities for survivors living in the same community either onsite or offering them transportation to the site and back. ● Additional Psychosocial Support Sessions. The design of future activities targeting migrant GBV survivors should include additional sessions with a psychosocial support specialist since opening these channels of communication takes time, and each case is different. xii ● Active Use of Monitoring Tools with Service Providers. Service providers have been crucial allies of BTG4VM’s one-stop shop of GBV services and WELCOME’s advocate￾centered model. Engaging the service providers to understand, gain access, and use the monitoring data can be an effective strategy to increase their ownership of cases and their agency to understand each case holistically. EQ3. To what extent are the activity clusters sustainable? ● Capacity for Sustainability. During design and implementation, implementing agencies might want to consider approaches that will foster greater sustainability for the program, such as integrating a network of partners into the project, building, and practicing fund-raising skills, or securing project coordination with larger agencies. IMPLEMENTATION EVALUATION EQ1. Is the activity based on local context and flexible to achieve results on the ground? ● Continue to encourage and fund initial needs assessment stages so that groups have time and resources to redesign proposed activities to meet current context￾specific needs. Allow sufficient time and fund specialist intervention research expertise so groups can consult with proposed project beneficiaries, local stakeholders and examine relevant international practices. ● Carefully reassess the role of the advocate and the scope of work required to fulfill the expectations of this position for a fair remuneration. Program funders, implementers and grantees should agree on either a scope of work that a volunteer could achieve or create part- or full-time basis positions to avoid burnout and shortfalls in advocacy follow up and case closure. ● For advocate-model interventions to be more successful, funders should also support work to engage with local service providers, train them in MEL practices, and spearhead fundraising efforts with common incentives to collaborate. Funders and implementation partners should promote and label funds and time to identify relevant local actors as well as their strengths, limitations, needs, interest, and leverage points with local authorities and other decision makers. This strategy may foster smoother transitions from the advocate to the actual services and follow up on their needs in the medium and long run. ● Promote participatory and co-produced intervention development. Based on needs assessments and needs to adapt implementation, donors should provide adequate funding and technical support for groups to engage and pay beneficiary representatives to undertake intervention co-development processes. Tools such as user- or human-centered design of activity components can help generate well-targeted and more effective interventions that are informed by actual users. The experience with WELCOME showed that GBV survivor involvement is necessary to improve the advocate model, so that these advocates can gain a more holistic approach to their users’ needs. These include a non-linear approach to psychosocial support and its implications on legal, job training, and other relevant components. EQ2. Is the activity reaching participants they are meant to target? ● Promote migrant and non-migrant inclusive engagement. Based on findings from host community engagement (Section 5.1), make further investment in joint migrant/non￾migrant programming and corresponding research to understand the potential added value of breaking down barriers between groups and find common spaces of growth and incorporation into the host community. xiii ● Promote gender-inclusive programming. Design future programming to ensure providers are prepared to respond to the needs of female, male, and gender non-binary GBV survivors. Initial research and ongoing monitoring should be designed to identify accessibility, acceptability, and emerging effectiveness of service provision by gender and other influential factors, such as language, ethnicity, migration status, cultural background, education level, and socioeconomic status. WELCOME found that it was necessary to train a male advocate as detailed under flexibility (Section 5.1) ● Improve strategies to ensure wide awareness of and easy access to services. If services are capable of managing a substantial caseload, future activities should allocate funds for grantees to investigate the various ways survivors might learn about their services (e.g., beyond word of mouth), and use such channels to increase awareness, without compromising the safety and security of GBV survivors and others seeking services. Consider different modes of assistance, including the possible range of remote sessions and mobile technology to reach migrants. EQ3. Is the activity achieving sustainability? ● Future funding schemes for GBV survivor programming need to integrate support for fundraising. Small organizations like La Casita lack the experience, time, and staff to work on fundraising. Fundraising and grant development skills-building can be included in the grant requirements. Sales and marketing activities are short-term tactics that have been useful to secure small-scale funding. ● Strengthen cross-organization collaboration. Improving an organization’s network of local and international partners can foster greater sustainability by creating potentially mutually supportive relationships and introducing the possibility of a shared workload and joint funding. ● Avoid relying on volunteer time or short-term funded jobs. Few good programs, if any, can rely primarily on community volunteers. Similarly, short-term jobs that last only for the life of the funding cycle often prove wasteful for the program, unfair for survivor-clients, and disappointing for the persons who are trained to do the work. At the very least, funders, implementers, and the activity partners, at design and implementation stages, should establish measures to help trained individuals be hired by other relevant organizations if the activities will not be sustained. 1 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS NORC at the University of Chicago (NORC) is carrying out a portfolio performance evaluation (PPE) of the Democracy, Human Rights, and Governance Learning, Evaluation, and Research (DRG-LER) II Activity for the United States Agency for International Development (USAID) Gender Equality and Women’s Empowerment Hub (GenDev) in the Bureau for Development, Democracy, and Innovation (DDI). The purpose of the evaluation is to understand what is working, identify facilitators and barriers to activity effectiveness where knowledge still needs to be developed, and what can be improved upon in GenDev’s GBV portfolio. The activity was designed so that USAID and its partners were involved in co-creation of the evaluation scope of work and the evaluability assessment. NORC was contracted to evaluate four activity clusters (ACs), which include: 1. Better Together Challenge (BTC) with GBV prevention and response interventions; 2. Collective Action to Reduce Gender-Based Violence (CARE-GBV) small grants activities; 3. The Resilient, Inclusive & Sustainable Environments (RISE): A Challenge to Address Gender￾Based Violence in the Environment; and 4. The Women’s Economic Empowerment (WEE) activities directly funded by GenDev integrating GBV prevention and response activities. This report focuses on the first activity cluster, the Better Together Challenge (BTC) implemented by Resonance between September 2019 and September 2022. Resonance1 was responsible for coordinating the technical selection of Better Together Challenge grantees, allocating funds, providing implementation and capacity-building support to grantees, serving as a liaison between funders (USAID GenDev and the Inter-American Development Bank), and complying with reporting and learning outcomes. The funding from GenDev served as a buy-in to the larger Better Together Challenge, which sat within the Innovation, Technology and Research Hub at USAID as a part of Resonance´s The Catalyst project, which was launched by the Hub. The BTC evaluation is based on data from activities in three countries: Guyana, Panama, and Trinidad and Tobago. The activities were Bridging the Gap for Venezuelan Migrants (BTG4VM), implemented by the National Coordinating Coalition (NCC); Shifting Power Dynamics: Engaging Men in Gender-Based Violence Reduction (SPD), implemented by the Hebrew Immigrant Aid Society (HIAS); and Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME), implemented by Democracy International (DI) in partnership with La Casita. After discussions with GenDev, NORC dropped two other activities from the evaluation on completion of the Evaluability Assessment.2 The team conducted field data collection from February to April 2023. NORC engaged with multiple respondents, including GenDev staff, the Resonance team, grantees, service providers, and program users, including Venezuelan migrants and nationals of the host countries. Our sampling approach emphasized the inclusion of Venezuelan migrants who are also GBV survivors. The evaluation team included NORC staff Carlos Echeverria-Estrada, Ph.D., Camille Smith, Paige Pepitone, Mithila Iyer, and 1 Resonance is a Vermont-based global consulting and implementing firm that helps companies, governments, and NGOs solve impact, development, and business problems. https://www.resonanceglobal.com/. All mentions of Resonance in this report refer to staff supporting the Catalyst Project, launched by USAID´s ITR Hub. 2 The GBV PPE evaluability assessment report can be found here. 2 Laura Ortiz Salazar, who are all based in the United States; and consultants Andrea Bolaños (Panama), Kerry Burris (Trinidad & Tobago), and Christel Bamfield (Guyana). This report is part of a series produced by NORC that includes the evaluation results of the CARE￾GBV, RISE, and WEE clusters, as well as the portfolio evaluation report, which has its own evaluation questions and compares findings across all the ACs. The key audiences for this report are the governments of Guyana, Panama, and Trinidad & Tobago, USAID and GenDev, other civil society organizations (CSOs), implementation partners (IPs), funders, and local and international experts on GBV and migration. NORC addressed the following evaluation questions in Table 3, which were co-created with GenDev to guide the final performance evaluation of the BTC activity cluster on two levels: the activity cluster level and Implementation Evaluation of the WELCOME activity. Table 3. Evaluation Questions Evaluation Question Topics and Sub-questions Activity Cluster Questions 1. Are the activity clusters based on context-specific and international evidence? • Needs assessment and intervention evidence: How well were needs assessments conducted and intervention evidence collected to inform the cluster activities? • Assumptions: What assumptions were made to design and implement the activity clusters? How accurate were any assumptions? • Causal pathways: What causal pathways or theories of change were articulated for the activity clusters? • Monitoring and adaptations: How well are interventions monitored and are emerging findings contributing to intervention adaptations or improvements? • Role of host community: Is there a role for members of the host communities in the activity?* • Approach to migrants: Has your organization changed its approach to migrants after your experience with the activity? Please explain.* 2. To what extent are each of the activity clusters achieving the targeted GBV results? • Outcomes: Are the stated outcomes realistic and achievable within the timeframe of the AC? What progress is being made toward achieving the outcomes? • Planning and activity designs: How and how well were activity plans and designs developed to achieve different GBV outcomes? • Intervention implementation: How well are interventions implemented to reach their target groups and influence change? • Mechanisms: What are the most effective aspects of the intervention? How do these “active ingredients” operate in each AC? 3. To what extent are the ACs sustainable? • Sustainability: What aspects of the ACs contributed to their sustainability? What components are needed for greater sustainability? • Replicability, transferability, and adaptability: In what ways are the ACs replicable in the same contexts? Adaptable for other contexts? • Scalability: What aspects of the ACs are most amenable to be scaled up? • Lessons on host community engagement: Are there any lessons learned on how to engage host community members?* 3 Evaluation Question Topics and Sub-questions Implementation Evaluation Questions 4. Is the activity design based on the local context and flexible to achieve results on the ground? • Design: What factors contributed to the design of the activity? How were priority GBV problems identified? • Re-design factors: What were the main reasons for re-designing the activity? • Implementation: What are the key intervention methods to achieve objectives? • Flexibility: Is there sufficient staffing to respond to local priorities? Is there flexibility to change approaches to respond to lessons and changing challenges in the local environment? 5. Is the activity reaching participants they are meant to target? • Target participants: What are the barriers to reaching participants? • Target population needs: How did your organization identify the needs of the target population after redefining it?* • Monitoring of results: Is the activity collecting evidence on what is working, not working and what could be done differently to achieve results? 6. Is the activity achieving sustainability? • Sustainability: What plans are in place for sustainability? What is the evidence of potential sustainability? • Host community support: Have any members of the host communities supported or expressed enthusiasm for this work?* • Local group ownership: Are there any migrant groups or any other social agreements in the implementation area that could promote these efforts in the future?* *Note: In addition to the AC questions asked to all four clusters, BTC-specific questions are shown in italics. 2. ACTIVITY CLUSTER BACKGROUND Venezuela continues facing a political, humanitarian, and economic crisis that has led to the migration of over 7 million Venezuelans, making it the largest external displacement in the history of the Western Hemisphere.3 In 2019, USAID, in partnership with the Inter-American Development Bank (IDB), launched the Better Together Challenge (BTC), or Reto Juntos es Mejor in Spanish. BTC is part of the inter-institutional response to the Venezuelan migrant crisis that has weakened service delivery, infrastructure, labor markets, and community relations in neighboring counties, including Guyana, Panama, and Trinidad and Tobago. USAID and IDB developed BTC to crowdsource, fund, and scale forward-thinking solutions worldwide to help the lives of Venezuelan migrants and communities they reside in. Thirty-three organizations received 35 awards to address the Venezuelan crisis through the BTC, which provided a platform for partnership building and collaboration to support shared goals of improving the lives of Venezuelans and host communities across Latin America and the Caribbean. The BTC offered an opportunity for organizations to use flexible programming to meaningfully improve economic, social, physical, and mental well-being in these communities. The main objectives of the BTC grants were to: ● Elevate Venezuelan voices and ingenuity to convey and answer their needs across the region; 3 Inter-Agency Coordination Platform for Refugees and Migrants from Venezuela (2023). Available at https://www.r4v.info/en/home. 4 ● Connect Venezuelans, host communities, and the world’s collective genius to develop innovative solutions; ● Expand networks across communities and countries to promote relationships and collaboration; ● Fund, test, and scale solutions; and ● Build a marketplace of tested, market-ready solutions. The theory of change for BTC is that if USAID funds, tests, and scales innovative ideas and solutions by Venezuelans and regional actors and expands networks across communities and countries, then Venezuelan voices and ingenuity will be heard; relationships and collaboration will be promoted; and a marketplace of tested, market-ready solutions will be built. Exhibit 1. Organization of the BTC Activities Evaluated Two unevaluated solutions were implemented in Panama (Conectadas 4.0 and Proyecto Emprendedores) and two in Trinidad and Tobago (RESET and Ayúdate GBV Bilingual Hotline). Additional countries where BTC solutions were implemented but not included in this evaluation were Argentina (2), Brazil (5), Chile (3), Colombia (4), Ecuador (3), Peru (5), and Venezuela (10). These grants, or solutions, were implemented by innovators funded through the broader BTC mechanism. Table 4 below summarizes the purpose, components, target populations, sites, and implementation dates of the three activities, by grantees, included in this evaluation: 1. Bridging the Gap for Venezuelan Migrants (BTG4VM), implemented by the NCC in Guyana; 2. Engaging Men in Gender-Based Violence Reduction (SPD), implemented by HIAS, in Panama; and 3. Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME), implemented by DI and La Casita Hispanic Cultural Center, in Trinidad and Tobago. 5 Table 4. Summary of Evaluated Activities ACTIVITY COMPONENTS SITES TARGET POPULATIONS ACTIVITY LIFECYCLE BTG4VM, one￾stop shop focused on GVB to access migrant services ● Mapping of GBV service providers ● One-stop shop for GBV services ● Awareness campaign ● Data collection, communication, and use Essequibo Islands￾West Demerara (region 3), and Georgetown, Demerara￾Mahaica, (region 4), Guyana Venezuelan and Guyanese women February – December 2021 SPD, series of workshops to address issues of toxic masculinity and gender inequalities among migrants ● Engaging men in GBV reduction workshops ● Gender dialogues with men and women ● Gender inclusion training for national police ● National dialogue table on continuing to involve men in preventing GBV Panama City, San Miguelito, and Panama West (La Chorrera), Panama Refugee, vulnerable migrant, and local men and women January – September 2021 WELCOME, advocate￾centered services to survivors of GBV ● Recruiting and training advocates ● Establishing a trusted referral network ● Matching advocates with survivors ● Advocates supporting survivors ● Social media campaign for GBV awareness ● Supporting scalability and sustainability Borough of Arima, Trinidad and Tobago Venezuelan GBV survivors July 2020 – August 2021 BRIDGING THE GAP FOR VENEZUELAN MIGRANTS (NCC GUYANA) Under the buy-in by GenDev to the BTC, the National Coordinating Coalition (NCC) partnered with the feminist research organization, Ladysmith, to conduct a Rapid Gender-Based Violence Assessment on the situations of Venezuelan and Guyanese women in Guyana in March 2021. Findings indicated that Venezuelan and returning Guyanese women and girls are at a high risk of GBV, including intimate partner violence, violence in the context of coerced sexual labor, abuse by state authorities, trafficking, and targeted harassment associated with negative stereotypes about the refugee and migrant population. Additionally, the assessment found poor awareness of GBV services and gaps in service provision in Guyana, which further exacerbates risks of GBV. Drawing on these findings, NCC designed Bridging the Gap for Venezuelan Migrants (BTG4VM) to address the needs of Venezuelan migrants affected by GBV and to address GBV service gaps in Guyana. The project aimed to leverage NCCs’ position as an umbrella network for non-governmental organizations (NGOs) and its trusted status amongst migrants and service providers in Guyana to consolidate and strengthen the referral pathways for GBV prevention and response. Further, the project was designed to ensure that Venezuelan migrant women, returnee women, and others at risk of GBV can access the services and resources they need. NCC sought to use its large sphere of contacts and 6 influence to close service gaps related to GBV prevention and response. NCC collaborated with government agencies and civil society organizations to strengthen GBV referral pathways and increase service access by liaising with new and existing providers. This collaboration also sought to address xenophobia regionally and nationwide, while also collecting data on the experiences of GBV among refugee and migrant women in Guyana. The project was implemented in communities with large Venezuelan migrant populations in Georgetown Regions 3 and 4. BTG4VM had three main activities. First, supporting coordination among GBV service providers by mapping and promoting GBV services and then developing a “one stop shop” for GBV services. Second, they launched a widespread communication campaign to strengthen solidarity between migrant and local communities and provide information about available GBV services. Finally, they collected data on GBV, including patterns of violence, barriers to access, and gaps in services; communicated findings to policymakers and services providers to inform GBV services; and leveraged the findings to inform the communication campaign. SHIFTING POWER DYNAMICS: ENGAGING MEN IN GBV REDUCTION (HIAS PANAMA) This activity aimed to reduce violence risks and mitigate the consequences of sexual violence, intimate partner violence, and child, early, and forced marriages and unions by offering participants safe spaces to reflect on and deconstruct ideas about gender. To achieve this, HIAS Panama used a Positive Masculinities curriculum, an innovative global model designed for refugee, vulnerable migrant, and local men. One of the central tenets was to involve adult and young men in the prevention of violence against women and girls. HIAS adapted the Positive Masculinities curriculum they had implemented in both Latin America and Africa for the Venezuelan migrant context in Panama. During the intervention, migrant men joined in-person sessions about unequal power relations that condone GBV, and they discussed using their role to prevent GBV and promote equality. Additionally, the project worked with women through Gender Dialogues, where they were given the space to discuss with men their ideas and experiences with gender relations and equality. Thus, the HIAS project aimed to help women strengthen peer support networks, better understand their right to live free of violence, and know how to work together to create safer communities. Moreover, HIAS arranged for the Women’s Ministry to work with the National Police of West Panama to improve community awareness of gender inclusion. At the end of the implementation, HIAS held a National Dialogue Table to identify strategies and resources to provide sustainability and continue the work of involving men in the prevention of violence against women and girls. WOMEN EXERCISING LEADERSHIP FOR COHESION & MEANINGFUL EMPOWERMENT (DI T&T) WELCOME initially sought to combat harassment and xenophobia. However, in the wake of the COVID-19 pandemic, the baseline assessment confirmed that GBV was a critical problem in Trinidad and Tobago (T&T), particularly for Venezuelans. Likewise, DI’s partners and networks in T&T began receiving growing reports of GBV (including domestic violence and intimate partner violence). In reviewing information from the baseline report, it was noted that for Venezuelan GBV survivors there are many barriers inhibiting access to support pathways and follow-up on referrals. In coordination with Resonance, the DI team decided to redesign the activity and involve a former contact from another 7 migrant-focused referral network project in T&T: La Casita, a local organization and cultural center focused on training and education as well as informally supporting migrant women. The WELCOME project was adapted to help formalize La Casita’s advocate-center model to provide survivors with access to GBV resources and services. DI created the advocate-center model so that La Casita had trained liaisons who could serve two purposes: 1) to access local GBV resources and service providers with more efficiency, rather than centralizing the contacts on La Casita’s director; and 2) to assist migrant women and ensure services were delivered by conducting regular follow-up on survivor cases. As a part of onboarding La Casita, DI trained both its staff and advocates. The new design aimed to train advocates from within the Venezuelan community in Arima to provide survivors with personalized, survivor-centered support by helping them overcome barriers that prevented them from receiving formal or informal support. The overall objective for WELCOME was to reduce barriers for Venezuelan GBV survivors to access services in T&T, including psychosocial, health, and legal services. This approach was tested with Venezuelan GBV survivors who sought help from La Casita in Arima. WELCOME’s key activities included: recruiting and training advocates, establishing a trusted referral network, matching advocates with survivors, advocates providing support for survivors, social media, and fostering scalability and sustainability. Advocates recruited by DI and La Casita included Venezuelan women living in T&T and working with migrants who were already a part of La Casita’s network. WELCOME advocates assisted between one and three women at a time, helping women experiencing violence in their homes access support services in T&T. WELCOME addressed La Casita’s capacity gaps to improve service provision and assist Venezuelan survivors of GBV in accessing support. These capacity gaps included the absence of a workplan, advocate manuals and training materials, standard protocols to assess GBV survivor needs, and institutional mechanisms to reach out to police authorities, among others. Overall, WELCOME sought to reduce their uncertainty about accessing care by connecting women to known and trusted referral services, providing social support, and giving survivors the information and help to use referral services. In addition, the WELCOME activity shared the resources it developed on social media, plus communicated the referral network members and other GBV resources in Trinidad to improve awareness of available services. At the end of the project, the activity created management and learning products, such as case management documentation, a Standard Operations and Procedures Manual for GBV survivor support, tasking forms and checklists, to promote scalability and sustainability. 3. METHODS AND LIMITATIONS The evaluation team used a mixed-methods strategy, in which it predominantly relied on qualitative data to address the evaluation questions and sub-questions displayed in Table 3. The initial stage of inquiry relied on a desk review of documents produced by Resonance, the BTC’s implementing partner (IP), and subgrantees. Details on the desk review are provided below. This stage also drew on a small group of informal, formative exchanges via email with Resonance and IP staff, as well as a semi-structured interview with USAID’s Activity Cluster Manager, to refine the evaluation questions and assess the methodological toolkit for the context of each activity. 8 The desk review and the formative interviews guided the data collection methods to address the evaluation questions. The mixed-methods strategy relied on combined qualitative and quantitative methods.4 Using qualitative methods, the evaluation team conducted key informant interviews (KIIs), semi-structured interviews with program users and GBV survivors—most of them Venezuelan migrants—and focus group discussions with activity staff and female partners of primary users of one of the activities. For the quantitative work, the team used web surveys to survey service providers that partnered with BTG4VM and WELCOME, in their respective service networks. Exhibit 2. Data Sources and Analysis DESK REVIEW For the desk review, as the NORC team reviewed the documents received from each activity, we extracted relevant data into a prepared matrix in MS Excel. Each one displayed a category of evaluation and sub-evaluation questions included in Table 3: Activity Cluster questions, Portfolio questions, and Implementation Evaluation questions. The team reviewed and analyzed each document, inputting relevant excerpts related to the sub-questions. In total, we reviewed 26 documents, which included MEL plans, final reports, work plans, GBV assessments, grant agreements, key performance indicators (KPIs) spreadsheets, and other activity deliverables created by Resonance, BTG4VM, SPD, and WELCOME. QUALITATIVE DATA COLLECTION: INTERVIEWS NORC staff conducted remote KIIs with USAID staff, the BTC team at Resonance, all three grantees, and BTC representatives at the IDB. The first rows of Table 5 indicate the KII respondents and their organizations; further details of these KIIs by activity can be found in Table 15(see Annex B). The evaluation team designed the KII guides to respond to the portfolio- and the cluster-level evaluation 4 Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. Sage Publications. 9 questions. Topics included: co-creation processes and relevance of activities to the implementation context; BTC’s overall effectiveness in GBV protection, prevention, and accountability; activity cluster implementation; monitoring data, learning and adaptation; sustainability; and lessons learned, among others. In addition, in-country consultants recruited and administered semi-structured interviews with 20 female GBV survivors in Guyana and Trinidad and with 14 male primary users in 3 Panamanian cities. The questions for program users focused on activity target strategies, user experience, and outcomes. NORC depended on contact information that grantees would provide, based on who consented to being contacted, which reduced the overall sample size compared to the number of users engaged in each activity. Table 5. Summary of KII and FGD Respondents KEY INFORMANT INTERVIEWS RESPONDENT TYPE SAMPLE USAID USAID AC Manager 1 Resonance Staff from the Catalyst Project dedicated to BTC 3 IADB IADB program managers (HQ and Guyana Office) 3 BTG4VM NCC Staff BTG4VM Female GBV Survivors* † 3 7 SPD HIAS Staff SPD Male Activity Users* 3 14 WELCOME DI/La Casita Staff WELCOME Female GBV Survivors* 3 13 Focus Group Discussions SPD Female Partners of Activity Users* 6 WELCOME Staff and Advocates 3 BTG4VM NCC Staff 6 Notes: *The evaluated activities had the following total counts of participants: BTG4VM – 48 Female GBV Survivors, SPD – 165 Male Activity Users and 153 Female Partners of Activity Users, WELCOME – 30 Female GBV Survivors. † The evaluation team obtained contact information of a limited number of BTG4VM users. NCC only provided information from GBV survivors whom they previously contacted to obtain their authorization to share their information with NORC. The contact information of multiple users was discontinued by the time of data collection. QUALITATIVE DATA COLLECTION: FOCUS GROUP DISCUSSIONS As displayed at the bottom of Table 5, in-country consultants coordinated three focus group discussions (FGDs): one with NCC frontline staff for BTG4VM, a second one with La Casita frontline staff for WELCOME, and one more with female partners of SPD male primary users in Panama. Each FGD lasted approximately 90 minutes. The NORC team followed the same protocol for data transcription and cleaning as for the program user and survivor interviews. While the evaluation design attempted to moderate three FGDs with partners of activity users in Panama –one per implementation site–, the evaluation team faced challenges with obtaining access to the female partners. The main obstacle that prevented the team from fielding three FGDs with female partners was the reluctance of most male users to allow the team to contact their female partners. As a result, our in-country consultant organized one FGD session with female partners from all three implementation sites and obtained rich information. Further details on methodological limitations appear in Annex B. 10 QUANTITATIVE DATA COLLECTION: SURVEY NORC designed two web survey instruments to collect information about BTG4VM and WELCOME5. NORC selected this method for two reasons: 1) the IPs of BTG4VM and WELCOME advised NORC to prevent using service providers’ time outside their work schedule to conduct interviews; and 2) the evaluation team could create close-ended items to collect information about provision of services, collection of monitoring data, and perceptions about activity sustainability from these third-party actors. Exhibits 3 and 4 display the respondents’ demographics. The surveys asked about the availability of GBV-related services in the community, characteristics of the services provided through the network, target population and participant uptake, context appropriateness, service outcomes, monitoring tools, and activity sustainability. Most survey items were close ended questions. Before launching, NORC programmed the surveys in Qualtrics and tested the tools internally. Exhibit 3. Web Survey demographics (BTG4VM) Exhibit 4. Web Survey demographics (WELCOME) Note: In Guyana (BTG4VM), two respondents and in Trinidad (WELCOME) one respondent did not indicate their gender. NORC fielded both web surveys in two stages through an iterative snowball sampling approach. NORC’s local Evaluation Specialists were provided with initial web survey sample frames which included contact information of the focal persons at the organizations that partnered with BTG4VM and WELCOME to provide services under the two activities (including legal services, job training, housing support, and other services). First, NORC distributed the survey to this initial list of service providers. Second, the Evaluation Specialists contacted each respondent or organization the IP shared and asked if they could nominate additional staff that worked on either BTG4VM or WELCOME to participate in the survey. The Evaluation Specialists also confirmed that the organizations the IP listed actually participated 5 HIAS implemented SPD directly with no service provider support. 2 in the activity. This snowball sampling enabled NORC to increase the scope of data collection and the range of responses at various levels in the service provision. To increase the response rate, NORC issued weekly reminders via the Qualtrics system and requested its Evaluation Specialists to follow up on non-complete survey cases via email, text, or a phone call to encourage respondent participation. As noted in Table 6, we achieved a 56 percent response rate for BTG4VM and 67 percent for WELCOME. Table 6. Web Survey Results Activity Initial Sample Snowball Sample Response Rates Data collection dates BTG4VM 11 7 56% (10/18) January 27, 2023 – April 19, 2023 WELCOME 5 10 67% (10/15) January 30, 2023 – March 31, 2023 LIMITATIONS The data collection strategy for BTC has some limitations. On the qualitative side, the sample frame of interviews with GBV survivors through BTG4VM was constrained due to the out-of-date contacts the grantee provided. Also, the recruitment for FDGs through SPD was limited because most male workshop participants did not provide their authorization to contact their partners. To mitigate these obstacles, NORC oversampled GBV survivors, including also Guyanese women who had used BTG4VM services. As to the small sample of FGD participants, NORC recruited female partners from all three implementation sites in one single session. The result was a rich discussion of the topic guide. While the team was not able to capture differences between communities, the analysis obtained insights from a variegated sample of respondents about their relationships with their male partners before and after the SPD workshops. Due to WELCOME’s financial shortages, most advocates and staff at La Casita did not volunteer or work with the activity at the time of data collection and had other work obligations that prevented them from participating in the staff FGD. On the quantitative side, the web survey samples are not representative of all service providers in the countries where these instruments were fielded. However, NORC included most of the service providers the grantees recognized as part of their partnering network in each country. Readers should interpret the web survey results with caution in assuming that the information represents the perspectives of all the providers of similar services throughout the country. The data reflect the perspectives of the small sample of respondents that recognized some association or coordination with BTG4VM and WELCOME, respectively, and those for which the grantees provided references to NORC for the web survey. In both cases, the retrospective approach of the interviews, FGDs, and web surveys may introduce biases in at least two ways. First, IP staff, grantees, and service providers may experience recall bias if they have produced or participated in data collection for other evaluation reports. Second, recollection of remote facts from the past tends to distort perceptions on accomplishments and failures, often increasing their dimensions in the respondent’s mind and lessening the respondent’s ability to nuance her descriptions. Annex B includes further details of these limitations. 3 4. FINDINGS, CONCLUSIONS & RECOMMENDATIONS 4.1 FINDINGS FOR THE ACTIVITY CLUSTER QUESTIONS Table 7 presents a summary of NORC’s responses to the evaluation questions for the Better Together Challenge (BTC). Detailed responses to the sub-questions and related themes are organized by evaluation questions in the rest of the section. Table 7. Summary of Activity Cluster Findings BTC Activity 1 BTG4VM BTC Activity 2 SPD BTC Activity 3 WELCOME EQ1: Are the activity clusters based on context-specific and international evidence? ● The grant covered a baseline assessment of Venezuelan GBV survivors’ needs, led by Ladysmith, as a follow up to the RV4 GBV Assessment. ● HIAS piloted and implemented the Positive Masculinities curriculum in other countries prior to Panama. ● WELCOME redesigned its activity to face COVID-19 challenges to its implementation. ● Assessment findings suggested facilitating access to psychological, social, and legal services primarily, to reduce migrant GBV survivors’ vulnerability. ● The assessment also informed a communications campaign to counteract xenophobia and promote migrant incorporation. ● The grantees’ long presence in Panama garnered support and interest in the SPD workshops among female partners, relatives, and friends of migrant men. All previous HIAS programs in Panama targeted female migrants and refugees, who helped HIAS recruit SPD male participants. ● The baseline assessment (during redesign) of the needs of Venezuelan GBV survivors in T&T revealed a vital need to access psychological, social, and legal services to support stabilization and incorporation of the host community. ● Resonance and DI refined the activity design using evidence from other countries. EQ2: To what extent are each of the activity clusters achieving the targeted GBV results? ● Activity cluster reported attaining the intended outcomes included in the grant: One-stop shop to serve 48 GBV survivors in regions 3 and 46, coordinate Resource Directory of 38 NGOs to improve the coordination of GBV services, innovative communication campaigns against xenophobia support￾ing migrant incorporation. ● Workshops held during the grant timeline in three predetermined sites, with 165 male and 161 female attendees. SPD exceeded their target goal of male participants and stayed below with female partners (n=165 in both cases).7 ● Absence of following up of participants limited the activity’s ability to capture outcomes. ● Outcomes focused on building La Casita’s capacity to implement WELCOME successfully by training staff and implementing standard operating procedures. ● WELCOME also formalized La Casita’s relationships with local police, anti-trafficking enforcement, and migration authorities. 6 NCC’s BTG4VM background documents did not indicate a target number of individuals. 7 HIAS MELP. 4 BTC Activity 1 BTG4VM BTC Activity 2 SPD BTC Activity 3 WELCOME ● BTG4VM faced challenges during implementation and funding extension. This included: difficulty in finding local service providers, need of upfront coordination to run safe houses for GBV survivors, and limited staffing in remote areas. ● Overall participant satisfaction with SPD’s curriculum, delivery, and logistical efforts. ● Despite positive perceptions about changes in male participants’ behavior before and after the workshops, the evaluation team (ET) found resistances and potential caveats in these attitudinal changes among male participants during fieldwork. ● WELCOME served 30 GBV survivors through personal advocates who managed their cases and referred them to needed services. This achieved WELCOME’s target goal of 30 beneficiaries.8 ● Financial constraints limited La Casita’s ability to staff WELCOME adequately to meet demand and provide GBV survivors with enough time from advocates for follow up and psychological safety. EQ3: To what extent are the activity clusters sustainable? ● BTG4VM obtained two-year funding from IDB to expand the one-stop shop from 2 to 5 regions. ● Funding is being used to strengthen local partnerships and develop the local ecosystems of GBV services. ● HIAS did not replicate the SPD workshops. ● SPD has not implemented any strategies to procure additional funds. In its report, SPD explored an interest from the National Police and the Women’s Ministry to adopt the curriculum, but this did not happen nor create additional funding. ● La Casita continues implementing WELCOME at a smaller scale using individual donations and organizing small procurement activities (selling Venezuelan dishes, summer camps, and other cultural activities). ● The limited funding constraints its ability to meet demand. EVALUATION QUESTION 1: ARE THE ACS BASED ON CONTEXT-SPECIFIC AND INTERNATIONAL EVIDENCE? NEEDS ASSESSMENTS AND INTERVENTION EVIDENCE: HOW WELL WERE NEEDS ASSESSMENTS CONDUCTED AND INTERVENTION EVIDENCE COLLECTED TO INFORM THE CLUSTER ACTIVITIES? According to a key informant, USAID developed the Better Together Challenges (BTC) in response to the rise in migration from Venezuela to its neighboring countries. Such phenomenon also carried increases in GBV against Venezuelan migrants, an area that USAID’s GenDev was interested to address through the BTC mechanism. One respondent indicated that during the design process, preliminary evidence on the social and economic impacts of migration and GBV raised questions such as how to bring 8 DI Work Plan. Evidence Used for Activity Design USAID-funded baseline needs assessments (BTG4VM, WELCOME). Previous pilots of workshop intervention in other contexts (SPD). Data on GBV in Latin America and the Caribbean (all). 5 together different types of organizations with different foci, such as migration or GBV, and integrate their work. Considering the intersection of gender, migration, safety, and income instability among Venezuelan female migrants, the BTC team included these conditions as important criteria to grant funds to BTG4VM and WELCOME. This informed the intentional flexible design of the activities and intra-cluster collaboration. Across project activities, grantees used different information sources such as needs assessments, baseline assessments, and previous program models, to inform the design of each intervention, using mostly local but also international experiences that had been piloted in the past. However, all three responses to the local context were diverse. Both BTG4VM and WELCOME conducted baseline studies to identify the GBV needs among Venezuelan migrants. While the former included budget and scope of work to collect data in Georgetown with Ladysmith, the latter faced procedural and logistical challenges during the COVID-19 pandemic. WELCOME conducted this baseline assessment under the initial activity design focused on xenophobia and public harassment. It wasn’t until DI tried to implement the initial activity design that they learned the activity needed to be adapted. In contrast, SPD had implemented multiple interventions working with Venezuelan women in Panama, and a similar experience of masculinities with Venezuelan men in Colombia in 2021/2022. While this grantee had no baseline data to inform the activity design, their voluminous groups of former activity participants generated substantial demand for the workshops on masculinities. BTG4VM. In Guyana, NCC partnered with Ladysmith to build NCC’s capacity to design and implement the research methodology. Ladysmith conducted a needs assessment of Venezuelan and Guyanese GBV survivors as a follow up to the Response for Venezuela’s GBV Assessment Report in Guyana9, which provided information for the technical guidance, tools, framework for the research, analysis, and presenting findings. As part of this process, Ladysmith and NCC conducted a literature review, KIIs, and FGDs with Venezuelan migrant and Guyanese GBV survivors to determine their challenges accessing GBV services, such as legal assistance or psychosocial support. One of the recommendations from the Rapid Gender Based Violence Assessment (R-GBV-A) was to have educational materials on alternative masculinities, which helped to guide the BTG4VM target audience. The rapid assessment confirmed that accessing GBV services was a major issue in Guyana (see Table 8), which informed the components including the one-stop shop, referral pathway, and advocacy work. Table 8. What are the top three resources, services, or programs that the migrant community in your city or neighborhood need most to prevent or combat GBV? RESOURCE OR SERVICE BTG4VM, GUYANA Count Economic empowerment services (including professional skills training and entrepreneurial opportunities) 6 Housing shelters and safe spaces 6 Legal assistance for protection orders 3 9 R4V (2019). Inter-Agency Rapid Gender-Based Violence Assessment Report in Guyana. Available at: https://data2.unhcr.org/es/documents/details/71648 6 RESOURCE OR SERVICE WELCOME, TRINIDAD AND TOBAGO Count Psychosocial social services 6 Economic empowerment services (including professional skills training and entrepreneurial opportunities) 4 Legal assistance for protection orders 4 N = 10 respondents in Guyana and 10 respondents in Trinidad and Tobago. Respondents may choose more than one option. No responses for SPD because the activity had no network of service providers. Source: Web-based survey by ET. According to BTG4VM’s partner network service providers, Venezuelan migrants in their own city or community urgently required the services described in Table 8, with a specific focus on economic livelihood training and opportunities to generate household income. The team found coincidences between the R-GBV-A and service providers’ perspectives on the needs of the migrant community in the Guyanese Regions 3 and 4 regarding legal services for GBV survivors, which was a priority for NCC in finding a provider. As to job training and entrepreneurship, and safe housing, NCC coordinated referrals with Catholic Charities Organization Guyana. NCC did not explicitly include any strategies to address the migrants’ need for job training and entrepreneurship support in the activity workplan. SPD. In Panama, HIAS piloted and implemented a male engagement model on Positive Masculinities used previously in Kenya and Colombia. They also used lessons learned on how to work with men on GBV topics from an eight-week men’s engagement pilot program in Panama in July 2020.10 HIAS has been working in Panama since 2010 managing multiple programs and has a list of former female project participants from the Venezuelan community living in the areas of previous interventions. These participants requested more workshops, especially for men in the community, such as their spouses. While HIAS selected the implementation sites that had a high density of Venezuelan migrants and refugees in Panama City, West Panama and San Miguelito, a few SPD activity participants noted that the SPD workshops should have been more accessible, and had they been consulted, they would have suggested holding them in popular community areas to reach more people directly. WELCOME. According to the WELCOME final report, the activity’s baseline assessment confirmed that GBV was a critical problem for Venezuelan migrants in Trinidad and Tobago. Initially, DI designed the project to increase attention to sexual and xenophobic harassment in public spaces and increase reports of that harassment. The model included “Supermarket Superheroine” monitors trained by DI, who would be ready to support women facing harassment in grocery stores and “call out” perpetrators.11 However, according to staff interviews and the WELCOME Final Report, with the disruption of the COVID-19 pandemic, the intervention needed to shift from a public intervention to one that responded to GBV in the home. When reviewing the baseline report, WELCOME found that there were many barriers that inhibited GBV survivors’ access to legal support and to follow-ups on police reports of gender-based violence. During the redesign of the project, DI identified La Casita 10 HIAS Concept Paper: Engaging men in Panama to reduce risk and mitigate consequences of violence against women and girls, with a specific focus on Venezuelan refugees and migrants. 11 WELCOME MEL Plan refers to “callouts” as people publicly acknowledging incidents of women being harassed While the original WELCOME MEL Plan does not define “calling out” specifically, this intervention design was aiming to measure these instances using two indicators: (i) Total number of harassment instances and (ii) total number of counter-harassment callouts. 7 Hispanic Cultural Center to become its partner and to use the advocate model to support migrant victims of GBV. The service providers to which WELCOME advocates referred GBV survivors indicated that Venezuelan migrants most needed psychosocial support services, followed by economic empowerment and legal services (Table 8 above). DI/La Casita included these services in the WELCOME advocate’s scope for referrals to the network of providers. ASSUMPTIONS: WHAT ASSUMPTIONS WERE MADE TO DESIGN AND IMPLEMENT THE ACTIVITY CLUSTERS? HOW ACCURATE WERE ANY ASSUMPTIONS? In developing BTC, USAID assumed that selecting activities with a wide geographic spread12 and high organizational capacity would lead to the most impact for Venezuelan migrants. As the main funder of the Better Together Challenge, IDB’s key assumption was that the private sector and civil society organizations needed to have a more active role for migrants to promote inclusive development and boost entrepreneurship. They learned that there was a large ecosystem of private sector and civil society organizations already working with Venezuelan migrants in several countries. In the process of activity design, BTC grantees were supposed to explicitly state their assumptions regarding their interventions and the context. Because of COVID-19, WELCOME needed to revisit its assumptions. Interventions that focused on addressing sexual harassment in public spaces now had to focus on GBV in closed spaces or spaces with a limited number of people. This made it impossible to implement the project as it had been initially conceived. While the rest of the assumptions in BTC were not proved incorrect, the challenge’s conceptualization was broad, and its assumptions were not granular enough for the GBV experience to test them. The box on the right summarizes the BTC grantees’ assumptions. BTG4VM. According to NCC staff, the rapid gender assessment done with Ladysmith in March 2021 informed the ultimate BTG4VM project. Ladysmith also helped NCC staff become more aware of their own cultural biases related to working with migrants. By May 2021, NCC staff articulated a central assumption for BTG4VM. They determined that a priority need was to provide women with information on GBV services so they would be more likely to access these services and improve GBV protection. NCC also assumed that for BTG4VM, bringing various government and international partners together could be leveraged to improve service provision and promote sustainability. NCC’s Monitoring, Evaluation, and Learning (MEL) Plan includes the following assumptions for BTG4VM: 12 BTC was implemented in Argentina, Brazil, Chile, Colombia, Ecuador, Peru, and Venezuela in addition to the evaluated activities in Guyana, Panama, and Trinidad & Tobago Key Assumptions – BTC Activity Cluster Large ecosystem of civil society and private sector with potential to implement solutions for female Venezuelan migrants to protect, prevent, and increase accountability of GBV in destination communities. Critical vulnerability of female Venezuelan migrants generates stable demand for GBV services. A need to reassess the limited access of Venezuelan migrants to basic services, including GBV protection/prevention. Imbalances in domestic relationships of power between genders. Local biases against Venezuelan migrant populations. 8 (i) There are GBV services and referral pathways available that can meet the increased demand from the Venezuelan migration crisis; (ii) When invited, local governments and civil society organizations will participate in the NCC’s GBV service coordination activities (for example, updating the GBV Resource Directory); (iii) Through improved coordination (of various GBV services), organizations can work together to provide more holistic services; and (iv) Service providers and key stakeholders will use data collected to improve the local GBV response systems. From interviews with BTG4VM staff and from the Activity reports, the evaluation team assessed that the first three assumptions have been correct; however, the BTG4VM team has faced challenges to replicate the model in other regions and extend the coverage when the activity was initiated in new regions. Engaging new local partners and facilitating the processes to combat GBV and offer safe conditions to Venezuelan migrants to protect them from GBV has also required local adaptations to the economic and social context of the communities where BTG4VM offers its services. The responses to the evaluation survey suggest the fourth assumption was not accurate; service providers have not had access to monitoring data, and they felt that the BTG4VM team does not use these data either. SPD. HIAS implemented its workshops on masculinity and balance of power dynamics between genders based on the assumption that heterosexual couples of migrants engage in power imbalances. Key informants from HIAS Panama remarked that there was a need to include men and women in workshops because of the high levels of GBV in their communities. One respondent said, “The truth is yes, because such workshops are necessary. If you see the newspaper [or] look at the news, you see so many atrocities, so many barbarities that are happening.” [SPD Activity participant KII] The key assumptions HIAS articulated in the activity’s MEL and work plans were: (i) Women’s organizations, government institutions, and other stakeholders are willing to participate in the project activities based on the methodology for the prevention and reduction of gender-based violence; (ii) Men are willing to participate and share their experiences during the modules and acquire knowledge in positive masculinities to build healthy and equal relationships with their community and partners; (iii) Men are willing to participate in the modules and are open to new ways of thinking about positive masculinity; (iv) Women are able to participate in the experience exchange without experiencing increased risk of violence; and (v) Men will apply the new knowledge and skills they have gained from the masculinity curriculum to their own personal lives. Overall, the assumptions had mixed levels of accuracy. While representatives from the Ministry of Gender and the National Police expressed interest in the curriculum, there were no concrete decisions made on funding. Key informants from Resonance, BTC’s implementing partner, believed there would be challenges with engaging men in the SPD workshops because they assumed migrant men may be more focused on finding income. Ultimately, though, they found it easy to engage men in the activity and felt that their engagement was effective in helping them reflect on their construction of masculinity and how they related to their female partners, for those who had one. Neither the evaluation team nor the 9 HIAS staff had evidence to assess the fourth assumption; it is relevant to indicate the activity had no mitigation strategies incorporated in its design for female partners of users. Whereas all male participants expressed high satisfaction with the curriculum, the evaluation team was not able to corroborate from their female partners how activity users had made changes in their personal lives using the workshop lessons. WELCOME. The initial assumptions for the WELCOME program were based on an experimental behavioral science approach initially developed to address harassment in supermarkets and later adapted to bars. The initial assumptions were that the “superheroines” would be able to identify and call out harassment in these spaces, leading to more awareness of and decreased instances of harassment. However, COVID made this intervention impossible due to limits on the number of people that could be in indoor public spaces. After changing the approach and doing research on the various kinds of GBV besides public harassment, DI determined that one size would not fit all in responding to the needs of migrant GBV survivors. A variety of services and assistance would be needed, so they designed a flexible approach to customize services to individual survivor preferences or needs, which is the advocate model. Initially, only women were trained as advocates for the WELCOME program; however, DI learned there was a need for support for male GBV survivors, too. Their assumption that survivors would participate proved accurate, as survivors did stay with the project long enough to measure project impact. TYPES OF GBV IN IMPLEMENTATION CONTEXT NORC collected information about the GBV survivors’ and grantee staff knowledge regarding the instances of GBV in their communities. Overall, GBV survivors in Guyana and Trinidad and Tobago emphasized the problem of verbal harassment and poor awareness of GBV. Many discussed the need for more interventions focused on preventing GBV in general. Respondents also described specific types of violence against female Venezuelan migrants and LGBTQIA individuals, as summarized below. Domestic violence. Several survivors, staff, and program users referred to the prevalence of domestic violence toward women in their communities. Survivors who participated in the WELCOME activity particularly stressed that ending domestic violence was a GBV priority in their community. SPD participants thought the workshops were an effective tool for changing ideas around machismo and preventing domestic violence. Institutional violence. Survivors participating in WELCOME recounted being treated poorly by the police and hospital workers when they went in for help with their cases. Some survivors feared interacting with the police at all due to their migrant status. It took the assistance of WELCOME to overcome this barrier to services. SPD program users commented that institutional actors tend to commit violence against migrants. Intimate partner violence. Several WELCOME participants were survivors of intimate partner violence. SPD program users recalled covering preventing intimate partner violence in the workshops. GBV against LGBTQIA+ individuals. The SPD workshops also touched on machismo and LGBTQIA+ equality. Multiple participants expressed that LGBTQIA+ individuals experience more harassment and discrimination than others. WELCOME staff shared that they assisted a man who identified as gay who 10 was a survivor of GBV. BTG4VM respondents shared that LGBTQIA+ individuals, especially transgender people, were particularly subject to GBV. Other GBV. Evaluation respondents expressed minimal or no references to child, early, and forced marriage, coercion, female infanticide, human trafficking, rape, and reproductive and sexual coercion. In addition, GBV survivors and activity staff reported sexual harassment, and sexual violence, which both constitute subsets of the GBV categories listed above. CAUSAL PATHWAYS: WHAT CAUSAL PATHWAYS OR THEORIES OF CHANGE WERE ARTICULATED FOR THE ACTIVITY CLUSTERS? BTC. The BTC theory of change indicated that if innovative ideas and solutions from local Venezuelans and relevant agencies were funded and tested, and if found effective, the ideas were then scaled and their networks were expanded, then this would elevate their voices and ingenuity, helping to create a marketplace of ready-made solutions based on effective approaches, evidence, tools, and lessons learned.13 Resonance staff explained that the BTC theory of change did not capture the intended outputs: for instance, that Venezuelans felt empowered to create better lives for themselves and others. While USAID’s GenDev did not participate in the genesis of the BTC and its theory of change, Resonance respondents said BTC’s purpose was to innovate in new fields and countries, and experiment with several types of interventions to obtain evidence of what works. Therefore, the theory of change is not as comprehensive of the needs and the intersectionality female migrants and refugees face in their host communities. One staff member noted: “Maybe the theory of change doesn’t capture the necessity or the work we did around the enabling environment to empower the Venezuelan migrants that we worked with” [Resonance Staff KII]. In particular for GenDev grants, BTC’s theory of change was not explicit about the needs of GBV survivors and did not align with the 2016 Update to United States Strategy to Prevent and Respond to Gender-based Violence Globally. The lessons from this first experience with the BTC will be instrumental to inform future programming and assumptions at IDB and USAID’s GenDev when working with migrants in the Western Hemisphere. The document review revealed that all three grantees had theories of change or similar content. BTG4VM and SPD offered a more thorough narrative of the outputs and outcomes they aimed to accomplish among their target population and at the community level. In contrast, WELCOME’s mission concentrated on outputs and short-term outcomes adapted during the redesign stage of the advocate￾centered model of assistance. BTG4VM. NCC articulated BTG4VM causal pathways in its combined MEL and work plan: “Through this intervention of the BG4VM [we] will improve access to and awareness of GBV services, promote solidarity between local and migrant communities, and support improved articulation and data collection among GBV service-providers. In doing so, the NCC will help reduce migrant women and girls’ vulnerabilities to GBV, while strengthening local GBV response systems for a more sustainable, long-term impact.” [BTG4VM MEL and Workplan] SPD. Broadly, the SPD theory of change hypothesized that by engaging participants in workshops on positive masculinities, instances of GBV would decrease. The combined MEL and work plan described 13 From the BTC Final Report: https://juntosesmejorve.org/wp-content/uploads/2023/02/BetterTogether-Final-Report.pdf 11 the main causal pathways of the SPD activity: “The objective of the intervention is for the participants to reflect on their power and privilege, understand how power relates to violence, and how it affects their relationships and communities. As a result, participants are anticipated to change awareness, beliefs and attitudes, and actions. These changes will in turn lead to a reduction of the risk of male violence against women and girls in emergency settings.” [HIAS MEL and work plan] WELCOME. The initial theory of change for the WELCOME project was that increasing attention to harassment and empowering bystanders to intervene would help reduce harassment and xenophobia toward women in T&T. Eventually, the theory of change for the WELCOME program drew on the advocate model to improve access to GBV services. As described by the DI Work Plan, “The WELCOME project will focus on using behavioral insights to help Venezuelan women who are experiencing violence in their homes in T&T to access support services. For this purpose, WELCOME will address La Casita’s capacity gaps to strengthen service provision and help Venezuelan survivors of GBV to overcome barriers to receiving support. Through WELCOME, DI will [connect] women to trusted referral services through known community members, provide social support to promote survivors’ feelings of self-efficacy, and provide the necessary information and support to survivors to receive referral services.” [DI work plan] Per DI’s work plan, before the redesign, the theory of change was first informed by previous DI experience integrating behavioral changes into programs around the world, such as through “nudges,” which use behavior science-based approaches to guide people to make positive choices and engender behavioral change through small, subtle interventions. This focus shifted with the incorporation of La Casita’s advocate-centered model. MONITORING AND ADAPTATIONS: HOW WELL ARE INTERVENTIONS MONITORED AND EMERGING FINDINGS CONTRIBUTING TO INTERVENTION ADAPTATIONS OR IMPROVEMENTS? Because the BTC challenge was implemented during the COVID-19 pandemic, Resonance staff were unable to conduct monitoring themselves and hired third-party monitors in each country. These external agencies had programmatic experience and GBV expertise to validate the activities using key performance indicator (KPI) data. Resonance staff described the monitoring as a data quality assessment (DQA) exercise and way to flag challenges. Data on activities, participants reached, and results were submitted every three to four months, and all grantees were given MEL templates, reporting data to a platform called Air Table. The evaluation found that Resonance approached the monitoring and learning strategy differently across AC grantees. While BTG4VM adopted USAID’s KPIs, SPD only reported workshop attendance. As to WELCOME, Resonance coordinated with the activity team to create an ad hoc monitoring system that relied on a brief qualitative log and a short set of quantitative questions. The monitoring system was ad hoc because it did not reflect the traditional KPIs of USAID. This log approach was created by the project. The qualitative portion about the GBV survivor’s well-being informed any progress in their state of mind and in the problem that La Casita helped to address. The quantitative questions asked about the topic of the call or meeting, any challenges encountered by the advocate, time spent on the follow-up call, and expenses. The analysis suggests that Resonance worked intensely with all three grantees to create monitoring systems and promote peer learning with a flexible approach. Resonance supported 12 the three activities in sharing methodologies for supporting GBV survivors and approaches to training. However, the aggressive timeline, the local capacity, and the implementation challenges were the main obstacles to achieving a harmonized monitoring strategy across all target countries. Overall, all grantee staff indicated that they used the monitoring data to adapt activity approaches or components during implementation. However, the perception of service providers for the BTG4VM and WELCOME activity (who did not have access to these data) was that the activities did not make any adaptations based on monitoring tools. The document review for these two activities also suggests that neither activity planned a common learning agenda with their partnering networks. BTG4VM. Staff reported using spreadsheets and KPIs from USAID as monitoring tools for the project. Information collected from beneficiaries included age range, services provided, gender, and nationality. Staff said that the indicators were easy to use and adapt for their purposes. One staff member explained the usefulness of the monitoring tools: “I think it helps our partners share talents and resources and to provide best practices and to support Venezuelans. In some instances, we're looking specifically at the KPIs and how they align to Bridging the Gap Program. We were allowed to see what is useful and what’s not useful, so you report to what had context to what you were doing. So, in that regard, we were able to use them for what they were intended, and also some takeaways beyond the project.” [BTG4VM staff KII] Overall, service providers in both networks were not familiar with the monitoring strategies that the grantees put in place. Among the organizations that partnered with NCC in Guyana, only one in eight respondents said the BTG4VM requested monitoring data about referrals. That same respondent reported their organization shared data that included: diagnosis of project user needs (such as mental health counseling and housing support), changes in education outcomes over time, and changes in housing status over time. That same respondent found the data requests clear, but none said their organization had access to grantee’s monitoring system. The survey data contrast with the information gathered from interviews and FGDs. On the one hand, no service providers reported that the BTG4VM management team had adapted the activity based on the monitoring data. However, NCC staff expressed value in using the monitoring data according to their experience with BTG4VM. This disagreement could find its origin in the lack of access to the activity’s monitoring data for the network of referral services. SPD. The HIAS Monitoring and Evaluation Officer aggregated and analyzed all project data to support ongoing project management, learning from implementation, and reporting at regular intervals. SPD used attendance lists in Excel for its workshops as a monitoring tool to track attendance. Activity staff had male SPD participants respond to 17 items from the Gender Equitable Men (GEM) scale pre- and post￾activity. The internationally-used GEM scale is specifically designed to evaluate interventions addressing gender-related attitudes in men14. According to HIAS Panama's SPD final report, in the pre-test, the SPD participants had an average GEM score of 40.24/51. On the last day of program sessions, the average GEM score increased to 42.31/51. Additionally, several workshop participants noted that SPD asked them how workshops were going to receive suggestions and that the workshop leads adapted the sessions when necessary. One user said, “There were questions about what aspects to improve and the adaptability of the program.” [SPD activity 14 Kato-Wallace, J., et al. (2019). 13 participant KII] Respondents felt positively about this and that the workshops were responsive to their needs because of it. For instance, HIAS adapted the timing of the workshops to be more convenient to users’ schedule, based on feedback from participants. WELCOME. Staff drafted and finalized monitoring tools and templates, such as advocate logs and advocate and survivor surveys. They gathered data via La Casita advocates, who also kept an advocate log for every survivor meeting via an online form for their calls that occurred at least once per month. WELCOME had these regular check-ins to learn about survivor experiences and assess their well-being and need for additional support. The online form and calling protocol were revised throughout the project and found to be useful adaptations by program staff. Advocates also met regularly for an advocates’ meeting and were interviewed by WELCOME staff every two weeks. Then, every week staff would meet to check in on the progress of the project, who was and wasn’t receiving assistance, and issues related to safety and security. As staff collected monitoring data throughout the intervention, they adjusted approaches and provided feedback to the referral network. Of the agencies to which DI/La Casita referred GBV survivors, only 1 of 10 respondents said the WELCOME program requested monitoring data about referrals. This respondent reported only one data request, to which they shared project participant numbers, demographics of project participants, diagnoses of project user needs, number of appointments with project participants, and self-reported feelings of support. This respondent found the data requests clear enough but said they didn’t know if their organization had access to the grantee’s monitoring system. Further, no service providers said the WELCOME management team had adapted the activity based on the monitoring data. So, in general, BTG4VM and WELCOME did not involve service providers in the monitoring strategy. This may be explained by the focus of both NCC and DI/La Casita on monitoring the services these two organizations delivered directly –the referral pathway and matching GBV survivors to advocates– rather than on the services provided by other partnering organizations. Service providers weren’t in WELCOME’s scope, so there was never a plan to receive monitoring data from them. HOST COMMUNITIES: IS THERE A ROLE FOR MEMBERS OF THE HOST COMMUNITIES IN THE ACTIVITY? Both BTG4VM and SPD included host community members during implementation. The BTG4VM Work Plan and Final Report indicate that the activity was tailored to meet the needs of the host community in Guyana as well as Venezuelan migrants impacted by GBV. For example, promotional messages, video, and audio about NCC’s GBV services were created in both Spanish and English to reach both migrants and host community members based on findings from the Rapid GBV Assessment. The BTG4VM Final Report states that one of the achievements was strengthening solidarity between migrants and host community members and increasing knowledge on alternative masculinities, GBV, and xenophobia in both communities. This was partially achieved through the airing of a Radio Serial Drama to encourage positive behavior changes and support migrant and host community unity, which reached 6,500 people in Guyana. Additionally, BTG4VM received 1,527 views on their Facebook posts about xenophobia and 287 shares of these posts. Staff of SPD indicated that the activity intentionally included both Panamanians and Venezuelans to foster community cohesion and greater mutual understanding among migrant and host communities. SPD’s design involved male migrants as primary users but also their partners, who could be either migrants or 14 Panamanians. In practice, SPD received 38 Panamanian male participants (23 percent of the total primary users) and 16 Panamanian women (10 percent of the participants’ partners). WELCOME, in contrast, only targeted Venezuelan migrants, both female and male, and most of the activity’s advocates were also Venezuelan. According to the advocate-centered model, WELCOME had to work intensely with local authorities and service providers to address each GBV survivor’s needs. Further, WELCOME users indicated that staff told them that the services were only available to Hispanic migrants. Evaluation participants explained that they asked about program eligibility because host community members asked if Trinidadian nationals could also participate.15 CHANGING APPROACHES: HAVE GRANTEES AND SUBGRANTEES CHANGED THEIR APPROACH TO MIGRANTS AFTER BTC? Participating in the BTC was an opportunity for NCC and DI to transform their approaches to protecting GBV survivors and adapt them for addressing the intersectionality of gender, migration status, and income insecurity. KIs expressed that their assumptions on program delivery and staff training changed dramatically to interact effectively with migrants. Resonance was instrumental in this transition as both organizations faced challenges in the outreach strategies to migrant communities, difficult schedules, competing urgencies in the lives of migrants, language barriers, and understanding how vulnerable migrants are compared to the members of their host communities. HIAS, in contrast, has a long tradition of working with refugees and migrants globally. BTG4VM. Staff commented that they integrated monthly migrant outreach along with a social media campaign. Staff also recognized the need to hire more bilingual Spanish-speaking staff and prepare all staff to respond to calls for assistance from migrants. Moreover, BTG4VM ensured that staff received anti￾bias training on migrant issues and cultural differences. The BTG4VM Final Report shares that the program expanded its awareness campaign to target host communities due to demand of services. Among service providers in the BTG4VM network, survey respondents indicated their organizations adjusted their services to address the needs of Venezuelan migrants. In the BTG4VM activity, from the 10 web survey respondents whose organizations worked with Venezuelan migrants, there were 35 cases in which the service provider adjusted their services across all the indicated groups to some or great extent. This number was drawn from the number of times the 10 web survey respondents indicated that service providers adjusted their services for each activity listed by respondents. In particular, all 10 respondents mentioned their organization invested additional resources to work with Venezuelan migrant women; nine respondents said their organizations added resources for Venezuelan migrant teenage girls; and nine respondents said their organizations allocated more resources to engage with Venezuelan migrant teenage boys. In contrast, there were 17 instances of services that survey respondents said their organizations did not adjust them when providing those services to Venezuelan migrants. SPD. Due to its long tradition of serving refugees and migrants around the world, HIAS was already focused on serving migrants prior to the SPD activity. HIAS reported no adjustments to their approach to migrants and refugees in Panama as a consequence of the BTC grant. 15 The WELCOME activity was designed to target Venezuelan women, however services would be provided even if a survivor was not Venezuelan. 15 WELCOME. Though DI had previously worked on projects related to migration, which is how they identified La Casita as a partner, they adjusted their approach based on the work with Venezuelan migrants throughout the project. WELCOME staff reflected that one change in approach to working with migrants was having to ensure that there were Spanish-speaking bilingual staff. One challenge was getting in touch and following up with hard-to-reach migrants. This was hard for a variety of reasons, such as fear about migration status or unusual working hours. Among WELCOME service providers, all 10 respondents indicated their organizations worked with Venezuelan migrants. Section 5.1 on the Implementation Evaluation offers further details on the approach WELCOME service providers adopted toward Venezuelan migrants and other target groups. EVALUATION QUESTION 2: TO WHAT EXTENT ARE EACH OF THE ACTIVITY CLUSTERS ACHIEVING THE TARGETED GBV RESULTS? OUTCOMES: ARE THE STATED OUTCOMES REALISTIC AND ACHIEVABLE WITHIN THE TIMEFRAME OF THE AC? WHAT PROGRESS IS BEING MADE TOWARDS ACHIEVING THE OUTCOMES? The evaluation found all BTC activities funded by USAID/GenDev attained the outputs and short-term outcomes they had proposed in their project objectives and theories of change. The scope of outcomes and the extent to which medium- and long-term outcomes were achieved varied across activities. BTG4VM had the widest breadth of components and intended outcomes. Overall, BTG4VM respondents described the activity’s results with pride and satisfaction. They describe both the most visible results—the launching of the one-stop-shop approach in Guyana, leading the national referral pathway—and the positive feedback to the communication campaigns. However, several coordination and collaboration strategies continue evolving as BTG4VM continues expanding with the additional IDB funds and establishing relationships with local service providers. The small ecosystem of implementing partners seems to continue being a challenge for strengthening the referral pathway both at the national and regional levels. In Panama, the SPD workshops completed their implementation, and HIAS made no follow up with users. While participants express satisfaction with the workshops, the evaluation found behaviors among male participants that revealed resistance to allow their partners to speak to the Evaluation Specialist about the activity. This is an indicator that some behaviors towards gender imbalances remained in several male respondents. Meanwhile, WELCOME substantially transformed La Casita’s operational model and provided it with resources to become an institutional advocate for GBV survivors. La Casita continues implementing the advocate-centered model at a smaller scale and aims to obtain funds to implement the systems it created through the collaboration with DI. BTG4VM. The first main outcome of the BTG4VM project was increased access to GBV services and referral pathways. BTG4VM served 48 GBV survivors and did not indicate initial targets in its MEL Plan. Staff noted that development, completion, and implementation of the national referral pathway in Guyana helped to streamline GBV services. Staff highlighted the importance of the one-stop-shop model in improving delivery of services like psychological, medical, and legal support.16 BTG4VM documented 48 GBV survivors that received services from the one-stop shop between June 25 and December 10, 2021. From the total number of BTG4VM users, 26 received psychosocial support/counseling, 17 legal support, and 5 both psychosocial and legal support. While NCC closed 20 cases in 2021, 28 remained 16 The one-stop-shop is a model to provide comprehensive GBV services through a single point of contact, including psychological, legal, and medical services, among others. (NCC (2020) BTG4VM MEL and Workplan) 16 open through the period funded with the IDB grant in 2022. Additionally, BTG4VM staff argued that the launching of the one-stop-shop model helped improve awareness of GBV services. During the implementation under GenDev funding, BTG4VM received users from Regions 5 and 10, beyond the original area of service. NCC staff both in interviews and FGDs indicated that word of mouth within the migrant community helped disseminate the one-stop shop service. BTG4VM’s Final Report indicates the activity improved coordination and collaboration between government agencies, the police, and civil society organizations (CSOs) in Guyana. BTG4VM created a Resource Directory of 38 NGOs in June 2021 to improve the coordination of GBV services. NCC mapped services including a diversity of sectors such as health, social services, and law enforcement from both local and national government agencies and civil society organizations to improve coordination among these service providers and more easily refer survivor cases to the appropriate entity. However, according to KIIs with staff, this coordination is still slim outside of Regions 3 and 4, where BTG4VM has more experience. In 2022, BTG4VM expanded the one-stop-shop model to regions 1, 7, and 8; respondents reported challenges to implementation, in part due to the absence of service providers in those areas to implement the model. Health care workers and law enforcement in particular received trainings on GBV prevention topics, which BTG4VM staff indicated helped contribute to improved coordination and collaboration on GBV response. While NORC did not survey health care workers and law enforcement on their level of satisfaction with the GBV trainings, the local Evaluation Specialist contacted the directory of service providers and experienced several contacts with wrong or out-of-date information and a very slow turnaround of responses despite multiple attempts between February and April 2023. While this approach is not a direct measurement of network coordination, the process suggests that the coordination and collaboration efforts still have areas for improvement and strengthening. The project promoted awareness on strengthening solidarity between migrant and local communities. In collaboration with Merundoi, a women-led NGO specialized in behavior change communication, BTG4VM developed scripts for a radio serial drama and public service announcements covering xenophobia and alternative masculinities. Additionally, social media influencers shared positive messages against xenophobia to contribute to this outcome. BTG4VM reported the drama aired between September 7 and December 31, 2021, and it created 204 posts that received overall positive feedback from migrant and host communities. BTG4VM also reported that the number of calls to NCC’s helpline increased after the social media posts. NORC also asked the BTG4VM service provider network about the activity outcomes. Among those respondents who answered this section of the survey (n=8), only three said the BTG4VM Activity had a moderate or major effect in preventing GBV. They also assessed the effectiveness of each service in the network, separately. As displayed in Exhibit 5, service providers considered social media campaigns and the implementation of the one-stop shop model marginally more impactful in addressing GBV among Venezuelan migrants compared to radio and TV public announcements and a strengthened national GBV referral pathway. These findings, while modest, coincide with the perceptions from BTG4VM staff and the outcomes the activity reported. 17 Exhibit 5. Rate the effectiveness of BTG4VM program components in preventing GBV Source: Web-based survey by the ET. N=10. Separately, NORC asked service providers which component of the BTG4VM program led to the most positive impact for the migrant community. Three out of 10 respondents suggested it was the one-stop￾shop model for GBV services, one respondent felt the social media campaign increased awareness, and one indicated radio and public service announcements. SPD. The SPD project worked to achieve two key outcomes. The first aim was to reduce the risk of GBV among forcibly displaced women, young women, and people with diverse sexual and gender orientations. The second goal was to help displaced and host community men unlearn social norms and behaviors that contribute to or perpetuate GBV against women. HIAS effectively recruited 165 male participants to attend at least one of the four workshop sessions and an Inter-Gender Dialogue session; and 161 female participants –partners or family members of the male participants– to attend an Inter￾Gender Dialogue session. These numbers are mostly on par with the activity targets, which were 165 male participants and 165 female partners.17 Staff confirmed that a major influence was participation in workshops, trust building, and fostering respect among different identities. Several program users commented on the workshops, with one participant describing the helpful influence for them: “On a personal level: it helped me in self-esteem, in not staying in negativity. We were in a pandemic, but it brought us closer, there was camaraderie and contact were motivated, respecting freedom of expression.” [SPD activity participant KII] HIAS staff also felt that the project garnered attention around positive masculinity among male users. A male activity participant agreed, explaining: “Here we see that machismo is normal and HIAS showed us that machismo is not normal. It is not good, and we change, and with that we instill a change in values leading to a better social life; we change tradition.” [SPD activity participant KII] 17 HIAS MELP 18 Another participant commented that they perceived increases in awareness about the responsibility of men to respond to GBV. Another explained the workshop started to change her husband’s attitude toward LGBTQIA+ people to become more positive. Staff noted that though the time was too short to measure norms change, the intention was to establish a strong foundation about the imbalances of power between men and women and to promote the role of men in actively reducing them. WELCOME. The WELCOME program reached 30 direct Venezuelan GBV survivors, 100 percent of the activity’s target,18 and helped them to recognize their support options. Staff reported they assisted Venezuelan GBV survivors in T&T to feel supported to access the services they needed and reduced barriers preventing survivors from accessing support. WELCOME staff indicated the main activity outcomes were two: 1) Assisting migrant GBV survivors to gain access to the services they needed and had the right to receive, and 2) raising awareness among migrant populations of these services. Another staff member remarked that a major outcome of WELCOME was that DI and other service providers supported La Casita to improve its operations by creating operational and administrative protocols and workflows, as well as financial control documents. Before the BTC’s grant, La Casita lacked any organizational systems to support programming. La Casita also developed institutional relationships with the police and other ally organizations formalizing its internal processes with this support. La Casita transitioned from an individual person advocating for GBV survivors to an inter-institutional approach for the advocacy work. Exhibit 6. Rate the effectiveness of WELCOME network services providers in preventing GBV N = 10 Through a web survey, WELCOME service providers indicated their perceptions about the effectiveness of network services in preventing GBV. Overall, 5 out of 10 respondents that answered this section said the WELCOME program had a moderate or major effect in preventing GBV. Exhibit 6 shows that survey respondents felt that counseling and other psychosocial services and services for victims of human trafficking, refugees, and asylum seekers had greater impact in preventing GBV among Venezuelan migrants compared to youth services and healthcare. Similarly, three respondents to the 18 DI Work Plan 19 service provider web survey considered services for refugees and asylum seekers had the most positive impact in the community of Venezuelan migrants, two selected counseling and other psychosocial services, and one indicated establishing a trusted referral network had the greatest benefits. PLANNING AND ACTIVITY DESIGN: HOW AND HOW WELL WERE ACTIVITY PLANS AND DESIGNS DEVELOPED TO ACHIEVE DIFFERENT GBV OUTCOMES? At a high level, IADB, USAID, and Resonance, led the design of the BTC Challenge to fund innovative solutions. IDB staff explained that the overall project design aimed to ensure services were accessible to migrants and host community members. The Catalyst team led the due diligence and co-creation process with IPs, and the led monthly meetings with USAID and IPs to report on projects progress, address challenges, respond to questions, and consider successful approaches from other contexts. BTG4VM staff indicated that coordination was managed well with Resonance and USAID. Resonance staff noted the short timeline for implementing the BTC and how this led to pressure and time constraints. BTG4VM and WELCOME reported numerous steps to adapt their design to the context. BTG4VM created a new approach to provide GBV services and engage a network of service providers, most of them not recipients of GenDev funds. WELCOME changed the design—and therefore outcomes—due to the COVID-19 pandemic. In contrast, SPD launched the workshops as planned and during outreach and recruitment, adjusted strategies to target the intended communities and reach the largest possible number of male participants in the target communities. BTG4VM. Ladysmith’s rapid gender assessment and GBV training helped guide activity planning and design. The design was also influenced by the various government, civil society, and international partnerships leveraged by NCC and the services these partners could facilitate, such as access to safe shelters and pro bono legal services. Ultimately, this structure aimed to improve coordination among GBV services providers and increase access to these services and referral pathways. Staff created social media campaigns to start changing attitudes toward migrants and strengthen solidarity between migrants and host communities. BTG4VM realized throughout the intervention that it needed to change its design to meet participant needs, such as having a dedicated lawyer for legal support cases. Like others, staff noted it would likely have been more effective for the project period to be longer. Staff also identified the need for services to translate documents for migrants. While most surveyed service providers (n=6) said BTG4VM was adequate to address their communities, a small group (n=3) remained neutral. Service provider organizations that partnered with BTG4VM shared their perspectives about services and whether the activity and its network of services were adequate and adapted to their community context. For BTG4VM, only six out of nine web survey respondents who answered this section said its services were a good fit in their community. Only three respondents agreed or strongly agreed that the BTG4VM activity and its service network staff was well adapted to their community; three others expressed neutrality about BTG4VM’s adaptation. The adaptations included addressing problems innovatively using the perspectives of people in need and addressing requested services. Further, the remaining respondents either did not know or provided no answer. SPD. HIAS Global provided HIAS Panama with the workshop design, which was described in the proposal to USAID. The activity is guided by the approach of the U.S. Strategy to Prevent and Respond 20 to GBV Globally. Staff then participated in a co-creation process with USAID. One staff member reflected, “USAID provided HIAS with valuable feedback. Their experts had a very clear idea of what they wanted and provided good feedback on masculinity concepts and considerations for the workshops.” [SPD staff KII] The SPD activity was designed to engage forcibly displaced men to help reduce risk and mitigate consequences of GBV by offering participants safe spaces to reflect and deconstruct ideas about gender. The activities promoted a reflexive understating of gender roles, how these contribute to violence, and men’s role in preventing and eliminating GBV. Thus, HIAS held workshops in private spaces to promote trust and community. Several program users explained that the workshops were planned well, and the facilitators were well trained. However, other activity participants did feel that timing and location of workshops could be improved to make the workshops more accessible to their communities. Another caveat is the selection of users by age. HIAS staff indicated they perceived mixed results in male participants’ attitudes towards gender balance according to their age. These respondents found deeper and more durable effects in younger migrants, especially under 25 years old, from the workshops, than among older ones, especially those 65 and older. In general, attitudes towards gender roles grow intractable and rigid as individuals age, independently of their migration status. WELCOME. The main actors involved in the WELCOME design were DI and La Casita, with support from USAID. La Casita advocates also helped influence planning and activity design by giving feedback on how to reach, support, and resolve situations for survivors. When the program decided to change approaches, a major shift in design was addressing GBV in the home rather than public forms of abuse. Due to COVID-19, GBV in private spaces became more frequent than in public. DI designed a women￾focused solution to reduce the barriers for Venezuelan migrant women experiencing GBV to access the support they need in T&T. Asked if they would design the activity differently again, DI/La Casita staff said that the project would benefit from more money and time. Also in the future, they would recruit advocates who aren’t already working full time. When asked about the WELCOME referral services, 7 out of 10 respondents said the BTG4VM services were a good fit in their community. Also, six respondents agreed or strongly agreed that the WELCOME Activity and its service network staff adapted to the context of their community. Their adaptations included listening to client needs as they arose and providing an interpreter for counseling sessions when the counselor did not speak Spanish. The remaining respondents either expressed not knowing about WELCOME’s ability to adapt or provided no answer. INTERVENTION AND IMPLEMENTATION: HOW WELL ARE INTERVENTIONS IMPLEMENTED TO REACH THEIR TARGET GROUPS AND INFLUENCE CHANGE? Resonance staff explained that the short implementation window was limiting and that a lesson learned would be to set expectations up front about the impacts that can be achieved within such a short timeframe. There were varying levels of understanding and capacity, so handling these all at once was a challenge. One Resonance staff member said that BTC should have promoted more activities in which grantees learned from each other. 21 While the design of the BTC activities seemed to include necessary components to attain intended outcomes, the evaluation found that numerous challenges affected the implementation of all activities. A common challenge for BTG4VM and WELCOME was the limited staff for the one-stop shop and advocate model. Several GBV survivors reported their referral staff and advocates had very busy schedules, and they were not able to follow up on their progress with services as they needed. The SPD workshops provided means of transportation to participants and accommodated Zoom access during the pandemic. However, the SPD design included no follow-up strategies, which in the participants’ perspective, did not promote long-term effects from the intervention. BTG4VM. A small group of survivors who received services from BTG4VM commented that communication and follow-up could have been improved across a number of different services, such as legal and psychosocial services. One survivor commented that they would have appreciated BTG4VM’s follow-up because she and her family continued feeling unsafe as she participated in legal assistance and psychosocial support. Among the service providers that answered NORC’s web survey in Guyana, the most frequently mentioned challenges to implementation for BTG4VM are displayed in Table 9. Key challenges included: limited BTG4VM financial resources, operational and referral issues, limits in the access to certain services for migrants due to lack of means of identification, and language barriers. This last challenge aligns with findings from program users about the limited follow-up advocates were able to conduct in some cases. Table 9. Main Implementation Challenges (Service Providers’ Perspective) BTG4VM Count Under-resourced service providers 2 Budgeting level of effort for lawyers and psychologists 1 Slow pace of information gathering 1 Documentation barriers for migrants to receive COVID-19 vaccines and enter public spaces 1 Language barriers 1 No challenges 1 Don’t know 5 WELCOME Count Difficulty of survivors to commit to schedules 3 Limited transportation/mobility of survivors 2 Small number of trained advocates 2 No challenges 2 COVID-19 limiting service availability 1 Lack of capacity to support male survivors 1 Under-funded WELCOME activity 1 N = 8 for BTG4VM and 9 for WELCOME. Respondents were able to select more than one challenge. Respondents who selected “Don’t know” represent only one count. Source: Web-based survey by ET. SPD. HIAS used a multimode recruitment approach for the SPD program: social media, local NGOs, and referrals through participants in the initiative targeting female migrants in Panama. Female partners also played an active role in recruiting male partners for the SPD program. In interviews, participants agreed that the workshops responded well to the values, traditions, and customs of the community. Some changes were made to make remote workshops accessible via Zoom due to the pandemic. Staff 22 thought that easy-to-find workshop locations, providing meals, and transportation was a facilitating factor for intervention and implementation effectiveness, which was confirmed during a few participant interviews. Despite this, several activity participants noted that the program lacked the follow-up they desired. Another noted that language facilitation could have been better; for example: “The name of the project’s mobile app ‘El Man de Hoy’ (Today’s Guy) adopts the idiom ‘Man,’ which is Colombian and Panamanian, not Venezuelan.” [SPD activity participant KII] WELCOME. Staff reflected that La Casita did not have enough staff to implement the project effectively. La Casita relies heavily on the work of their well-known Director to manage activities. With one person so central to the functioning of the whole organization, widening the reach of their activities was difficult on staff. For example, when migrants were turned away from the police, the Director personally had to advocate for them to open a report because they knew her. The short implementation window was also very challenging, as a DI staff member explained that completing advocate logs sufficiently and on-time was difficult. However, when discussing their ability to contact Venezuelan migrants GBV survivors, a DI/La Casita staff member remarked that there was no difficulty reaching them. Staff felt that the dedication of the advocates was a facilitating factor to intervention and implementation effectiveness. Other contextual challenges to the implementation of WELCOME are displayed in Table 9, based on findings from referral service providers. Like the BTG4VM partners, service providers in Trinidad and Tobago identified the users’ busy schedules, their limited mobility, and the small number of trained advocates as the main challenges for WELCOME. This aligns with staff comments about the challenges advocates experienced when following up on each of their cases, reaching out to authorities and services providers, and completing their logs. MECHANISMS: WHAT ARE THE MOST EFFECTIVE ASPECTS OF THE INTERVENTION? HOW DO THESE “ACTIVE INGREDIENTS” OPERATE IN EACH AC? Different mechanisms were used to effectively reach intended outcomes; according to IP respondents, grantee staff, and activity participants, the strategies responded to the nature of the program and the challenges of the context. The evaluation found that legal assistance and financial support were key elements to improve GBV survivors’ conditions in Guyana. In Panama, SPD’s technical approach to men and the reflection about their own identity, their home, their family, and their partner resonated with several program users that responded to interviews. WELCOME's effectiveness in working with GBV survivors relied on the intervention of an advocate who understood the GBV survivor’s life condition and provided her with different solutions by walking her through the process. As Resonance and the grantees and subgrantees implemented the BTC initiatives during the COVID-19 pandemic, the USAID responded reflected that both GenDev and the Resonance promoted that all of the activity mechanisms had to be flexible, given the nature of the COVID pandemic. BTG4VM. A staff member acknowledged the value of the support provided through BTG4VM but emphasized the effectiveness of the way legal services were provided. They described how having a dedicated lawyer to protect the interest of the survivors through legal proceedings was an effective mechanism for improving trajectories for GBV survivors. Another added that providing financial support for transportation costs, services, and shelter improved survivor outcomes. When asked what they 23 would share with friends about the activity, one survivor described their satisfaction with the various activity components, “I’d tell them it’s a good service because it helped me with the lawyers, [and] to get child support. They were very friendly and warm with me. They listened to me. They provide a place for me to stay.” [BTG4VM GBV survivor KII] IDB staff remarked that NCC’s integrated approach to improving service delivery for GBV is inclusive and carefully considers the vulnerabilities of both Guyanese and migrant populations. SPD. An activity participant remarked that the “the best part” of the SPD workshop was addressing the needs of couples. Men were able to share their experiences and reflect on improved collaboration at home. The participant felt the effectiveness came from the dynamic and psychological approach of the workshop. Most participants felt they had increased knowledge of masculinity and GBV, and for that reason the workshops were effective. Another participant added, “The diversity of people was what I liked the most. There were people of different social classes, of different educational levels, and of different sexual affinities. That was what I liked the most.” [SPD activity participant KII] Moreover, USAID staff added that the well-facilitated workshops were effective because of the way they were tapping into the experiences of men. Survivor interviews confirmed this, with two survivors explaining, “These courses have helped us.... Before I had to do most of the things ... but with these courses they have helped us greatly; we do things together.... These courses should be given more to many men so that all women people be equal...” [SPD activity participant KII] “My husband did not cook, he did not clean, he did not wash, only when things fell behind. Now he has begun to clean, cook, wash.... The masculinity workshops refreshed his memory ... and he understood gender equality .... and even began to be a couples counselor..." [SPD partners of participants FGD] During implementation, HIAS asked SPD activity participants for their input on how to improve workshops, which they felt was a useful mechanism to improve the intervention. WELCOME. In the FGD with staff and advocates, participants indicated that having personal advocates, especially those who were bilingual, was an effective mechanism for making survivors feel supported and protected. Several survivors remarked that the WELCOME program was effective at helping them stay safe and access support. One survivor sha red, “After I made my complaint, I came here. The girls showed them the report, they took me back to the police, they accompanied me. Everything was excellent, I mean, I can’t complain: they were very attentive [of] me, they advised me on how to request an appointment for a psychologist to attend [to] me.” [WELCOME GBV survivor KII] Another shared that feeling safe with La Casita staff and having the support of law enforcement helped them after experiencing domestic violence. La Casita’s service coordination helped survivors feel secure 24 and empowered. Regarding the services received from the WELCOME program, one survivor in an interview said, “They gave me strength.” HOST COMMUNITY ENGAGEMENT: ARE THERE ANY LESSONS LEARNED ON HOW TO ENGAGE HOST COMMUNITY MEMBERS? PLEASE, PROVIDE AN EXAMPLE. BTC. Resonance staff said that finding a trusted partner in the host community was critical. Trusted partners support the community by facilitating resources and can amplify voices to share about the opportunities that each program generates to engage positively with migrants. BTG4VM. In Guyana, BTG4VM learned that Venezuelan migrants are less represented and coordinated than the host populations, so migrants needed a coordinating advocacy body among the groups of migrants themselves. Also, by training service providers on GBV response, the activity found that there were a number of biases and misconceptions about migrant populations, (such as concerns about cultural differences and the idea that Venezuelans were receiving unfair benefits from Guyanese society) among Guyanese health care workers and police that needed addressing. BTG4VM identified and included details of the steps to be taken at each entry point when a survivor accesses a GBV service. To prevent potentially biased treatment of migrants by the service providers in the network, NCC identified basic protocols on how GBV survivors should be referred. BTG4VM also included details of the steps to be taken at each entry point when a survivor accesses a GBV service. For example, suppose a survivor presents for service at the health entry point. In that case, the health care worker must receive and inform the survivor about all the available options and support, based on the survivor's needs and availability at the facility. Moreover, NCC accompanied survivors to the Police Station when making a report to make sure they were acknowledged. NCC also guided Police officers where necessary to ensure that reports were accurately documented. SPD. The workshops included a small fraction of Panamanian men and women. According to SPD staff, the participation of Panamanians added valuable insights to the conversations among men about masculinity and GBV in Panamanian society. Their participation was also an opportunity to create allies in the host community. SPD respondents said they would include members of the host community in any future experiences of SPD. WELCOME. The main reason for the WELCOME redesign was COVID-19, so DI had to revise its strategy to meet the needs of migrant populations during lockdown. Lessons from the co-design and from WELCOME’s baseline assessment revealed the existence of additional barriers to access basic services to prevent and protect them from GBV, compared with local communities. By using an advocate approach to engage difficult-to-reach migrants, the program enabled an instrument to address their needs with the more holistic approach that migrants required. EVALUATION QUESTION 3: TO WHAT EXTENT ARE THE ACS SUSTAINABLE? SUSTAINABILITY: WHAT ASPECTS OF THE ACS CONTRIBUTED TO THEIR SUSTAINABILITY? WHAT COMPONENTS ARE NEEDED FOR GREATER SUSTAINABILITY? Currently, there are no plans to launch another version of BTC. The evaluation found that the activity cluster IP, Resonance, implemented strategies to promote the sustainability of the funded interventions. However, those efforts were limited; the short timeline of the Challenge, the demanding capacity 25 building to establish the program mechanisms of BTG4VM and WELCOME, and the adaptation to attain successful outreach for the SPD workshops competed with Resonance’s efforts to focus on sustainable strategies in all three grantees. However, sustainability varied across BTC grants. BTG4VM secured two years of additional funding from IDB, and La Casita continued implementing the advocate-centered model at a smaller scale by relying on small donations and fundraising activities before receiving a prime award to continue the work from the WELCOME program. SPD has not been implemented again, and HIAS Panama had no replication plans. As to integrating programmatic work with local and national institutions as a strategy for sustainability, BTG4VM has contributed to formalize the referral pathway system for GBV survivors, but the evaluation reports remaining challenges to attain smooth operations of a one-stop shop for female users in the new regions of implementation. SPD was not able to incorporate its curriculum on masculinities in the Ministry of Gender's or the National Police’s capacity building agendas. WELCOME’s approach did not include incorporating the advocate model in the public GBV services. SUSTAINABILITY FACTORS BTC. From a regional perspective, all three IDB respondents agreed that the need for GBV-focused services for Venezuelan migrants in the region will continue. Therefore, the need for continuous programming to address the short- and long-term needs of Venezuelan migrant communities will promote further dialogue and funding. However, neither USAID nor IDB respondents expressed any plans for a BTC replication. At the local level, staff at BTG4VM and WELCOME, and the USAID Activity Cluster Manager, expressed that the network approach benefits do not diminish. Respondents explained that organizations such as NCC and La Casita cannot administer all the required solutions. Therefore, building strong collaboration networks is a key ingredient to achieve sustainability. In Guyana, focus group respondents described that the pathway referral system allows for the continuation of other network services despite one partner stopping its service. “So, we always promote networking and shared resources that is part of our core value because no one entity at no one time has all the resources to address all the needs. So, we find that the more we do that kind of networking like [staff member] is speaking to, we are creating more sustained programs for sustained support for survivors.” [BTG4VM staff FGD] BTG4VM. Specifically in Guyana, the BTG4VM activity still had two additional years of funding from IDB at the time of data collection. During this time, staff members mentioned NCC is exploring the possibility of contracting with the Guyanese Government to continue providing the one-stop shop services. One NCC respondent mentioned their experience of governmental contracting in which they provided HIV and TB testing and treatment with public funding. A respondent added the organization could match funds to provide services in the areas of public interest, so that “at least we should be able to come to the table with something, and then ask for more to help” [NCC staff KII]. While NCC gained experience on this “social contracting” and respondents expressed there is concrete need for GBV￾focused services, the conversations had not produced any concrete agreements by the time of data collection. A Resonance respondent highlighted how the IDB’s contribution was outstanding in such a limited ecosystem of GBV funding in Guyana. 26 SPD. The HIAS Panama Head explained that they created a mobile app “El Man de Hoy” to make some of the SPD workshop contents available to a larger population in Panama and Colombia. However, the respondent recognized the app was launched toward the end of the workshops and the activity funding. The respondent did not have information about how often the app was updated and for how long it would continue being available. At the time of this report, the app is available in both the Apple Store for iPhone and Google Play for Android mobile phones. WELCOME. DI identified two areas in which they could help La Casita foster sustainability. The first strategy was to institutionalize the relationship between La Casita and the police, particularly with the Counter-trafficking Unit of the Trinidad and Tobago Police Service. DI staff explained that La Casita’s Head came to the program with a strong personal relationship with that Unit. The activity increased the visibility of La Casita’s staff, making it easy for the police unit officers to recognize the advocates and receive GBV victims’ complaints from staff vs. only accepting them from La Casita’s head. La Casita staff said they were satisfied with the technical assistance from DI for advocate training. Building this capacity was crucial to formalize support for GBV survivors and strengthen the links to service providers beyond a one-woman effort. The second area that is likely to lead to greater sustainability was building the team’s capacity to submit funding proposals to international organizations. For example, DI helped La Casita’s head prepare and submit a proposal to UNHCR. At the time of data collection, the proposal had enabled La Casita to be included in the list of UNHCR potential grantees in Trinidad and Tobago. STRATEGIES FOR SUSTAINABILITY Resonance staff explained that the agency stopped following up with HIAS, and DI/La Casita after their grants expired. According to BTC documents and respondents from USAID and Resonance, the BTC Cluster was not designed to provide post-funding oversight or support. All strategies for sustainability and self-sufficiency should have been incorporated during the life cycle of the grants. Resonance provided the IPs with a webinar on fundraising basics in the last three months of BTC implementation. Resonance respondents did not assess the success of this training. At the activity level, each program differed in their needs and plans. BTG4VM. Staff respondents mentioned that during the two years of IDB funding that remain, they plan to implement active fundraising strategies with businesses and to create a specialized team within BTGFVM, called Advocacy Support Group. This team will include, among other advocacy issues, an assessment of the long-term needs of female Venezuelan migrants in Guyana. FGD respondents explained that the plan is to translate their needs into specific requests for “financial support, businesses’ support, and collaboration with other entities to really support long-term sustainability together.” [BTG4VM participant FGD] SPD. Respondents from Resonance indicated that HIAS Panama proposed to implement in Panama an extension of a toxic masculinity approach in Colombia but did not contemplate its sustainability. Main Challenges to Sustainability Small funding environment for GBV programs (Guyana) Absence of sustainability strategies in design and IP’s approach to SPD (Panama) Access to very small-scale funding opportunities (T&T) Lack of resources to keep experienced advocates and training survivors to become advocates (T&T) 27 WELCOME. La Casita has continued administering the WELCOME activity, at a small scale, after USAID’s funding and DI’s support ended in 2021. NORC’s Evaluation Specialist conducted an on-site visit, which indicated that La Casita, in addition to managing the WELCOME activity and its advocates, also provides preschool services for children of GBV survivors. For this initiative and to maintain a small-scale operation, La Casita relies on small donations from other organizations based in Trinidad and from social media apps, such as La Casita’s Facebook page, for funding the costs of managing individual cases of GBV survivors. La Casita organized a festival to sell prepared meals to benefit the organization in 2021, and a Spanish summer camp to charge TT$600 (approximately $88 USD) per participant child in 2022. La Casita also promotes donations through any available interviews in local media. The internal management of WELCOME also reflects the small-scale efforts of WELCOME and La Casita and its considerable limitations to sustain this effort. A respondent from Resonance that followed the grant closely said: “La Casita runs on an engine of a woman who with interest […] and sheer passion, which you can call sustainable, but I'm concerned that the level of interest and visibility she’s having with the community is going to overwhelm her, her current capacity, and resources.” [Resonance Staff KII] A DI staff member also expressed internal resources constraints and explained that La Casita needed further funding to train and build a cadre of advocates from the group of GBV survivors who received the WELCOME services in the past. At the time of data collection, the WELCOME advocates did not receive a salary from La Casita. The head of the organization provided a stipend to recognize the advocates’ time and dedication. The lack of resources to retain experienced advocates and train new ones threatens WELCOME’s sustainability. Initially, WELCOME tested the advocate role as a volunteer role for financial sustainability, however the role was too demanding to be sustainable without pay. REPLICABILITY, TRANSFERABILITY AND ADAPTABILITY: IN WHAT WAYS ARE THE AC’S REPLICABLE IN THE SAME CONTEXTS? ADAPTABLE FOR OTHER CONTEXTS? Respondents from Resonance explained that the case management systems that both BTG4VM and WELCOME built to refer their clients, either through the one-stop shop or the advocates, were a good example of a replicable component. With WELCOME in particular, the design was client-centered so that the advocate served as a means to access the services each survivor required. The case management approach customized each advocate’s approach according to the GBV survivor’s needs. The SPD workshops could be replicated in the same cities of implementation and transferred to other areas of Panama. However, respondents in Panama suggested targeting younger male cohorts to strengthen any future SPD’s impact in migrant communities. Strong demand for all three services remained, as activity participants and grantees agreed that there was a continuous need for support for survivors to access services. Further there was a persistent need for masculinity and gender equity workshops among Venezuelan males, especially those 25 years old or younger. It is important to highlight that a few activity participants in Guyana and T&T suggested the need for shelters where GBV survivors can live while their legal cases are taking place when they are in danger at home. A group of these participants and staff members emphasized this component should be ingrained in the client-centered approach because safety risks are very common among GBV survivors who migrated. They also indicated the need to adapt both programs on job training. This latter 28 topic is described below as part of the program adaptation strategies BTG4VM and WELCOME followed. BTG4VM. Resonance and NCC respondents believe the one-stop model is replicable in other areas of Guyana where Venezuelan migrants have settled. During 2021, although the activity´s target areas were Regions 3 and 4, BTG4VM also assisted residents from Regions 5, 6, and 10. Such demand for services could be addressed by replicating the services there. However, the activity´s work plan and the comments from NCC leading staff suggest that replicating or scaling up these services would involve intense coordination with local service providers. Such efforts are uneven across regions and strongly depend on the availability of reliable partners in each region. The experience that NCC gained with the first round of BTG4VM prepared them to engage with a provider ecosystem that had low technical capacity, similar to other services in the region. The model required engaging with local organizations that plan to be onsite in the long term so that they continually build capacity of new staff and weaker organizations. In some areas of implementation, the BTG4VM activity had to adapt its approach to the particular community and train a Community Support Officer. This was necessary in mining areas where Venezuelan women were partners or spouses of miners. These officers run safe houses where female clients spend some time while receiving assistance from the activity, when their safety is compromised at home. The Community Support Officers are focal points trusted by migrant women. 29 Exhibit 7. Which activity components do you think could be implemented in other communities in the country? N = 10 respondents in Guyana and 10 respondents in Trinidad and Tobago NCC also adapted the program design by adding services to help female migrants empower themselves through capacity-building. Two strategies implemented under IDB funding emerged from the interviews. The first one is professional development and job hunting. This assistance activity included “CV building up, writing an application, and how to present for an interview,” to be more effective when searching for jobs. NCC was also working with their collaborating service providers to organize skills-building training for female migrants and women who are GBV survivors. Concretely, NCC worked with a local 30 service provider to buy a recycling machine and obtain a space where participant women can recycle cardboard paper and produce goods the market demands, such as egg crates, paper cups, paper plates, etc. The second one was an effort to work with the host communities to build allies among Venezuelan migrants. This comment from a NCC staff members expresses the approach: “We felt it was going to be crucial to empower migrants more, something that increased the migrants’ ability to earn for themselves. And we also thought we would do some kind of periodic intervention that really brings migrants and host communities together while still doing everything that we’re doing. So those were two pieces we thought would have been crucial, like the entrepreneurship component and a cross cultural one.” [BTG4VM staff FGD] The web survey explored service providers’ perceptions on the replicability of activity components. As displayed on the left side of Exhibit 7 above, service providers in Guyana agreed with BTG4VM staff that the one-stop shop model and the links that unite partners under such model are elements that could continue being implemented in the same context. With IDB funding since 2022, these components continued implementation in Regions 3 and 4 and have been transferred to other contexts, including Regions 1, 7, and 8. SPD. Respondent IP staff agreed on the feasibility of replicating the masculinity workshops, using the 2021 design, in other cities in Panama, like Chiriquí, or even in other Central American countries. However, an SPD respondent recognized that the workshop influenced the attitudes of male participants differently depending on age. The experience in Panama suggests that the current methodology works better with participants between 18 and 25 years old. Thus, the workshops could be replicated in a series focused on that target population to attain stronger intended outcomes. Another limitation SPD respondents identified is the absence of field offices in additional areas of Panama where Venezuelan migrants have arrived, especially in the coastal city of Colón, in the Atlantic Ocean end of the Canal. Overall, respondents agreed that there was a demand for the workshops, both in the same cities and in new ones, including Colón and Bulgaria. HIAS technical experts added that a future version of the workshops should be open to Panamanian participants, too (host community members), not only migrants. They said such openness would have positive results in the insights men and their partners can have with the program and their own role in their partnerships, households, and communities. Several male participants agreed with their partners about the advantage of adding a job-skills component or to supervise a concrete activity in which participants can live the experience of what they learned. A program that supports migrants, such as SPD, should incorporate a practical component that helps build concrete skills. The “El Man de Hoy” mobile app is another element that SPD could replicate in other contexts. This app contains workshop materials in a format of daily tips for users. Replicating the app in other countries would require adjustments in language for countries such as Peru, Chile, or Argentina, where, for instance, there are different words to refer to a guy other than “man.” Replicating in other cultural contexts would also require adjustments to the content to make it relevant to both the migrant community and, especially, the host communities. Such updates, SPD respondents said, require labor and other costs that are not too high, but that are worthwhile investments to maintain interest in the app and gain more users. 31 Independent of the technical ability to replicate the model or an altered version of the model and the app, staff expressed funding constraints that limit their capacity to implement the workshops again and also expressed their willingness to apply for more funds, when those become available. A group of primary activity participants also mentioned the program should, at some point, cover areas that suffer high levels of insecurity, where organized crime operates more intensively, in particular over the coastal areas in the Atlantic Ocean. A male participant expressed this as follows: “To go the deepest neighborhoods where, right now, nobody can really visit due to fear or for being a red zone of crime.” [SPD activity participant KII] WELCOME. As noted previously, the WELCOME design was critically modified to adapt the intervention to the obstacles that female Venezuelan GBV survivors faced in T&T when accessing social services. Both Resonance and DI made substantial adjustments to design in Trinidad, and respondents from both organizations, especially Resonance, expressed praise and pride in the results attained: “It’s so, it’s so innovative. Yeah, we were really excited about this project, that was just mainly affected by external factors, that external factor being COVID, but that they completely redesigned their MEL plan. I mean they had to re-create a new result framework, set of indicators, and learning agenda.” [Resonance staff KII] Due to a change in design, WELCOME’s team had to continue adapting even during implementation. The main adaptations made during implementation were to identify, recruit, and train a male advisor for the male GBV victims La Casita started to receive early on. After finalizing and starting to implement the advocate model, the focus of the program was on learning from its new experiences, committing to grant accountability and obtaining alternative funding sources. Overall, the evaluation found no plans or intentions to plan for replicability of the model in any other site in Trinidad. The activity’s financial restrictions limit La Casita’s ability to plan for replicating the program elsewhere, as effective advocates require a significant amount of training hours, preferably paid. However, the advocate model has the potential to be used in other sites as the advocates acquired the skills to adapt to the legal and policy context, to help GBV survivors gain access to the services legally available to them, due to their migration status. As to the perspective of service providers, reflected in Exhibit 7 above, only a fraction of respondents indicated any WELCOME components that could be replicated. Among those, four consider that six activity elements were ready for replication: youth services, emergency shelters, services for human traffic victims, legal and health services, and sharing resources on social media. However, only two respondents selected the referral pathway. This perspective is convergent with the findings from the qualitative approach to WELCOME staff and GBV survivors. The financial restrictions of the activity have limited La Casita’s ability to pay the activity advocates. These mentors, who volunteered a substantial amount of their time during the grant, are the fuel of the intervention; without them and without training more advocates, La Casita replicate nor expand WELCOME. SCALABILITY: WHAT ASPECTS OF THE ACS ARE MOST AMENABLE TO BE SCALED UP? Resonance and IDB respondents described the ecosystems of social services in Guyana and Trinidad and Tobago as small and with low capacity to provide the services Venezuelan migrants need. They recognized that it is difficult to launch initiatives to support Venezuelan migrants in the Caribbean due to the limited capacity there. They praised the work both NCC and DI/La Casita have done in both 32 countries with the referral systems and emphasized the need for continuous capacity-building that NCC has been spearheading in Guyana and for the partnerships of La Casita in its locality. The IDB’s interest would be to fund initiatives in which multiple organizations work together in synergy to impact migrant communities at a larger scale and in the longer term. However, the IDB panel that participated in this evaluation provided no concrete plans for this in the region. Only one activity, BTG4VM, was able to scale up due to funding from IDB between 2022 and 2023. According to staff, BTG4VM expanded the one-stop shop to regions 1, 7, and 8. However, they continue to rely on service providers in regions 3 and 4, while establishing new partnerships in the new locations. This has overextended the original service providers. Meanwhile, SPD and WELCOME do not have available resources to scale up. While SPD staff expressed enthusiasm and interest in increasing the size of the effort and adapting it to improve its results, the organization did not have plans for replication or scaling up. WELCOME has downsized its operations due to limited funding. BTG4VM. BTG4VM secured two-year funding in 2022 from IDB to continue implementing the activity and expand its coverage from the Guyanese regions 3 and 4 to regions 1, 7, and 8. This expansion involved adapting the referral system of services to each new region. In this process, NCC worked with service providers that were part of the referral pathway system in the original regions to partner with them in the new territories. In the words of NCC’s Director, “What is needed is holistic implementation of the referral pathway […]. That is the juncture at which I think will really assist at a national level.” [NCC staff KII] To scale BTG4VM in three additional regions, NCC trained and onboarded two “case navigators,” who seem to serve a role similar to advocates for GBV migrants in the new regions of coverage. NCC respondents suggested that local capacity to operate the program in new regions is still a work in progress, and local staff had to make important efforts to assist GBV survivors in the new regions. The remote coordination of services has also been a challenge. In the new regions, there were periods of time when NCC had no safe houses to provide shelter to GBV survivors, so the team relied on hotels. After a year of implementing the expansion, NCC reported some improvements to implementation in the new regions, and its director suggested this experience will be helpful to continue expanding the pathway referral model through NCC’s regional partners in the Dominican Republic and Trinidad and Tobago.19 The providers of referral services for BTG4VM also indicated their perceptions on the services they could scale up. Web survey respondents mentioned that they would like to scale up the sharing the development and implementation of a one-stop-shop model for GBV services (6 respondents), and social media campaigns to increase awareness of xenophobia, GBV services, and alternate masculinities (6 respondents). The selection of a one-stop-shop model for GBV services is in sync with the plans that NCC has undertaken since 2022. SPD. Staff mentioned there was interest of the Ministry of Gender and some Police Authorities in adopting the SPD methodology for their own staff and, potentially, to deliver it at schools. However, HIAS had not been able to reach a concrete agreement with any of these agencies by the time of data 19 The respondent provided no information, however, regarding funding for these initiatives. 33 collection. Both male primary users and female partners of the program users identified demand for the workshops in neighborhoods and cities in Panama that SPD had not covered. These respondents emphasized the importance of using members of the migrant community as promoting agents to recruit participants and to work as coaches in post-intervention periods (this last point refers to sustainability above). These coaches should receive training and updates over time. This is something HIAS Panama has done successfully with other programs that targeted female migrants in the past. As part of scaling up, a HIAS respondent said it would be necessary to create a series of workshops that targeted male migrants by age cohorts. Working with male migrants over 35 years old would require adjusting the methodology and the workshop curriculum. This segmentation is seen as desirable because the dimensions of masculinity and men’s role in balancing the power relationships between genders change according to age. This comment captures the sentiment: “To a man between 26 and 30, or even 35 or older? No, we saw the workshops have no effect on them. It’s not the same a 55-year-old Venezuelan man that has been here for 25 years than a younger one who just arrived…. If we want to get stronger results, we should change the methodology and many things.” [HIAS staff KII] The HIAS team added that scaling up the workshops would require piloting this new series with a small group, based on age-segmented targets. Further, primary users and their partners agreed that scaling up the workshops would require more extensive advertising campaigns in conventional and social media to find the participants required to scale it up. In general, staff and participant respondents agreed on the need of and interest in these types of interventions among migrants. In this effort, a group of male participants said Panamanian men should also be invited to participate. This is consistent with the findings from staff regarding the importance of host communities to improve these approaches to masculinity and the role of men in strengthening gender equity. WELCOME. GBV survivors who participated in this activity identified the need for a larger coverage of WELCOME in Trinidad, including Chaguana, Port of Spain, and Sangre Grande. However, the ability of the advocate model to scale up is highly dependent on the availability of committed and well-trained advocates on the services to which GBV survivors have legal access, both conditions that were inexistent by the time of data collection. Additional concerns included the lack of funding to train and pay these advocates, and the limited media outreach to advertise the program. Neither DI nor WELCOME staff shared any concrete plans to scale up the activity. The providers of referral services for WELCOME also indicated their perceptions on the services they could scale up. Four web survey respondents said the top resources they would like to scale up were sharing information and resources on social media; three respondents mentioned services for victims of human trafficking; and three respondents mentioned emergency shelter services. Based on the responses from WELCOME staff and GBV survivors, the WELCOME program seems to not be at a stage in which La Casita can scale up its services further than what the activity has been able to attain after they received the GenDev funding. 4.2 CONCLUSIONS FOR THE BTC ACTIVITY CLUSTER EQ1: Are the activity clusters based on context-specific and international evidence? 34 What worked: Initial needs assessments were extremely beneficial to guide BTG4VM and WELCOME targets and content. Findings from the review of the needs assessments conducted by the different teams indicate that these were generally valuable uses of time and resources. For example, groups recognized the need for materials on alternative masculinities. This research period also helped inform the advocacy models used by several groups. Further, this period was necessary for several groups to shift their plans because of COVID-19. HIAS’ experience with the Positive Masculinities curriculum and their strong presence in the main Panamanian cities, at the neighborhoods with largest migrant density, were instrumental to inform the approach that resonated with male migrants and their partners. Challenges: For WELCOME, the COVID-19 pandemic forced a change of strategy to target participants—Venezuelan GBV survivors in Trinidad and Tobago—and thus, understand their needs and context. EQ2: To what extent are each of the activity clusters achieving the targeted GBV results? What worked: Overall, the outcomes of the cluster were obtained within the timeline of the Better Together Challenge and the MEL plans of each activity. While BTG4VM had no predetermined target of GBV survivors, it provided 48 of them with increased access to services and improved service coordination for GBV survivors in Guyana. The SPD program met its target goal of 165 male participants to increase awareness of GBV and masculinity while building community ties between Panamanians and Venezuelan migrants in Panama. DI and La Casita redesigned, developed, and adapted WELCOME into a successful client-focused model based on advocates to handle 30 cases of GBV survivors with a customized strategy, according to the individual’s support needs. Customized approaches and direct services, including one-stop-shops and advocate-centered assistance are important support approaches for GBV survivors, particularly when programs aim to include all genders. Most findings indicated that advocates and other support approaches tailored their case management to meet the individual client’s needs. In WELCOME, for instance, DI indicated that men and LGBTQIA+ groups experiencing violence also needed support, and services needed to be designed to meet their specific needs and circumstances. Additionally, including men in GBV programming is generally desirable and welcome by men and women. A valuable finding from these ACs is that both women and men appreciated efforts to include men in GBV “masculinity” activities to identify more equitable gender roles in the home. Moreover, changing attitudes about migrants is possible. Most of the service providers that responded to the web survey for BTG4VM believed the social media campaigns shifted views about migrants. Further, participation in mixed workshops with migrants and non-migrants (SPD) seemed to build trust between individuals. Challenges: Financial concerns at the migrants’ homes are often a priority and can hinder participation, especially among male migrants. Not surprisingly, for migrants, income source is a common concern that organizations must consider when designing GBV programs. Migrants are likely to prioritize activities that help them generate income over psychosocial or counseling support if the solutions only include those stand-alone components. In older male participants, attitudes and behaviors towards gender roles resulted in more rigid and overall long-term attitudes to GBV that promote its perpetuation. Further efforts of incorporation of 35 these initiatives into policies in the public and private sector continue being crucial to strengthen positive messages to build allies of gender equity, especially among men. Overall, while performance monitoring strategies seemed to provide limited benefits to the programs for adaptation and learning, grantees were more likely to use these strategies for reporting purposes. Most findings suggested that data from monitoring exercises did not necessarily inform program adaptations. Data was used primarily for quality assurance or KPIs. One grantee noted that they used monitoring data to revise their online form and calling protocol and had regular advocate staff meetings, which informed their client approaches. Further, most service providers were likely to report that they did not have access to the monitoring data. EQ3: To what extent are the ACs sustainable? What worked: Cross-organization coordination and service networks can foster better access to services and collaboration between partners to maintain an active network of services and promote sustainability. Through their activities, several organizations built relationships with other service providers and established stronger referral processes that helped GBV survivors access the services they needed. Some of these relationships are likely to persist beyond the grant period. NCC was able to secure IDB funding for two additional years for the BTG4VM program and expand the one-stop shop model from two to five Guyanese regions where migrants have settled. While IBD respondents did not confirm the reason why the additional funds were granted, since the start of BTC, NCC implemented two BTC activities, one directly funded by IDB. This could have been one of the reasons BTG4VM secured additional funds. Resonance was custodian of these innovations, and their staff witnessed BTG4VM’s and WELCOME’s contributions to strengthen the local capacity on GBV and the enrichment of the social service ecosystem. Challenges: Short-term funding generally precluded project sustainability, especially for small organizations. The timeframe and scope of the funding made it difficult for groups to root their activities as sustainable practices. This form of funding did not leave sufficient time for groups to embed their new practices, for example, by meaningfully instituting practices within the community, building sustainable partnerships to garner ‘strength in numbers’ or raising further funds to continue the activities. Moreover, not uncommonly, when activities are built from limited funding, they often rely on ‘volunteers.’ In such cases, donors, implementation partners, and grantees must take one additional step during the design stage to assess whether the scope of work and the pace of the task can be successfully accomplished in a volunteering role. Without such assessment, large caseloads and fast pacing of cases can lead to quick burnout and consequently, cause high turnover and regular lost learning. In the case of SPD, the program was not able to build skills or strategies that would be sustainable. SPD has no plans for new implementation in Panama, despite the high interest and need among Venezuelan migrants. On the other hand, for WELCOME, La Casita continues its operation, at a considerably smaller scale, thanks to receiving funding from the DI to continue the work of the WELCOME program as prime. While IDB funded the extension of BTG4VM, SPD and WELCOME only implemented one BTC grant each and had no interaction with any other BTC funders. 36 Both funders, USAID and IDB, explained that the local capacity in the Caribbean is still relatively low and reliable partners are scarce, making it difficult to scale up to meet the needs of large and growing numbers of Venezuelan migrants. 4.3 RECOMMENDATIONS FOR THE BTC ACTIVITY CLUSTER ● Capacity for Sustainability. Donors need to consider the implications of short-term funding for sustainability, especially for small organizations. During design and implementation, implementing agencies might want to consider approaches that will foster greater sustainability for the program, such as integrating a network of partners into the project, building fund-raising skills, or securing project coordination with larger agencies. For instance, while the BTC included sustainability indicators, the evaluation found small capacity to procure funds and make the activities survive and thrive after the USAID funding ended. ● Temporary Refuge for Legal Services Users. Due to frequent intimate partner violence Venezuelan migrants experience, some GBV survivors shared that these interventions should provide refuge or temporary housing to legal service users that need legal protection from their partners. This would decrease the danger of living in the same place with their partners. ● Additional Psychosocial Support Sessions. Based on comments from those using these support services, the design of future activities should include additional sessions with a psychosocial support specialist since opening these channels of communication takes time, and each case is different. ● Address Migrants’ Transportation Needs. Due to migrants’ financial and logistical difficulties related to commuting, if groups want migrants to be able to join the activities, they will need to consider ways of reaching them without costs or hardships for the migrants. For instance, it may be possible to schedule activities for groups of women living in the same community either onsite or offer them transportation to the site and back. Moreover, traveling together will make their journey safer. Some male program users said that male workshop attendees living in “red zone” areas of crime would also benefit from providing a means of transportation to travel to the workshop site and back. ● Specific to SPD. The workshop on masculinities opens the channel to new questions and psychosocial needs in male participants, particularly the youngest ones. For that reason, an ideal path for psychosocial support should include referrals to support groups to improve self￾esteem, work on trauma healing, provide therapy for couples—and also to prevent and heal sexual child abuse. ● Strengthening future GBV programming. The evaluation team identified several network￾building activities that could be integrated into proposed activities, which may both improve services for migrant GBV survivors and perhaps achieve more sustainable activities. NORC recommends that in future programs, USAID encourages groups to integrate strategies that strengthen local alliances between service providers or promote a network of services for GBV survivors, which would include, for instance: 37 o Housing. Include options for medium- to long-term housing to support migrants to become homeowners rather than tenants. A large proportion of migrants’ income is allocated to rent, which prevents them from settling at a property they own for long periods of time or for the rest of their lives in the host country. o Occupational training, entrepreneurship skills, and livelihoods assistance. GBV survivors indicated that GBV prevention activities were useful but also incomplete. A majority of user respondents and some implementation staff suggested that future GBV activities should include job-training, entrepreneurship, and livelihood components. Support to promote income-generating activities is not only a priority for most migrants, but improved financial security may also be protective against GBV in the host countries. Skills that users mentioned to be relevant in the local labor markets include: barbering and beauty shop skills and business planning to open a nursery as well as to prepare and sell meals at a small and medium scale. 5. IMPLEMENTATION EVALUATION FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 5.1 FINDINGS FOR THE IE OF THE WELCOME ACTIVITY This section responds to the evaluation questions on implementation for the WELCOME activity. EQ1: IS THE ACTIVITY DESIGN BASED ON THE LOCAL CONTEXT AND FLEXIBLE TO ACHIEVE RESULTS ON THE GROUND? DESIGN: WHAT FACTORS CONTRIBUTED TO THE DESIGN OF THE ACTIVITY? HOW WERE PRIORITY GBV INTERVENTIONS IDENTIFIED? DI designed the WELCOME activity to respond to GBV in Trinidad and Tobago, particularly GBV against Venezuelan migrants. Because of COVID-19, in June 2021, DI pivoted the WELCOME activity from addressing sexual harassment in public places and instead aimed to reach women experiencing violence in their homes. Originally, DI planned to use a behavioral science-based approach to stop harassment in grocery stores using the model of “Supermarket Superheroines.” After abandoning this model, DI then planned to address harassment in bars. However, DI ultimately decided they wanted to address violence against women in their homes instead of in public. Staff recognized they could not go door to door to reach women but needed a way to safely interact with them. This led them to partner with La Casita Hispanic Cultural Centre. The WELCOME baseline assessment and FGD and KII respondents discussed the prevalence of sexual violence against Venezuelan women, highlighting intimate partner violence and abuse in the home. Baseline evidence suggested that after seeking assistance, Venezuelan GBV survivors have many challenges to follow service referrals. Barriers hindering further assistance include competing priorities to meet basic needs for themselves and their families, language barriers, low self-efficacy and/or fear and distrust of the government and unfamiliar organizations. Thus, the WELCOME advocate design aimed to help Venezuelan women experiencing GBV access support. 38 Exhibit 8. WELCOME’s Advocates Model Exhibit 8 summarizes the advocate model, drawing on WELCOME’s workplan, information from the qualitative data collection, and data collected during visits to La Casita. The intervention development process was iterative, consisting of biweekly calls or visits between survivors and their WELCOME advocate. According to advocates, they followed their survivors for several weeks and sometimes months after their official advocacy work ended. They used short calls and mostly WhatsApp messages to continue checking in and learning about survivors’ life milestones, including new jobs, news about their children, etc. During the assistance period, advocates said they followed up with survivors mainly in the form of telephone calls, visits, telephone interpreting services, and accompaniment to police stations. Advocates assessed which of the services that La Casita was not able to provide themselves, such as legal or food assistance, and coordinated among the network of service providers to assist the survivor. The most common services to which La Casita made referrals were legal assistance and food assistance. In the web-based survey, service providers that partnered with DI and La Casita were asked what are the top three resources, services, or programs that are needed the most by the migrant community to combat GBV and to what extent the WELCOME activity and its network of service providers were able to provide these services. As seen in Table 10 below, six respondents selected psychosocial support services, five selected housing shelters and safe spaces, four selected legal assistance for protection orders and economic empowerment services, and three selected legal assistance for police reports. However, the extent to which these services are provided varies substantially. 39 Table 10. Top three migrant community needs to prevent or combat GBV (Count) TOP THREE MIGRANT COMMUNITY SERVICE NEEDS SERVICE PROVISION Psychosocial support services (6) Somewhat or to a great extent (5) Don’t know (1) Housing shelters and safe spaces (5) Very little or not at all (2) Don’t know (3) Economic empowerment services (4) Somewhat or to a great extent (2) Don’t know (2) Legal assistance for protection orders (4) Somewhat or to a great extent (2) Very little or not at all (1) Don’t know (1) Legal assistance for police report (3) Somewhat or to a great extent (1) Don’t know (2) Health care services (2) Don’t know (2) Referral pathways (1) Don’t know (1) Legal assistance for custody matters (1) Very little or not at all (1) Institutional accompaniment – advocate work (1) Very little or not at all (1) Hotline services (1) Somewhat or to a great extent (1) Other (specify) (1) Somewhat or to a great extent (1) Source: Web-based survey by ET. KEY IMPLEMENTATION METHODS: WHAT ARE THE KEY IMPLEMENTATION METHODS TO ACHIEVE OBJECTIVES? The WELCOME activity’s main components included: recruiting and training advocates, matching advocates with survivors, advocates supporting survivors, and local capacity building. Both survivors and staff said the WELCOME advocates were well trained and responsive to the needs of survivors. Before WELCOME, most of La Casita’s success in advocacy work hinged on the role of the director as the primary advocate. Staff noted that the most effective method WELCOME implemented for the protection approach of the project was having advocates, especially the ones that were bilingual, bridge the gaps between survivors and the services they needed. Survivors also noted the usefulness of advocates who could translate for the GBV survivors. Besides having a larger cadre of trained advocates, another component of WELCOME’s success in reaching the migrant community was media coverage and networking that La Casita’s Director undertook with other organizations to spread the word about the services. Its director, a Venezuelan national who migrated due to the difficult life conditions in her native Venezuela, cultivated long and tight relationships with police agents and other authorities since she arrived in Trinidad and Tobago. For over a decade, she had acted as a personal advocate of Venezuelan GBV survivors to obtain police protection for and children custody on behalf of Venezuelan GBV survivors. Another method that facilitated implementation was a formalized referral processes for survivors. Both survivors and staff confirmed that WELCOME provided safe spaces for survivors and a variety of 40 support options, including food assistance, psychosocial support, or help accessing employment opportunities. One survivor described the effects of the services, “I was able to take advantage in some way of the benefits that the program gave me, because not only did it help me [with] my assistance with the psychologists [and] help me deal with the process of what I had lived through as a result of the aggression, but also, I was able to open up with things that I needed, like healing in myself, in my previous life. So, in a way the program helped me to feel supported, to feel that I was not really alone in this country. And well, here I am whenever I can, I stop by.” [WELCOME GBV survivor KII] Further, service providers who partnered with DI/La Casita reported the services they provided to GBV survivors at the time of data collection, as displayed in Table 11. Most provided counseling and psychosocial services, while only one provided training on safety protocols. Table 11. Services available for Venezuelan migrants who are GBV survivors SERVICE PROVIDED RESPONDENTS Counseling and other psychosocial services 6 Services for refugees and asylum seekers 3 Services for victims of human trafficking 2 Advocate training – Information on GBV and its context in Trinidad and Tobago 4 Advocate training – Survivor and Advocate safety protocols 1 Advocate training – Building capacity to provide psychological first aid 2 Advocate training – Survivor referral information and processes 2 Advocate training – Self-care 3 Developing referral pathways 3 Sharing information and resources on social media 4 Additional Training of Trainers 3 Knowledge and expertise on data collection tools 3 N = 10 respondents. Respondents could select multiple response options. Survey item asked: “Indicate what services are available for Venezuelan migrants who are GBV survivors in your community, including those that you or your organization provides”. Source: Web-based survey by ET. WELCOME staff also shared the main challenges to implementation. One most frequently mentioned was learning how to interact with service providers, especially the police and hospitals. Often, Venezuelan migrants were turned away or ignored at the police department if they went to report an issue or would receive inadequate care at the hospital. It took advocacy and training sessions for police officers and hospital staff by La Casita staff to address this issue and encourage cooperation. Other issues were varying language abilities and burnout among overworked advocates. Not all WELCOME advocates were fully fluent in Spanish, creating some barriers to communication with survivors and efficiency managing cases. WELCOME then made an effort to recruit advocates who were fluent in Spanish, and staff emphasized the importance of this in KIIs. 41 Staff suggested that a remedy to burnout could be higher pay and making the position of advocate into a more official role, so advocates did not feel they had to choose between their own full-time jobs and the work with survivors. Advocates worked as volunteers and received only a small stipend to thank them for their work. Survivors reported that staff availability affected the services they received. A couple of examples illustrate this challenge. In one case, a WELCOME advocate who was supposed to accompany a GBV survivor to the police did not show, and she had to be assigned a new advocate. In another, the survivor said she never received follow up on psychological services, and another mentioned the same for legal services. In the Final Report, DI recommended that future implementers train a larger number of advocates from project onset to fill a backup roster and consider making the advocate role full time with a salary. FLEXIBILITY: IS THERE SUFFICIENT STAFFING TO RESPOND TO LOCAL PRIORITIES: IS THERE FLEXIBILITY TO CHANGE APPROACHES TO RESPOND TO LESSONS AND CHANGING CHALLENGES IN THE LOCAL ENVIRONMENT? After the redesign of the activity in March 2021, its staff became more capable of adapting the program to respond to contextual changes or challenges. However, they noted that they had not considered the need for a male advocate. At the beginning of the implementation, male GBV survivors started asking for support, which put pressure on advocates that was not anticipated by decision-makers. Once DI/La Casita identified this need in the target population, DI staff were able to recruit and train a male advocate for the WELCOME team. Grantee staff explained, however, that although personnel and resources were sufficient at the beginning of the project, they were unable to change and adapt to the growing reach of WELCOME. Staff said that with more financial resources, this issue could be resolved. La Casita also developed a job referral and training network. Such additional services arose because survivors and advocates expressed a desire for employment, and financial assistance and accessible services for men. Moreover, advocate and survivor interviews made clear there is a substantial desire by survivors for employment and financial assistance. Staff explained that in response, the team began to develop a job referral and training network, which was not included in the original advocate model. La Casita was able to connect to other CSOs and companies in the area that were hiring and refer survivors to available job opportunities. During 2021, because of the COVID-19 pandemic, WELCOME needed health safety precautions. Thus, a number of services and meetings between advocates and survivors were scheduled remotely via Zoom or similar platforms. No staff or participants mentioned this impacting participation. EQ 2: IS THE ACTIVITY REACHING PARTICIPANTS THEY ARE MEANT TO TARGET? TARGET BENEFICIARIES: WHAT ARE THE BARRIERS TO REACHING BENEFICIARIES? Though the initial target beneficiaries for the WELCOME activity were migrant Venezuelan women who were survivors of GBV in T&T, according to KIIs with staff, the project adapted to include male Activity Adaptations (during Implementation) Recruiting and training a male advocate for male GBV survivors Job referral system and job training network for GBV survivors Remote advocate meetings with survivors during pandemic 42 survivors, as well. Staff indicated that the project reached groups in addition to Venezuelans and supported Cuban and Dominican female survivors as well. Staff also indicated that migrants can be hard to track down and may work unusual hours due to their immigration status and formal or informal recognition of their skills. Not all WELCOME staff had previous experience working with migrants, so this presented an additional learning curve and effort for the program. Survivors who participated in semi-structured interviews said they wanted the project to assist more beneficiaries in the future. Some survivors thought that WELCOME needed a better awareness campaign for their services. Interviewees mostly cited a lack of awareness among the migrant community as the reason why certain participants did not engage in the services. A number of survivors found out about the services via word of mouth from a friend, family member, or someone in the community. An important challenge the activity staff encountered with the targeting strategy was communicating with victims safely. Sometimes survivor phones were in possession of their abusive partners or there were not good times during the day for the GBV survivors to talk to an advocate. In response, WELCOME staff took extra precautions to prevent increasing any risks for survivors through their communication. This entailed working closely with survivors to predetermine communication mode and timing for their next follow-up, and not giving any information on the cases except in confidence with the survivor. In the final report for WELCOME, DI noted barriers to effectively reaching beneficiaries. These included the location of services, the disruption of COVID-19, scheduling conflicts with GBV survivors due to work and other responsibilities, lack of standard operating procedures at La Casita, language barriers between English-speaking advocates and Spanish-speaking survivors, and initially not having male staff. When asked in the web-based survey whether anyone in their community should have received services from WELCOME but did not, only two out of ten WELCOME service providers indicated this as an issue. The survey of service providers asked them about the demographic groups they serve, the services each group needed, the adjustments they made to their services according to each target group, and whether they allocated additional resources to make such adjustments. Overall, service providers indicated all seven target groups of Venezuelan migrants identified in the survey needed the following services: counseling and other psychosocial services, services for refugees and asylum seeker (i.e., resettlement, temporary housing, English lessons), services for victims of human trafficking, and services to prevent or protect from GBV. Moreover, these survey respondents said all groups, except for males under 18 years old and non-binary individuals needed legal services. Service providers expressed that only male minors, adult women, male members of the LGBTQIA+, and migrants self-identified as non-binary needed shelter services in the communities they serve. And interestingly, these respondents only identified a need for healthcare among females under 18 years old and both males and females who identify as members of the LGBTQIA+ community. These responses may reflect the experience that these service providers had with Venezuelan migrants in recent years. Table 21(see Annex E) offers further details on target groups –including both Venezuelan and Trinidadians–, their identified needs, as well as to what extent the partner organizations adapted their services, and whether they allocated extra resources to them. 43 In the GBV survivors’ perspective, WELCOME’s psychosocial services and the healthcare received through the network were highly appreciated. However, several users said WELCOME should have offered in-person sessions with counselors rather than remote consultation. Usually, the quality of their wireless services made remote interaction cumbersome, if not impossible when the service was off. Some users also expressed their preference for a more personal connection with their counselors in a personal visit. Another area that was not fully addressed was the advocates’ ability to understand and follow up on the recipients’ legal cases. This comment from a GBV survivor explains that trauma is an obstacle to making good legal decisions in the aftermath of a traumatic experience. Trauma can influence survivors’ perspectives about themselves and their relationship with a GBV perpetrator. In some cases, those influences can lead survivors to make decisions that do not benefit them, like for example, staying with their intimate partners and continuing suffering violence. This suggests that GBV survivors need advocates to make sure lawyers and psychosocial counselors in charge of their cases exchange and propose beneficial options for the activity users: “They would have been able to help me more because they did not understand what I was going through; they thought I just did not want to leave that house, and they closed my case. But it was not like that, the psychological damage I have was deep and nobody has helped me overcome it. Even if I want to get out of there, I can’t.” [WELCOME GBV survivor KII] While WELCOME provided legal support and food assistance, the advocates were only able to show a proactive lead and engage more actively at the beginning of the work on each case. Several GBV survivors described how advocates did not follow up on the received services. For instance, a GBV survivor who received assistance to file a complaint with the police never received a resolution of her case because the advocate did not have access to the legal information. “I would have liked that they had information about my case with the police. […] They never told me anything else and why they didn’t I never got it. And I did not want to insist”. WELCOME did not design strategies to make sure all advocates were able to conclude all the cases, whatever services included in them, with concrete results for GBV survivors. Advocates should provide the status of cases to GBV survivors, including instances where authorities have not provided resolution, and pursue other avenues. Additionally, some GBV survivors that participated in WELCOME also mentioned their children needed health care, a service that WELCOME did not provide. Exhibit 9 below displays, in each line, the main groups of Venezuelan migrants the WELCOME service providers target, either through WELCOME referrals or as part of the services they deliver in general. Both charts refer to the same target groups. The top chart suggests most of the surveyed service providers had to adjust their service to meet the needs of each specific target group. The counts are particularly high among those who referred to services targeting female minors (n=10) and adult women (n=9). Most service providers targeting male minors and female members of the LGBTQIA+ community (n=8) also adjusted services. While one fewer count, services for non- LGBTQIA+ and LGBTQIA+ males also said their services adjusted to address user needs. 44 Exhibit 9. Tailor-made Approaches and Allocated Resources by Group of Service Users (Service Providers) Note: Numbers in individual axis legends reflect the number of respondents who indicated that response option. In some cases, not all respondents selected that response option (e.g., occasionally n=10 vs. n=6) 45 The bottom chart displays the distribution of service providers according to the additional resources they allocated to serve the target group of Venezuelans. We see some consistency between the adjustments of services and allocating additional funds somewhat or to a great extent to serve female minors (n=6) and adult women (n=8). Half of survey respondents (n=5) said they allocated additional resources somewhat or to a great extent for services targeting female members of the LGBTQIA+ community. Services targeting non-LGBTQIA+ men also consumed additional resources, according to five respondents. However, there is no consistency between both charts when it comes to services targeting male minors, for which adjustments did not necessarily involve allocating additional resources somewhat or to a great extent (n=3); most respondents allocated very little or no additional resources (n=6) to services that target male minors. Further, while most service providers targeting LGBTQIA+ men said they adjusted their services, the number of those who said they allocated additional resources somewhat or to a great extent is the same as those who allocated very little or no resources (n=4). Services targeting non-binary Venezuelans were fewer in general (n=6) and most of them also adjusted their services. However, not all respondents that adjusted their services for this group allocated additional resources too, only three did it somewhat or to a great extent. MONITORING OF RESULTS WELCOME respondents reported they trained advocates to write comprehensive logs to monitor results. This strategy did not use USAID’s key performance indicators (KPIs), but rather involved documenting with narratives of every advocate check-in session with survivors, which occurred every two weeks during implementation. Some GBV survivors explained that during these meetings, staff were interested in feedback on how the program could improve. The narratives, according to DI staff, were easier and more familiar to handle for the advocates and La Casita staff. Thus, DI agreed with La Casita to use this monitoring strategy and the translated narratives in English, to comply with the activity's MELP. One staff member described how the advocate logs for WELCOME were adapted from La Casita’s Director’s regular intake process: “She had her intake form. She had a grid like a spreadsheet to track what she was doing. So, in terms of monitoring, it was more that the data gathering helped you monitor. One of the things we did was to prepare an online survey so that advocates could log. We created everything to be used online so you could fill out the form on your phone. And then when you fill it out, it automatically got stored.” [WELCOME staff KII] Another staff member discussed the monitoring process and how data from the advocate logs were used to inform the WELCOME program interventions: “So, we would go first to the case manager, and the case manager passes it to us. We do the first call to the victims, and we have a conversation with them. Then we pass that information to the case manager through logs. Through the online program, we help the victims to make police reports or to file a hospital report, depending on the case. And then we pass this information to the case manager. She is also in charge of calling the victims and following up with us.” [WELCOME staff KII] Overall, La Casita’s Director and activity staff expressed their satisfaction with this monitoring system based on logs of biweekly check in visits with GBV survivors. The Director expressed the activities’ ability to react to these logs and adapt immediately when it was required. For example, advocates felt 46 very uncomfortable about sharing their personal phone numbers with survivors for security concerns, so this comment from a WELCOME staff member encapsulates the adaptive experience: “Through [the logs], we were able to track to learn more in real time like what is going well, what is not going well. It was just really important for us to have that kind of immediate feedback. And then we were able to adapt.” [WELCOME staff KII] USAID and Resonance required DI/La Casita to submit data every quarter. After the redesign of the project to the advocate-centered model, the WELCOME’s MELP and staff said that there were no modifications of the performance indicators. WELCOME translated all advocate logs from Spanish to English so that DI staff could also monitor implementation. EQ 3: IS THE ACTIVITY ACHIEVING SUSTAINABILITY? Staff and survivors alike commented on the need for more funding for the sustainability of the WELCOME Activity. Initially using volunteers was intended to be a financially sustainable strategy, however it became clear that more funding would be necessary to sustain the work of advocates. Both staff and survivors also noted that they would like to see geographic expansion of WELCOME in migrant-dense areas of Trinidad, so more survivors can access support services. La Casita staff were applying for new funding and managed to be listed as a UNHCR potential grantee for future funding opportunities. However, they have not secured continued funding for their activities yet. La Casita staff explained, “I think that if we get to work together again, the budget should be a little broader to give us that expansion, to be able to give us the ability to have enough staff that I can dedicate myself or have a specific person who is in charge to train and to know what needs to be done to continue with sustainability” [WELCOME staff KII] Other staff members discussed the need for more funding and time for implementation, which would support sustainability of the activity: “Somebody needs to give her [the La Casita Director] money to implement for like a five-year period with the right amount of staff. She needs staff at La Casita, and she needs a proper advocate pool. Then you can take the time to train people properly. You can even start building your cadre of advocates from survivors. You can start doing courses for survivors. You can change the way the daycare works for the survivors so you can do so much more. If you had a longer implementation period and way more money.” [WELCOME staff KII] Some FGD respondents emphasized they would have liked to collect more data to attain more conclusive findings that would demonstrate the success of the approach. Proving a successful approach would help the organization secure more funding, contributing to more sustainability. There were several indications that the WELCOME model might be sustainable, including improved coordination among service providers. According to staff and advocates, La Casita plans to formalize and standardize its relationships and operations with service providers so it can link survivors to services more effectively. In addition, according to both interviews and the WELCOME Final Report, DI supported La Casita to standardize and codify written policies and standard operating procedures. 47 Some service providers in WELCOME’s referral network expressed the status of their own services. Respondents were first asked to select all the services they are currently providing in coordination with the WELCOME activity and then indicate the existing status of service provision—if it is still being provided with WELCOME, if it has been suspended, if survivors are referred to another service provider or any other status. According to the survey results in Table 12, only a fraction of service providers is currently providing services with WELCOME, especially counseling, and other psychosocial services and working on developing referral pathways. Also, several have suspended services. Table 12. Status of Service Provision (WELCOME Referral Network), Count Service Continues with WELCOME Suspended Referral to another provider Other Don’t Know Counseling and other psychosocial services (6) 4 1 0 0 1 Services for refugees and asylum seekers (3) 2 0 0 1 0 Services for victims of human trafficking (2) 1 0 0 0 1 Advocate training – Information on GBV and its context in T&T (4) 1 3 0 0 0 Advocate training – Survivor and advocate safety protocols (1) 0 1 0 0 0 Advocate training – Building capacity to provide psychological first aid (2) 0 1 0 0 1 Advocate training – Information on migrant rights (1) 0 1 0 0 0 Advocate training – Survivor referral information and processes (2) 0 1 0 1 0 Advocate training – Self-care (3) 0 2 0 0 1 Developing referral pathways (3) 3 0 0 0 0 Sharing information and resources on social media (4) 2 0 0 1 1 Additional Training of Trainers (3) 0 2 0 1 0 Other, specify (3) 0 0 0 2 1 Note: Number of respondents that mentioned each service provided in parentheses. Total respondents: 10. Respondents from these partner organizations expressed they would like counseling and other psychosocial services (7), establishing a trusted referral network (6), and health care (5) to continue being available in the future. 5.2 CONCLUSIONS FOR THE IE EQ1: Is the activity design based on the local context and flexible to achieve results on the ground? What worked: The needs assessment WELCOME conducted proved particularly useful because they used the findings to shift their design from an approach that promoted accountability of perpetrators of sexual harassment in public spaces to a client-centered initiative that customized their advocate techniques to meet client needs. DI and La Casita demonstrated flexibility and adaptability to meet the 48 needs of migrant Venezuelan women in Trinidad & Tobago under the redesign initiated because of COVID-19. They redesigned their initial plans and worked to develop the local partner’s capacity (La Casita) to deliver a client-centered, tailored advocate model. Challenges: Because of the COVID-19 pandemic, the original design to work in public spaces was no longer feasible. Once redesigned, WELCOME staff had difficulties gaining the cooperation of certain service providers, such as police officers and hospital staff. Moreover, La Casita advocates were limited in their capacity to take on several cases due to the volunteer nature of their positions. EQ2: Is the activity reaching beneficiaries they are meant to target? What worked: The WELCOME program was successful at reaching Venezuelan migrant women who are survivors of GBV, who mostly found out about the program through word of mouth. In addition, the WELCOME program was able to reach Cuban and Dominican female survivors as well as some male survivors, all of whom were originally not targeted by the activity. Challenges: Feedback from staff and survivors indicates that the WELCOME activity has room to reach numerous beneficiaries. Survivors suggested expanding awareness campaigns to be more inclusive. There were some difficulties reaching beneficiaries because of scheduling, language barriers, COVID-19, and weak standard operating procedures. Additionally, staff had to quickly adapt to include support for male survivors of GBV, who were not originally foreseen as part of their client-base. EQ3: Is the activity achieving sustainability? What worked: DI and La Casita used GenDev funds to improve relationships with local organizations that provide critical GBV services, built local capacity for case management, grew their network, and developed a referral pathway, based on their client-centered advocacy approach. Challenges: While their activities improved coordination, management, and relationship-building, La Casita’s reach, sustainability, and potential scale-up were limited by funding challenges. Funding opportunities have been scarce in Trinidad and Tobago, according to the director and other staff. The evaluation found that sustainability strategies are part of the key performance monitoring indicators and Resonance supported La Casita to become listed as an UNHCR’s potential grantee. At the same time, the need for services by GBV survivors expands, including the need for shelter and job training, psychosocial support, police and legal assistance, and health services. The program’s existence relies heavily on the willingness of La Casita’s Director to continue helping Venezuelan women in Trinidad. For a while, funding and the continuation of La Casita’s work was uncertain, but for now, La Casita has received a prime award to continue providing services and is seeking out more sources of funding. 5.3 RECOMMENDATIONS FOR THE IE EQ1: IS THE ACTIVITY DESIGN BASED ON THE LOCAL CONTEXT AND FLEXIBLE TO ACHIEVE RESULTS ON THE GROUND? ● Continue to encourage and fund initial needs assessment stages so that groups have time and resources to redesign proposed activities to meet current context-specific 49 needs. It will be of regular benefit for donors and recipients alike for donors to allow sufficient time and funded specialist intervention research expertise so groups can consult with proposed project beneficiaries, local stakeholders and examine relevant international practices. This early intervention-focused and context-specific research can make the difference between an effective activity versus a missed opportunity. Initial needs assessments also motivate activities to think evaluatively and can serve as baseline information to compare with data collected during and after implementation. ● Carefully reassess the role of the advocate and the scope of work required to fulfill the expectations of this position for a fair remuneration. Underpaying case managers and advocates to undertake important work, like supporting violence survivors, severely undervalues their time and the work that they do. Underfunding their labor leads to high turnover and poor program sustainability. Program funders, implementers and grantees should agree on either a scope of work that a volunteer could achieve or create part- or full-time basis positions to avoid burnout and shortfalls in advocacy follow up and successful closure of cases, at least from the advocate’s side. ● For advocate-model interventions, include strategies to engage with local service providers, train them in MEL practices, and spearhead fundraising efforts with common incentives to collaborate. To cultivate strong relationships with local partners and service providers, funders and implementation partners should promote and label funds to identify relevant local actors as well as their strengths, limitations, needs, interest, and leverage points with local authorities and other decision makers. From this analysis, which ideally should be part of the baseline assessment, the activity team could identify the strategies to promote collaboration, identify comparative advantages and areas in which partners complement each other. With this strategy, the grantee can foster smoother transitions from the advocate to the actual services and follow up on their needs in the medium and long run. ● Promote participatory and co-produced intervention development. This worked well in the case of BTG4VM and to some degree with WELCOME. Given the benefits of early needs assessments and consultations found in other similar approaches, donors should provide adequate funding and technical support for groups to engage and pay beneficiary representatives to undertake intervention co-development processes. Tools such as user- or human-centered design of activity components can help generate well-targeted and more effective interventions that are informed by actual users. The experience with WELCOME showed that GBV survivor involvement is necessary to improve the advocate model, so that these advocates can gain a more holistic approach to their users’ needs. These include a non-linear approach to psychosocial support and its implications on legal, job training, and other relevant components. EQ2: IS THE ACTIVITY REACHING BENEFICIARIES THEY ARE MEANT TO TARGET? ● Promote migrant and non-migrant inclusive engagement. One important lesson from this activity was the potential value of bringing together migrant and non-migrant groups. USAID and other donors should make further investment in joint migrant/non-migrant programming and corresponding research to understand the potential added value of breaking down barriers between groups and find common spaces of growth and incorporation into the host community. 50 For further detail on the sources of this recommendation, see section 5.1 under Key Implementation Methods. ● Promote gender-inclusive programming. Future programming should be designed to ensure providers are prepared to respond to the needs of female, male, and gender non-binary GBV survivors. Program models will need to consider the different ways survivors might learn about services, decide to access them and how they want to engage with case workers. Initial research and ongoing monitoring should be designed to identify accessibility, acceptability, and emerging effectiveness of service provision by gender and other influential factors, such as language, ethnicity, migration status, cultural background, education level, and socioeconomic status. For further detail on the sources of this recommendation, see section 5.1 under Flexibility. ● Improve strategies to ensure wide awareness of and easy access to services. Findings indicated that word of mouth was a common way survivors learned about services. If services are capable of managing a substantial caseload, future activities should allocate funds for grantees to investigate the various ways survivors might learn about their services (e.g., beyond word of mouth), and use such channels to increase awareness, without compromising the safety and security of GBV survivors or others potentially seeking GBV services. Similarly, because it is not uncommon for migrants to have difficulty navigating and freely moving in new locations, especially if they cannot leave job sites, etc., organizations should consider different modes of assistance, including the possible range of remote sessions and mobile technology. EQ3: IS THE ACTIVITY ACHIEVING SUSTAINABILITY? ● Future funding schemes for GBV survivor programming need to integrate support for fundraising. Small organizations like La Casita lack the experience, time, and staff to work on fundraising. It is not realistic for small organizations to simultaneously carry out such important survivor work and invest in funding efforts—especially if they do not have expertise in fundraising or grant-writing. Fundraising and grant development skills-building can be included in the grant requirements. Especially in resource-limited locations like Trinidad and Tobago, donors must consider strategies to support small organizations to secure resources to properly pay staff and to continue their work beyond the life of the original funding. Future GBV-client￾centered donations might, for example, include additional funds for a dedicated fundraising initiative, with guidance from development professionals experienced with international funding. ● Strengthen cross-organization collaboration. Improving an organization’s network of local and international partners can foster greater sustainability by creating potentially mutually supportive relationships and introducing the possibility of a shared workload and joint funding. Donors should encourage grantees to submit joint funding proposals. Additionally, when donors are funding large international organizations, donors should be cautious to support truly equitable partnerships between large organizations and local groups versus funding for large organizations that commission poorly funded short-term work by local organizations. ● Don’t rely on volunteer time or short-term funded jobs. As noted above, few good programs, if any, can rely primarily on community volunteers. Especially among resource-poor 51 individuals, for large donor agencies to ask them to give their time for free is at the least impractical and at the worst, unethical. Similarly, short-term jobs that last only for the life of the funding cycle often prove wasteful for the program, unfair for survivor-clients, and disappointing for the persons who are trained to do the work. Building the knowledge and skills of workers and spurring their enthusiasm to support survivors raises hopes and expectations and creates an important resource that often simply disappears when the funding ends. At the very least, the activity, at design and implementation stages, should establish measures to help trained individuals be hired by other relevant organizations if the activities are not sustained. See the first recommendation “Capacity for Sustainability” in Section 4.3. 52 ANNEX A. EVALUATION STATEMENT OF WORK 53 Gender Based Violence: Portfolio Performance Evaluation Scope of Work Final Version 1. Background USAID’s Gender Equality and Women’s Empowerment Hub (GenDev) in the Bureau for Development, Democracy, and Innovation (DDI), advances gender equality and women’s empowerment (GEWE) as fundamental for the realization of human rights, and key to effective and sustainable development outcomes. To achieve Gender Equality and Women’s Empowerment globally, GenDev collaborates with Operating Units (OU) across the Agency supporting USAID’s programming in all sectors. Preventing and responding to gender-based violence (GBV) is a U.S. government (USG) priority. GenDev supports USAID’s efforts to prevent and respond to GBV in more than 60 countries through its thought leadership, training and technical assistance, and programming initiatives. GenDev has contracted NORC at the University of Chicago (NORC) to carry out a performance evaluation of its GBV portfolio comprising four activity clusters: (a) women’s economic empowerment (WEE) activities directly funded by GenDev integrating GBV prevention and response activities; (b) Collective Action to Reduce Gender-Based Violence (CARE-GBV) small grants activities; (c) Resilient, Inclusive & Sustainable Environments (RISE): A Challenge to Address Gender￾Based Violence in the Environment; and (d) Better Together Challenge (BTC) activities funded by GenDev integrating GBV prevention and response interventions. This Scope of Work (SOW) 2 document specifies the objectives of the performance evaluation, the activities that will be included in the evaluation, the evaluation questions, possible data collection methods, the timeline/period of the performance and implementation evaluation from Phase 3b20 onwards, reporting, and deliverables. Definitions: Since GenDev included the four activity clusters (ACs) based on a need for further monitoring and evaluation (M&E), the term portfolio is used only to discuss the four ACs together. Activities funded under each AC are referred to as activities to align with the Agency definition. 2. PPE Objectives This Portfolio Performance Evaluation (PPE) will examine the effectiveness of the portfolio/ACs in achieving their objectives and outcomes, the lessons learned and gaps that are currently not being addressed. Within each AC, NORC will assess if the goal for each AC is being met and how specific projects are being implemented, their quality and challenges. In addition, NORC will conduct an implementation evaluation for a limited set of activities (perhaps one from each AC if feasible), examining how the specific activity is working (if it is on the right pathway to achieving end outcomes), for whom, and in what context. This work will consider the programmatic assumptions, identify intervention challenges and facilitators, and explore engagement with beneficiaries and partners. Findings will inform recommendations for USAID’s future programming and guide future monitoring and evaluation approaches to strengthen USAID’s evidence for decision-making. 20 The Portfolio Performance Evaluation includes several Phases: (1). Scope of Work 1, (2). Evaluability Assessment, (3a). Scope of Work 2, (3b). Evaluation Design Report, (4). Portfolio and Activity Cluster Evaluation and Reporting, (5). Implementation Evaluation and Reporting, (6) Overall PPE Report, Evaluation Debriefing and Dissemination. 54 3. Activities Included in the Evaluation The following activities will be included in the portfolio and activity cluster level evaluation. NORC will also determine one activity within each cluster that will be the target of the implementation evaluation, if appropriate. Table 13: Activities under each Activity Cluster ACTIVITY CLUSTER LIST OF EVALUABLE ACTIVITIES COUNTRY Better Together Challenge 1. Democracy International’s (DI) Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME) 2. HIAS’s Shifting Power Dynamics: Engaging Men in Gender-Based Violence Reduction 3. NCC’s Bridging the Gap for Venezuelan Migrants (BTG4VM) 1. Guyana 2. Panama 3. Trinidad & Tobago CARE-GBV 1. Žene sa Une (ZSU) 2. Women Against Rape (WAR) 3. Sexual Offences Awareness and Response Initiative (SOAR) 4. Crisis Center Hope (CCH) 5. Sexual Violence Research Initiative (SVRI) 1. Bosnia & Herzegovina 2. Botswana 3. Nigeria 4. North Macedonia 5. Global RISE Challenge 1. Creative Capacity Building to Address Gender Based Violence in the Artisanal and Small-Scale Mining Sector in Colombia 2. Resource-ful Empowerment: Elevating Women’s Voices for Human and Environmental Protection in Congolese Small-Scale Mining. 3. Conservation of the Alto Mayo Landscape without Gender Violence 4. Tz’unun: Ending Environmental Violence Against Indigenous Women in Guatemala through Empowerment in Community Forestry, Agroecology and Collective Healing Spaces 5. Combatting Gender-based Violence in Vietnamese Conservation 6. Advancing Equitable Gender, Social and Power Norms in Community Conservancies in Kenya. 7. Gender Empowerment and Transformation: Tackling Resource-Based Conflict and Gender-based Violence in Fiji 8. Rising Up!: Promoting Congolese Women’s Land Access and Preventing GBV in eastern DRC 9. Securing Land Rights & Ending Gender Exclusion 1. Colombia 2. Democratic Republic of Congo 3. Peru 4. Guatemala 5. Vietnam 6. Kenya 7. Fiji 8. Democratic Republic of Congo 9. Uganda WEE 1. Global Labor Program: Levi-Strauss Partnership 2. Engendering Utilities (WAGE) 3. A Micro-Journey to Self-Reliance 4. Enabling Environment for Economic Empowerment of Women 5. New Partnerships Initiative (NPI): Latin America 1. Lesotho 2. Global 3. Benin 4. Burundi 5. Guatemala, Honduras, Mexico 4. Evaluation questions Table 14 includes the evaluation questions and sub-questions at the portfolio, activity cluster and individual activity level. 55 Table 14: Evaluation Questions (SOW Final Version) EVALUATION QUESTION EQ-SUB-QUESTIONS PORTFOLIO QUESTIONS 1. How are the USG’s guiding principles and priorities to end GBV being incorporated into the four activity clusters (AC)? Prevention: In what ways are the USG activity portfolio contributing to reduced risks? Protection: How does the portfolio contribute to accessible, effective services for violence survivors? Accountability: How does the portfolio contribute to ending impunity? 2. To what extent are the USG objectives being achieved across the 4 ACs? Coordination: How are the GBV prevention and response efforts being coordinated and managed at the Agency, Activity Cluster and Activity levels?? Integration: How are GBV prevention and response efforts being integrated into current and future GenDev work and informing related programs? Data. How is GenDev’s GBV portfolio collecting, analyzing, and using data and research to enhance prevention and response efforts? Expansion: How is GenDev’s GBV portfolio helping to expand and improve GBV programming? 3. What lessons are being learned and to what extent is there sharing of best practices, lessons, and information across the 4 ACs? Foundations: Are lessons regarding foundations of GBV being shared with AC implementing partners? Populations: What types of populations are being engaged in the AC? Which vulnerable and underserved populations are been included? Stakeholders: Which stakeholders are being engaged to achieve results? 4. What pervasive gaps still exist in understanding GBV and addressing specific types of GBV? Intervention planning and design: What are important knowledge and practice gaps in planning and designing GBV interventions? Forms of violence: What are important knowledge and practice gaps in addressing specific forms of GBV? Reach and effectiveness: How is the GBV portfolio influencing the reach and effectiveness of interventions? ACTIVITY CLUSTER QUESTIONS 1. Are the activity clusters based on context-specific and international evidence? Needs assessment and intervention evidence: How well were needs assessments conducted and intervention evidence collected to inform the cluster activities? Assumptions: What assumptions were made to design and implement the activity clusters? How accurate were any assumptions? Causal pathways: What causal pathways or theories of change were articulated for the activity clusters? Monitoring and adaptations: How well are interventions monitored and emerging findings contributing to intervention adaptations or improvements? 56 EVALUATION QUESTION EQ-SUB-QUESTIONS 2. To what extent are each of the activity clusters achieving the targeted GBV results? Outcomes: Are the stated outcomes realistic and achievable within the timeframe of the AC? What progress is being made towards achieving the outcomes? Planning and activity designs: How and how well were activity plans and designs developed to achieve different GBV outcomes? Intervention implementation: How well are interventions implemented to reach their target groups and influence change? Mechanisms: What are the most effective aspects of the intervention? How do these ‘active ingredients’ operate in each AC? 3. To what extent are the ACs sustainable? Sustainability: What aspects of the ACs contributed to their sustainability? What components are needed for greater sustainability? Replicability, transferability and adaptability: In what ways are the ACs replicable in the same contexts? Adaptable for other contexts? Scalability: What aspects of the ACs are most amenable to be scaled up? IMPLEMENTATION EVALUATION QUESTIONS 1. Is the activity design based on the local context and flexible to achieve results on the ground? Design: What factors contributed to the design of the activity? How were priority GBV problems identified? Implementation: What are the key intervention methods to achieve objectives? Flexibility: Is there sufficient staffing to respond to local priorities? Is there flexibility to change approaches to respond to lessons and changing challenges in the local environment? 2. Is the activity reaching beneficiaries they are meant to target? Target beneficiaries: What are the barriers to reaching beneficiaries? Monitoring of results: Is the activity collecting evidence on what is working, not working and what could be done differently to achieve results? 3. Is the activity achieving sustainability? Sustainability: What plans are in place for sustainability? What is the evidence of potential sustainability? 5. Possible Data Collection Methods The evaluation will comply with USAID Evaluation requirements as stated in the ADS and the USAID Evaluation Policy. The expected evaluation type is a Performance Evaluation. The evaluation team will use a comprehensive evaluation design and methodology, using a mixed method approach (e.g., desk review, interviews, focus group discussions, key informant interviews, monitoring indicators, web-based survey, etc.), that will generate the highest quality and most credible evidence on each evaluation question, subject to budget constraints across the full portfolio evaluation. Other data collection methods such as outcome harvesting, and most significant change may also be considered and will be explored by NORC. Note: Considering the ongoing COVID-19 pandemic situation worldwide, the evaluation team must consider an alternative plan for fieldwork, including employment of local consultants and usage of IT tools and approaches to remote evaluation. 57 6. Exhibit 10. Evaluation Timeline Task July - Sept Oct￾Dec Jan￾March April￾June July￾Sept 2022 2022 2023 2023 2023 Phase 3B – Evaluation Design Report Phase 4 – Portfolio and Activity Cluster Performance Evaluations & Reporting Project Document Review Finalizing instruments for KIIs, FGDs, and Surveys Data Collection Transcription, Coding and Data Analysis Phase 5 – Implementation Research Reporting Project Document Review Finalizing instruments for KIIs, FGDs, and Surveys Data Collection Transcription, Coding and Data Analysis Phase 6 – PPE Report, Evaluation Debriefing & Dissemination Report Writing Dissemination 7. Reporting and Deliverables Evaluation Design: The report will indicate the three levels of evaluation and a detailed approach and methodology to answer the evaluation questions. Implementation Evaluation Report: This report will include an overview chapter as well as 3-4 separate chapters/sections for each of the individual activity implementation evaluations. Performance Evaluation Report: This report will include an overall synthesis report and 4 separate chapters corresponding to each GBV AC. Post evaluation action plan: This report will include various agreed-upon product(s) to debrief the evaluation activities, disseminate findings, discuss recommendations, and follow-up programming actions responding to recommendations. Knowledge sharing and dissemination: The team will present findings to key stakeholders, including policy briefs, webinars and re-usable slide deck. 58 ANNEX B. EVALUATION METHODS AND LIMITATIONS 59 QUALITATIVE FIELDWORK NORC designed both Key Informant Interview guides and Focus Group Discussion guides for audiences of USAID, Resonance, and IDB staff, NCC, DI/La Casita, and HIAS staff, and survivors and program users for each respective activity. The design of the guides was mapped to reflect the evaluation questions. The key informant guides (KIIs) were used to examine program users’ experience of project recruitment and engagement, and their perspectives about their participation. Other topics examined include needs assessment activities and assumptions in Activity design, Activity flexibility and adaptation, and user outcomes. NORC worked with consultants in each country to interview program users and GBV survivors who participated in the activities. Each organization, DI/La Casita, HIAS, and NCC, provided contact information for program users and survivor participants. NORC prepared the sample of respondents and assigned this to our local consultants for fieldwork (via Secure File Transfer Protocol). For WELCOME users, DI/La Casita provided a confidential participant list using codes, which was communicated to our local consultant offline, and only for the recruitment process. In Guyana, all interviews took place at the RCBG office. In Panama, interviews were held at HIAS in Panama City. In Trinidad and Tobago, eleven interviews were held at La Casita in Arima, and two were held via Zoom. To maintain confidentiality, participant codes were used, and all personal identification information was removed. Referrals for GBV support were prepared and shared with activity participants before the interviews took place. To recruit participants, consultants contacted them by telephone to schedule their interviews. In Trinidad and Tobago, a Spanish interpreter was also on the line for outreach in addition to the interview. In some cases, WhatsApp was utilized when participants could only communicate via text message, or a follow-up reminder was needed. Calling was still preferred as the first means of contact as it allowed for the development of trust, open communication, and transparency. In Guyana, NORC’s consultant did follow-up calls to participants after the interviews to make sure they were feeling well as some of them struggled when discussing their situation during the interviews. Overall, the NORC team followed a set of procedures for transcribing and cleaning all interview and FGD audio. For all audio, consultants uploaded audio files to the Secure File Transfer Protocol (SFTP), and when NORC members received these files, they uploaded them to Amberscript software for automatic transcription and manual cleaning of personal information and transcription errors. There were 20 interviews transcribed in English and 33 transcribed in Spanish. The evaluation team obtained transcripts from MS Teams in English and Spanish, and from Zoom in English only. An analyst cleaned the transcriptions to make sure all contents were comprehensible and well transcribed. For analysis of KIIs, semi-structured interviews, and FGDs, NORC used a team of two coders for the BTC qualitative data. One coder was assigned to Spanish materials and one to the English materials. Coders analyzed, at a first stage, one hundred percent of the qualitative transcriptions using the codes related to the Activity-Cluster evaluation questions (Table 3). At a second stage, coders analyzed only the WELCOME-related transcripts categorizing excerpts according to the Implementation Evaluation codes (Table 3). NORC managed a handful of challenges in the data collection process. The most common barriers to field the KIIs were no shows, remote locations of respondents, holidays, childcare, and busy work schedules. NORC helped provide strategies to overcome these barriers by reimbursing respondents 60 for transportation costs, providing meals, confirming interviews multiple times, and conducting remote interviews. In cases where selected interviewees could not participate, new potential interviewees were contacted from a list of replacements. Table 15. Qualitative Fieldwork Method Respondents Language Sample Dates KII USAID Staff English 1 12/16/22 KIIs Resonance-The Catalyst Project Staff English 3 12/16/22-12/20/22 Small￾group KII IADB Staff English 3 1/23/23 KIIs NCC Staff (BTG4VM) English 3 12/15/22-1/23/23 KIIs DI/La Casita Staff (WELCOME) 4 English, 1 Spanish 5 12/20/22-3/21/23 KII and small￾group KII HIAS Staff (SPD) Spanish 3 11/15/22-12/9/22 KIIs GBV Survivors with NCC (BTG4VM) 5 English, 2 Spanish 7 3/13/23-3/30/23 KIIs GBV Survivors with La Casita (DI), WELCOME Spanish 13 2/15/23-3/18/23 KIIs Program users with HIAS (SPD) Spanish 14 3/1/23-3/14/23 FGD Female partners with HIAS (SPD) Spanish 6 3/14/23 FGD Staff and advocates with DI (WELCOME) English and Spanish 3 2/15/23 FGD NCC Staff (BTG4VM) English 6 3/17/23 WEB SURVEY FIELDING NORC designed two web survey instruments to collect information about BTG4VM and WELCOME from the service providers that partnered with NCC in Guyana and DI/La Casita in Trinidad and Tobago, respectively, to refer Venezuelan migrants to a pathway of services. During the instrument design phase, NORC clearly mapped in Excel each survey item to evaluation questions and sub-questions to ensure that the tool elicited sufficient and relevant information from service providers. Questions related to topics such as the availability of GBV-related services in the community, characteristics of the services provided through the network, target population and participant uptake, context appropriateness, service outcomes, monitoring tools, and activity sustainability. NORC used this map to produce versions of the web survey in Word that grouped questions by theme and indicated display instructions and skip logic for programming. Most survey items were close ended. NORC incorporated feedback from USAID and the IPs to refine the survey items and response options. NORC programmed the surveys in Qualtrics and tested the tools internally before data collection was launched. NORC followed an iterative snowball sampling approach for the web surveys. NORC received initial Web Survey sample frames from the NCC and DI/La Casita. The sample frames included the contact information of the focal persons at the organizations that partnered with NCC and DI/La Casita to 61 provide services under the two activities (including legal services, job training, housing support, and other services). In the first phase, NORC distributed the survey to this initial list of service providers. NORC’s Evaluation Specialists in each country acted as web survey monitors. In the second phase of sampling, the Evaluation Specialists contacted each respondent or organization the IP shared and asked if they could nominate additional staff that worked on either BTG4VM or WELCOME to participate in the survey. The Evaluation Specialists also confirmed the organizations the IP listed actually participated in the activity. This snowball sampling enabled NORC to increase the scope of data collection and gather a wider range of responses from individuals involved at various levels in the activity implementation. To increase the response rate, NORC issued weekly reminders via the Qualtrics system. The Evaluation Specialists also followed up on non-complete survey cases via email, text, and over the phone to encourage respondent participation. For Guyana, 10 of 18 targeted users responded for a 56 percent response rate. Table 16 and Table 17 display the results of both survey exercises. Table 16. Web Survey Sample -WELCOME ORGANIZATION COMPLETES INCOMPLETES TOTAL SAMPLE Rape Crisis Society 6 0 6 Families in Action 2 0 2 International Organization for Migration 2 1 3 Institute of Gender and Development Studies (IGDS) at University of the West Indies 0 1 1 TTV Solidarity Network 0 1 1 United Nations High Commissioner for Refugees 1 0 1 Family Planning Association of Trinidad and Tobago 0 1 1 TOTAL 11 4 15 Table 17. Web Survey Sample - BTG4VM ORGANIZATION COMPLETES INCOMPLETES TOTAL SAMPLE Catholic Charities Organization Guyana - Migrant Support Services 3 0 3 Food for the Poor 1 0 1 Hebrew Immigrant Aid Society 1 2 3 Hope Foundation 2 0 2 International Organization for Migrants 1 0 1 NCC Legal Consultant 1 1 2 NCC Psychosocial Consultant 0 2 2 United Nations High Commission for Refugees 1 3 4 62 ORGANIZATION COMPLETES INCOMPLETES TOTAL SAMPLE Voices Gy 0 1 1 TOTAL 10 9 19 We include the survey respondents’ demographics in Table 18. Table 18. Demographic Information of Web Survey Respondents DEMOGRAPHICS GUYANA (N) TRINIDAD AND TOBAGO (N) Gender Male 3 1 Female 5 8 Age 25-34 4 - 35-44 1 6 45-54 1 2 Language English 8 8 Trinidad English - 4 Spanish 3 2 Guyanese Creole 3 - Portuguese 1 - Education Level Complete secondary 1 - Some higher education 1 1 Complete higher education 1 - Advanced degree 4 7 Current career position Mid-career 3 3 Senior 2 4 LIMITATIONS Sample Frame of GBV Survivors in Guyana. Through a series of attempts between January and March 2023, NORC received a list of 17 female BTG4VM users who were GBV survivors from NCC. From those, only five were of Venezuelan origin. Overall, the contact information of nine survivors was outdated and NCC had no other resources to locate these survivors. One potential respondent had no recollection of receiving services from NCC. Limited geographic coverage of focus groups in Panama. Due to NORC’s ethical standards of transparency to program users, our methodology required the primary user’s authorization to contact their female partner for FGD recruitment. The access to female partners was highly restricted for two reasons: (i) several male respondents who attended the SPD workshops were single or had no partner at the time of data collection; and (ii) a large proportion of those who were partnered did not facilitate access to their partners. For this reason, the evaluation team was able to recruit a number of female partners large enough to moderate one session with six participants. This smaller sample prevented the evaluation team from capturing any differences in perceptions about changes in male-partner behavior towards their partners across communities. 63 Web Survey Respondent Availability. As the web survey respondents were frontline workers and service providers, the Evaluation Specialists reported challenges contacting respondents over the phone to remind them to take the surveys. Reasons for lack of availability included staff working in the field, academics being on sabbatical, low to nonexistent interest in answering about an initiative that did not fund their own organization, and phones being turned off. After four weeks of self￾administered data collection in Guyana, NORC asked the Evaluation Specialist to administer the surveys over the phone and mark the respondent’s answers in Qualtrics on their behalf. The phone￾assisted approach increased the response rate from 4 to 10, and NORC ended up meeting our benchmark of a response rate greater than 50 percent. Web Survey Small Sample Frame. Over the course of survey testing, NORC learned the lists of service providers who could speak to their participation in the activities were shorter than the IPs had suggested at EDR stage. In Trinidad and Tobago, the Evaluation Specialist confirmed that 3/10 organizations the DI/La Casita shared did not play a role in program implementation. In Guyana, the Evaluation Specialist could not establish contact with 3/12 organizations shared by the IP, either because no contact phone number was on file or because the organization was no longer operating in Guyana. 64 ANNEX C. DATA COLLECTION INSTRUMENTS 65 PE of USAID GBV Portfolio – Better-Together Challenge Cluster Protocol to interview USAID’s Activity Cluster Manager Respondent Name: Date: Start Time: Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of the Performance Evaluation of the USAID's GBV Portfolio. USAID commissioned NORC, an independent and non-partisan organization, to collect data that will inform current and future USAID-funded programming focused on GBV prevention and response. As evaluators, we maintain neutrality on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Your participation is voluntary. If you are unable or prefer not to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming focused on GBV prevention and response. All the information you decide to share with me today will be handle confidentially. This means that only I will know the personal information of respondents, including you. We will anonymize all transcriptions and the analysis we conduct of them so that any comments or quotes NORC includes in reports or publications will not be linked to any person or household. Today’s interview is planned for 60 minutes. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Carlos Echeverria-Estrada, the lead of this study at echeverria￾carlos@norc.org or at the phone number +1(312) 759 2658. With your permission, I’d like to record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID, nor the implementing partners or any third party. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [OBTAIN CONSENT] [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? [OBTAIN CONSENT] Thank you. Before we jump in, I’d like to get to know you bit. Introduction Today we are going to discuss one of the USAID Activity Clusters, the funded activities within the Better-Together Challenge (BTC). We would like to know more about the activities USAID funded and you helped manage, its design effects in the communities, your experience managing the clusters, and lessons for future programming and sustainability at USAID. 66 Topic Question 1. Relevance, Design & Planning In our first section, I’d like to talk about how this cluster was designed and planned. Assumptions Could you comment on the elements of the theory of change for the BTC cluster? I will pull it up so that we can all see it and let us walk through it together and comment on how it worked in practice.[INTERVIEWER: Show or read aloud] - Did the underlying assumptions and reasoning prevail? - What was the capacity of the implementing partners to deliver services? - Were there availability partners and network to bring the ToC to life? - Overall, was the uptake of services well accepted or were there resistances to it? Needs Assessment and Intervention Evidence Were grantees in the activity cluster required to conduct pre￾implementation assessments? Needs Assessment and Intervention Evidence What assessments were conducted, and could you comment on their usefulness and robustness? Design What USG principles and policy measures drove the choice of included interventions in the BTC cluster? 2. Cluster Co-creation, Coordination, & Planning Thank you very much for your responses so far, they are really helpful. Now we’re going to talk about how the cluster coordinated the activities and the communication flow. Coordination Could you please tell me about the management structure for the BTC activity cluster? Coordination How did the information flow from grantees up to GenDev? Coordination How does this information influence decision-making? Please, provide an example. [If necessary, probe: past re-design, future programming, adjustment to current programming]. Coordination What is your assessment of the co-creation processes undertaken between USAID and IPs? Are there any lessons learned from these experiences? Co-creation Were grantees briefed, at some point, on USAID strategies and priorities around GBV? Or the lessons learned from the BTC cluster? Integration Have you coordinated initiatives from this cluster with any other GBV efforts from USAID? If so, what types of collaboration took place and how well did they work? What did not work so well? Integration What benefits or learning emerged from that collaboration? Integration How does this activity cluster fit into the full range of USAID's GBV programs? What gendered aspects or diverse gender populations are being considered in GBV programing? What aspects are being prioritized? Are other intersections being considered? 67 Topic Question Integration What are the primary features of the BTC cluster that might differ or complement the other GBV activity clusters? Please identify two or three. Reach and effectiveness How do the 4 ACs together influence the reach and effectiveness of other GBV interventions, even those not funded by USAID? 3. Target Groups and Engagement I want to thank you again for your collaboration so far. Now, let’s explore the cluster targeting and engaging important actors. Populations Could you tell me what are the most vulnerable or underserved populations who participated in the BTC programming? Populations Who do you think these activities might have missed? Why? Populations Are there certain populations that you would recommend that should have been considered in the BTC activities? Stakeholders What other stakeholders were you able to engage to accomplish the goals of BTC? Stakeholders What activities were able to effectively engage relevant local or regional stakeholders? Please, provide one or two examples. Stakeholders In contrast, what activities were less successful in engaging important local actors? Also, please provide one or two examples. 4. Effectiveness in GBV Protection, Prevention & Accountability Thanks for your responses to these questions, we really value your insight. Now, I’d like to know about how effective this activity was. Protection Thinking about the BTC cluster, how have its grants affected access to effective services for GBV survivors? Why? Protection What approaches were most effective in increasing access to services? Which were least effective? Why? Prevention What have been the most important contributions of the BTC cluster in preventing GBV? Prevention Which activity approaches were effective in preventing GBV? Why? Accountability [Ask only if the cluster funded any strategies to combat impunity of GBV perpetrators] In your opinion which strategy implemented by the BTC cluster to stop or punish GBV perpetrators was most effective, and why? Outcomes Overall, were the outcomes stated in activity designs for this activity cluster realistic? Why? Outcomes Was the length of the grants sufficient to achieve these outcomes? Why? Outcomes Which activities were most successful in achieving the outcomes in their activity design, and which were less successful? Can you identify facilitating factors and challenges either way? 68 Topic Question Outcomes Were there any unanticipated positive or negative effects from this activity? Intervention planning and design Were there particular gaps related to gender diversity? If yes, please describe. Forms of violence What are important knowledge and practice gaps in addressing specific forms of GBV? 5. Monitoring, Learning & Adaptation Again, I really appreciate your answers, thank you. I’d like to move on to a few questions about activity monitoring and adaptation. Data How did GenDev use data collected from the BTC cluster? Data Are lessons and information from the BTC cluster ever used to inform higher level decision making across GenDev programs? Could you provide some examples of this? Expansion How have lessons from the BTC cluster affected other GBV programming? Reach and effectiveness How will you take advantage of the lessons learned or knowledge gained from the 4 ACs for future work? 6. Replicating, Adapting, Transferring & Scaling Up Thanks for this helpful information. Now I want to discuss if and how you think this activity could be replicated, adapted, transferred, or scaled up. Replicability, transferability, and adaptability Are there any interventions or approaches in the BTC cluster that you think could be replicated in or adapted to other contexts? Replicability, transferability, and adaptability Has the experience with the BTC cluster informed the potential for scale up or replicability elsewhere? Please explain. Replicability, transferability, and adaptability Are there any that you think could not be replicated or adapted? Why? Scalability Which interventions in the BTC cluster do you see as having potential for scaling up? Are there any that you think would be very costly or difficult to scale up? Why? Scalability Conversely, are there interventions that you would cut? If yes, why? 8. Sustainability Thank you. To finish, I have some additional questions regarding the sustainability of this activity. Sustainability What components or approaches in the BTC cluster ended? Why? (PROBE: funding, political willingness, cultural competency, etc.) 69 Topic Question Sustainability What strategies were absent or could have been improved to increase sustainability for those activities or approaches that ended? Sustainability Are there any lessons learned for the activity cluster about sustainability in the current context and existing stakeholders? 9. Closure I don’t have any more questions. Is there anything else you would like to add to what we have discussed today? Thank you. I have learned a lot and I thank you for your participation and comments. Before leaving, do you have any questions for me? 70 PE of USAID GBV Portfolio – Better-Together Challenge Cluster KII Guide with IP Leads (Resonance, DI, HIAS, NCC, La Casita) and two Technical Advisors (HIAS) Respondent Name, Position, Institution Date: Start Time: Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of the [ACTIVITY NAME] activity, funded by the Office of Gender Equality and Women’s Empowerment at the United States Agency for International Development (USAID). USAID commissioned NORC, an independent and non-partisan organization, to collect data that will inform current and future USAID-funded programming focused on GBV prevention and response. While NORC does a lot of work with USAID, we do not work for USAID. We are neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming focused on GBV prevention and response. All the information you decide to share with me today will be handled confidentially. This means that only I will know the personal information of respondents, including you. We will anonymize all transcriptions and the analysis we conduct of them so that any comments or quotes NORC includes in reports or publications will not be linked to any person or household. While the questions I have for us to discuss today focus on your experience with the [ACTIVITY NAME] activity, this form includes the information I just read to you, as well as a list of counseling and psychosocial support resources available to you [HAND THE PRINTED COPY TO RESPONDENT]. Today’s interview is planned for 60 minutes. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Carlos Echeverria-Estrada, the lead of this study at echeverria￾carlos@norc.org or at the phone number +1(312) 759 2658. With your permission, I’d like to record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID, nor the implementing partner [NAME OF ORGANIZATION] or any third party. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. Before we jump in, I’d like to get to know you bit. Could you please give tell me your name, position at [ORGANIZATION], your role in the [ACTIVITY] activity, and or how long have you worked here? 71 Topic Question Introduction Today we are going to discuss the USAID funding of [ACTIVITY]. We would like to know more about the activity you helped implement, its design and management, its effects in the community, and your experience working with USAID. 1. Co-creation & Relevance Let me start with the stage prior to implementation, its design. Design Could you speak to how this activity was designed? What factors influenced this design? What priorities were established? Who was involved in these design decisions? Would you change anything about the design? Needs Assessment and Intervention Evidence What pre-implementation assessments did [ORGANIZATION] do for this activity? Were they useful? How did those help your team plan and implement the activity? Needs Assessment and Intervention Evidence Would you recommend any other kinds of research before implementing similar activities? Please, give me an example. Assumptions [ONLY IF NO TOC AVAILABLE]: Could you please tell me about your theory of change? Assumptions I have taken the time to review the activity’s theory of change and have a few follow up questions about it. According to document review, the theory of change is… [Interviewer: Read TOC] When formulating the TOC of the [ACTIVITY] activity, what were the main assumptions about?: a. the ability of the organization to deliver the activities; b. who were the potential participant to use the services; c. how would the services result in the desired outcomes? d. How relevant were the assumptions? Causal Pathways According to the theory of change, the overarching causal pathways for the [ACTIVITY] activity should lead from [PROGRAM INPUTS] TO [OUTCOMES]. Upon implementation, was this TOC applicable to the [ACTIVITY] activity? Were there any caveats? Please, explain. Causal Pathways Once you started implementation did you need to adjust your TOC? If yes, what adjustments were necessary? Planning and Activity Designs If you were designing the activity again, is there anything that you would do differently? Planning and Activity Designs What are the main lessons learned from designing the ACTIVITY] activity? Forms of Violence [Specify for each activity] What are important knowledge and practice gaps in addressing specific forms of GBV? 2. Coordination & Target Groups 72 Topic Question Thank you very much for your responses so far, they are really helpful. Now, I’d like to talk about how this activity was coordinated and planned. Planning and Activity Designs Did you carry out a co-creation process with USAID to design the [ACTIVITY] activity? How did the co-creation process on activity design work for your organization? Could you identify advantages and down sides of this approach? Foundations Have you been briefed on USAID strategies and priorities around GBV? [IF NOT]: Would you be interested in such a briefing? Coordination What other activities could USAID implement to benefit IPs from collaboration? Coordination Have you or your organization participated in meetings with other IPs via USAID's GenDev? a. If so, what was the purpose of these meetings? Please, provide examples if the meetings served multiple purposes. b. What information, ideas or other benefits did you obtain from this experience(s)? Would you recommend more exchanges between IPs and USAID? Why? Or why not? Populations Could you please tell me about the populations or groups served by your activity? a. Are there any underserved or especially vulnerable groups that your activity has reached? b. If so, what are the approaches or strategies the [ACTIVITY] activity has implemented to address the needs of these groups? How effective have these strategies been? Stakeholders What other stakeholders were you able to engage to accomplish the goals of the [ACTIVITY] activity? Stakeholders What were the contributions of these additional stakeholders? Was their participation valuable? Stakeholders Do you have any lessons learned that you could share about this? 3. Effectiveness in GBV Protection, Prevention & Accountability I want to thank you again for your collaboration so far.. Now, let’s talk about the activity’s effectiveness in preventing GBV, protecting from it and, if applicable, keep perpetrators accountable. Outcomes What were the primary outcomes of the X activity? Outcomes Please describe one or two of the main activity outcomes so far. Protection Can you provide some examples of program effectiveness in providing or facilitating access to these services? Prevention Now, I would like to talk about prevention of GBV. What has been the most important contribution of the [ACTIVITY] activity to preventing 73 Topic Question GBV among Venezuelan migrants in [COMMUNITY]? Please, explain how. Prevention What have been the most effective aspects of your strategies or approaches to prevent GBV in the [ACTIVITY] activity? Please, elaborate on the reasons you consider these aspects effective. Probe, if necessary: Could you give me an example? Accountability [Dependent on activity focus] How has your activity, directly or indirectly, contributed to increasing perpetrator accountability? Accountability [Dependent on activity focus] What else could the activity or future programming in a similar area do to foster further perpetrator accountability? Outcomes Were the outcomes stated in activity design realistic and achievable? Why? Outcomes Are there outcomes that you would have liked to see but were not feasible to accomplish? Why? Outcomes Was the length of the grant sufficient to achieve these outcomes? Protection-Prevention￾Accountability From your experience implementing the [ACTIVITY] activity, what are the main lessons about prevention, protection, and accountability of GBV that you could share? Now we’re going to talk about how relevant this activity was for the setting and context. 4. Implementation Thanks for your responses to these questions, we really value your insight. Now, a few points about the implementation of [ACTIVITY]. Implementation Were there specific challenges or enabling factors in implementing this activity? Please explain. Were the challenges overcome? And how? Target Participants Who were the main target individuals of this activity? Did you have any difficulty reaching them? If so, what were the difficulties? Target Participants Are there particular sub-groups of target individuals that are difficult to reach? How do you think these barriers could be mitigated in the future? Flexibility Was there sufficient staffing to respond to local priorities? Was there flexibility to change approaches to respond to lessons and changing challenges in the local environment? 5. Monitoring & Adaptation Again, I really appreciate your answers, thank you. I’d like to move on to a few questions about activity monitoring and adaptation. Monitoring and Adaptations What was your overall strategy for activity monitoring and adapting the activities based on emerging findings? 74 Topic Question Monitoring and Adaptations Could you share some examples of how you might have changed any aspects of activities based on emerging evidence from monitoring? Monitoring and Adaptations How accessible and user friendly were the USAID monitoring tools/templates provided to you? Monitoring and Adaptations Are there any lessons learned from your experience with the monitoring tools? 6. Replicating, Adapting, Transferring & Scaling Up Thanks for this helpful information. Now I want to discuss if and how you think this activity could be replicated, adapted, transferred, or scaled up. Replicability, transferability, and adaptability What components or approaches do you think could be replicated in other communities or countries? Replicability, transferability, and adaptability What approaches, components, or tools of the [ACTIVITY] activity would need to be adapted for a different context? Replicability, transferability, and adaptability Where else would you recommend implementing this activity? Scalability If you were to scale up your activity, which components of your intervention would you focus on? Scalability Are there any that you would drop? What changes would you make? Scalability What are the main challenges for scaling the activity up in your country/region? 7. Sustainability Thank you. I have some additional questions regarding the sustainability of this activity. Sustainability Do you think that this activity is sustainable moving forward? Sustainability What practices or activities have taken place to support the sustainability of X activity? Sustainability [IF ACTIVITY ENDED]: What strategies could have enhanced sustainability of the activity? Sustainability What have been the primary facilitators and barriers to the sustainability of the X activity? 8. Lessons Learned Thanks again. I’m almost finished with my questions; I have some final things I want to ask about lessons learned from this activity. Prevention Based on implementing the [ACTIVITY] activity, what are the main lessons learned about prevention that you could share? BTC Cluster Has your organization changed its approach to migrants after experience with the activity? Please explain? 75 Topic Question BTC Cluster Are there any lessons learned on how to engage host community members? Please, provide an example. 9. Closure I don’t have any more questions. Is there anything else you would like to add to what we have discussed today? Thank you. I have learned a lot and I thank you for your participation and comments. Before leaving, do you have any questions for me? Implementation evaluation Items [ONLY FOR T&T] 9A. IMPLEMENTATION EVALUATION As you know, we are also assessing the implementation of this grant. I have a few additional questions on that regard. BTC IE Please, identify the main factors that determined the re-design of the activity. BTC IE How did your organization identify the needs of the target population after redefining it? BTC IE Have any members of the host communities supported or expressed enthusiasm for this work? Please, describe. BTC IE Are there any migrant groups or any other social agreements in the implementation area that could promote these efforts in the future? Please, explain. Intervention Implementation Was your activity able to reach the participants that it was designed to reach? How did you identify changemakers in the org/community for X activity? Intervention Implementation Were there any challenges in reaching the right people to influence change? Intervention Implementation Who else should be engaged and was not in the X activity? Mechanisms What do you think were the most effective components of your activity? Please, identify one or two. Why? 76 PE of USAID GBV Portfolio – Better-Together Challenge Cluster FGD Topic guide with grantee staff (La Casita) Respondent Names, Roles, Institutions: Date: Start Time: Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be facilitating today’s discussion. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this group discussion, which is part of an evaluation of the Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME) Activity, funded by the Office of Gender Equality and Women’s Empowerment at the United States Agency for International Development (USAID). USAID commissioned NORC, an independent and non-partisan organization, to collect data in several countries, including Guyana that will inform current and future USAID-funded programming focused on GBV prevention and response. While NORC does a lot of work with USAID, we do not work for USAID. We are neutral on all the issues we will be talking about, and we’re just here to learn about your perspectives and experiences. That means none you don’t need to worry about making us happy or hurting our feelings. Please be candid in your comments and answers. However, this is a safe space to all perspectives and opinions as long as those are respectful of each other’s. Feel free to respond to the topics I will be bringing up and to comment on what others say. While you don’t have to wait for me to call you, let’s keep only one person speaking at a time. Your participation is voluntary. If you are unable to answer a question, you may wait to the next topic or stop your participation at any time; there will be no repercussions for this not this would change your relationship to NCC, NORC, or USAID. However, your comments will be very useful in helping in informing future USAID-funded programming focused on GBV prevention and protection. NORC will handle the information you decide to share with me confidentially. This means that, at the organization, only I will know your personal information as participants. But that does not mean that any of the participants today may repeat anything of what is said today. Please, consider that when deciding what you would feel comfortable sharing with me. NORC will anonymize all transcriptions so that any comments or quotes NORC includes in reports or publications will not be linked to any person or organization. Today’s discussion is planned for 50 minutes. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Carlos Echeverria-Estrada, the lead of this study at echeverria-carlos@norc.org or at the phone number +1(312) 759 2658. With your permission, I’d like to record today’s discussion. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID, nor NCC or any third party. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [GET EVERYONE’S CONFIRMATION AROUND THE ROOM] [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. A FEW GROUND RULES FOR TODAY’S DISCUSSION: ● If you need to get up for any reason, you may feel free to do so and don’t need to ask for permission. When you return, please wait until the next question to jump back in. 77 Topic Question ● Since our meeting is only 45 minutes, and we have a lot of ground to cover, we kindly ask that you cover 1-2 points in your responses to give others time to respond as well. This is meant to be a free￾flowing discussion, but for ease of facilitation, please raise your hand if you would like to join the discussion. ● Say what you believe, even if it’s not what everyone thinks. There are no bad answers, just different opinions, and we want to hear them all. If you agree with what has been said, please say that. Otherwise, please share your points. ● From time to time, I may have to interrupt you to finish on time. I’m not trying to be rude, but there’s a lot to cover in a limited amount of time, so I apologize in advance if that happens. Now, let’s get started. Before we jump in, I’d like to get to know you bit. Could you please give tell me your name, position at La Casita, your role in the WELCOME program, and or how long have you worked here? Introduction We would like to know more about the activity you helped or are helping to implement, its design and management, its effects in the community, and your experience working with Democracy International and Resonance. 1. Design & Implementation In our first section, I will ask you to discuss how the activity contributes to GBV Protection and Prevention Design Who could you speak to how this activity was designed? What factors influenced this design? What priorities were established? Who was involved in these design decisions? Planning and Activity Designs If you were designing the activity again, is there anything that you would do differently? Implementation During implementation, were there any specific challenges or factors that facilitated its success? Please explain. Implementation [LIST CHALLENGES AND DISCUSS EACH OF THEM]: Were the challenges overcome? And how? Target Participants Who were the main target individuals of this activity? Did you have any difficulty reaching them? If so, what were the difficulties? Target Participants Have there been any particular sub-groups of target individuals that are difficult to reach? How do you think these barriers could be mitigated in the future? Staffing Has there been sufficient staffing to respond to local priorities? Flexibility Has there been flexibility to change approaches to respond to lessons and changing challenges in the local environment? 2. Monitoring We are aware that USAID requires their grantees to monitor their activities. I would like to talk about the monitoring system in place. Monitoring and Adaptations What was your overall strategy for activity monitoring and adapting the activities based on emerging findings? Monitoring and Adaptations [TO THE MONITORING STAFF]: How accessible were the USAID monitoring tools/templates provided to you? Monitoring and Adaptations [TO THE MONITORING STAFF]: How user-friendly were the USAID monitoring tools/templates provided to you? 78 Monitoring and Adaptations Are there any lessons learned from your experience with the monitoring tools? 3. Results I appreciate everyone’s contributions so far. Now, I would like to hear about the results of the activity including outcomes and adaptation. Outcomes Please describe one or two main outcomes of the WELCOME program so far. Facilitating factors Can you describe any facilitating factors to accomplish these outcomes? Protection Who would like to share with me how does the WELCOME activity has provided Venezuelan GBV survivors with access to services that protect them? Protection What specific protection components or approaches of the WELCOME activity were the most effective? Prevention Now, I would like to talk about prevention of GBV. What has been the most important contribution of the BTG4VM activity to preventing GBV among Venezuelan migrants in the areas of implementation in Guyana? 5. Lessons Learned Thank you again to everyone for your participation. I have just a one final point about the future of this activity and what lessons were learned. Sustainability Do you think that this activity is sustainable moving forward? Why? Are there any plans to find new funds? Closure I don’t have any more topics to discuss. Does anyone have anything else you would like to add to what we have discussed today? Thank you. I have learned a lot and I thank you for your participation and comments. Before leaving, do you have any questions for me? 79 PE of USAID GBV Portfolio – Better-Together Challenge Cluster FGD Topic guide with grantee staff (NCC) Respondent Names, Roles, Institutions: Date: Start Time: Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be facilitating today’s discussion. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this group discussion, which is part of an evaluation of the Bridging the Gap for Venezuelan Migrants (BTG4VM) Activity, funded by the Office of Gender Equality and Women’s Empowerment at the United States Agency for International Development (USAID). USAID commissioned NORC, an independent and non-partisan organization, to collect data in several countries, including Guyana that will inform current and future USAID-funded programming focused on GBV prevention and response. While NORC does a lot of work with USAID, we do not work for USAID. We are neutral on all the issues we will be talking about, and we’re just here to learn about your perspectives and experiences. That means none you don’t need to worry about making us happy or hurting our feelings. Please be candid in your comments and answers. However, this is a safe space to all perspectives and opinions as long as those are respectful of each other’s. Feel free to respond to the topics I will be bringing up and to comment on what others say. While you don’t have to wait for me to call you, let’s keep only one person speaking at a time. Your participation is voluntary. If you are unable to answer a question, you may wait to the next topic or stop your participation at any time; there will be no repercussions for this not this would change your relationship to NCC, NORC, or USAID. However, your comments will be very useful in helping in informing future USAID-funded programming focused on GBV prevention and protection. NORC will handle the information you decide to share with me confidentially. This means that, at the organization, only I will know your personal information as participants. But that does not mean that any of the participants today may repeat anything of what is said today. Please, consider that when deciding what you would feel comfortable sharing with me. NORC will anonymize all transcriptions so that any comments or quotes NORC includes in reports or publications will not be linked to any person or organization. Today’s discussion is planned for 50 minutes. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Carlos Echeverria-Estrada, the lead of this study at echeverria-carlos@norc.org or at the phone number +1(312) 759 2658. With your permission, I’d like to record today’s discussion. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID, nor NCC or any third party. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [GET EVERYONE’S CONFIRMATION AROUND THE ROOM] [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. A FEW GROUND RULES FOR TODAY’S DISCUSSION: ● If you need to get up for any reason, you may feel free to do so and don’t need to ask for permission. When you return, please wait until the next question to jump back in. 80 Topic Question ● Since our meeting is only 45 minutes, and we have a lot of ground to cover, we kindly ask that you cover 1-2 points in your responses to give others time to respond as well. This is meant to be a free￾flowing discussion, but for ease of facilitation, please raise your hand if you would like to join the discussion. ● Say what you believe, even if it’s not what everyone thinks. There are no bad answers, just different opinions, and we want to hear them all. If you agree with what has been said, please say that. Otherwise, please share your points. ● From time to time, I may have to interrupt you to finish on time. I’m not trying to be rude, but there’s a lot to cover in a limited amount of time, so I apologize in advance if that happens. Now, let’s get started. Before we jump in, I’d like to get to know you bit. Could you please give tell me your name, position at NCC, your role in the BTG4VM activity, and or how long have you worked here? Introduction Today we are discussing the USAID funding of BTG4VM. We would like to know more about the activity you helped or are helping to implement, its design and management, its effects in the community, and your experience working with USAID. 1. Protection & Prevention In our first section, I will ask you to discuss how the activity contributes to GBV Protection and Prevention Protection Who would like to share with me how does the BTG4VM activity has provided or facilitated access for Venezuelan GBV survivors to services that protect them? Protection Which specific activity component or approach of the BTG4VM activity was most effective? Prevention Now, I would like to talk about prevention of GBV. What has been the most important contribution of the BTG4VM activity to preventing GBV among Venezuelan migrants in the areas of implementation in Guyana? 2. Design & Implementation Thank you for all of your responses so far, they’re really valuable to us. Now, I’d like the group to think about the design and implementation of this activity. Design Who could you speak to how this activity was designed? What factors influenced this design? What priorities were established? Who was involved in these design decisions? Planning and Activity Designs If you were designing the activity again, is there anything that you would do differently? Implementation During implementation, were there any specific challenges or factors that facilitated its success? Please explain. Implementation [LIST CHALLENGES AND DISCUSS EACH OF THEM]: Were the challenges overcome? And how? Target Participants Who were the main target individuals of this activity? Did you have any difficulty reaching them? If so, what were the difficulties? Target Participants Have there been any particular sub-groups of target individuals that are difficult to reach? How do you think these barriers could be mitigated in the future? Staffing Has there been sufficient staffing to respond to local priorities? Flexibility Has there been flexibility to change approaches to respond to lessons and changing challenges in the local environment? 3. Monitoring We are aware that USAID requires their grantees to monitor their activities. I would like to talk about the monitoring system in place. Monitoring and Adaptations What was your overall strategy for activity monitoring and adapting the activities based on emerging findings? 81 Topic Question Monitoring and Adaptations [TO THE MONITORING STAFF]: How accessible were the USAID monitoring tools/templates provided to you? Monitoring and Adaptations [TO THE MONITORING STAFF]: How user-friendly were the USAID monitoring tools/templates provided to you? Monitoring and Adaptations Are there any lessons learned from your experience with the monitoring tools? 4. Results I appreciate everyone’s contributions so far. Now, I would like to hear about the results of the activity including outcomes and adaptation. Outcomes Please describe one or two of the main activity outcomes so far. Facilitating factors Can you describe any facilitating factors to accomplish these outcomes? 5. Lessons Learned Thank you again to everyone for your participation. I have just a one final point about the future of this activity and what lessons were learned. Sustainability Do you think that this activity is sustainable moving forward? Closure I don’t have any more topics to discuss. Does anyone have anything else you would like to add to what we have discussed today? Thank you. I have learned a lot and I thank you for your participation and comments. Before leaving, do you have any questions for me? 82 PE of USAID GBV Portfolio Protocol to focus groups discussion with partners of project participants (HIAS-SPD) Respondent Name: Date: Start Time: Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be facilitating today’s discussion. I want to thank you for coming and participating in this exercise, which is part of an evaluation of the “Shifting the Power Dynamics: Engaging Men in Gender-Based Violence Reduction” Activity, funded by the Office of Gender Equality and Women’s Empowerment at the United States Agency for International Development (USAID). I understand most of you participated in the inter-gender dialogues of this Activity. NORC was contracted as an external, independent organization to collect data that will inform current and future USAID-funded projects focused on preventing and responding to gender-based violence. While we do a lot of work with USAID, I do not work for USAID. NORC is completely neutral on all the issues we will be talking about, and we’re just here to learn about your experiences. That means none you don’t need to worry about making us happy or hurting our feelings. Please be candid in your comments and answers. This is a safe space to all perspectives and opinions as long as those are respectful of each other’s. Feel free to respond to the topics I will be bringing up and to comment on what others say. While you don’t have to wait for me to call you, let’s keep only one person speaking at a time. INFORMED CONSENT: Your participation is voluntary. If you are unable to answer a question, you may wait to the next topic or stop your participation at any time; there will be no repercussions for this not this would change your relationship to HIAS, NORC, or USAID. However, your comments will be very useful in helping in informing future USAID-funded programming focused on GBV prevention and protection. NORC will handle the information you decide to share with me confidentially. This means that, at the organization, only I will know your personal information as participants. But that does not mean that any of the participants today may repeat anything of what is said today. Please, consider that when deciding what you would feel comfortable sharing with me. NORC will anonymize all transcriptions so that any comments or quotes NORC includes in reports or publications will not be linked to any person or organization. Today’s discussion is planned for 50 minutes. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Carlos Echeverria-Estrada, the lead of this study at echeverria-carlos@norc.org or at the phone number +1(312) 759 2658. If any of you had further questions about your rights as participant, please contact the University of Panama’s Bioethics Committee (CBUP) at comitebioetica.invup@up.ca.pa or dialing to (+507) 523 5769. Do you agree to participate? [GET EVERYONE’S CONSENT AROUND THE ROOM] CONSENT TO RECORD: With your permission, I’d like to record today’s discussion. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID, nor HIAS or any third party. It will be kept within this research team and destroyed at the end of this study. Do you agree to participate in today’s study and to have this interview recorded? [GET EVERYONE’S CONFIRMATION AROUND THE ROOM] [START RECORDING] The recording has started. Could you please confirm for me one more time on the tape that you agree to participate in this study and have this interview recorded? Thank you. 83 Topic Question A FEW GROUND RULES FOR TODAY’S DISCUSSION: If you need to get up for any reason, you may feel free to do so and don’t need to ask for permission. When you return, please wait until the next question to jump back in. Our meeting is only 50 minutes long and we have a lot of ground to cover. Thus, we kindly ask that you cover 1-2 points in your responses to give others time to respond as well. This is meant to be a free￾flowing discussion, but for ease of facilitation, please raise your hand if you would like to join the discussion. Say what you believe, even if it’s not what everyone thinks. There are no bad answers, just different opinions, and we want to hear them all. If you agree with what has been said, please say that. Otherwise, please share your points. From time to time, I may have to interrupt you to finish on time. I don’t mean to be rude, but there’s a lot to cover in a limited amount of time, so I apologize in advance if that happens. Now, let’s get started. Before we jump in, I’d like to get to know you bit. Could each of you please give tell me your name, where are you from, for how long have you been living in Panama, and what is your occupation here? I. Introduction Today, I would like to talk about the “Shifting the Power Dynamics: Engaging Men in Gender-Based Violence Reduction” program, implemented by HIAS and funded by USAID. We would like to know more about the activity, the services it provided, its role in the engaging with the community of Venezuelan migrants and members of the communities that host them, as well as your experience and your partners’ experience with it. I will refer to it as the HIAS workshops henceforth. Awareness First off, I would like to know how you learned about the HIAS workshops. Participation What activities did you participate in with the HIAS workshops? [INTERVIEWER: If not coming spontaneously, explore Inter-gender dialogues] Participation Did your spouse participate? [IF YES]: What did you hear from your spouse about these workshops? Did your spouse share anything they learned from the workshops? Did your spouse do anything differently after these workshops? 2. Relevance and Beneficiaries Thank you for your responses. Now, I would like to know your perspectives about the HIAS program in detail. Target Participants Do you know of anyone, man or woman, who wanted to participate in the HIAS workshops but couldn’t? If so, were they invited to join? [IF YES]: What prevented those people from participating? Target Participants [INTERVIEWER: Only if there are Inter-Gender Dialogue participants in the room]: When you decided to participate in the Inter-Gender Dialogues, how did you decide whether or not to participate? Did you have any concerns about participating? Was there anything specific that you hoped would be offered? Stakeholders During your participation in the HIAS workshops, did the HIAS staff ask you about aspects of the program that could be improved? [IF YES]: Do you believe the HIAS staff listened to your opinions about the workshops? What makes you believe that? Please explain Needs Assessment and Intervention Evidence Do you think the HIAS workshops responded to the needs and priorities of male participants? [IF YES]: Could you give me an example? [IF NOT]: Can you tell us what things you would have wanted them to discuss? 84 Topic Question What about your needs and priorities as a female participant? How well did the HIAS program talk about things you thought were important? Please, help me understand that with an example. 3. Activity Implementation We appreciate your thoughts so far. Now, my questions will talk about the actual implementation or hands-on practice of the HIAS workshops. Implementation What activities do you think the HIAS program did very well? Which ones could be improved? Which ones did you think were most important or relevant for you or your community? Could you help me understand the reason you think that? Needs Assessment and Intervention Evidence Are there any activities that you wish had been included in the HIAS workshops but were not? Assumptions Do you think that the HIAS workshops were responsive to the values and traditions of the community where you live? Please, could you explain the reason you think that? Please, give me an example. Assumptions [INTERVIEWER: Only if Target Participants was exclusive or incomplete] Do you have any recommendations on how the HIAS workshops could have reached Venezuelan migrants and their partners better? Assumptions Let’s talk now about challenges to the HIAS workshops. What have been the main challenges or difficulties in your community, to administer these workshops or reach other Venezuelan migrants in the area? Staffing and content [INTERVIEWER: Only if there are Inter-Gender Dialogue participants in the room]: Do you think that HIAS had enough staff to conduct the dialogues? [INTERVIEWER: Only if there are Inter-Gender Dialogue participants in the room]: - Did you like the facilitator? Tell me how she conducted the sessions - Did she talk about things that were important or relevant to you? - Were the sessions interesting or were you bored? - Did you think she knew her subject well? - Did you think she was respectful of all participants? 4. Activity Outcomes We are close to the end of the interview. Thank you so much for the comments made so far. All of them are very valuable. I want to talk briefly about the goals of the program. Outcomes What’s your understanding of the HIAS workshops´ goals? Outcomes To what extent did the HIAS workshops accomplish these results? [INTERVIEWER: DISCUSS EACH OUTCOME MENTIONED BY RESPONDENT] Protection What is the most important contribution of the HIAS workshops to protect Venezuelan migrants from gender-based violence? [INTERVIEWER: If not coming spontaneously, probe]: In addressing sexual violence, inter-partner violence, or forced early marriages? 85 Topic Question Prevention In your opinion, have the HIAS workshops helped to prevent acts of gender-based violence in your community? If yes, please explain. [INTERVIEWER: If not coming spontaneously, probe into sexual violence, inter-partner violence or forced early marriages] Prevention What other things could the HIAS workshops have offered to your community to prevent gender-based violence? Accountability Do you think that participating in the HIAS workshop increases men’s awareness about the responsibility of gender-based violence perpetrators in these incidents? Could you help me understand with an example? 5. Community Needs & Sustainability You have provided with great information. Thank you so much! I have a few last questions about your community needs and the future of the program. Design In your opinion, what problems related to gender-based violence should be prioritized in your community? Design Do you think the HIAS workshops prioritized the most important problems? [IF NOT]: Please, elaborate on it and give me an example. Design [IF YES]: Were the resources the program invested adequate to address that problem? Sustainability Considering your experience with the Inter-Gender Dialogues and your knowledge of your partner’s experience with the HIAS workshops, do you think that the sessions will have a long-lasting effect? Do you think these types of sessions will be important for your community in the future? Please, explain which ones and what is the reason you think so. Sustainability Last question, what should your community do to continue having those services? Closure I don’t have any more topics to discuss. Does anyone have anything else you would like to add to what we have discussed today? Thank you. I have learned a lot and I thank you for your participation and comments. Before leaving, do you have any questions for me? 86 Protocol to interview project participants (BTG4VM) Respondent code/name# ________________________________ Date: ________________________________ Start Time: ________________________________ Continued (if applicable) Date: ________________________________ Start Time: ________________________________ Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and am a researcher working for NORC at the University of Chicago. I appreciate you being here and sharing your thoughts and experiences with me. Today you’ll be participating in an interview as part of an evaluation of the Bridging the Gap for Venezuelan Migrants (BTG4VM) program for the Office of Gender Equality and Women’s Empowerment at the United States Agency for International Development, USAID. USAID contracted NORC as an external, independent organization to collect data that will inform USAID on their current and future funded projects, focused on preventing and responding to gender-based violence. While we do a lot of work with USAID, I do not work for USAID. NORC is completely neutral on all the issues we will be talking about and we’re just here to learn about your experiences. That means you don’t need to worry about making us happy or hurting our feelings. I am just a facilitator of this interview, and I am here to listen to you and keep the conversation moving. INFORMED CONSENT: Our interview is planned for 60 minutes. Your participation in this study is voluntary. If you are unable or prefer not to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in informing current and future USAID￾funded projects focused on GBV prevention and response. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. If you have questions, please e-mail Carlos Echeverria-Estrada, the director for this research at +1(312) 759 2658 or at echeverria-carlos@norc.org. If you have any other concerns about your rights as a research participant that have not been answered by the investigators, you may contact [NAME] at the Ministry of Health’s Institutional Review Board (IRB) and Ethics Committee at XXX-XXXX or guyanamohirb@gmail.com. Do you agree to participate? [RECEIVE CONSENT OR WITHDRAWAL, HAND A PRINTED COPY OF THE INFORMED CONSENT TO PARTICIPANT] CONSENT TO RECORD: Thank you very much for your participation. We would like to record this interview with your permission. This will enable us to go back and substantiate our notes. The recording and notes will never be shared with USAID or anyone outside of this research team. Once we’ve compared the recording and notes, we will destroy the recording. [START RECORDER] Do you agree to participate and to have this session recorded? [OBTAIN VERBAL CONSENT AND AVOID RECORDING ANY NAMES] I. Introduction Today, I would like to talk about the Bridging the Gap for Venezuelan Migrants (BTG4VM) program and your participation in it. I will call this program “BTG4VM” hereafter. I would like to learn more about the activity, the services it provides, your experience with it, and its influence on the role you play in your community, among both migrants and hosts. Awareness First, I would like to know how you learned about the BTG4VM program? Outreach When you were invited to participate, did you have any concerns or questions about participating? What type of concerns or questions did you have? Participation Could you let me know which of the following services you received from the BTG4VM program? [If necessary, probe the following: ● Referral pathways? 87 Topic Question ● Legal assistance for protection orders? ● Legal assistance for police report? ● Legal assistance for custody and other family related matters? ● Hotline services? ● Psychosocial support services? ● Housing shelters and safe spaces? ● Economic empowerment services (including professional skills training and entrepreneurial opportunities)? ● Immigration documentation support services ● Healthcare services (including sexual and reproductive health rights)? ● Document translation? ● Provide other information (migration, translation services, and local transportation assistance for school age children)? ● Other?] 2. Activity Implementation Thank you for sharing that with me. Now, I would like to ask about your experience with each of the services you received from BTG4VM more in detail. Is that OK? [INTERVIEWER: Keep the list of services received at hand and administer the rest of this section for each of them.] Service delivery Was it easy to access the [USED SERVICE]? If not, how could it have been easier to access? Cultural competency Do you think that the [USED SERVICE] you received from BTG4VM was respectful of and responsive to the values and traditions of your community? Could you explain the reason you think that? Please, give me an example. What about the values and traditions of your host community? Did the [USED SERVICE] you received from BTG4VM consider those too? Please, give me an example. Service target groups Do you think that there are other people in your community who did not receive the [USED SERVICE] from BTG4VM and should have received it? If so, what prevented those people from receiving the service? [NOTE: if they don’t know or cannot recall, ask]: ‘Can you think of some ways that BTG4VM might be able to help other female Venezuelan migrants like you?’ Staffing What do you think about the [USED SERVICE] staff? [PROBE:] a. What did you like about their work? b. What could they have done better to help you? c. Did they listen to you to understand your problem? d. What could they have done better to help you? Was the staff member who worked with you knowledgeable of the [USED SERVICE] you received? Service uptake and suitability Did the [USED SERVICE] you received from BTG4VM program meet the needs you had at the time you approached the program? [INTERVIEWER: If ‘don’t know’ or ‘cannot recall’, probe on what the needs were.] Service quality In your opinion, what type of things could BTG4VM do to improve the delivery of the [USED SERVICE] that you received? Could you give me an example of this? 3. Responsiveness and Adaptability Thank you for sharing that with me. Now, I would like to ask about your experience with these services and BTG4VM regarding the way they use feedback and experience with migrants. Is that OK? [INTERVIEWER: The following two sections refer to summative questions, ask them to refer to the activity itself, not isolated services, to the extent possible] Responsiveness During your participation in BTG4VM, did program staff ask you about aspects of the service that could improve? If yes, what did you suggest? If not, what would you have suggested? 88 Topic Question Monitoring and adaptations [IF YES to above]: Do you believe the program staff listened to your opinions about the services you received from the program? What makes you believe that? Monitoring and adaptations [IF YES to above]: Do you think BTG4VM program was able to adapt the [USED SERVICE] when it was working well to address your needs? Could you give an example, please? [IF NO to above]: Could you share with me why you think that way? Please give me an example. Staffing Do you think that BTG4VM had enough staff to provide the services you received? Needs Assessment and Intervention Evidence Are there any things you wish had been included in the [USED SERVICES] service but were not? Please, explain. 4. Program Results I appreciate your thoughts so far. Now, my questions will talk about the results of your participation in BTG4VM. [INTERVIEWER: The following two sections refer to summative questions, ask them to refer to the activity itself, not isolated services, to the extent possible] Perception If someone asked you, what would you tell your friends about BTG4VM? Do you think you’d want to participate in BTG4VM again? Satisfaction Which of the services you received were the most useful for you? Which were least useful? Please, share with me an example of these experiences. Outcomes What were the main changes that happen after your participation in the services you received from BTG4VM? Were these changes good for you? Did your interaction with your community of migrants change after you received the [USED SERVICE] from BTG4VM? Could you help me understand and share an example, please? What about your interaction with the host community of Panamanians? Did your relationships with them changes too? If so, could you give me an example? [INTERVIEWER: if they don’t know or cannot recall ask “Can you give me some examples of services you can remember?”] Needs Assessment and Intervention Evidence Are there any things you wish the BTG4VM or its services could have accomplished but did not? Please explain. Others’ perception Did you hear other people talking about BTG4VM? If yes, what were they saying? 5. Community Needs & Sustainability You have provided with great information. Thank you so much! I have a few last questions about your community needs and the future of the program. Design In your opinion, what problems related to gender-based violence still should be prioritized in your community? Outcome When thinking about BTG4VM, can you think of anything that changed for anyone in your community because of the it? Sustainability Considering your experience with the services that BTG4VM provided, does your community, including your host community, still need any of these services in the future? Please, explain which ones and what is the reason you think so. Sustainability Last question, what should this community do to continue having those services? I don’t have any more topics to discuss. I have learned a lot and I thank you for your participation and comments. Before leaving, is there anything else you think we should talk about that we have not done so far? Again, I appreciate the time, experiences, and perspective you shared with me today. 89 Protocol to interview project participants (WELCOME) Respondent code/name# ________________________________ Date: ________________________________ Start Time: ________________________________ Continued (if applicable) Date: ________________________________ Start Time: ________________________________ Topic Question INTRO / ACKNOWLEDGE: Hello. My name is ______ and am a researcher working for NORC at the University of Chicago. I appreciate you being here and sharing your thoughts and experiences with me. Today you’ll be participating in an interview as part of an evaluation of the Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME) program for the Office of Gender Equality and Women’s Empowerment at the United States Agency for International Development, USAID. USAID contracted NORC as an external, independent organization to collect data that will inform USAID on their current and future funded projects, focused on preventing and responding to gender-based violence. While we do a lot of work with USAID, I do not work for USAID. NORC is completely neutral on all the issues we will be talking about and we’re just here to learn about your experiences. That means you don’t need to worry about making us happy or hurting our feelings. I am just a facilitator of this interview, and I am here to listen to you and keep the conversation moving. INFORMED CONSENT: Our interview is planned for 60 minutes. Your participation in this study is voluntary. If you are unable or prefer not to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in informing current and future USAID￾funded projects focused on GBV prevention and response. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. If you have questions, please e-mail Carlos Echeverria-Estrada, the director for this research at +1(312) 759 2658 or at echeverria-carlos@norc.org. If you have any other concerns about your rights as a research participant that have not been answered by the investigators, you may contact April Baker, IRB Director at NORC at the University of Chicago, at +1 (312) 759-4014 or irb@norc.org. Do you agree to participate? [RECEIVE CONSENT OR WITHDRAWAL, HAND A PRINTED COPY OF THE INFORMED CONSENT TO PARTICIPANT] CONSENT TO RECORD: Thank you very much for your participation. We would like to record this interview with your permission. This will enable us to go back and substantiate our notes. The recording and notes will never be shared with USAID or anyone outside of this research team. Once we’ve compared the recording and notes, we will destroy the recording. [START RECORDER] Do you agree to participate and to have this session recorded? [OBTAIN VERBAL CONSENT AND AVOID RECORDING ANY NAMES] I. Introduction Today, I would like to talk about the Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME) program and your participation in it. I will call this program “WELCOME” hereafter. I would like to learn more about the activity, the services it provides, your experience with it, and its influence on the role you play in your community, among both migrants and hosts. 90 Topic Question Awareness First, I would like to know how you learned about the WELCOME program? Outreach When you were invited to participate, did you have any concerns or questions about participating? What type of concerns or questions did you have? Participation Could you let me know which of the following services you received from the WELCOME program? [If necessary, probe the following: ● Counseling and other psychosocial services ● Health care ● Legal services ● Services for refugees and asylum seekers ● Services for victims of human trafficking ● Emergency shelter services ● Youth services ● Other? 2. Activity Implementation Thank you for sharing that with me. Now, I would like to ask about your experience with each of the services you received from WELCOME more in detail. Is that OK? [INTERVIEWER: Keep the list of services received at hand and administer the rest of this section for each of them.] Service delivery Was it easy to access the [USED SERVICE]? If not, how could it have been easier to access? Cultural competency Do you think that the [USED SERVICE] you received from WELCOME was respectful of and responsive to the values and traditions of your community? Could you explain the reason you think that? Please, give me an example. What about the values and traditions of your host community? Did the [USED SERVICE] you received from WELCOME consider those too? Please, give me an example. Service target groups Do you think that there are other people in your community who did not receive the [USED SERVICE] from WELCOME and should have received it? If so, what prevented those people from receiving the service? [NOTE: if they don’t know or cannot recall, ask]: ‘Can you think of some ways that WELCOME might be able to help other female Venezuelan migrants like you?’ Staffing What do you think about the [USED SERVICE] staff? [PROBE:] e. What did you like about their work? f. What could they have done better to help you? g. Did they listen to you to understand your problem? h. What could they have done better to help you? Was the staff member who worked with you knowledgeable of the [USED SERVICE] you received? Service uptake and suitability Did the [USED SERVICE] you received from WELCOME program meet the needs you had at the time you approached the program? [INTERVIEWER: If ‘don’t know’ or ‘cannot recall’, probe on what the needs were.] Service quality In your opinion, what type of things could WELCOME do to improve the delivery of the [USED SERVICE] that you received? Could you give me an example of this? 3. Responsiveness and Adaptability Thank you for sharing that with me. Now, I would like to ask about your experience with these services and WELCOME regarding the way they use feedback and experience with migrants. Is that OK? [INTERVIEWER: The following two sections refer to summative questions, ask them to refer to the activity itself, not isolated services, to the extent possible] 91 Topic Question Responsiveness During your participation in WELCOME, did program staff ask you about aspects of the service that could improve? If yes, what did you suggest? If not, what would you have suggested? Monitoring and adaptations [IF YES to above]: Do you believe the program staff listened to your opinions about the services you received from the program? What makes you believe that? Monitoring and adaptations [IF YES to above]: Do you think WELCOME program was able to adapt the [USED SERVICE] when it was working well to address your needs? Could you give an example, please? [IF NO to above]: Could you share with me why you think that way? Please give me an example. Staffing Do you think that WELCOME had enough staff to provide the services you received? Needs Assessment and Intervention Evidence Are there any things you wish had been included in the [USED SERVICES] service but were not? Please, explain. 4. Program Results I appreciate your thoughts so far. Now, my questions will talk about the results of your participation in WELCOME. [INTERVIEWER: The following two sections refer to summative questions, ask them to refer to the activity itself, not isolated services, to the extent possible] Perception If someone asked you, what would you tell your friends about WELCOME? Do you think you’d want to participate in WELCOME again? Satisfaction Which of the services you received were the most useful for you? Which were least useful? Please, share with me an example of these experiences. Outcomes What were the main changes that happen after your participation in the services you received from WELCOME? Were these changes good for you? Did your interaction with your community of migrants change after you received the [USED SERVICE] from WELCOME? Could you help me understand and share an example, please? What about your interaction with the host community of Panamanians? Did your relationships with them changes too? If so, could you give me an example? [INTERVIEWER: if they don’t know or cannot recall ask “Can you give me some examples of services you can remember?”] Needs Assessment and Intervention Evidence Are there any things you wish the WELCOME or its services could have accomplished but did not? Please explain. Others’ perception Did you hear other people talking about WELCOME? If yes, what were they saying? 5. Community Needs & Sustainability You have provided with great information. Thank you so much! I have a few last questions about your community needs and the future of the program. Design In your opinion, what problems related to gender-based violence still should be prioritized in your community? Outcome When thinking about WELCOME, can you think of anything that changed for anyone in your community because of the it? Sustainability Considering your experience with the services that WELCOME provided, does your community, including your host community, still need any of these services in the future? Please, explain which ones and what is the reason you think so. Sustainability Last question, what should this community do to continue having those services? 92 I don’t have any more topics to discuss. I have learned a lot and I thank you for your participation and comments. Before leaving, is there anything else you think we should talk about that we have not done so far? Again, I appreciate the time, experiences, and perspective you shared with me today. 93 WOMEN EXERCISING LEADERSHIP FOR COHESION AND MEANINGFUL EMPOWERMENT (WELCOME) OPERATIONAL STAFF SURVEY QUESTIONNAIRE NOTES This instrument be administered to the service providers for WELCOME in Trinidad and Tobago. [Blue text in brackets] are survey programmer instructions. [Purple text in brackets] indicate when a survey question or set of response options will need to be adjusted for country-specific use. 94 CONSENT PURPOSE OF STUDY NORC at the University of Chicago, a non-partisan research institution based in the United States, is carrying out a performance evaluation of the portfolio of the gender-based violence activities funded by the United States Agency for International Development (USAID). One of the activity clusters is the Better Together Challenge (BTC), a USAID-funded initiative to crowdsource, fund, and scale innovative solutions from anywhere in the world to improve the lives of Venezuelan migrants and communities hosting them in Latin America and the Caribbean. We are inviting you to participate in this evaluation because of your role in implementing the activities under “Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME)”, a BTC activity. The purpose of this study is to understand what is working, challenges faced, knowledge gaps, and what can be improved in the USAID’s GBV portfolio. DESCRIPTION OF THE STUDY PROCEDURES If you agree to be in this study, you will be asked to answer survey questions about your views on activity design, service provision, implementation, uptake, and monitoring. The online survey will take approximately 15-20 minutes to complete. RISKS/DISCOMFORTS OF PARTICIPATING IN THIS STUDY Your participation in this study does not involve any risks other than what you would encounter in a normal workday at your workplace. If you are uncomfortable, you are free to not answer or to skip to the next question. However, we will always confirm you want to continue without providing an answer; your responses are very valuable to this study and for improving future grant opportunities. BENEFITS OF PARTICIPATING IN THE STUDY Your participation is important to help us and USAID learn more about the implementation of the GBV activities funded, including lessons learned and areas for improvement. You will receive no economic or material incentive for participating. CONFIDENTIALITY Your responses to this survey will be kept strictly confidential. We will report all results as averages. We will never share any information that could be used to identify you outside of the research team. At the end of the study, we may share the data with USAID or others outside the study team. Before sharing the data, we will remove all details that could be used to identify you, such as name, employer, or IP address used to answer the survey. As such, no one will know whether you participated in the survey or which answers are yours. Since no one will know which answers are yours, we ask that you answer all questions honestly. RIGHT TO REFUSE OR WITHDRAW The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any time. You have the right not to answer any single question, as well as to withdraw 95 completely from the study at any point during the process; additionally, you have the right to request that I delete your answers. There are no penalties for refusing or withdrawing. RIGHT TO ASK QUESTIONS AND REPORT CONCERNS You have the right to ask questions about this research study and to have those questions answered by me before, during or after the research. If you have any further questions about the study, feel free to contact Carlos Echeverria-Estrada at echeverria-carlos@norc.org or by telephone at +1(612) 876 6339. If you have any other concerns about your rights as a research participant that have not been answered by the investigators, you may contact April Baker, NORC’s Senior Institutional Review Board Manager, at irb@norc.org. Consent. Do you agree to participate in this survey? 1. Yes 2. No SCREENER RespType. What institution(s) or organization(s) are you affiliated with in implementing the Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME) program? You may select up to three. 1. La Casita Hispanic Cultural Center 2. Rape Crisis Society of T&T 3. Families in Action 4. National Family Services 5. Living Water Community 6. Family Planning Association of Trinidad & Tobago 7. TTVSOLNET (TTV Solidarity Network) 8. International Organization for Migration 9. United Nations High Commissioner for Refugees 10. Counter Trafficking Unit (T&T Police Service) 11. Institute of Gender and Development Studies (IGDS) at University of the West Indies COMMUNITY SERVICES To start, we will ask you a few questions about the availability of services in the community. Service_avail. From the following list, please indicate what services are available for Venezuelan migrants who are GBV survivors in your community, including those that you or your organization provides. [select all that apply] 1. Referral pathways 2. Legal assistance for protection orders 3. Legal assistance for police report 4. Legal assistance for custody and other family related matters 5. Hotline services 6. Counseling services 7. Psychosocial support services 96 8. Emergency shelter services 9. Economic empowerment services (including professional skills training and entrepreneurial opportunities) 10. Immigration documentation support services 11. Healthcare services (including sexual and reproductive health rights) 12. Services for refugees and asylum seekers 13. Services for victims of human trafficking 14. Youth services 15. Document translation 16. Provide other information (migration, translation services, and local transportation assistance for school age children) 17. Other [Specify] 18. Don’t know 19. No response Service_use. [For each service selected in service_avail] To your knowledge, have Venezuelan migrants who are survivors of GBV used or received this service: [selected service]? 1. Yes 2. No 3. Don’t know 4. No response Thinking about Venezuelan migrants who have received GBV survivor support services… Service_access. To your knowledge, to what extent are they satisfied with their overall access to services in the community? 1. Very satisfied 2. Satisfied 3. Dissatisfied 4. Very dissatisfied 5. Don’t know 6. No response Service_effect. To what extent are they satisfied with the overall effectiveness of these community services? 7. Very satisfied 8. Satisfied 9. Dissatisfied 10. Very dissatisfied 11. Don’t know 12. No response Learn_satisfaction. [If service_effect = 5 or 6] To what extent do you agree that learning about GBV survivors’ satisfaction with the services provided is important for improving these services? 1. Strongly agree 2. Agree 3. Neutral (neither agree nor disagree) 97 4. Disagree 5. Strongly disagree 6. Don’t know 7. No response 98 WELCOME NETWORK COMPONENTS Thank you. Now, we will ask you some questions specifically about the WELCOME program and the network of services provided. Service_provided. From the following list, please select the services you or your organization(s) provided in partnership with the WELCOME program. [Select all that apply] Direct services to GBV survivors 1. Counseling and other psychosocial services 2. Health care 3. Legal services 4. Services for refugees and asylum seekers 5. Services for victims of human trafficking 6. Emergency shelter services 7. Youth services Advocate Recruitment and Training 8. Recruiting Advocates 9. Advocate training – Information on GBV and its context in Trinidad and Tobago 10. Advocate training – Survivor and Advocate safety protocols 11. Advocate training – Building capacity to provide psychological first aid 12. Advocate training – Information on migrant rights 13. Advocate training – Survivor referral information and processes 14. Advocate training – Self-care 15. Providing ongoing training of Advocates Partnerships & Organizational Capacity Building 16. Developing referral pathways 17. Matching Advocates with Survivors 18. Sharing information and resources on social media 19. Additional Training of Trainers 20. Knowledge and expertise on data collection tools 21. Other [Specify:] 22. Don’t know 23. No response Resource_needed. What are the top three resources, services, or programs that the migrant community in your city or neighborhood need most to prevent or combat GBV? We are interested specifically in identifying gaps in community service provision, which include lack of a particular service or inadequate provision of a service. [Select up to three] 1. Referral pathways 2. Legal assistance for protection orders 3. Legal assistance for police report 4. Legal assistance for custody and other family related matters 5. Hotline services 6. Counseling services 99 7. Psychosocial support services 8. Emergency shelter services 9. Economic empowerment services (including professional skills training and entrepreneurial opportunities) 10. Immigration documentation support services 11. Healthcare services (including sexual and reproductive health rights) 12. Services for refugees and asylum seekers 13. Services for victims of human trafficking 14. Youth services 15. Document translation 16. Provide other information (migration, translation services, and local transportation assistance for school age children) 17. Other [Specify] 18. Don’t know 19. No response [for each selected service] Service_provision. To what extent did the WELCOME program and its network of partnering organizations provide this service via advocate work or referral: [selected service]? 1. To a Great Extent 2. Somewhat 3. Very Little 4. Not at All 5. Don’t know 6. No response [for each selected service that was provided to some extent] Service_use. To what extent did GBV survivors use this service: [selected service]? 1. To a Great Extent 2. Somewhat 3. Very Little 4. Not at All 5. Don’t know 6. No response [for each selected service that was used to some extent] Service_satisfy. In your opinion, to what extent were GBV survivors satisfied with this service: [selected service]? 1. Very satisfied 2. Satisfied 3. Dissatisfied 4. Very dissatisfied 5. Don’t know 6. No response TARGET GROUPS 100 Now, we are going to ask you some questions about the groups of people your organization(s) serves. Group. Please, select the groups with which your organization(s) works [select multiple] o Venezuelan migrants: girls and female teenagers (up to age 17) o Venezuelan migrants: boys and male teenagers (up to age 17) o Venezuelan migrants: women o Venezuelan migrants: men o Venezuelan migrants: female members of the LGBTQIA+ community o Venezuelan migrants: male members of the LGBTQIA+ community o Venezuelan migrants: non-binary or non-gender conforming individuals o Trinidadian or Tobagonian nationals: women o Trinidadian or Tobagonian nationals: men o Trinidadian or Tobagonian nationals: female members of the LGBTQIA+ community o Trinidadian or Tobagonian nationals: male members of the LGBTQIA+ community o Trinidadian or Tobagonian nationals: non-binary or non-gender conforming individuals o Individuals recruited and trained to be Advocates o La Casita employees included in all training activities o Other [Specify] o Don’t know o No response [LOOP FOR EACH GROUP SELECTED]: Group_needs. In what ways the WELCOME program addressed [Group’s] needs? [Select multiple] 1. Providing referral services 2. Counseling and other psychosocial services 3. Health care 4. Legal services 5. Services for refugees and asylum seekers 6. Services for victims of human trafficking 7. Emergency shelter services 8. Youth services 9. Promoting services to prevent or protect from GBV 10. Other [Specify] 11. Don’t know 12. No response Group_approach. To what extent has your organization(s) used a tailored approach in delivering its services to [Group]? 1. To a Great Extent 2. Somewhat 3. Very Little 4. Not at All 5. Don’t know 6. No response 101 Group_resources. To what extent has your organization had to invest additional resources since 2020 to satisfy [Group]’s needs? 1. To a Great Extent 2. Somewhat 3. Very Little 4. Not at All 5. Don’t know 6. No response [If 1 or 2 selected]: Group_resources_specify. Please, indicate the additional resources that were invested [Select multiple] 1. Additional staffing 2. Additional funding 3. Additional activities 4. Additional planning 5. Longer implementation time 6. Additional community outreach 7. Other [Specify] 8. Don’t know [exclusive] 9. No response [exclusive] Thank you for your responses. We would like to ask you a few more questions about these groups’ access to and participation in GBV services. As a reminder, the groups mentioned are: [pipe in Groups response options]. Group_access. Did all groups have equal access to the services your organization(s) provided through WELCOME program, or were there differences in the access across groups? 1. They were even 2. There were differences 3. Don’t know 4. No response [IF DIFFERENCES:] Group_access_dif. If there were differences in access, please name which groups did not participate as much. (open-ended) Group_outcomes. Did the services your organization(s) provided through WELCOME program impact all target groups equally, or were there differences in the results across groups? 5. They were even 6. There were differences 7. Don’t know 8. No response [IF DIFFERENCES:] Group_outcomes_dif. Please briefly describe the differences in results of your WELCOME program related between the target groups (open-ended) Group_participation. Are there people in your community who you think should have received the WELCOME services but did not? 102 1. Yes 2. No 3. Don’t know 4. No response [If Yes]: Please specify the group (open ended) Other_providers. Are there organizations in your community that you think should have provided services for the WELCOME network but did not? 1. Yes 2. No 3. Don’t know 4. No response [If Yes]: Please specify who (open ended) CONTEXT APPROPRIATENESS [Transition] Thank you very much for your responses this far. We’re also interested in learning about your views on how the WELCOME program fit the community and any challenges encountered during implementation. Activity_fit. To what extent did the services provided as part of the WELCOME program fit well in your community ? 1. To a Great Extent 2. Somewhat 3. Very Little 4. Not at All 5. Don’t know 6. No response Adaptation. To what extent do you agree with this statement: The WELCOME program and its service network staff adapted to the context of my community? 1. Strongly agree 2. Agree 3. Neutral (neither agree nor disagree) 4. Disagree 5. Strongly disagree 6. Don’t know 7. No response Adaptation_explain_y. [If “Strongly Agree” or “Agree”]. Please, briefly elaborate on your answer above to describe how the staff adapted to the context of your community. (open￾ended) Adaptation_explain_n. [If “Disagree” or “Strongly Disagree”]. Please, briefly elaborate on your answer above to describe how the staff failed to adapt to the context of your community. (open-ended) 103 Challenge_context. What were the three main challenges in implementing services in partnership with the WELCOME program? [select up to three]? 1. Limited transportation/mobility of Survivors 2. Difficulty of Survivors to commit to schedules 3. Scheduling conflicts with Advocate training 4. Small number of trained Advocates 5. Advocates and Survivors not living in the same general locations 6. Language barriers between Advocates and Survivors 7. Lack of capacity to support male survivors 8. COVID-19 limiting service availability 9. Lack of standardized processes 10. Slow pace of information gathering 11. Difficulty in documenting if the beneficiary received referral service 12. Organizational barriers with the main grantee 13. No challenges [exclusive] 14. Other [Specify:] 15. Don’t know [exclusive] 16. No response [exclusive] GBV PREVENTION [Transition] Now, we are going to ask about your community values and the WELCOME program’s role in GBV prevention. Prevent_conflict. Are GBV prevention efforts in conflict with any values or traditions in your community? 1. Yes 2. No 3. Don’t Know 4. No response [If Yes]: Prevent_conflict_sp What community values or traditions are most in conflict with GBV prevention efforts? (open-ended) Prevent_gbv. To what extent has the WELCOME program helped prevent GBV in your community? 1. No effect 2. Minor effect 3. Moderate effect 4. Major effect 5. Don’t know 6. No response Groups_change. In your community, what are the top three groups that have the most power in influencing change to promote GBV protection and prevention? [select up to three] 1. Funders/donors 2. National authorities 104 3. Local authorities 4. Neighbor associations 5. Immigrant groups or associations 6. Survivor groups or associations 7. Non-profit organizations 8. Operational staff 9. Other, specify 10. Don’t know 11. No response Prevent_effect. Rate the effectiveness of WELCOME network services providers in preventing GBV. 1. Counseling and other psychosocial services 2. Health care 3. Legal services 4. Services for refugees and asylum seekers 5. Services for victims of human trafficking 6. Emergency shelter services 7. Youth services [For each component above, rate:] a. No effect b. Minor effect c. Moderate effect d. Major effect e. Don’t know f. No response Service_impact. Overall, from the following list, please identify the component of the WELCOME program that led to the most positive impact on the migrant community? 1. Advocate training and matching 2. Establishing a trusted referral network 3. Counseling and other psychosocial services 4. Health care 5. Legal services 6. Services for refugees and asylum seekers 7. Services for victims of human trafficking 8. Emergency shelter services 9. Youth services 10. Sharing information and resources on social media 11. Don’t know 12. No response MONITORING SYSTEM [Transition] We are nearing the end of the survey. Now, we would like to ask you a few questions about your familiarity with the monitoring system implemented by the WELCOME program. 105 Monitor_familiar. In your work with the WELCOME program, have you or your team members been asked by Democracy International to share data on the beneficiaries that use your organization(s)’s services? 1. Yes 2. No [Skip block] 3. Don’t know [Skip block] 4. No response [Skip block] [IF YES]: Monitordata_type. Please, indicate what kinds of data you or your team members have shared with Democracy International for the WELCOME program. Select all that apply. [select multiple] 1. Number of program participants 2. Demographics of project participants (gender, age, nationality, address, etc.) 3. Diagnosis of project user needs (immigration regularization, mental health counseling, housing support, food insecurity, etc.) 4. Monetary investment per project participant 5. Number of appointments/contacts with project participant 6. Changes in income outcomes/results over time 7. Changes in education outcomes/results over time 8. Changes in migration status over time 9. Changes in housing status over time 10. Participant self-reported self-efficacy 11. Participant self-reported feeling of support 12. Participant self-reported access to service 13. Other [Specify] 14. Don’t know [exclusive] 15. No response [exclusive] Data_understand. To what extent do you consider the data requests from Democracy International clear? 1. To a Great Extent 2. Somewhat 3. Very Little 4. Not at All 5. Don’t know 6. No response Data_accessible. Did you or your team have access to the grantee monitoring system that uses the data your organization(s) provided? 1. Yes 2. No [skip to ‘Monitor_freq’] 3. Don’t know [skip to ‘Monitor_freq’] 4. No response [skip to ‘Monitor_freq’] [IF YES] 106 Monitor_practical. To what extent do you consider the system practical in terms of entering the data? 1. To a Great Extent 2. Somewhat 3. Very Little 4. Not at All 5. Don’t know 6. No response Monitor_effective. To what extent do you consider the system useful to extract information to tailor your organization’s services to beneficiary needs? 1. To a Great Extent 2. Somewhat 3. Very Little 4. Not at All 5. Don’t know 6. No response Monitor_freq. Did the management of the WELCOME program use these data of your activity to adapt the activity? 1. Yes 2. No [Skip block] 3. Don’t know [Skip block] 4. No response [Skip block] [IF YES] Please, provide an example of such adaptations (open ended) SUSTAINABILITY Thank you so much for the answers you have provided so far. To finish our questions about the WELCOME program, we are interested in your thoughts about the current and future implementation. [Loop for each service marked in ‘Service_provided”] Sustain_current. What is the current status of this service: [service]? 1. We continue providing [service] with WELCOME program 2. We suspended [service] with the WELCOME program 3. We refer the WELCOME program beneficiaries to another service provider for this service 4. Other (please, specify: ) 5. Don’t know 6. Refuse [If ‘Sustain_current’ = 1] Sustain_sources_open. Please, indicate the source of funding for this service: [open￾ended] 107 Sustain_likelihood. In your opinion, what is the likelihood that your organization(s) will continue to provide [service] for the next five years?. 1. Very likely 2. Likely 3. Neutral (neither likely nor unlikely) 4. Unlikely 5. Very unlikely 6. Don’t know 7. No response Sustain_desire_comp. Which of the WELCOME components you would like to see continue in your organization/community after this activity has ended? [Select multiple] 1. Advocate training and matching 2. Establishing a trusted referral network 3. Counseling and other psychosocial services 4. Health care 5. Legal services 6. Services for refugees and asylum seekers 7. Services for victims of human trafficking 8. Emergency shelter services 9. Youth services 10. Sharing information and resources on social media 11. None 12. Don’t Know 1. No response [FOR EACH YES:] Sustain_resource. What do you think your organization/community needs to do to make sure this activity continues? (open￾ended response) Scalability. Which activities of the WELCOME program do you think could increase their coverage (scale up) in Trinidad and Tobago? [Select multiple] Advocate training and matching Establishing a trusted referral network Sharing information and resources on social media Health care Legal services Services for refugees and asylum seekers Services for victims of human trafficking Emergency shelter services Youth services Sharing information and resources on social media None [exclusive] Don’t Know [exclusive] 1. No response 108 Replicability. Which activities of the WELCOME program do you think could be implemented in other communities in Trinidad and Tobago? [Select multiple] 1. Advocate training and matching 2. Establishing a trusted referral network 3. Sharing information and resources on social media 4. Health care 5. Legal services 6. Services for refugees and asylum seekers 7. Services for victims of human trafficking 8. Emergency shelter services 9. Youth services 10. Sharing information and resources on social media 11. None [exclusive] 12. Don’t Know [exclusive] 13. No response Replicability_where. Where in Trinidad and Tobago should [activity] be replicated? 1. Couva–Tabaquite–Talparo 2. Diego Martin 3. Mayaro–Rio Claro 4. Penal–Debe 5. Princes Town 6. San Juan–Laventille 7. Sangre Grande 8. Siparia 9. Tunapuna–Piarco 10. Don’t know DEMOGRAPHICS We appreciate your invaluable insights. Before letting you go, we’d like to ask the following questions for statistical purposes. Remember that your answers are confidential and we will use the information in the following section to analyze trends and present aggregate results. Dem_gender. What is your gender? 1. Man 2. Woman 3. Other, specify [open-ended] 4. Don’t know 5. No response Dem_dob. About how old are you? 1. 18-24 2. 25-34 3. 35-44 4. 45-54 109 5. 55-64 6. 65-74 7. 75+ 8. Prefer not to say Dem_lang. Which languages are you capable of speaking fluently? (Check all that apply.) 1. English 2. Trinidad English 3. Spanish 4. French Creole 5. Hindi 6. Other [Specify] 7. Prefer not to say Dem_edu. What is the highest degree or level of education you have completed? 1. Never attended school 2. Some primary school 3. Complete primary school 4. Some secondary school 5. Complete secondary school 6. Some higher education 7. Complete higher education 8. Advanced degree 9. Technical school 10. Prefer not to say [For each organization the respondent is affiliated with] RespType1.a. What is your role (title) in that [organization]? [open-ended] RespType1.c. How many years have you worked with your [organization]? 1. Less than one year 2. One to three years 3. Four to six years 4. Seven or more years 5. Don’t know 6. No response Dem_region. Where does [organization] work in Trinidad and Tobago? 1. Couva–Tabaquite–Talparo 2. Diego Martin 3. Mayaro–Rio Claro 4. Penal–Debe 5. Princes Town 6. San Juan–Laventille 7. Sangre Grande 8. Siparia 110 9. Tunapuna–Piarco 10. Prefer not to say RespType1.b Overall, at what level is your current position? 1. Entry-level or early career 2. Mid-career 3. Senior 4. Don’t know 5. No response End of Survey Message: Close. You have now completed the survey. Thank you for your participation. 111 BRIDGING THE GAP FOR VENEZUELAN MIGRANTS (BTG4VM) (BETTER TOGETHER CHALLENGE) OPERATIONAL STAFF SURVEY QUESTIONNAIRE NOTES This instrument be administered to the service providers for BTG4VM in Guyana. [Blue text in brackets] are survey programmer instructions. [Purple text in brackets] indicate when a survey question or set of response options will need to be adjusted for country-specific use. 112 CONSENT PURPOSE OF STUDY NORC at the University of Chicago, a non-partisan research institution based in the United States, is carrying out a performance evaluation of the portfolio of the gender-based violence activities funded by the United States Agency for International Development (USAID). One of the activity clusters is the Better Together Challenge (BTC), a USAID-funded initiative to crowdsource, fund, and scale innovative solutions from anywhere in the world to improve the lives of Venezuelan migrants and communities hosting them in Latin America and the Caribbean. We are inviting you to participate in this evaluation because of your role in implementing the activities under “Bridging the Gap for Venezuelan Migrants (BTG4VM)”, a BTC activity. The purpose of this study is to understand what is working, challenges faced, knowledge gaps, and what can be improved in the USAID’s GBV portfolio. DESCRIPTION OF THE STUDY PROCEDURES If you agree to be in this study, you will be asked to answer survey questions about your views on activity design, service provision, implementation, uptake, and monitoring. The online survey will take approximately 30 minutes to complete. RISKS/DISCOMFORTS OF PARTICIPATING IN THIS STUDY Your participation in this study does not involve any risks other than what you would encounter in a normal workday at your workplace. If you are uncomfortable, you are free to not answer or to skip to the next question. However, we will always confirm you want to continue without providing an answer; your responses are very valuable to this study and for improving future grant opportunities. BENEFITS OF PARTICIPATING IN THE STUDY Your participation is important to help us and USAID learn more about the implementation of the GBV activities funded, including lessons learned and areas for improvement. You will receive no economic or material incentive for participating. CONFIDENTIALITY Your responses to this survey will be kept strictly confidential. We will report all results as averages. We will never share any information that could be used to identify you outside of the research team. At the end of the study, we may share the data with USAID or others outside the study team. Before sharing the data, we will remove all details that could be used to identify you, such as name, employer, or IP address used to answer the survey. As such, no one will know whether you participated in the survey or which answers are yours. Since no one will know which answers are yours, we ask that you answer all questions honestly. RIGHT TO REFUSE OR WITHDRAW The decision to participate in this study is entirely up to you. You may refuse to take part in the study at any time. You have the right not to answer any single question, as well as to withdraw completely from the study at any point during the process; additionally, you have the right to request that I delete your answers. There are no penalties for refusing or withdrawing. 113 RIGHT TO ASK QUESTIONS AND REPORT CONCERNS You have the right to ask questions about this research study and to have those questions answered by me before, during or after the research. If you have any further questions about the study, feel free to contact Carlos Echeverria-Estrada at echeverria-carlos@norc.org or by telephone at +1(612) 876 6339. If you have any other concerns about your rights as a research participant that have not been answered by the investigators, you may contact Dr. Reeta Gobin with the Guyana Ministry of Health’s Institutional Review Board (IRB) and Ethics Committee at 592-226-1224 or guyanamohirb@gmail.com. Consent. Do you agree to participate in this survey? 3. Yes 4. No SCREENER RespType. What institution(s) or organization(s) are you affiliated with? You may select up to three. 12. NCC 13. International Organisation for Migrants 14. Catholic Charities Organisation Guyana -Migrant Support Services 15. Food for the Poor 16. Guyana Responsible Parenthood Association 17. Hebrew Immigrant Aid Society 18. Help and Shelter 19. Hope Foundation 20. Ministry of Human Serivces & Social Security (Sexual Offences & Domestic Violence Policy Unit) 21. United Nations High Commission for Refugees 22. United Nations Population Fund 23. Voices Gy COMMUNITY SERVICES To start, we will ask you a few questions about the availability of services in the community. Service_avail. From the following list, please indicate what services are available in your community for Venezuelan migrants who are GBV survivors. These may include services that you or your organization(s) provides. [select all that apply] 20. Referral pathways 21. Legal assistance for protection orders 22. Legal assistance for police report 23. Legal assistance for custody and other family related matters 24. Hotline services 25. Psychosocial support services 26. Housing shelters and safe spaces 114 27. Economic empowerment services (including professional skills training and entrepreneurial opportunities) 28. Immigration documentation support services 29. Healthcare services (including sexual and reproductive health rights) 30. Document translation 31. Provide other information (migration, translation services, and local transportation assistance for school age children) 32. Other [Specify] 33. Don’t know 34. No response Service_use. [For each service selected in service_avail] To your knowledge, have Venezuelan migrants who are survivors of GBV used or received this service: [selected service]? 5. Yes 6. No 7. Don’t know 8. No response Thinking about Venezuelan migrants who have received GBV survivor support services… Service_access. To your knowledge, to what extent are they satisfied with their overall access to services in the community? 13. Very satisfied 14. Satisfied 15. Dissatisfied 16. Very dissatisfied 17. Don’t know 18. No response Service_effect. To what extent are they satisfied with the overall effectiveness of these community services? 1. Very satisfied 2. Satisfied 3. Dissatisfied 4. Very dissatisfied 5. Don’t know 6. No response Learn_satisfaction. [If service_effect = 5 or 6] To what extent do you agree that learning about GBV survivors’ satisfaction with the services provided is important for improving these services? 8. Strongly agree 9. Agree 10. Neutral (neither agree nor disagree) 11. Disagree 12. Strongly disagree 13. Don’t know 115 14. No response 116 BTG4VM NETWORK COMPONENTS Thank you. Now, we will ask you some questions specifically about the BTG4VM program and the network of services provided. Service_provided. From the following list, please select the services you or your organization(s) provided in partnership with the BTG4VM program. [select multiple] 24. Referral pathways 25. Legal assistance for protection orders 26. Legal assistance for police report 27. Legal assistance for custody and other family related matters 28. Hotline services 29. Psychosocial support services 30. Housing shelters and safe spaces 31. Economic empowerment services (including professional skills training and entrepreneurial opportunities) 32. Immigration documentation support services 33. Healthcare services (including sexual and reproductive health rights) 34. Document translation 35. Provide other information (migration, translation services, and local transportation assistance for school age children) 36. Financial support 37. Other [Specify] 38. Don’t know [Exclusive] 39. No response [Exclusive] Resource_needed. What are the top three resources, services, or programs that the migrant community in your city or neighborhood need most to prevent or combat GBV? We are interested specifically in identifying gaps in community service provision, which include lack of a particular service or inadequate provision of a service. [Select up to three] 1. Referral pathways 2. Legal assistance for protection orders 3. Legal assistance for police report 4. Legal assistance for custody and other family related matters 5. Institutional accompaniment 6. Hotline services 7. Psychosocial support services 8. Housing shelters and safe spaces 9. Economic empowerment services (including professional skills training and entrepreneurial opportunities) 10. Immigration documentation support services 11. Healthcare services (including sexual and reproductive health rights) 12. Document translation 13. Provide other information (migration, translation services, and local transportation assistance for school age children) 14. Other [Specify] 15. Don’t know [Exclusive] 16. No response [Exclusive] 117 [LOOP FOR EACH SELECTED SERVICE] Service_provision. To what extent did the BTG4VM program and its network of partnering organizations provide this service: [selected service]? 7. To a Great Extent 8. Somewhat 9. Very Little 10. Not at All 11. Don’t know 12. No response [If selected service was provided “To a Great Extent” or “Somewhat”] Service_use. To what extent did GBV survivors use this service: [selected service]? 7. To a Great Extent 8. Somewhat 9. Very Little 10. Not at All 11. Don’t know 12. No response [If selected service was used “To a Great Extent” or “Somewhat”] Service_satisfy. In your opinion, to what extent were GBV survivors satisfied with this service: [selected service]? 1. Very satisfied 2. Somewhat satisfied 3. Somewhat dissatisfied 4. Very dissatisfied 5. Don’t know 6. No response TARGET GROUPS Now, we are going to ask you some questions about the groups of people your organization(s) serves. Group. Please, select the groups with which your organization(s) works [select multiple] ● Venezuelan migrants: girls and female teenagers (up to age 17) ● Venezuelan migrants: boys and male teenagers (up to age 17) ● Venezuelan migrants: women ● Venezuelan migrants: men ● Venezuelan migrants: female members of the LGBTQIA+ community ● Venezuelan migrants: male members of the LGBTQIA+ community ● Venezuelan migrants: non-binary or non-gender conforming individuals ● Guyanese nationals: women 118 ● Guyanese nationals: men ● Guyanese nationals: female members of the LGBTQIA+ community ● Guyanese nationals: male members of the LGBTQIA+ community ● Guyanese nationals: non-binary or non-gender conforming individuals ● Other [Specify] ● Don’t know [exclusive] ● No response [exclusive] [LOOP FOR EACH GROUP SELECTED]: Group_needs. In what ways has the BTG4VM program addressed [Group’s] needs? [Select multiple] 13. Providing referral services 14. Providing psychosocial support 15. Providing legal support 16. Providing safe housing 17. Promoting awareness of Xenophobia 18. Promoting services to prevent or protect from GBV 19. Other [Specify] 20. Don’t know [exclusive] 21. No response [exclusive] Group_approach. To what extent has your organization(s) used a tailored approach in delivering its services to [Group]? 7. To a Great Extent 8. Somewhat 9. Very Little 10. Not at All 11. Don’t know 12. No response Group_resources. To what extent has your organization(s) had to invest additional resources since 2020 to satisfy [Group]’s needs as GBV survivors? 1. To a Great Extent 2. Somewhat 3. Very Little 4. Not at All 5. Don’t know 6. No response [If 1 or 2 selected]: Group_resources_sp. Please, indicate the additional resources that were invested [Select multiple] 10. Additional staffing 11. Additional funding 12. Additional activities 13. Additional planning 14. Longer implementation time 119 15. Additional community outreach 16. Other [Specify] 17. Don’t know [exclusive] 18. No response [exclusive] Thank you for your responses. We would like to ask you a few more questions about these groups’ access to and participation in GBV services. As a reminder, the groups mentioned are: [pipe in Groups response options]. Group_access. Did all groups have equal access to the services your organization(s) provided through BTG4VM program, or were there differences in the access across groups? 9. They were even 10. There were differences 11. Don’t know 12. No response [IF DIFFERENCES:] Group_access_dif. If there were differences in access, please name which groups did not participate as much. (open-ended) Group_outcomes. Did the services your organization(s) provided through BTG4VM program impact all target groups equally, or were there differences in the results across groups? 1. They were even 2. There were differences 3. Don’t know 4. No response [IF DIFFERENCES:] Group_outcomes_dif. Please briefly describe the differences in results of your BTG4VM program related between the target groups (open-ended) Group_participation. Are there people in your community who you think should have received the BTG4VM services but did not? 1. Yes 2. No 3. Don’t know 4. No response [If Yes]: Please specify the group (open ended) Other_providers. Are there organizations in your community that you think should have provided services for the BTG4VM network but did not? 5. Yes 6. No 7. Don’t know 8. No response [If Yes]: Please specify who (open ended) 120 CONTEXT APPROPRIATENESS [Transition] Thank you very much for your responses this far. We’re also interested in learning about your views on how the BTG4VM program fit the community and any challenges encountered during implementation. Activity_fit. To what extent did the services provided as part of the BTG4VM program fit well in your community? 7. To a Great Extent 8. Somewhat 9. Very Little 10. Not at All 11. Don’t know 12. No response Adaptation. To what extent do you agree with this statement: The BTG4VM program and its service network staff adapted to the context of my community. 8. Strongly agree 9. Agree 10. Neutral (neither agree nor disagree) 11. Disagree 12. Strongly disagree 13. Don’t know 14. No response Adaptation_explain_y. [If “Strongly Agree” or “Agree”]. Please, briefly elaborate on your answer above to describe how the staff adapted to the context of your community. (open￾ended) Adaptation_explain_n. [If “Disagree” or “Strongly Disagree”]. Please, briefly elaborate on your answer above to describe how the staff failed to adapt to the context of your community. (open-ended) Challenge_context. What were the three main challenges in implementing services in partnership with the BTG4VM program? [select up to three] 17. Knowledge gaps in on what resources/services the Ministry of Human Services and Social Security was willing to offer clients 18. Slow pace of information gathering 19. Referral organizations under-resourced 20. Difficulty in documenting if the beneficiary received referral service 21. Unexpected rise in transportation costs 22. Budgeting the level of effort for the lawyer and psychologist 23. Documentation barriers for migrants to receive COVID-19 vaccines and enter public spaces 24. Language barriers 25. Organizational barriers with the main grantee 26. Buy-in from stakeholders 27. No challenges [exclusive] 121 28. Other [Specify] 29. Don’t know [exclusive] 30. No response [exclusive] GBV PREVENTION AND RESULTS [Transition] Now, we are going to ask about your community values and the BTG4VM program’s role in GBV prevention. Prevent_conflict. Are GBV prevention efforts in conflict with any values or traditions in your community? 1. Yes 2. No 3. Don’t Know 4. No response [If Yes]: Prevent_conflict_sp What community values or traditions are most in conflict with GBV prevention efforts? (open-ended) Prevent_gbv. To what extent has the BTG4VM program helped prevent GBV in your community? 7. No effect 8. Minor effect 9. Moderate effect 10. Major effect 11. Don’t know 12. No response Group_change. In your community, what are the top three groups that have the most power in influencing change to promote GBV protection and prevention? [select up to three] 1. Funders/donors 2. National authorities 3. Local authorities 4. Neighbor associations 5. Immigrant groups or associations 6. Survivor groups or associations 7. Non-profit organizations 8. Operational staff 9. Other, [Specify] 10. Don’t know [exclusive] 11. No response [exclusive] Prevent_effect. Rate the effectiveness of BTG4VM program components in preventing GBV. 2. Create linkages to health and social services through a strengthened National GBV referral pathway, including updating the National GBV Referral Directory and designing National GBV Referral pathway 3. Development and implementation of a One-Stop-Shop model for GBV services, including legal support, psychosocial support, and safe housing 122 4. A radio serial drama to strengthen solidarity between migrant and host communities 5. Social Media campaign to increase awareness of Xenophobia, GBV Services and Alternative Masculinities 6. Radio and television Public Service Announcements to increase awareness of Xenophobia, GBV Services and Alternative Masculinities [For each component above, rate:] a. No effect b. Minor effect c. Moderate effect d. Major effect e. Don’t know f. No response Service_impact. Overall, from the following list, please identify the component of the BTG4VM program that led to the most positive impact on the migrant community? 1. Create linkages to health and social services through a strengthened National GBV referral pathway, including updating the National GBV Referral Directory and designing National GBV Referral pathway 2. Development and implementation of a One-Stop-Shop model for GBV services, including legal support, psychosocial support, and safe housing 3. A radio serial drama to strengthen solidarity between migrant and host communities 4. Social Media campaign to increase awareness of Xenophobia, GBV Services and Alternative Masculinities 5. Radio and television Public Service Announcements to increase awareness of Xenophobia, GBV Services and Alternative Masculinities 6. Don’t know 7. No response MONITORING SYSTEM [Transition] We are nearing the end of the survey. Now, we would like to ask you a few questions about your familiarity with the monitoring system implemented by the BTG4VM program. Monitor_familiar. In your work with the BTG4VM program, have you or your team members been asked by NCC to share data on the beneficiaries that use your organization’s services? 5. Yes 6. No [Skip block] 7. Don’t know [Skip block] 8. No response [Skip block] [IF YES]: Monitordata_type. Please, indicate what kinds of data you or your team members have shared with NCC for the BTG4VM program. Select all that apply. [select multiple] 1. Number of project participants 2. Demographics of project participants (gender, age, nationality, address, etc.) 123 3. Diagnosis of project user needs (immigration regularization, mental health counseling, housing support, food insecurity, etc.) 4. Monetary investment per project participant 5. Number of appointments/contacts with project participant 6. Changes in income outcomes/results over time 7. Changes in education outcomes/results over time 8. Changes in migration status over time 9. Changes in housing status over time 10. Other [Specify] 11. Don’t know [exclusive] 12. No response [exclusive] Data_understand. To what extent do you consider the data requests from NCC clear? 7. To a Great Extent 8. Somewhat 9. Very Little 10. Not at All 11. Don’t know 12. No response Data_accessible. Did you or your team have access to the grantee monitoring system that uses the data your organization(s) provided? 5. Yes 6. No [skip to ‘Monitor_freq’] 7. Don’t know [skip to ‘Monitor_freq’] 8. No response [skip to ‘Monitor_freq’] Monitor_practical. To what extent do you consider the system practical in terms of entering the data? 7. To a Great Extent 8. Somewhat 9. Very Little 10. Not at All 11. Don’t know 12. No response Monitor_effective. To what extent do you consider the system useful to extract information to tailor your organization’s services to beneficiary needs? 7. To a Great Extent 8. Somewhat 9. Very Little 10. Not at All 11. Don’t know 12. No response Monitor_adapt. Did the management of the BTG4VM program use these data from your organization adapt the program activity? 124 5. Yes 1. No [Skip block] 6. Don’t know [Skip block] 7. No response [Skip block] [IF YES] Please, provide an example of such adaptations (open ended) SUSTAINABILITY AND REPLICABILITY Thank you so much for the answers you have provided so far. To finish our questions about the BTG4VM program, we are interested in your thoughts about the current and future implementation. [Loop for each service marked in ‘Service_provided”] Sustain_current. What is the current status of this service: [service]? 7. We continue providing [service] with the BTG4VM program 8. We suspended [service]with the BTG4VM program 9. We refer the BTG4VM program beneficiaries to another service provider for this service 10. Other (please, specify: ) 11. Don’t know 12. Refuse [If ‘Sustain_current’ = 1] Sustain_sources_open. Please, indicate the source of funding for this service: [open-ended] Sustain_likelihood. In your opinion, what is the likelihood that your organization(s) will continue to provide [service] for the next five years? 8. Very likely 9. Likely 10. Unlikely 11. Very unlikely 12. Don’t know 13. No response Sustain_desire_comp. Which BTG4VM components would you like to see continue in your community after this activity has ended?. [Select multiple] 2. Create linkages to health and social services through a strengthened National GBV referral pathway, including updating the National GBV Referral Directory and designing National GBV Referral pathway 3. Development and implementation of a One-Stop-Shop model for GBV services, including legal support, psychosocial support, and safe housing 4. A radio serial drama to strengthen solidarity between migrant and host communities 5. Social Media campaign to increase awareness of Xenophobia, GBV Services and Alternative Masculinities 125 6. Radio and television Public Service Announcements to increase awareness of Xenophobia, GBV Services and Alternative Masculinities 7. None [exclusive] 8. Don’t Know [exclusive] 9. No response [exclusive] [FOR EACH OPTION MARKED IN Sustain_desire_comp:] Sustain_resource. What do you think your community needs to do to make sure this activity continues? (open-ended response) Scalability. Which activities of the BTG4VM program do you think could increase their coverage (“scale up”) in Guyana? [Select multiple] 1. Development and implementation of a One-Stop-Shop model for GBV services, including legal support, psychosocial support, and safe housing 2. A radio serial drama to strengthen solidarity between migrant and host communities 3. Social Media campaign to increase awareness of Xenophobia, GBV Services and Alternative Masculinities 4. Radio and television Public Service Announcements to increase awareness of Xenophobia, GBV Services and Alternative Masculinities 5. None [exclusive] 6. Don’t Know [exclusive] 7. No response [exclusive] Replicability. Which activities of the BTG4VM program do you think could be implemented in other communities in Guyana? [Select multiple] 2. Create linkages to health and social services through a strengthened National GBV referral pathway, including updating the National GBV Referral Directory and designing National GBV Referral pathway 3. Development and implementation of a One-Stop-Shop model for GBV services, including legal support, psychosocial support, and safe housing 4. A radio serial drama to strengthen solidarity between migrant and host communities 5. Social Media campaign to increase awareness of Xenophobia, GBV Services and Alternative Masculinities 6. Radio and television Public Service Announcements to increase awareness of Xenophobia, GBV Services and Alternative Masculinities 7. None [exclusive] 8. Don’t Know [exclusive] 9. No response [exclusive] [FOR EACH OPTION MARKED IN REPLICABILITY] Replicability_where. Where in Guyana should this component be replicated? (open-ended response) DEMOGRAPHICS 126 We appreciate your invaluable insights. Before letting you go, we’d like to ask the following questions for statistical purposes. Remember that your answers are confidential and we will use the information in the following section to analyze trends and present aggregate results. Dem_gender. What is your gender? 1. Man 2. Woman 3. Other, specify [open-ended] 4. Prefer not to say Dem_dob. About how old are you? 9. 18-24 10. 25-34 11. 35-44 12. 45-54 13. 55-64 14. 65-74 15. 75+ 16. Prefer not to say Dem_lang. Which languages are you capable of speaking fluently? (Check all that apply.) 1. English 2. Guyanese Creole 3. Spanish 4. Portuguese 5. Hindi 6. Chinese 7. Other [Specify] 8. Prefer not to say [exclusive] Dem_edu. What is the highest degree or level of education you have completed? 11. Never attended school 12. Some primary school 13. Complete primary school 14. Some secondary school 15. Complete secondary school 16. Some higher education 17. Complete higher education 18. Advanced degree 19. Technical school 20. Prefer not to say [For each organization the respondent is affiliated with] RespType1.a. What is your current role (title) in that [organization]? [open-ended] RespType1.c. How many years have you worked with [organization]? 127 1. Less than one year 2. One to three years 3. Four to six years 4. Seven or more years 5. Prefer not to say Dem_region. Where is [organization] located in Trinidad and Tobago? 1. Caroni. 2. Mayaro. 3. Nariva. 4. Saint Andrew. 5. Saint David. 6. Saint George. 7. Saint Patrick. 8. Victoria 9. Prefer not to say RespType1.b. Overall, at what level is your current position? 6. Entry-level or early career 7. Mid-career 8. Senior 9. Don’t know 10. No response End of Survey Message: Close. You have now completed the survey. Thank you for your participation. 128 ANNEX D. SOURCES OF INFORMATION 129 DOCUMENTS REVIEWED Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. Sage publications. Better Together Challenge (2022). The Better Together Challenge/Juntos es Mejor Cluster Final Report Democracy International (2021). Email: WELCOME Revised Design Democracy International (2021). Fixed Amount Award (FAA) Agreement Democracy International (2021). PMP Table Democracy International (2021). WELCOME Revised Design for Resonance Review Democracy International (2020). Women Exercising Leadership For Cohesion & Meaningful Empowerment (Welcome) Project: Democracy International Final Report Democracy International (2020). Women Exercising Leadership For Cohesion & Meaningful Empowerment (Welcome) Project: Democracy International Work Plan Democracy International (2021). Women Exercising Leadership For Cohesion & Meaningful Empowerment (Welcome) Project: Democracy International Work Plan (Revised) Democracy International (2020). Women Exercising Leadership For Cohesion & Meaningful Empowerment (Welcome) Project: Monitoring, Evaluation, and Learning Plan Kato-Wallace, J., Barker, G., Garg, A., Feliz, N., Levack, A., Ports, K., & Miller, E. (2019). Adapting a global gender-transformative violence prevention program for the US community-based setting for work with young men. Global social welfare, 6, 121-130 National Coordinating Coalition (2021). Activity Plan National Coordinating Coalition (2021). Bridging The Gap For Venezuelan Migrants (BTG4VM) Monitoring, Evaluation, And Learning Plan And Work Plan National Coordinating Coalition (2021). NCC Final Report National Coordinating Coalition (2021). Grant Agreement National Coordinating Coalition (2021). KPI Tracker National Coordinating Coalition (2021). Rapid Gender Based Violence Assessment The Hebrew Immigrant Aid Society (2020). Environmental Assessment The Hebrew Immigrant Aid Society (2022). Equilibrando La Balanza: Involucrar A Los Hombres En La Reducción De La Violencia De Género: HIAS Informe Final The Hebrew Immigrant Aid Society (2021). Fixed Amount Award (FAA) Agreement 130 The Hebrew Immigrant Aid Society (2021). Mesa De Diálogo Sobre Masculinidades Positivas En El Contexto De Panamá The Hebrew Immigrant Aid Society (2021). BlogMasculinidades The Hebrew Immigrant Aid Society (2021). Concept Paper: HIAS Panama - Engaging Men In Panama To Reduce Risk And Mitigate Consequences Of Violence Against Women And Girls, With A Specific Focus On Venezuelan Refugees And Migrants The Hebrew Immigrant Aid Society (2021). Shifting Power Dynamics: Engaging Men In Gender-based Violence Reduction: HIAS Periodic Report The Hebrew Immigrant Aid Society (2021). Shifting Power Dynamics: Engaging Men in Gender-Based Violence Reduction: Monitoring, Evaluation, and Learning Plan and Work Plan The Hebrew Immigrant Aid Society (2020). USAID Global Development Lab Gender Analysis Template The Hebrew Immigrant Aid Society (2022). United States Strategy to Prevent and Respond to Gender-Based Violence Globally SITES VISITED Apple Store. https://apps.apple.com/us/app/el-man-de-hoy/id1604608067 Democracy International. https://democracyinternational.com/ Doodnat, A. (2021). Building back better: NGO founder heals from shooting, rebuilds centre. Available at: https://tt.loopnews.com/content/building-back-better-ngo-founder-heals-shooting￾rebuilds-centre?fbclid=IwAR1eCRZ1fauKZzHwW5kEMEEuJmcbj6kJxQ9GQ8WMr5kr5Oi￾LHY6mzv7q9g National Coordinating Coalition. https://nccgy.org/ R4V Inter-Agency Coordination Platform for Refugees and Migrants from Venezuela. https://www.r4v.info/en USAID BTC. https://www.usaid.gov/BetterTogetherVE US Dept. of State 2016 Update to United States Strategy To Prevent and Respond to Gender-based Violence Globally. https://2009- 2017.state.gov/s/gwi/priorities/gbv/258536.htm#:~:text=The%20goal%20of%20U.S.%20Government,d evelopment%2C%20security%2C%20and%20prosperity. KEY INFORMANTS NORC obtained qualitative information through key informant interviews, focus group discussions, and semi-structured interviews from the following organizations: 131 Table 19. Key Informants ORGANIZATIONAL AFFILIATION NAME TITLE USAID Jamie Small Activity Cluster Manager HIAS Roberto Mera Montenegro Country Head HIAS Leandro Cardozo Technical Expert HIAS Yazcari Murillo Technical Expert DI Molly Knapp Monitoring and Evaluation DI Naomi Rasmussen Director La Casita Andreina Briceño Brown Director NCC Simone Sills Director NCC Caressa Henry Project Manager NCC Arielle Gordon M&E Resonance-The Catalyst Project Chelsea Kay BTC Director Resonance-The Catalyst Project Shanna O’Reilly BTC Project Manager Resonance-The Catalyst Project Ana Acosta BTC M&E Specialist IDB Juan Pablo López Gross Senior Specialist IDB Vashtie K. Dookiesingh Project Team Leader IDB Kaimlall Chattergoon Project Manager-Guyana Additionally, NORC collected data on service providers’ perceptions and knowledge about the BTG4VM and WELCOME program using a web survey. The organizations to which respondents mentioned being affiliated are displayed in Table 20. Table 20. Affiliated Organizations GUYANA TRINIDAD AND TOBAGO NCC Families in Action International Organisation for Migrants International Organization for Migration Catholic Charities Organisation Guyana - Migrant Support Services Rape Crisis Society of T&T Food for the Poor United Nations High Commissioner for Refugees Hebrew Immigrant Aid Society - Hope Foundation - Voices Gy United Nations High Commission for Refugees 132 ANNEX E. ADDITIONAL TABLES 133 Table 21. Service Providers’ Immigrant Target Groups, Needs, and Approaches and Resources Used to Satisfy Needs TARGET GROUP TARGET GROUP NEEDS TAILOR-MADE APPROACH ADDITIONAL RESOURCES Venezuelan migrants: girls and female teenagers (up to age 17) (10) ● Providing referral services ● Counseling and other psychosocial services ● Services for refugees and asylum seekers ● Services for victims of human trafficking ● Promoting services to prevent or protect from GBV ● Health care ● Legal services Somewhat or to a great extent (9) Don’t know (1) Somewhat or to a great extent (6) Very little or not at all (3) Don’t know (1) Venezuelan migrants: boys and male teenagers (up to age 17) (10) ● Providing referral services ● Counseling and other psychosocial services ● Services for refugees and asylum seekers ● Services for victims of human trafficking ● Emergency shelter services ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (8) Very little or not at all (1) Don’t know (1) Somewhat or to a great extent (3) Very little or not at all (6) Don’t know (1) Venezuelan migrants: women (10) ● Providing referral services ● Counseling and other psychosocial services ● Legal services ● Services for refugees and asylum seekers ● Services for victims of human trafficking ● Emergency shelter services ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (10) Somewhat or to a great extent (8) Very little or not at all (1) Don’t know (1) Venezuelan migrants: men (10) ● Providing referral services ● Counseling and other psychosocial services ● Legal services ● Services for refugees and asylum seekers ● Services for victims of human trafficking ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (7) Very little or not at all (3) Somewhat or to a great extent (5) Very little or not at all (4) Don’t know (1) Venezuelan migrants: female members of the LGBTQIA+ community (9) ● Providing referral services ● Counseling and other psychosocial services ● Health care ● Legal services ● Services for refugees and asylum seekers ● Services for victims of human trafficking ● Emergency shelter services ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (8) Very little or not at all (1) Somewhat or to a great extent (5) Very little or not at all (3) Don’t know (1) 134 TARGET GROUP TARGET GROUP NEEDS TAILOR-MADE APPROACH ADDITIONAL RESOURCES Venezuelan migrants: male members of the LGBTQIA+ community (9) ● Providing referral services ● Counseling and other psychosocial services ● Health care ● Legal services ● Services for refugees and asylum seekers ● Services for victims of human trafficking ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (7) Very little or not at all (2) Somewhat or to a great extent (4) Very little or not at all (4) Don’t know (1) Venezuelan migrants: non-binary or non-gender￾conforming individuals (6) ● Providing referral services ● Counseling and other psychosocial services ● Services for refugees and asylum seekers ● Services for victims of human trafficking ● Emergency shelter services ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (5) Very little or not at all (1) Somewhat or to a great extent (3) Very little or not at all (2) Don’t know (1) Trinidadian and Tobagonian nationals: women (10) ● Providing referral services ● Counseling and other psychosocial services ● Legal services ● Services for victims of human trafficking ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (7) Very little or not at all (2) Don’t know (1) Somewhat or to a great extent (4) Very little or not at all (4) Don’t know (2) Trinidadian and Tobagonian nationals: men (10) ● Providing referral services ● Counseling and other psychosocial services ● Services for victims of human trafficking ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (5) Very little or not at all (4) Don’t know (1) Somewhat or to a great extent (2) Very little or not at all (6) Don’t know (2) Trinidadian and Tobagonian nationals: female members of the LGBTQIA+ community (9) ● Providing referral services ● Counseling and other psychosocial services ● Services for victims of human trafficking ● Youth services ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (6) Very little or not at all (2) Don’t know (1) Somewhat or to a great extent (4) Very little or not at all (3) Don’t know (2) 135 TARGET GROUP TARGET GROUP NEEDS TAILOR-MADE APPROACH ADDITIONAL RESOURCES Trinidadian and Tobagonian nationals: male members of the LGBTQIA+ community (9) ● Providing referral services ● Counseling and other psychosocial services ● Health care ● Services for victims of human trafficking ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (4) Very little or not at all (4) Don’t know (1) Somewhat or to a great extent (2) Very little or not at all (5) Don’t know (2) Trinidadian and Tobagonian nationals: non-binary or non-gender￾conforming individuals (7) ● Providing referral services ● Counseling and other psychosocial services ● Services for victims of human trafficking ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (4) Very little or not at all (3) Somewhat or to a great extent (2) Very little or not at all (4) Don’t know (1) Individuals recruited and trained to be advocates (4) ● Promoting services to prevent or protect from GBV ● Other Somewhat or to a great extent (2) Very little or not at all (2) Somewhat or to a great extent (1) Very little or not at all (2) Don’t know (1) La Casita employees included in all training activities (2) ● Promoting services to prevent or protect from GBV Somewhat or to a great extent (1) Very little or not at all (1) Very little or not at all (1) Don’t know (1) Notes: N = 10. Number of service providers (in parentheses) in the first column according to the target group their organizations serve. Depending on the number of respondents that identified each target group, the third and fourth columns show respondents (in parentheses) according to the extent to which their organizations tailored the services in the second column, and whether the organizations invested additional resources in such services to serve the target group. Source: Web-based survey by ET. 136 ANNEX F. DISCLOSURE CONFLICTS OF INTERES 137 Name Carlos A. Echeverria-Estrada Title Research Scientist Organization NORC at the University of Chicago Evaluation Position? ☒Team Leader ☐ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / 7200AA18M00016, Tasking N054 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION OF BETTER TOGETHER CHALLENGE ACTIVITY CLUSTER I have real or potential conflicts of interest to disclose. Yes☐ No ☒ If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Carlos Alberto Echeverria-Estrada Date June 12, 2023 138 Name Camille Smith Title Research Associate II Organization NORC at the University of Chicago Evaluation Position? ☐ Team Leader ☒Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / 7200AA18M00016, Tasking N054 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION OF BETTER TOGETHER CHALLENGE ACTIVITY CLUSTER I have real or potential conflicts of interest to disclose. ☐ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Camille Smith Date 6/12/2023 139 Name Paige Pepitone Title Research Associate II Organization NORC at the University of Chicago Evaluation Position? ☐ Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / 7200AA18M00016, Tasking N054 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION OF BETTER TOGETHER CHALLENGE ACTIVITY CLUSTER I have real or potential conflicts of interest to disclose. ☐ Yes☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Paige Pepitone Date June 12, 2023 140 Name Mithila Iyer Title Research Associate II Organization NORC at the University of Chicago Evaluation Position? ☐ Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / 7200AA18M00016, Tasking N054 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION OF BETTER TOGETHER CHALLENGE ACTIVITY CLUSTER I have real or potential conflicts of interest to disclose. ☐ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Mithila Iyer Date June 12, 2023 141 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523