This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT Draft: September 16, 2022, Revised October 4, and November 1, 2022 Prepared under Contract No.: GS-10F-0033M / Order No. 7200AA18M00016, Tasking N054 DRG LEARNING, EVALUATION, AND RESEARCH II ACTIVITY GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT SEPTEMBER 16, 2022, Revised October 4, and November 1, 2022 Prepared under Contract No.: GS-10F-0033M / Order No. 7200AA18M00016, Tasking N054 Submitted to: USAID DRG Center Submitted by: Ritu Nayyar-Stone (Team Lead) Cathy Zimmerman (GBV Lead) Vaiddehi Bansal (CARE-GBV Cluster Lead) Carlos Echeverría (Better Together Challenge Cluster Lead) Ridhi Sahai (WEE Cluster Lead) Katrina Kamara (WEE Cluster Lead) Jessica Wallach (RISE Cluster Lead) Contractor: NORC at the University of Chicago 4350 East West Highway, 8th Floor Bethesda, MD 20814 Attention: Matthew Parry Tel: 301-634-5444; E-mail: parry-matthew@norc.org DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | i TABLE OF CONTENTS LIST OF ACRONYMS III BACKGROUND 1 BETTER TOGETHER CHALLENGE: EVALUATION DESIGN AND DATA COLLECTION 5 INTRODUCTION 5 EVALUATION DESIGN 5 DATA COLLECTION METHODS 7 COLLECTIVE ACTION TO REDUCE GENDER-BASED VIOLENCE: EVALUATION DESIGN AND DATA COLLECTION 9 INTRODUCTION 9 EVALUATION DESIGN 10 DATA COLLECTION METHODS 11 RESILIENT, INCLUSIVE AND SUSTAINABLE ENVIRONMENTS 13 INTRODUCTION 13 EVALUATION DESIGN 15 DATA COLLECTION METHODS 17 WOMEN’S ECONOMIC EMPOWERMENT 18 INTRODUCTION 18 EVALUATION DESIGN 20 DATA COLLECTION METHODS 22 ETHICAL CONSIDERATIONS 23 DATA ANALYSIS 25 REPORTING AND DISSEMINATION 26 ANNEX A 29 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | ii LIST OF TABLES Table 1: Evaluation Questions........................................................................................................................................1 Table 2: Summarized Data Collection Across all Four Activity Clusters............................................................4 Table 3: BTC Activities Included in the PPE................................................................................................................5 Table 4: BTC Specific Instrument Matrix.....................................................................................................................6 Table 5: Data Collection Methods for BTC................................................................................................................8 Table 6: Summary of CARE-GBV Activities Subject to Performance Evaluation...............................................9 Table 7: CARE-GBV Specific Instrument Matrix......................................................................................................11 Table 8: Data Collection Methods for CARE-GBV.................................................................................................12 Table 9: RISE Activities Included in the PPE..............................................................................................................14 Table 10: RISE Specific Instrument Matrix.................................................................................................................16 Table 11: Data Collection Methods for RISE............................................................................................................18 Table 12: Data Sources by WEE Activity...................................................................................................................19 Table 13: WEE Specific Instrument Matrix................................................................................................................21 Table 14: Data Collection Methods for WEE...........................................................................................................22 Table 15: Ethics Committees Regulating this Evaluation........................................................................................24 Table 16: Data Analysis Methods for GBV PPE........................................................................................................26 Table 17: Evaluation Timeline.......................................................................................................................................27 Table A. 1: Instrument Matrix for Portfolio, Activity Cluster, and Implementation Evaluation...................29 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | iii LIST OF ACRONYMS AC Activity Cluster ASOCUCH Association of Organizations of the Cuchumatanes APESSA Association Pour l’Education, la Sexualité et la Santé en Afrique BTC Better Together Challenge (Juntos Es Mejor) BTG4VM Bridging the Gap for Venezuelan Migrants CARE-GBV Collective Action to Reduce Gender-Based Violence CATI Computer Assisted Telephonic Interview CCH Crisis Center Hope CEL Communications, Evidence, and Learning – Women's Land Rights CEFM Child, Early, and Forced Marriage CI Conservation International COP Chief of Party CREW Center for Rights Education and Awareness CRS Catholic Relief Services DI Democracy International DDI Development, Democracy, and Innovation DPI Development Professionals Inc. DRC Democratic Republic of the Congo DRG Democracy, Human Rights, and Governance E4W Enabling Environment for Economic Empowerment of Women EDR Evaluation Design Report EESIVMR Employment, Entrepreneurship, and Social Innovation for Venezuelan Migrants and Refugees FCT-SGBV-RT FCT Sexual and Gender-Based Violence Response Team FFI Fauna and Flora International FGD Focus Group Discussion FGM/C Female Genital Mutilation/Cutting FIDA Federation of Women Lawyers in Lesotho GEM Gender Equitable Men GBV Gender-Based Violence GBVH Gender Based Violence and Harassment GenDev Gender Equality and Women’s Empowerment Hub HIAS Hebrew Immigrant Aid Society IBEDC Ibadan Electricity Distribution Company IDUL Independent Democratic Union of Lesotho IE Implementation Evaluation IFDP Innovation et Formation pour le Développement et la Paix IHSA Integrated Health Services Activity ILRG Integrated Land and Resource Governance – Women's Land Rights IP Implementing Partner CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | iv IPV Intimate Partner Violence IRB Institutional Review Board IYF International Youth Foundation KII Key Informant Interview KWCA Kenya Wildlife Conservancies Association LEMU Land and Equity Movement in Uganda LER Learning, Evaluation, and Research MCI Making Cents International MEL Monitoring, Evaluation, and Learning MIT D-Lab Massachusetts Institute of Technology Development Lab MSH Management Sciences for Health, Inc. MOH Ministry of Health NACTWU National Allied Clothing and Textiles Workers Union NCC National Coordination Coalition NGO Non-governmental organization PPE Portfolio Performance Evaluation PROMSEX Center for the Promotion and Defense of Sexual and Reproductive Rights REDD+ Reducing Emissions from Deforestation and Forest Degradation RISE Resilient, Inclusive & Sustainable Environments SDP Shifting Power Dynamics SIMN Scalabrini International Migration Networks SOAR Sexual Offences Awareness and Response Initiative SOCADIDO Soroti Catholic Diocese Integrated Development Organization SOW Scope of work SPD Shifting Power Dynamics: Engaging Men in Gender-Based Violence Reduction SVRI Sexual Violence Research Initiative SWC Sera Wildlife Conservancy TOC Theory of Change TOT Training of trainers UNITE United Textiles Employees USAID United States Agency for International Development USG United States Government WAR Women Against Rape WCT Wellness Check Tool WEE Women’s Economic Empowerment WEEM Women’s Economic Empowerment in Mesoamerica WELCOME Women Exercising Leadership for Cohesion and Meaningful Empowerment WFW Women for Women WJI Women’s Justice Initiative WLSA Women and Law in Southern Africa – Lesotho WRW Workers’ Rights Watch CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | v ZSU Žene sa Une CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 1 BACKGROUND Under the Democracy, Human Rights, and Governance Learning, Evaluation, and Research (DRG-LER) II Activity, the United States Agency for International Development’s (USAID) Gender Equality and Women’s Empowerment Hub (USAID/GenDev) in the Bureau for Development, Democracy, and Innovation (DDI) has contracted NORC at the University of Chicago (NORC) to carry out a portfolio performance evaluation (PPE) of its gender-based violence (GBV) activities comprising four activity clusters (ACs) as follows: 1. Better Together Challenge (BTC) with GBV prevention and response interventions; 2. Collective Action to Reduce Gender-Based Violence (CARE-GBV) small grants activities; 3. The Resilient, Inclusive & Sustainable Environments (RISE): A Challenge to Address Gender-Based Violence in the Environment; and 4. The Women’s Economic Empowerment (WEE) activities directly funded by USAID/GenDev integrating GBV prevention and response activities. The purpose of this study is to understand what is working, challenges faced, knowledge gaps, and what can be improved in USAID/GenDev’s GBV portfolio. The PPE was structured to initially include a period of co-creation with USAID and its partners in developing the evaluation scope of work and undertaking an evaluability assessment. These phases have been completed. This evaluation design report details the approaches and methodologies that NORC will use to answer the evaluation questions for each cluster and activity. The evaluation questions at the portfolio, AC, and individual activity level are provided in Table 1 below. Table 1: Evaluation Questions Evaluation Question EQ-Sub-questions PORTFOLIO QUESTIONS 1. How are the USG’s guiding principles and priorities to end GBV being incorporated into the four activity clusters (AC)? · Prevention: In what ways is the USG activity portfolio contributing to reduced risks? · Protection: How does the portfolio contribute to accessible, effective services for violence survivors? · Accountability: How does the portfolio contribute to ending impunity? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 2 Evaluation Question EQ-Sub-questions 2. To what extent are the USG objectives being achieved across the 4 ACs? · Coordination: How are the GBV prevention and response efforts being coordinated and managed at the Agency, Activity Cluster and Activity levels?? · Integration: How are GBV prevention and response efforts being integrated into current and future USAID/GenDev work and informing related programs? · Data. How is USAID/GenDev’s GBV portfolio collecting, analyzing, and using data and research to enhance prevention and response efforts? · Expansion: How is USAID/GenDev’s GBV portfolio helping to expand and improve GBV programming? 3.What lessons are being learned and to what extent is there sharing of best practices, lessons, and information across the 4 ACs? · Foundations: Are lessons regarding foundations of GBV being shared with AC implementing partners? · Populations: What types of populations are being engaged in the AC? Which vulnerable and underserved populations are being included? · Stakeholders: Which stakeholders are being engaged to achieve results? 4.What pervasive gaps still exist in understanding GBV and addressing specific types of GBV? · Intervention planning and design: What are important knowledge and practice gaps in planning and designing GBV interventions? · Forms of violence: What are important knowledge and practice gaps in addressing specific forms of GBV? · Reach and effectiveness: How is the GBV portfolio influencing the reach and effectiveness of interventions? ACTIVITY CLUSTER QUESTIONS 1.Are the activity clusters based on context-specific and international evidence? · Needs assessment and intervention evidence: How well were needs assessments conducted and intervention evidence collected to inform the cluster activities? · Assumptions: What assumptions were made to design and implement the activity clusters? How accurate were any assumptions? · Causal pathways: What causal pathways or theories of change were articulated for the activity clusters? · Monitoring and adaptations: How well are interventions monitored and are emerging findings contributing to intervention adaptations or improvements? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 3 Evaluation Question EQ-Sub-questions 2. To what extent are each of the activity clusters achieving the targeted GBV results? · Outcomes: Are the stated outcomes realistic and achievable within the timeframe of the AC? What progress is being made towards achieving the outcomes? · Planning and activity designs: How and how well were activity plans and designs developed to achieve different GBV outcomes? · Intervention implementation: How well are interventions implemented to reach their target groups and influence change? · Mechanisms: What are the most effective aspects of the intervention? How do these ‘active ingredients’ operate in each AC? 3. To what extent are the ACs sustainable? Sustainability: What aspects of the ACs contributed to their sustainability? What components are needed for greater sustainability? Replicability, transferability, and adaptability: In what ways are the ACs replicable in the same contexts? Adaptable for other contexts? Scalability: What aspects of the ACs are most amenable to be scaled up? IMPLEMENTATION EVALUATION QUESTIONS 1. Is the activity design based on the local context and flexible to achieve results on the ground? Design: What factors contributed to the design of the activity? How were priority GBV problems identified? Implementation: What are the key intervention methods to achieve objectives? Flexibility: Is there sufficient staffing to respond to local priorities? Is there flexibility to change approaches to respond to lessons and changing challenges in the local environment? 2. Is the activity reaching participants they are meant to target? Target participants: What are the barriers to reaching participants? Monitoring of results: Is the activity collecting evidence on what is working, not working and what could be done differently to achieve results? 3. Is the activity achieving sustainability? Sustainability: What plans are in place for sustainability? What is the evidence of potential sustainability? Table 2 below shows a summary of the proposed data collection across all four ACs. Details are provided below under each AC. Following this section, we present the evaluation design and data collection methods for each AC, followed by a section on data analysis, ethical considerations, reporting and dissemination, as well as the timeline for this evaluation. We provide the first draft of instrument questions at the portfolio, activity cluster, and implementation evaluation level in Annex A. In addition, each AC section also includes a few activity-specific draft instrument questions. In addition, NORC will prepare a final set of instrument questions for each AC by type of respondent and share it with USAID for final sign off. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 4 Table 2: Summarized Data Collection Across all Four Activity Clusters Activity Cluster Document Review No. of Key Informant Interviews No. of Focus Group Discussions Survey(s) (estimate of N) BTC 20-24 documents 42 with activity participants (24 females / 18 males), six with network service providers, one with USAID / GenDev, one with Resonance, six with IP leadership Two – IP staff (Guyana and T&T) Three – secondary activity participants in Panama (female partners) Two web-based, (backed by local monitors) with network service providers (N=40- 50 CARE-GBV 60-80 documents across all 5 grantees 7 KIIs total with USAID, MCI, IP senior staff, IP partner org, and IP staff selected for IE (SOAR) N/A 1 web-based with all grantee staff + external stakeholders who participated in workshop/training course/webinar (N=795) RISE Approx. 140 documents across all 7 grantees 40 KIIs total with USAID, Resonance, IP staff, and local GBV experts who can contextualize activities into the larger picture of GBV programming in their contexts 23 FGDs with activity participants One web-based survey (Vietnam) and one face-to￾face or phone survey (Fiji) with activity participants (N=approx. 30-50 for both) WEE Approx. 96 documents across all 5 grantees1 38 KIIs total with IP senior leadership, representatives from local partner organizations, USAID/GenDev staff, and activity participants 14 FGDs with IP staff, activity participants, and representatives from local partner organizations 2 CATI surveys with activity participants in Benin (N=150) and Burundi (N=12,000); 2 face￾to-face surveys in Guatemala2 and Lesotho (N=5,000) 1 We are waiting to receive additional documents, including MEL data, for Global Labor Program: Levi-Strauss Partnership and A Micro-Journey to Self-Reliance: Economic Reintegration for Victims of GBV. 2 The total number of program participants for the WEEM activity is still being confirmed. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 5 BETTER TOGETHER CHALLENGE: EVALUATION DESIGN AND DATA COLLECTION INTRODUCTION Since 2013 Latin America has experienced the largest migration crisis in its history. The Venezuelan exodus of over 6.8 million people has strained the delivery of basic services, infrastructure, and labor markets as well as community relations in neighboring small countries, including Guyana, Panama and Trinidad and Tobago. The Better Together Challenge (BTC), launched in September 2019, is a global initiative implemented by Resonance Global to crowdsource, fund, and scale forward-thinking solutions from anywhere in the world to improve the lives of Venezuelans and communities hosting them across Latin America and the Caribbean affected by the regional crisis. Co-sponsored by USAID, BTC’s theory of change (ToC) hypothesizes that if USAID BTC funds, tests, and scales innovative ideas and solutions by Venezuelans and regional actors, and expands networks across communities and countries, then Venezuelan voices and ingenuity to answer needs in their home country and across the region will be heard; relationships and collaboration will be promoted; and a marketplace of tested, market-ready solutions will be built. In 2021, USAID/GenDev funded six GBV-oriented activities to empower Venezuelan migrant women, and create better lives for participants, their families, and others. Of these, three activities will be the focus on this performance evaluation: Table 3: BTC Activities Included in the PPE No. Activity Organization Country Types of Respondents 1 Building the Gap for Venezuelan Migrants (BTG4VM) National Coordinating Coalition Inc. (NCC) Guyana - IP (grantee) leadership - IP (grantee) frontline staff - Network service providers - Program participants (Venezuelan migrant GBV survivors) 2 Shifting Power Dynamics: Engaging Men in Gender-Based Violence Reduction (SDP) Hebrew Immigrant Aid Society (HIAS) Panama - IP leadership and technical staff - Male primary activity participants - Female secondary activity participants 3 Women Exercising Leadership for Cohesion and Meaningful Empowerment (WELCOME)* Democracy International (DI) Trinidad & Tobago - IP (grantee) leadership - IP (subgrantee) frontline staff - Network service providers - Program participants (Venezuelan migrant GBV survivors) Note: * Activity selected for implementation evaluation. EVALUATION DESIGN For the portfolio and cluster level evaluation questions in Table 1, NORC will use a mixed methods approach to conduct the performance evaluation of the BTC activities. Questions regarding processes CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 6 and activity design, adoption, and implementation will be asked to USAID/GenDev staff, as well as staff of grantees, and subgrantees. We will obtain information to answer evaluation questions that deal with activity outcomes from activity recipients, to obtain direct feedback on their experience with specific interventions. We will begin with a desk review of activity documents, and meetings with grantees, subgrantees, and relevant partner staff. After having further discussions with the IPs for the three BTC activities, NORC proposes undertaking the implementation evaluation (IE) for the WELCOME activity in Trinidad and Tobago – documenting the re-design and shift in implementation, due to the disruption of the COVID-19 pandemic. We suggest a qualitative approach for the IE, working closely with DI and its subgrantee La Casita. This will include document review, KIIs with grantee and subgrantee staff, and FGDs with frontline workers at La Casita to focus on the strategy the grantee and the subgrantee deployed to adjust their intervention so that it addressed the needs of Venezuelan women in Arima, T&T. This retrospective inquiry is valuable to document the institutional knowledge the grant enhanced, as well as the lessons learned from this process, practices that could be replicated, and improvements in future programming. We include draft instrument questions at the portfolio and AC level in Annex A under Table A.1: Instrument Matrix for Portfolio, Activity Cluster, and Implementation Evaluation. In addition, Table 4 below includes a few more instrument questions that are specific to the BTC cluster. Table 4: BTC Specific Instrument Matrix Evaluation Question Activity-Specific Questions ACTIVITY CLUSTER QUESTIONS Are the activity clusters based on context-specific and international evidence? - Is there a role for members of the host communities in the activity? - Has your organization changed its approach to migrants after experience with the activity? Please explain? To what extent are each of the activity clusters achieving the targeted GBV results? - Are there any lessons learned on how to engage host community members? Please, provide an example. IMPLEMENTATION EVALUATION QUESTIONS Is the activity design based on the local context and flexible to achieve results on the ground? - Please, identify the main factors that determined the re-design of the activity. - How did your organization identify the needs of the target population after redefining it? Is the activity achieving sustainability? - Have any members of the host communities supported or expressed enthusiasm for this work? Please, describe. - Are there any migrant groups or any other social agreements in the implementation area that could promote these efforts in the future? Please, explain. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 7 DATA COLLECTION METHODS Data collection activities for the portfolio and activity cluster evaluation questions will take place from October 2022 through January 2023. The IE will take place in January and February 2023. The evaluation team will conduct most KIIs with USAID/GenDev staff and grantee staff via Zoom, Teams, or any other remote communication platform for respondents’ convenience (for those with access to the internet). NORC will rely on local consultants in each country to arrange and conduct KIIs with heads of local partners and service providers and two FGDs with front-line service providers staff for all three activities, including the activity selected for the implementation evaluation. The local coordinator will also moderate three FGDs with secondary activity participants in Panama, which the team will find through the primary activity users. For all in-person meetings, NORC will follow standard public safety measures, including the use of a mask, sanitizing gel, and keeping a six-feet distance between participants. Sampling: As displayed in Table 5, NORC will obtain insights on user experience from Venezuelan migrants who are survivors of GBV and participated in the initiatives funded by USAID through grants awarded to DI-WELCOME and NCC-BTG4VM. In each case, our consultants will schedule and administer between 10 and 12 one-on-one interviews with them, following NORC’s and in-country IRB protocols. In both cases, DI-La Casita and NCC have offered their generous collaboration to assist our consultants in sharing the users’ information to identify and recruit respondents. For both locations, NORC will provide a guide of counseling services for respondents to contact in case of distress or upsetting reactions from narrating their experience with the program. NORC is making its best efforts to hire consultants with experience on basic psychosocial support, particularly related to the migration experience. HIAS has provided our team with a full list of the 165 male participants, including phone numbers, and data on demographic characteristics and treatment uptake. NORC will use this information to select a list of potential respondents, including three replacements for each of the 18 interviews. The consultant will call and recruit respondents according to the three lists, one for each locality. NORC plans to administer two web-based surveys via Qualtrics, accessible through desktop and mobile devices, for the staff of DI-WELCOME’s and NCC-BTG4VM’s networks partners to provide their perspectives on these activities. Web survey respondents will include grantee and subgrantee staff, as well as staff from organizations that partnered with these two activities as providers of legal, job training, housing support, and other administrative services. As members of these one-stop shops for Venezuelan migrants – who are GBV victims or vulnerable to it – NORC plans to collect information about their experience working with this group of migrants, collaborating with the main grantee, implementing the activity, and witnessing any outcomes or changes in these communities. Almost all survey items will be close-ended and will capture topics such as the availability of GBV-related services in the community, characteristics of the services provided through the network, target population and participant uptake, context appropriateness, service outcomes, monitoring tools, and activity sustainability. Local consultants will facilitate the engagement of web-survey respondents in each country. They will contact the respondents, get their consent to participate in the evaluation, and remind those who have not completed the survey, via email, text, and over the phone to do it, to improve response rates. With this approach, NORC expects a response rate greater than 50 percent. The team suggests the use of web-based surveys to allow working service providers the convenience of responding when convenient and over a period of time. This will avoid imposing a significant time burden on respondents. Both IPs in CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 8 Guyana and Trinidad and Tobago have confirmed the ability of staff having phones and cell phone coverage to respond to the online surveys. Table 5: Data Collection Methods for BTC Data Collection Methods Data Source(s) Desk Review Documentation provided by grantees and subgrantee for each activity, including current evaluation workplans, annual, quarterly, baseline, and endline reports, MEL data records, theories of change, logic models, workplans, assessments, internal documents, public outreach and communications documents and webpages. Key Informant Interviews Eight KIIs with grantees and subgrantee senior leadership and technical staff, regional implementation agency, USAID/GenDev staff, and participants facilitated by team lead or local consultant: ● One KII with IP COP (NCC-BTG4VM) ● Two-to-three KIIs with IP COP and technical experts (HIAS￾SDP) ● Two KIIs with grantee and subgrantee (DI-WELCOME) ● One group interview with Resonance ● One KII with BTC Cluster Manager ● Three-to-four KIIs with managers of NCC-BTG4VM service providers ● Three-to-four KIIs with managers of DI-WELCOME service providers ● 35-42 semi-structured interviews with female Venezuelan migrants who survived GBV and used activity services, as well as with male partners of Venezuelan migrants: ● 10-12 interviews with NCC-BTG4VM participants in the Georgetown region ● 10-12 interviews with DI-WELCOME participants in Arima, T&T ● 15-18 interviews with primary male HIAS-SPD users in Panama City, San Miguelito, and Panama West (5-6 in each) Focus Group Discussions ● Three FGDs with female partners of primary HIAS-SPD participants (one in each site: Panama City, San Miguelito, East Panama) ● Two FGDs with 6-8 service staff members, one with NCC￾BTG4VM and the other one with DI-WELCOME Survey ● Two Qualtrics surveys with operational staff of each activity (NCC-BTG4VM and DI-WELCOME) Field Observation and Site Visit Site visit to La Casita to observe activity components in Arima, T&T for the implementation evaluation of the WELCOME activity under current funding (post-USAID). CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 9 COLLECTIVE ACTION TO REDUCE GENDER-BASED VIOLENCE: EVALUATION DESIGN AND DATA COLLECTION INTRODUCTION The Collective Action to Reduce Gender-Based Violence (CARE-GBV) activity implemented by Development Professionals, Inc.-Making Cents International (DPI-MCI) supports USAID/GenDev in the development of guidelines, strategic plans, training, and professional networking support for its GBV programming. To support GBV organizations capacity building, CARE-GBV awarded grants from $50,000 to $125,000 over a one-year period (July 2021 - July 2022) to five organizations through an open call. The grants program, which is the focus of this evaluation, awards small grants to new, local, and under-utilized partners to improve staff wellness and resiliency in GBV programming, fill global data gaps related to self- and collective care and wellness for staff of GBV organizations, and promote learning. Each of the final organizations selected for the CARE-GBV Small Grants Program is led by women, including women who identify as survivors of GBV. All five grantees confirmed interest and availability to participate in the portfolio and activity cluster evaluation. The table below presents the five activities that will be included in the CARE-GBV evaluation. Table 6: Summary of CARE-GBV Activities Subject to Performance Evaluation No. Activity Organization Country Types of Respondents 1. Supporting Innovative Practices in Self-Care, Wellness, and Resiliency among GBV Workers in North Macedonia Crisis Center Hope (CCH) North Macedonia - CCH staff - Staff of other organizations/institutions (working with domestic violence victims) who participated in 2-day workshop hosted by CCH 2. Promoting Staff Wellness and Resilience for Effective Response to Sexual and Gender-Based Violence Programming* Sexual Offences Awareness and Response Initiative (SOAR) Nigeria - SOAR staff - Staff of other organizations and members of the sexual and GBV response team who participated in SOAR’s training 3. We Care - Institutionalizing Accessible Staff Wellness and Resilience Policies, Tools, and Practices for the GBV Field Sexual Violence Research Initiative (SVRI) Global - SVRI staff - External stakeholders who completed the pilot course developed by SVRI 4. Thuso Ya Bathusi (Enhancing Staff Resilience and Wellness) Women Against Rape (WAR) Botswana - WAR staff 5. Žene sa Une Staff Wellness Program Žene sa Une (ZSU) Bosnia and Herzegovina - ZSU staff Note: * Activity selected for implementation evaluation. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 10 EVALUATION DESIGN NORC will use a mixed-methods approach to conduct the performance evaluation of the CARE-GBV cluster, beginning with a desk review of activity documents. We will conduct KIIs with USAID/GenDev staff, MCI staff, and IP senior staff and partner organizations. We will also conduct web surveys with all organization staff and external stakeholders who participated in the training on dealing with vicarious trauma. In Bosnia and Herzegovina, we will contract a local translator to translate interview guides and web surveys from English to Bosnian. They will also translate transcripts and survey responses from Bosnian to English. Since the direct participants of CARE-GBV are grantee staff and other local GBV responders (or global in the case of SVRI), we do not require clearance from any in-country institutional review board. KII and web survey questions will be specific to the program design, whether it met their needs, any challenges in achieving the desired results, lessons learned, and sustainability of the program to improve staff wellness, selfcare and resilience among GBV responders. Based on conversations with Chai Shenoy, USAID/GenDev Point of Contact (POC) for the CARE-GBV cluster and to align with USAID priorities, we considered two activities for the implementation evaluation: (i) Promoting Staff Wellness and Resilience for Effective Response to Sexual and Gender￾Based Violence Programming implemented by SOAR in Nigeria, and (ii) Žene sa Une Staff Wellness Program implemented by ZSU in Bosnia and Herzegovina. Although both SOAR and ZSU expressed interest and willingness to participate in the implementation evaluation, upon further consideration, we selected SOAR’s activity for the following reasons: 1. SOAR has a unique focus on providing counseling services to survivors of child sexual abuse in Nigeria. Since child sexual abuse is a widespread issue, SOAR highlighted the need to build capacity and well-being of GBV responders so they can better serve this vulnerable group. 2. SOAR had wider reach since the activity also engaged member organizations of the FCT Sexual and Gender-Based Violence Response Team (FCT SGBV-RT) in monthly staff wellness and self-care workshops. SOAR worked in consultation with FCT SGBV-RT to develop i) Training manual on trauma counselling and psychosocial support for child survivors of GBV and ii) Wellness and selfcare training manual for GBV prevention & response workers. Draft instrument questions at the portfolio and AC level can be found in Annex A under Table A.1: Instrument Matrix for Portfolio, Activity Cluster, and Implementation Evaluation. In addition, Table 7 below includes a few more instrument questions that are specific to the CARE-GBV cluster. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 11 Table 7: CARE-GBV Specific Instrument Matrix Evaluation Question Activity-Specific Questions ACTIVITY CLUSTER QUESTIONS 1. Are the activity clusters based on context-specific and international evidence? ● In what ways were the courses, trainings, and overall activity components designed to meet the unique needs of grantee staff and other GBV responders based on the survivor group (e.g., domestic violence, child sexual abuse, refugee, trafficking victims, etc.) and the local context? ● In addition to organization staff, grantees collaborated with external stakeholders including GBV experts, counselors/psychologists, and partner organizations working on GBV prevention, among others. What were the contributions of these additional stakeholders and was their participation valuable? 2. To what extent are each of the activity clusters achieving the targeted GBV results? ● CARE-GBV focuses on building staff wellness and resilience among GBV responders. Which interventions have had an effect? Which interventions have not performed as anticipated? Why? ● What do you think is the biggest obstacle in minimizing vicarious trauma? How did your program address this? 3. To what extent are the ACs sustainable? ● What are the primary ways in which this activity has changed the way that grantees address vicarious trauma? ● How sustainable are online courses, self-reported wellness tools, workshops, and other such mechanisms developed and implemented under this activity? Do you envision any barriers to continued implementation and uptake of these resources? IMPLEMENTATION EVALUATION QUESTIONS 1. Is the activity design based on the local context and flexible to achieve results on the ground? ● There is no common theory of change for the CARE-GBV cluster. What prompted SOAR to develop an independent theory of change for the activity? How is this grounded in the local context? 2. Is the activity reaching participants they are meant to target? ● SOAR provides counseling services to survivors of child sexual abuse. Can you tell me if and how this unique focus was incorporated into the study design? What factors were considered so they can better support this group of GBV survivors? ● What are the specific self-care and wellness needs of GBV responders working with child survivors of sexual violence? DATA COLLECTION METHODS NORC will conduct desk review of activity documentation and collect mixed-methods data for all five activities included in the evaluation. NORC will collect portfolio and activity cluster evaluation data between October 2022 and January 2023 and conduct the implementation evaluation in February 2023. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 12 The research team will conduct virtual KIIs with USAID/GenDev staff, MCI staff, IP senior staff, and local partners via Zoom or Microsoft Teams. Most interviews will be conducted in English, and a few will be conducted with a real-time translator from Bosnian to English and vice versa. We will seek informed consent prior to the interviews. Sampling: In addition to the KIIs, NORC will administer one web-based survey via Qualtrics, accessible through desktop and mobile devices. The survey will be administered to all IP staff and GBV responders who participated in program activities. This gives us a sample of 36 respondents for CCH, 36 for SOAR, 39 for WAR and 14 for ZSU. Since SVRI piloted their course globally their samples is 670. Since IP senior staff confirmed that all participants have internet access to complete the survey, we will share a unique link with each participant via email. For ZSU staff, the survey will be translated into Bosnian. The survey will include a consent script and participants will be requested to provide their consent before starting the survey. The research team will monitor survey responses and send email reminders to participants who have not completed it. Table 8: Data Collection Methods for CARE-GBV Data Collection Methods Data Source(s) Desk Review Activity data and documents ● Monthly/quarterly/progress reports ● Training/webinar attendance sheets ● Training manual/curriculum/slides ● Training report ● Theory of change3 ● Indicator tracking sheet Key Informant Interviews 15 key informants to be interviewed: ● 1 USAID/GenDev staff (Chai Shenoy) ● 2 MCI staff ● 2 IP senior staff for each of the five activities (total 10) ● 1 IP sub-contractor/partner org for each relevant activity (total 2) Web Surveys 1 web survey with all grantee staff + external stakeholders (n=795) who participated in workshop/training course/webinar Note: This is an approximate number as grantees are obtaining consent from external participants. 3 The CARE-GBV activity was not asked to develop a theory of change for the small grants initiative and so there is no common theory of change for this activity cluster. However, two grantees, SOAR and SVRI, developed their own theory of change to guide their implementation. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 13 RESILIENT, INCLUSIVE AND SUSTAINABLE ENVIRONMENTS INTRODUCTION The Resilient, Inclusive, & Sustainable Environments (RISE) Challenge is a program that supports organizations to adapt and implement approaches addressing gender-based violence in environmental programming. The Challenge aims to: ● Raise awareness of linkages between GBV and environmental programming; ● Test new programming approaches; ● Share learning on interventions and policies; and ● Increase attention to this issue with other organizations, implementing partners, and donors for collaboration and co-investment. The program fosters partnerships across sectors to reduce rates of gender-based violence related to access to and control over natural resources in contexts that are affected by environmental degradation and climate change. The evaluation of the RISE activities was supposed to include all nine grantees from the Challenge. However, NORC reached out to the grantees to confirm the feasibility of their participation, two of the nine indicated that it would not be feasible for them to participate due to changes in activity staff, limited staff bandwidth, and concerns about research fatigue with participants. The activities that declined participating in the evaluation include Resourceful Empowerment: Elevating Women’s Voices for Human and Environmental Protection in Congolese Small-Scale Mining and Tz’unun: Ending Environmental Violence Against Indigenous Women in Guatemala through Empowerment in Community Forestry, Agroecology and Collective Healing Spaces. The latter had previously been selected for the implementation evaluation, in part because USAID/GenDev is interested in understanding how to improve their partnerships with Indigenous organizations and small￾scale local partners. Since they declined to be evaluated, we propose a different activity for the implementation evaluation – Rising Up!: Promoting Congolese Women’s Land Access and Preventing GBV in Eastern DRC – for reasons provided below. Table 9 below presents the organizations and types of participants that we will collect data from for the seven RISE grants included in our evaluation. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 14 Table 9: RISE Activities Included in the PPE No. Activity Organizations Country Types of Respondents 1 Creative Capacity Building to Address Gender Based Violence in the Artisanal and Small￾Scale Mining Sector Massachusetts Institute of Technology Development Lab (MIT D-Lab), Advocacy Capacity Building (ARM) Colombia -Gender equality advocates who participated in advocacy network building and organizing -Community members in mining communities in the zone of influence of the activity 2 Conservation of the Alto Mayo Landscape without Gender Violence Conservation International (CI); PROMSEX Peru Participants of training sessions Community members who have not been directly involved with the activity 3 Combatting Gender-based Violence in Vietnamese Conservation WildAct Vietnam; CARE International, Vietnam Association of National Parks and Protected Areas, Women in Conservation Canterbury Network Vietnam Conservation organization staff not directly involved with the activity Activity participants from partner NGOs 4 Advancing Equitable Gender, Social and Power Norms in Community Conservancies in Kenya Kenya Wildlife Conservancies Association (KWCA); Fauna & Flora International (FFI), Sera Wildlife Conservancy (SWC), CARE International in Kenya (CARE), Centre for Rights Education and Awareness (CREAW) Kenya Conservancy staff not directly involved with the activity Activity participants from partner NGOs CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 15 No. Activity Organizations Country Types of Respondents 5 Gender Empowerment and Transformation: Tackling Resource￾Based Conflict and Gender-based Violence in Fiji Marstel Day; Wi-Her, University of the South Pacific, the Fiji Environmental Law Association, Live & Learn Environmental Education, Fiji’s Reducing Emissions from Deforestation, and forest Degradation (REDD+) Program Fiji Community members who have participated in sensitizations about the Enhanced Feedback Grievance and Redress Mechanism Participants of sensitization activities 6 Rising Up!: Promoting Congolese Women’s Land Access and Preventing GBV in Eastern DRC Women for Women International (WfW); Innovation et Formation pour le Développement et la Paix (IFDP) Democratic Republic of the Congo Women who have started but not yet completed the process of land titling Male leaders who received training Women change agents Land reflection groups members 7 Securing Land Rights & Ending Gender Exclusion in Uganda* Trócaire, SOCADIDO, and LEMU Uganda Community influencers trained by the activity Community members not directly involved in the activity Women who have started or gone through the process of land titling Note: * Activity selected for implementation evaluation. EVALUATION DESIGN We are proposing to evaluate the project, Rising Up!: Promoting Congolese Women’s Land Access and Preventing GBV in Eastern DRC project for the implementation evaluation. We will go through NORC’s ethical review board approval process to be able to collect data from project participants, which will give us sufficient information to assess project outcomes and answer individual activity level questions. Correspondence with the grantee and desk research revealed that we will not need local IRB approval in the DRC since we are not conducting biomedical research. We had previously proposed to evaluate the Securing Land Rights and Ending Gender Exclusion in Uganda activity for the implementation evaluation. Nevertheless, after corresponding with implementing partners and doing our own desk research, we determined that securing local ethical review board CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 16 approval would take a minimum of six months which is not feasible for the timeline of our evaluation. Therefore, we will include the Ugandan activity in our performance evaluation by collecting data from implementing partner staff in order to answer our portfolio level and activity cluster level evaluation questions. However, without local IRB approval, we will not be able to collect data from program participants to answer individual activity level questions, which is necessary for the implementation evaluation. Both activities in Uganda and the DRC center around land tenure governance reform, so there are similar lines of inquiry around the sustainability of these social norm change interventions. In addition, implementing partner staff with Women for Women International indicated in a recent call that they are keenly interested in learning from us as they plan to expand their gender-based violence programming with similar intervention approaches in other geographic contexts. NORC will exclusively collect qualitative data for five out of the seven RISE activities included in our evaluation. For these five activities, we will collect information about respondents’ personal experiences from community members who participated in social norm change activities and potentially survivors of gender-based violence. Therefore, in-person interviews or focus group discussions with local moderators trained in trauma-informed research practices were determined to be most feasible and appropriate. Since many participants are in remote locations with limited internet access and speak Indigenous or non-colonial languages, NORC will recruit local moderators that speak native tongue languages and understand cultural contexts. The local moderators will receive training in trauma-informed research practices for all qualitative data collection with participants so that interview respondents feel comfortable and safe while sharing their experiences. In addition, we will secure clearance from local ethical review boards. In addition, NORC will conduct mixed-methods data collection for two activities that have indicated that a survey would be feasible and appropriate. Surveys will be translated into local languages and administered only to organization staff and participants in sensitization activities. Respondents will be asked about their experiences with the activity, not their personal experiences. Draft instrument questions at the portfolio and AC level can be found in Annex A under Table A.1: Instrument matrix for Portfolio, Activity Cluster, and Implementation Evaluation. In addition, Table 10 below includes a few more instrument questions that are specific to the RISE cluster. Table 10: RISE Specific Instrument Matrix Evaluation Question Activity-Specific Questions PORTFOLIO QUESTIONS 1. What pervasive gaps still exist in understanding GBV and addressing specific types of GBV? ● Have you made changes to the TOC or approach of RISE in designing RISE II? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 17 Evaluation Question Activity-Specific Questions ACTIVITY CLUSTER QUESTIONS 3. To what extent are the ACs sustainable? ● How did you hear about the opportunity to participate in the RISE Challenge? ● What, if any, changes would you like to make to the proposal submission process? What features of this activity were most suited to change the way that your organization operates? Which were most effective? ● What operational challenges did your organization have to implement the activity? IMPLEMENTATION EVALUATION QUESTIONS 4. Is the activity design based on the local context and flexible to achieve results on the ground? ● What are the barriers and potential facilitators to sustainable reform of land tenure/natural resource governance systems so that they contribute to the prevention of GBV? How did your program address these? 5. Is the activity reaching participants they are meant to target? ● Among the groups of people in your community/zone of influence, who do you think most benefited and who might have been left out or not benefited sufficiently? [Probe on different kinds of populations] Why or why not? 6. Is the activity achieving sustainability? ● Have you experienced any resistance from community members or leadership to the changes promoted by this activity? Please describe them. ● Have community members supported or expressed enthusiasm for this work? Describe? ● Have there been any unexpected consequences (good or bad) to these reforms or activity activities in general? DATA COLLECTION METHODS NORC will conduct desk review of activity documentation and collect qualitative data for all seven RISE activities included in this evaluation. For these activities, we will conduct interviews with (1) Resonance, the Prime partner for all RISE grants; (2) implementing partner staff; (3) GBV experts in the activity contexts or countries that can comment on how the activities fit into the bigger picture of GBV programming in that context; and (4) relevant USAID bilateral mission and USAID/GenDev staff. In addition, we will gather quantitative data for two activities that focus on capacity building for conservation organizations (Combatting Gender-based Violence in Vietnamese Conservation) or capacity building for a conservation mechanism (Gender Empowerment and Transformation: Tackling Resource-Based Conflict and Gender-based Violence in Fiji). Sampling: Vietnam: The sample will consist of conservation organization staff whose contacts will be provided by the implementing partner. The Vietnamese consultant will help mobilize the web-survey contacting respondents over the phone and by email to remind them to take the web-survey. IP staff CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 18 indicated that there are no problems with internet connectivity or access to computers to take the survey. Fiji: The sample will consist of community members who were sensitized about the enhanced Feedback Grievance and Redress Mechanism in Fiji as well as participants of virtual sensitization activities (these are stakeholders such as REDD+ staff, partner organization staff, and government officials). For community members, our data collection firm will conduct the survey either over the phone or in person depending on feasibility and whether respondents have access to cell phones. For participants of virtual sensitization activities, the survey will be conducted over the phone because they will have access to phones. We expect community members to make up about 1/3 of the sample. Table 11: Data Collection Methods for RISE Data Collection Methods Data Source(s) Desk Review Documentation provided by Resonance for each activity, including annual, quarterly, and baseline/midline/endline reports; MEL data records; MEL plan and work plan; formative studies such as gender, environment, or conflict analyses; guidance for staff and partners; meeting notes; documentation on public outreach and communications; webpages; and other publications or outputs, such as tools, curricula, or policies Focus Group Discussions or One-on-one Interviews with participants FGDs or one-on-one interviews with participants facilitated by a local moderator, with at least 2 additional FGDs or interviews for the activity that will be the target for the implementation evaluation Key Informant Interviews ● Panel interview with Resonance (Prime) ● 16 KIIs with IP staff: at least 2 for each of the 7 activities, with two additional for the implementation evaluation activity ● 6 KIIs with USAID Staff: 3 KIIs with USAID/GenDev staff (Corinne Hart, Chai Shenoy, Cathy Odera) and 2-3 Other USAID Staff TBD (Bilateral Mission Staff) ● 14 KIIs with local GBV experts to discuss TOC and appropriateness of activity designs (2 per activity) Field Observation and Site Visits None—we are working through local partners and moderators, NORC staff will be training them but not going in-person for data collection Survey 2 web-based, phone or in-person surveys in Vietnam and Fiji4 WOMEN’S ECONOMIC EMPOWERMENT INTRODUCTION The WEE cluster evaluation will cover five out of the 16 activities from the list of USAID/GenDev WEE activities. All IPs confirmed their availability and interest to participate in the evaluation and provide necessary documentation and support. Table 12 presents the activities and types of respondents we will 4 Implementing partners in Fiji and Vietnam confirmed that their participants (conservation organization staff not directly involved with the activity and people who have used the enhanced Feedback Grievance and Redress Mechanism) would be able to answer survey questions either on the web (Vietnam) or by phone/face to face methods (Fiji) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 19 collect data from for the five WEE grants included in this evaluation. These activities encompass a broad range of interventions to overcome barriers and foster women’s economic participation through directly working with grassroots participants or enabling systems-level and environmental change related to gender equality. Activity 5 has been selected for an implementation evaluation and will shed light on potential ways to improve the implementation of GBV components of activities. Table 12: Data Sources by WEE Activity No. Activity Organizations Country Types of Respondents 1 A Micro￾Journey to Self-Reliance: Economic Reintegration for Victims of GBV ● IP: Management Sciences for Health, Inc. (MSH) ● Local Partners: Ministry of Health, Ministry of Justice, Association Pour l’Education, la Sexualité et la Santé en Afrique (APESSA). Benin ● MSH senior leadership ● Representatives of local partner organizations ● USAID/GenDev staff ● Women survivors of intimate partner violence (IPV) ● Spouses of survivors 2 Enabling Environment for Economic Empowerment of Women (E4W) ● IP: Freedom House ● Local Partners: Search for Common Ground, Catholic Relief Services (CRS), Kahawatu Foundation Burundi ● Freedom House senior leadership ● Representatives of local partner organizations ● USAID/GenDev staff ● Socioeconomically disadvantaged women and women experiencing homelessness ● Married and unmarried women seeking financial independence 3 Engendering Industries5 ● IP: Tetra Tech, Inc. ● Local Partner: Ibadan Electricity Distribution Company (IBEDC) Nigeria ● Tetra Tech senior leadership ● Tetra Tech staff ● Representatives of IBEDC ● USAID/GenDev staff ● Women and men who participated in Intensive or Accelerated Programs, Male Engagement programs, or grant-funded gender initiatives ● IBEDC employees who have attended grant-funded trainings or programs 5 Due to the broad scope of this activity, the team will focus the performance evaluation of Engendering Industries on one utility company in Nigeria, based on the recommendation of the IP. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 20 No. Activity Organizations Country Types of Respondents 4 Global Labor Program: Levi￾Strauss Partnership ● IP: Solidarity Center ● Local Partners: Federation of Women Lawyers in Lesotho (FIDA), Women and Law in Southern Africa – Lesotho (WLSA), Workers’ Rights Watch (WRW), Independent Democratic Union of Lesotho (IDUL), United Textiles Employees (UNITE), National Allied Clothing and Textiles Workers Union (NACTWU) Lesotho ● Solidarity Center senior leadership ● Representatives of Nien Hsing ● Representatives of local partner organizations ● USAID/GenDev staff ● Women and men employed in 2 Nien Hsing factories that have participated in the activity 5 Women’s Economic Empowerment in Mesoamerica (WEEM)* ● IP: Rainforest Alliance ● Local Partners: Association of Organizations of the Cuchumatanes (ASOCUCH), Women’s Justice Initiative (WJI) Guatemala ● Rainforest Alliance senior leadership ● Representatives of local partner organizations ● USAID/GenDev staff ● Sixteen to 60-year-old women with reduced economic resources in the Huehuetenango Department ● Men who are gender champions in the activity Note: * Activity selected for implementation evaluation. EVALUATION DESIGN We will use a mixed-methods approach for four out of the five activities, beginning with a desk review of activity documents. The desk review will help us finalize the draft instrument questions for the KIIs and FGDs with USAID/GenDev staff, IP staff, local partner staff, and/or participants for each activity, as well as web surveys of activity participants (if applicable). Based on feedback from the IP, the activity Engendering Industries will not involve a survey and findings will be based on the analysis of primary qualitative data and existing secondary data from baseline and endline studies. Qualitative data will include KIIs with the 7 Intensive and Accelerated Program participants6 and FGDs with employees who have attended grant-funded gender initiatives, trainings, or programs. FGDs will be moderated either by NORC staff or by local consultants. Table 14 below shows the breakdown of different primary data collection sources. Local consultants will be hired in Benin, Burundi, Guatemala, Lesotho, and Nigeria to conduct in-person FGDs and KIIs and administer two computer-assisted telephone interview (CATI) surveys in Benin and Burundi and two face-to-face surveys in Guatemala and Lesotho. Local consultants will contact respondents, get their informed consent to participate in the evaluation, and follow up with respondents 6 These are the 7 champions within the organization who underwent the Intensive and Accelerated Program. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 21 to ensure we meet our response targets. We will recruit local consultants who speak participants’ local language. For the WEEM activity, whose participants speak a variety of Indigenous languages, we will work with the IP to identify interpreters. We propose the Women’s Economic Empowerment in Mesoamerica (WEEM) activity for the implementation evaluation for the following reasons: 1. The IP Rainforest Alliance has expressed great interest and willingness to participate in the evaluation and is currently in its third year of implementation out of five, so recommendations would be useful for the activity at this stage. 2. This activity has a unique focus on developing the capacity of local government (municipalities, Indigenous People’s offices, Women’s Divisions, police departments) related to GBV prevention and rehabilitation. 3. Most of the activity’s participants are Indigenous women (85-90%) who speak 20 different languages, providing an opportunity for deeper learning around implementation targeting linguistically diverse Indigenous communities. Draft instrument questions at the portfolio and AC level can be found in Annex A under Table A.1: Instrument Matrix for Portfolio, Activity Cluster, and Implementation Evaluation. In addition, Table 13 includes a few more instrument questions that are specific to the WEE cluster. Table 13: WEE Specific Instrument Matrix Evaluation Question Activity-Specific Questions ACTIVITY CLUSTER QUESTIONS 1. Are the activity clusters based on context-specific and international evidence? ● Why and how were GBV-focused activities integrated into the larger IHSA program? ● What additional support is provided to participants after the business trainings to foster sustainability of their ventures? ● What was the community reaction to these activities? ● Could you please discuss the theory of change for Engendering Industries? Can you describe how the causal pathway was conceived and any assumptions that were made? 2. To what extent are each of the activity clusters achieving the targeted GBV results? ● This activity is focused on preventing instances of GBV in communities. Which interventions have had an effect? Which interventions have not performed as anticipated? Why? ● Are women belonging to ethnic minorities and other disadvantaged groups engaged in community dialogue? What strategies does the organization employ to engage these groups? ● Were the courses, trainings, and activity components suited for the organizational structure and culture of IBEDC? ● What were the obstacles to influencing GBVH at Nien Hsing and how do you think the Global Labor Program activity is addressing these? ● Have you used the anti-GBVH information line? If so, why did you call? What was your experience with the information line staff? Did WRW follow up on your case? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 22 Evaluation Question Activity-Specific Questions IMPLEMENTATION EVALUATION QUESTIONS 1. Is the activity design based on the local context and flexible to achieve results on the ground? ● We understand that COVID-19 had a significant impact on the activity. Could you describe some of these challenges for WEEM in Guatemala? How did the program adapt to these challenges? What additional support was provided to participants during this period? ● What are the key enabling factors that improved the implementation of the activity? DATA COLLECTION METHODS The following data collection activities will be conducted from October 2022 through February 2023. Most KIIs with USAID/GenDev staff, IP staff, and local partners will be conducted virtually via Zoom (for respondents with access to the internet), while FGDs and KIIs with activity participants will be conducted in-person in participants’ local language. Sampling: According to recent meetings with IPs, there are a total of 17,150 participants across three of the four activities for which we are conducting surveys: 150 from A Micro-Journey to Self-Reliance; 12,000 from E4W; and 5,000 from Global Labor Program: Levi-Strauss Partnership. The target sample size for each activity is 100 respondents. We will randomly sample up to 125 respondents for each of the activities to accommodate a non-response rate of up to 20% and then randomly sample additional respondents if non-response exceeds this amount. The team will assign weights to the study samples during the data cleaning and analyses stages to ensure the representativeness of the sample to the population of program participants for each activity. The two CATI surveys will be administered to activity participants with the help of local data collection firms for each activity. We will use face-to-face surveys for activities with program participants who may have difficulty taking a survey over the phone. For WEEM participants, many of whom have not completed primary school, the IP has recommended a face-to-face survey that will be conducted by a team of local consultants and interpreters. The IP for the Global Labor Program: Levi-Strauss Partnership has also indicated a face-to-face survey would be more appropriate for program participants. Table 14: Data Collection Methods for WEE Data Collection Methods Data Source(s) Desk Review Documentation provided by IPs for each activity, including annual, quarterly, baseline, and endline reports, MEL data records, theories of change, logic models, assessments, internal documents for staff, public outreach and communications documents, and webpages. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 23 Data Collection Methods Data Source(s) Focus Group Discussions 14 FGDs with IP staff, activity participants, and representatives from local partner organizations facilitated by a local moderator ● 2 FGDs with IP staff for Engendering Industries ● 10 FGDs with activity participants (3 for WEEM; 3 for E4W; 4 for Engendering Industries) ● 2 FGDs with local partner organizations for Global Labor: Levi￾Strauss Partnership Key Informant Interviews 38 KIIs with IP senior leadership, representatives from local partner organizations, USAID/GenDev staff, and activity participants facilitated by team leads or local moderator ● 10 KIIs with IP senior leadership facilitated by team leads (2 per activity) ● 10 KIIs with representatives from local partner organizations facilitated by team leads (2 for Engendering Industries, 2 for E4W, 2 for A Micro-Journey to Self-Reliance, and 4 for WEEM) ● 3 KIIs with USAID/GenDev staff facilitated by team leads (Catherine Odera, Corinne Hart, Jennifer Hansel) ● 15 KIIs with activity participants in their local language facilitated by local moderator and/or team leads (7 KIIs with all of the Intensive and Accelerated program participants for Engendering Industries; 4 KIIs with activity participants of Global Labor Program: Levi-Strauss Partnership; 4 KIIs with activity participants from A Micro-Journey to Self-Reliance) Field Observation and Site Visits 8 days of site visits to observe activity components in Huehuetenango Department for the implementation evaluation of the WEEM activity by the local consultant Survey 2CATI surveys of activity participants in Benin and Burundi, and 2 face￾to-face surveys of activity participants in Guatemala and Lesotho ETHICAL CONSIDERATIONS Institutional Review Boards (IRB) have jurisdiction over research involving human subjects. This category of research includes obtaining information about behavior that occurs in a context in which an individual can reasonably expect that no observation or recording is taking place, and that information provided will not be made public (e.g., sexual abuse or harassment descriptions). Since research conducted about survivors of GBV normally involves inquiring about respondents’ private information, it is considered human subjects’ research and subject to an IRB. NORC’s IRB is charged with the responsibility of reviewing and overseeing all human subject research related activities undertaken by the organization. The IRB review process is designed to protect the rights and welfare of human subjects by ensuring equitable subject selection, assuring adequate informed consent, assessing and minimizing risks, and maintaining privacy and confidentiality. No human-subject CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 24 research activities at NORC can be conducted without the approval of the IRB. Once IRB approval is granted, the IRB conducts a continuing review of the research at intervals appropriate to the degree of risk, but not less than once a year. Our evaluation team will work to comply with the IRB’s policies and regulations across instrument development and all stages of data collection and analysis. The team is taking concrete measures to minimize respondents’ risks; ensure that any risks to subjects from participating are reasonable in relation to the benefits; select respondents with equity; request and document informed consent from each respondent; protect the privacy of subjects and maintain the confidentiality of data; provide support routes in cases of distress or adverse reactions to instrument items; and provide additional safeguards to prevent coercion or undue influence of responses when engaging vulnerable populations (e.g., minors, unauthorized immigrants, mentally disabled individuals, or economically or educationally disadvantaged persons). In addition to complying with the IRB, the evaluation team will work in coordination with USAID Country Missions (as needed), USAID/GenDev and IPs in each country to determine whether the research protocol should be approved by an in-country IRB that conducts oversight of research activities. Table 15 summarizes the activities and countries for which the evaluation team will request approval from a local institutional review board to conduct data collection. Table 15: Ethics Committees Regulating this Evaluation Activity Cluster IP/Activity Ethics Institution for Social Science Research Country BTC NCC – Building the Gap for Venezuelan Migrants (BTG4VM) Institutional Review Board (IRB) and Ethics Committee, Ministry of Health (MOH) Guyana HIAS - Shifting Power Dynamics: Engaging Men in Gender-Based Violence Reduction (SDP) Committee of Research on Bioethics, Instituto Conmemorativo Gorgas de Estudios de la Salud Panama RISE Creative Capacity Building to Address Gender Based Violence in the Artisanal and Small-Scale Mining Sector Comité Institucional de ética en investigación en humanos Universidad CES Colombia Conservation of the Alto Mayo Landscape without Gender Violence Social Science Institutional Review Board with Grupo de Análisis para el Desarrollo Peru Advancing Equitable Gender, Social and Power Norms in Community Conservancies in Kenya African Population and Health Research Center Kenya Gender Empowerment and Transformation: Tackling Resource￾Based Conflict and Gender-based Violence in Fiji The University of the South Pacific IRB #1 Fiji CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 25 Activity Cluster IP/Activity Ethics Institution for Social Science Research Country WEE A Micro-Journey to Self-Reliance: Economic Reintegration for Victims of GBV Ministry of Health Benin Enabling Environment for Economic Empowerment of Women (E4W) Comité National d'Ethique au Burundi Burundi Engendering Industries National Health Research Ethics Committee; University of Ibadan Nigeria Activities not included in the table above either do not require approval from any in-country institutional review board or the nature of the research do not match the description of the activities under jurisdiction of the existing institutional bodies. DATA ANALYSIS NORC will apply a mixed-methods analysis approach for data collected across the four clusters. The evaluation team will analyze findings from each data source listed in the sections above and align the evidence to each evaluation question and sub-question at the portfolio, AC, and implementation evaluation level. Quantitative analysis will include generating frequencies and descriptive statistics of relevant program indicators and producing descriptive statistics on output and outcome indicators based on web survey data and a longitudinal analysis of trends in any time series data, where available. For qualitative data analysis, the NORC team will use an inductive, data-driven approach to analyze information from document review as well as notes and transcripts from KIIs and/or FGDs. This approach involves first closely reading the text in detail until familiar with the content, then developing a scheme of emerging categories or themes to be used as codes in the analysis. Once the code frame is complete, the team will produce brief descriptions and definitions for each code and go through an iterative test-coding process to ensure inter-coder reliability. The final code book and transcript files will be loaded into Dedoose or NVivo qualitative data software for analysis. Analysts will then code sections of text according to key themes of interest and tag transcript files according to participant demographics (e.g., gender, stakeholder type). Once coding is complete, all coded segments will be exported, organized by the code applied and the key demographics of the speaker, ensuring that all feedback is anonymized for confidentiality. Both quantitative and qualitative strands of data will be then triangulated within each theme of findings to increase the validity of data with rich contextual information. Table 16 below summarizes the key data analysis methods the NORC team will use across the ACs. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 26 Table 16: Data Analysis Methods for GBV PPE Data Analysis Methods Data Source(s) Description Comparison analysis (comparing activities across the cluster) Document review, qualitative interviews, and web surveys ● Qualitative analysis using Dedoose or Nvivo software, with excerpts coded and extracted by theme and evaluation question or sub-question ● Analysis of documentation using coding matrix in Excel ● Statistical analysis using Excel and Stata software; data visualization using Excel, Vizzlo, or Infogram Trend/Pattern analysis Qualitative interviews, field observation notes ● Qualitative analysis using Dedoose or Nvivo software, with excerpts coded and extracted by theme and evaluation question or sub-question Content analysis Qualitative interviews, FGDs, document review, field observation notes ●Activity cluster (performance) ● Qualitative analysis using Dedoose or Nvivo software; excerpts coded and extracted by theme and evaluation question/sub-question Frequency and descriptive analysis Program monitoring data, web surveys ● Statistical analysis using Excel and Stata software, data visualization using Excel, Vizzlo, or Infogram Information flow and validation analysis Qualitative interviews and validation feedback from IP Staff and USAID ● Qualitative analysis using Dedoose or Nvivo software, with excerpts coded and extracted by theme and evaluation question or sub-question ● Sharing of drafts with respondents, and validation meetings to discuss REPORTING AND DISSEMINATION Following data analysis, NORC will produce two evaluation reports. All NORC products will comply with Section 508 of the Rehabilitation Act (29 U.S. Code § 794 (d)). 1. A PPE report with an overall synthesis chapter including findings, conclusions, and recommendations across the portfolio and four separate chapters corresponding to each AC. Findings and recommendations will be based on analysis from both quantitative and qualitative data across the different approaches used for the evaluation. 2. An implementation evaluation report with an overview chapter as well as separate chapters for the four identified activities: WELCOM activity in Trinidad and Tobago (BTC), SOAR in Nigeria (CARE￾GBV), Securing Land Rights & Ending Gender Exclusion in Uganda (RISE), and WEEM in Guatemala (WEE). CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 27 The NORC team will have workshops to brainstorm and validate recommendations emerging from the evaluation. Workshops will involve representatives from the USAID/GenDev team, IP staff, and where feasible, other GBV programmers and practitioners. The objective is to confirm the findings by the evaluation team and ensure that recommendations are realistic/feasible. For activities that are still completing implementation, the recommendations will suggest mid-course corrections to achieve better outcomes. Some implementing partners have also indicated that even though their activities are complete, they plan to have future GBV programming in similar contexts and thus our findings will help them refine their approaches. The recommendations and key findings from the above reports will inform post-evaluation action planning with the USAID/GenDev team, as well as policy briefs and/or infographics for each activity cluster. Finally, the team will actively disseminate the findings of the PPE through workshops, webinars, policy briefs, and infographics. Table 17: Evaluation Timeline Task July - Sept Oct - Dec Jan - March April - June July -Aug 2022 2022 2023 2023 2023 Phase 3B – Evaluation Design Report Submit final EDR Phase 4 – Portfolio and Activity Cluster Performance Evaluations & Reporting Activity document review IRB application preparation and submission Data collection (KIIs, FGDs, and surveys) Quantitative data cleaning and descriptive analysis Qualitative coding Phase 5 – Implementation Research Reporting Data collection (KIIs, FGDs, surveys, and observations) Quantitative data cleaning and descriptive analysis Qualitative coding Phase 6 – PPE Report, Evaluation Debriefing & Dissemination Report writing and submission of initial draft of final report Post-evaluation action plan Submit final report CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N054 USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 28 Task July - Sept Oct - Dec Jan - March April - June July -Aug 2022 2022 2023 2023 2023 Submit initial draft of policy brief and infographics Submit final policy brief and infographics Dissemination USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 29 ANNEX A Table A. 1: Instrument Matrix for Portfolio, Activity Cluster, and Implementation Evaluation Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* PORTFOLIO QUESTIONS 1. How are the USG’s guiding principles and priorities to end GBV being incorporated into the four activity clusters (AC)? Prevention: In what ways is the USG activity portfolio contributing to reduced risks? What USG principles and policy measures have driven the choice of included interventions in the X cluster? What have been the most important contributions of the X cluster in preventing GBV? Which approaches from grantee activities were effective in preventing GBV? Why? What has been the most important contribution of your activity to preventing GBV? Please, explain how. What have been the most effective aspects of your activities or approaches to prevent GBV in this activity? Why? What are the main lessons learned about prevention that you could share? In the past X months, what have been the most important contributions of the X activity to prevent GBV in your community/organization ? In your opinion how has the X activity helped your community prevent episodes of GBV, if it has at all? What other services could activity X provide to your community to prevent instances of GBV? To what extent has activity X helped prevent GBV in your community/org/country? (Likert scale) Rate which program components have been the most effective in preventing GBV (Likert scale for each pre￾set component) USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 30 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Protection: How does the portfolio contribute to accessible, effective services for violence survivors? Thinking about the X cluster, how have its grants affected access to effective services for GBV survivors? Why? What approaches were most effective in increasing access to services? Which were least effective? Why? Please, briefly describe how the activity has provided or facilitated access to services for GBV survivors? Which specific activity component or approach was most effective? Can you provide some examples of program effectiveness in providing or facilitating access to these services? Do you think that the X activity has helped to improve access to services for GBV survivors in your community/organization/ country? How? From the following list, please indicate: ● What services are available for GBV survivors in your community/organization? ● To your knowledge what services have GBV survivors used/received? [Selected services]: To your knowledge to what extent are GBV survivors satisfied with the service they received? (Satisfaction scale) USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 31 Accountabilit y: How does the portfolio contribute to ending impunity? *(Ask only if the cluster funded any strategies to combat impunity of GBV perpetrators) In your opinion which strategy implemented by X cluster to stop or punish GBV perpetrators was most effective, and why? *Dependent on activity focus: How has your activity, directly or indirectly, contributed to increasing perpetrator accountability? What else could the activity or future programming in a similar area do to foster further perpetrator accountability? In your experience with the X activity, have the activities helped or not helped to make sure perpetrators are stopped or punished? If yes, how so? If no, why not? What have been the most effective strategies to promote accountability of GBV perpetrators in this community/organization ? If currently not a focus, do you think there are any strategies the X activity should follow to hold perpetrators of GBV accountable in your community/org? Have there been any cases in which the X activity has stopped abuse or punished perpetrators of GBV in your community/org? [IF YES]: In how many instances can you recall? Have successful cases of stopping or punishing GBV perpetrators had any impact your community/org? (Impact scale) If yes, what? If no, why not? [If activity works on accountability]: * To your knowledge which activities has X activity put in place to raise accountability of GBV perpetrators (list of activities) Have you or your organization been responsible for keeping GBV perpetrators accountable? [IF YES]: To what extent was the structure of the X activity conducive to achieving your goals? (scale) To what extent has your organization been able to increase the accountability of perpetrators in the community? (scale) Indicate how many cases of GBV perpetrators has the X activity addressed: USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 32 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* 2. To what extent are the USG objectives being achieved across the 4 ACs? Coordination: How are the GBV prevention and response efforts being coordinated and managed at the Agency, Activity Cluster and Activity levels? Could you please tell me about the management structure for the X cluster? How does information flow from grantees up to USAID/GenDev? How does this information influence decision-making? Please, provide an example (probe past re-design, future programming, etc.). What is your assessment of the co-creation processes undertaken between USAID and IPs? Are there any lessons learned from these experiences? Have you or your organization participated in meetings with other IPs via USAID's USAID/GenDev? If so, what was the purpose of these meetings? Please, provide examples if the meetings served multiple purposes. What information, ideas or other benefits did you obtain from this experience(s)? Would you recommend more exchanges between IPs and USAID? Why? Or why not? What other activities could USAID implement to benefit IPs from collaboration? n/a n/a USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 33 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Integration: How are GBV prevention and response efforts being integrated into current and future USAID/GenDev work and informing related programs? Only USAID: Have you coordinated initiatives from this cluster with any other GBV efforts from USAID? If so, what types of collaboration took place and how well did they work? What did not work so well? What benefits or learning emerged from that collaboration? How does this activity cluster fit into the full range of USAID's GBV programs? What gendered aspects or diverse gender populations are being taken into account in GBV programing? What aspects are being prioritized? Are other intersections being considered? What are the primary features of X cluster that might differ or complement the other GBV activity clusters? Please identify two or three. n/a n/a n/a USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 34 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Data. How is USAID/GenDev ’s GBV portfolio collecting, analyzing, and using data and research to enhance prevention and response efforts? Only USAID: How does USAID/GenDev use data collected from the X cluster? Are lessons and information from the X cluster ever used to inform higher level decision making across USAID/GenDev programs? Could you provide some examples of this? n/a n/a n/a Expansion: How is USAID/GenDev ’s GBV portfolio helping to expand and/or improve GBV programming? <--Sub-EQ itself (only for USAID) How have lessons from the X activity cluster affected other GBV programming? N/a n/a n/a 3. What lessons are being learned and to what extent is there sharing of Foundations: Are lessons regarding foundations of GBV being shared with AC implementing partners? Have grantees been briefed on USAID strategies and priorities around GBV? Or the lessons learned from the cluster? Have you been briefed on USAID strategies and priorities around GBV? n/a n/a USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 35 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* best practices, lessons, and information across the 4 Activity Clusters? Populations: What types of populations are being engaged in the AC? Which vulnerable and underserved populations have been included? Could you tell me about the most vulnerable or underserved populations who have participated in X cluster programming? Who do you think these activities might have missed? Why? Are there certain populations that you would recommend be considered for future programming? Could you please tell me about the populations served by your activity? Are there any underserved or especially vulnerable groups that your activity has reached? If so, what are the approaches or strategies the X activity has implemented to address the needs of these groups? How effective have these strategies been? Which vulnerable groups susceptible to GBV has the activity not been able to reach? Why? n/a Please, indicate the groups with which your organization works (list of groups) *: [FOR EACH GROUP]: To what extent has your organization used a tailored approach in delivering its services to each group? (scale) To what extent has your organization had to invest additional resources to satisfy this group of survivors’ needs? (scale) Please, indicate the additional resources that were invested (list + other) In what ways did the program address this group of survivors’ needs? (List) USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 36 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Stakeholders: Which stakeholders are being engaged to achieve results? Was there a strategy for engaging a broad range of stakeholders for the X cluster? What activities were able to effectively engage relevant stakeholders? Please, provide one or two examples. In contrast, what activities were less successful? Also, please provide one or two examples. What other stakeholders were you able to engage to accomplish the goals of the X activity? What were the contributions of these additional stakeholders? Was their participation valuable? Do you have any lessons learned that you could share about this? During your participation in the X activity, did staff ask you about aspects of the activity that could improve? Do you believe your opinions about the X activity resonated with activity staff? Why? n/a 4. What pervasive gaps still exist in understandi ng GBV and addressing specific types of GBV? Intervention planning and design: What are important knowledge and practice gaps in planning and designing GBV interventions? <----Sub-EQ itself Were there particular gaps related to gender diversity? If yes, please describe. <----Sub-EQ itself n/a n/a USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 37 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Forms of violence: What are important knowledge and practice gaps in addressing specific forms of GBV? <----Sub-EQ itself, adapted to specific forms of GBV related to each cluster <----Sub-EQ itself, adapted to specific forms of GBV related to each activity n/a n/a Reach and effectiveness: How is the GBV portfolio influencing the reach and effectiveness of interventions? How do the 4 ACs together influence the reach and effectiveness of other GBV interventions, even those not funded by USAID? How will you take advantage of the lessons learned or knowledge gained from the 4 ACs for future work? Are there any USAID USAID/GenDev learning or dissemination of lessons that are relevant for other areas? Have the strategies used by the 4 ACs influenced your ability to plan or strategize GBV prevention and protection initiatives? n/a n/a ACTIVITY CLUSTER QUESTIONS USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 38 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* 1. Are the activity clusters based on context￾specific and international evidence? Needs assessment and intervention evidence: How were needs assessments conducted and intervention evidence collected to inform the cluster activities? Were grantees in the activity cluster required to conduct pre￾implementation assessments? What assessments were conducted, and could you comment on their usefulness and robustness? What pre-implementation assessments did you do for this activity? Were they useful? How did those help your team plan and implement the activity? Would you recommend any other kinds of research before implementing similar activities? Do you think activity X responded to the needs and priorities of your org/community? Why? What components do you remember most? Which were most useful to you? How were they useful? Are there any activities that you wish had been included in the activity? How well do the components of Activity X respond to the needs of your community? (list of components with Likert scale) USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 39 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Assumptions: What were the main assumptions among USAID/GenDev and its stakeholders to design, commission, and implement the activity clusters? How accurate were these assumptions? <--Sub-EQ itself (Probe to ask questions about the following types of assumptions: Could you comment on the elements of the theory of change for the X cluster and specifically discuss the underlying reasoning and assumptions? I will pull it up so that we can all see it, and let us please walk through it together and comment on how it worked in practice. -Theory of Change (if applicable) -Capacity of the NGO to deliver services -Availability of partners/network -Uptake of services and/or resistance to it [ONLY IF NO TOC AVAILABLE]: Could you please tell me about your theory of change? When formulating the TOC of the X activity, what were the main assumptions about: - the ability of the org to deliver the activities; of the potential participant s to access the services; of how the services would result in the desired outcomes? How relevant were the assumptions? Do you feel that the activity was responsive to the cultural context of GBV in your community/org? Please explain. Do you have any recommendations how the activity might have reached people in your community/org better? What were the main challenges in its context to deliver the intervention or reach participants? To what extent did the X activity fit well in your community/org? (scale) Is the ability to prevent GBV in conflict with any cultural patterns in your community/org? [IF POSITIVE]: What aspects are most in conflict (open￾ended) To what extent is the accountability of perpetrators of GBV valued in your community/org? (scale) To what extent was the X activity tailored to its context of implementation? * (scale) What were the main challenges in implementing X activity given its context?* (list of challenges) USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 40 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Causal pathways: What causal pathways or theories of change were articulated for the activity clusters? Not applicable to WEE: According to the document review, the theory of change or overarching causal pathways for the X activity cluster is [READ IF EXISTENT]. Was this TOC comprehensive of the cluster? What have been its caveats, if any? Not applicable to WEE: According to the document review, the theory of change or overarching causal pathways for the X activity cluster is [READ IF EXISTENT]. Was this TOC inclusive of the X activity that your organization implemented? What have been any caveats? Once you started implementation did you need to adjust your TOC? If yes, what adjustments were necessary? n/a n/a USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 41 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Monitoring and adaptations: How well are interventions monitored and emerging findings contributing to intervention adaptations or improvements? How have activities in the cluster monitored their implementation? Have cluster activities been able to adapt their implementation based on monitoring findings? Please, provide an example. What could be improved about activity monitoring? What was your overall strategy for activity monitoring and adapting the activities based on emerging findings? Could you share some examples of how you might have changed any aspects of activities based on emerging evidence? How accessible and user friendly were the monitoring tools/templates provided to the activity? Are there any lessons learned from your experience with the monitoring tools? During your interaction with the X activity staff, were there any changes in the activity? [IF YES:] Please, give me an example. [IF YES:] Why do you think these changes were implemented? Were there changes in the context of your community/org that led to these adjustments? [IF NOT:] Do you think there were adjustments that were necessary to the activity but were not implemented? Please, explain. *Ask all questions only to service providers Are you familiar with the monitoring system implemented by the X activity?* To what extent do you consider the system easy to understand? (Likert scale)* To what extent do you consider the system accessible? (Likert scale)* To what extent do you consider the system practical in terms of entering the data? (Likert scale)* To what extent do you consider the system effective to make decisions on the activity? (Likert scale)* How often did activity management use monitoring indicator data to adapt the activity? (scale)* Please, provide an example of such adaptations (open ended)* USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 42 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* 2. To what extent are each of the activity clusters achieving the targeted GBV results? Outcomes: Are the stated outcomes realistic and achievable within the timeframe of the AC? What progress is being made towards achieving the outcomes? Were the outcomes stated in activity designs for this activity cluster realistic? Was the length of the grants sufficient to achieve these outcomes? Why? Which activities were most successful in achieving the outcomes in their activity design, and which were less successful? Can you identify facilitating factors and challenges either way? Were there any unanticipated positive or negative effects from this activity? What were the primary outcomes of the X activity? Were the outcomes stated in activity design realistic and achievable? Why? Was the length of the grant sufficient to achieve these outcomes? Please describe one or two of the main activity outcomes so far. Can you describe any facilitating factors? Are there outcomes that you would have liked to see but were not feasible to accomplish? Why? What’s your understanding of the X activity goals? To what extent did the activity accomplish x results? Do you think that the activity objectives matched the actual results? Did the X activity do what it was supposed to do? Please indicate, for the following results, to what extent the X activity accomplished each of them in your community/org (4-point scale for each result, including Don’t Know) Please indicate, for the following results, to what extent the X activity accomplished each of them (4- point scale for each result, including Don’t Know)* USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 43 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Planning and activity designs: How and how well were activity plans and designs developed to achieve different GBV outcomes? What did the design process of the activity cluster look like? Who came up with the idea? Who contributed to it? Who made the decisions and how were these decisions adopted? Please, share elements of the activity design that you think were effective. What activity designs faced challenges? Why? Please think of any stage between the process of co-creation, adoption, and implementation. How did the co-creation process on activity design work for your organization? Could you identify advantages and down sides of this approach? If you were designing the activity again, is there anything that you would do differently? What are the main lessons learned from designing the X activity? n/a n/a USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 44 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Intervention implementati on: How well are interventions implemented to reach their target groups and influence change? Do you think that the activities in this cluster were able to reach the participants that they were designed to reach? Did reaching the target groups help influence change around GBV in each context? Were there any challenges in reaching the right people to influence change? Was your activity able to reach the participants that it was designed to reach? How did you identify changemakers in the org/community for X activity? Were there any challenges in reaching the right people to influence change? Who else should be engaged and was not in the X activity? Who benefited most from this activity? Did certain groups benefit more than others? Are there any groups of people that you think should have been included, but were not? Are there people in your organization/community who you think should have been involved in X activity but were not? In your organization/community, which groups of people are important for influencing change? Were these results even for all target groups or were there differences in the access across groups?* [IF DIFFERENCES:] Please, select what groups presented a deficit in their participation (open-ended)* Mechanisms: What are the most effective aspects of the interventions? How do these ‘active ingredients’ operate in each AC? What were the most effective intervention components or approaches used by the grantees? Please, identify two or three. What were the concrete outcomes from these components or approaches? What do you think were the most effective components of your activity? Please, identify one or two. Why? What were the best and most helpful parts of this activity? Which components had the most positive results or impact? From the following list, please identify the components of X activity that led to the most positive impact in your org/community? (list of activities + Other and text box) USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 45 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* 3. To what extent are the ACs sustainable? Sustainability: What aspects of the ACs contributed to their sustainability? What components are needed for greater sustainability? What components or approaches in the X cluster have remained active even after funding ended? Please, identify two or three. Why did that happen? What components or approaches in the X cluster ended? Why? (PROBE: funding, political willingness, cultural competency, etc.) What strategies were absent or could have been improved to increase sustainability for those activities or approaches that ended? Are there any lessons learned for the activity cluster about sustainability in the current context and existing stakeholders? What components or approaches in the X cluster have remained active even after the USAID funding ended? Why did that happen? What components or activities in the X cluster ended? Why? (PROBE: funding, political willingness, cultural competency, etc.) What strategies could have enhanced sustainability of those components that ended? What have been the primary facilitators and barriers to the sustainability of the X activity? Do you think that positive results from the X activity will continue in the future? Please, identify one or two, if possible. Which activities are more likely to continue? Why? Is this desirable for your community/org? Which activities will be less likely to continue? Why? Would the community have received benefits if these activities continued? What do you think your organization/community needs to do to make sure these activities continue? (open-ended response) Identify which components you would like to see continue in your organization/community after this activity has ended. (list of activities) What do you think your organization/community needs to do to make sure these activities continue? (open￾ended response) Do you think the activities will continue? USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 46 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Replicability, transferability , and adaptability: In what ways are the ACs replicable in the same contexts? Adaptable for other contexts? Are there any interventions or approaches in the X cluster that you think could be replicated in or adapted to other contexts? Has the experience with the X cluster informed the potential for scale up or replicability elsewhere? Please explain. Are there any that you think could not be replicated or adapted? Why? What components or approaches do you think could be replicated in other communities or countries? Where else would you recommend implementing this activity? What approaches, components, or tools of the X activity would need to be adapted for a different context? n/a n/a Scalability: What aspects of the ACs are most amenable to be scaled up? Which interventions in the X cluster do you see as having potential for scaling up? Are there any that you think would be very costly or difficult to scale up? Why? Conversely, are there interventions that you would cut? If yes, why? If you were to scale up your activity, which components of your intervention would you focus on? Are there any that you would drop? What changes would you make? What are the main challenges for scaling the activity up in your country/region? what aspects of this activity that could be useful to women in other communities or regions? Which ones? Could they be expanded geographically? Please explain. Are there any that you think would not work well in other places? Why? Or that could not be scaled up? Why? Which activities of X activity do you think would be useful in other organizations/communities in your country? (list of activities)* IMPLEMENTATION EVALUATION QUESTIONS USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 47 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* 1. Is the activity design based on the local context and flexible to achieve results on the ground? Design: What factors contributed to the design of the activity? How were priority GBV problems identified? What was the process of activity design? What factors influenced the design? Who led the design process? What GBV issues were identified and prioritized? Could you speak to how this activity was designed? What factors influenced this design? What priorities were established? Who was involved in these design decisions? Would you change anything about the design? In your opinion, what problems related to GBV should be prioritized in your community? Do you think the activity prioritized the most salient problems? Was the approach– resources or programming–adequate to address the problem? If you were designing the activity, what would you have done differently? What activities would best address these problems? What resources or programs does your community need most to prevent GBV? (select from list) To what extent were these provided by the X activity? (5- point scale) USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 48 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Implementati on: What are the key intervention methods to achieve objectives? Is there a theory of change that links the intervention approaches and tools to expected outcomes? In your opinion which approaches/tools were most effective in achieving the objective of this activity? Were there specific challenges or enabling factors in implementing this activity? Please explain. Were the challenges overcome? And how? In your opinion which approaches/tools were most effective in achieving the objective of this activity? Were there specific challenges or enabling factors in implementing this activity? Please explain. Were the challenges overcome? And how? How did the X activity address GBV issues? What changes resulted in your community? What type of interventions do you feel the activity did very well? Which could be improved? n/a USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 49 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* Flexibility: Is there sufficient staffing to respond to local priorities? Is there flexibility to change approaches to respond to lessons and changing challenges in the local environment? <--Sub-EQ itself <--Sub-EQ itself Do you think that the X activity had enough staff to support the needs of your organization/community ? Was the support responsive to the needs of your organization/community ? Do you think the activity was able to adapt when programming didn’t work or wasn’t in line with what the community needed? Could you give some examples? To what extent do you agree with this statement: The X activity staff adapted to the context of your organization/community (Likert scale) USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 50 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* 2. Is the activity reaching participants they are meant to target? Target participants: What are the barriers to reaching participants? Who were the target participants for this activity? How were target participants for X activity identified? Whose input was part of the identification process? What, if any, challenges were there in reaching targeted participants? What could mitigate these challenges in the future? Who were the main target individuals of this activity? Did you have any difficulty reaching them? If so, what were the difficulties? Are there particular sub￾groups of target individuals that are difficult to reach? How do you think these barriers could be mitigated in the future? Did you have any concerns about participating in the activity? Why? Did you have any difficulty participating in the X activity? If so, what conditions would make participating easier? Do you know of anyone that wanted to participate but couldn’t? If so, why not? Did you experience any difficulties in participating in X activity? (from list) Monitoring of results: Is the activity collecting evidence on what is working, not working and what could be done differently to achieve results? <--Sub-EQ itself Do you have a process in place to use performance data for adaptive management? Please describe. <--Sub-EQ itself Do you know how to give feedback to the implementers of X activity? If you had feedback to share about X activity, do you think staff would listen? Do you know if anything about X activity has changed because of you or someone else giving feedback? If so, could you give an example. To what extent do you agree with these statements: o If I wanted to give feedback on X activity, I know who to talk to o If I wanted to give feedback on X activity, I know what to do o My feedback would be considered seriously. USAID.GOV GENDER-BASED VIOLENCE PORTFOLIO PERFORMANCE EVALUATION: EVALUATION DESIGN REPORT | 51 Evaluation Questions Evaluation Sub￾Questions Questions for Prime or USAID (Semi￾Structured Interview) Questions for Implementing Partner Staff (Semi-Structured Interview) General Focus Group Discussion or One￾on-One Interview Questions to Participants General Web-Survey Questions for Participants or Service Providers* 3.Is the activity achieving sustainability ? Sustainability: What plans are in place for sustainability? What is the evidence of potential sustainability? Are there any plans to ensure the sustainability of this activity? Has any evidence been gathered to support the potential sustainability of this activity? Do you think that this activity is sustainable moving forward? What practices or activities have taken place to support the sustainability of X activity? See Activity Cluster items See Activity Cluster items U.S. AGENCY FOR INTERNATIONAL DEVELOPMENT 1300 Pennsylvania Avenue, NW Washington, DC 20523