This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. GREATER INTERNET FREEDOM PERFORMANCE EVALUATION July 2023 Prepared under Contract No. GS-10F-0033M/7200AA18M00016, Tasking N087 Pexel DRG LEARNING, EVALUATION, AND RESEARCH ACTIVITY II GREATER INTERNET FREEDOM PERFORMANCE EVALUATION AUGUST 2023 Prepared under Contract No. GS-10F-0033M/7200AA18M00016, Tasking N087 Submitted to: Matthew Baker, USAID COR Submitted by: Anna Solovyeva, Team Lead Catherine Coyer, Subject-Matter Expert Jessica Wallach, Senior Research Associate Samantha Austin, Research Associate Contractor: NORC at the University of Chicago 4350 East West Highway, 8th Floor Bethesda, MD 20814 Attention: Matthew Parry Tel: 301- 634-5444; E-mail: Parry-Matthew@norc.org DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | i ABSTRACT This performance evaluation assesses selected thematic areas of the Greater Internet Freedom (GIF) project, a global activity designed to enhance digital security for civil society and media and increase citizen engagement in governing the Internet. The evaluation uses a mixed-methods approach including desk review, key informant interviews (KIIs), and a web-survey with implementing partners to provide evidence on GIF’s performance in areas of: (1) network development; (2) localization; (3) advocacy with governments and multilateral institutions; and (4) advocacy with the private sector. Based on emerging findings, the evaluation team suggests several practical recommendations for each thematic area. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | ii CONTENTS ABSTRACT..................................................................................................................................... I EXECUTIVE SUMMARY............................................................................................................. 1 1 EVALUATION PURPOSE AND QUESTIONS..............................................................5 1.1 BACKGROUND TO EVALUATION................................................................................................. 5 1.2 EVALUATION QUESTIONS................................................................................................................ 5 2 PROGRAM OVERVIEW: GIF OBJECTIVES AND ACTIVITIES ................................6 2.1 GIF OBJECTIVES...................................................................................................................................... 6 2.2 GIF PARTNERS ........................................................................................................................................ 7 2.3 GIF PRINCIPLES AND ACTIVITIES.................................................................................................... 9 3 METHODOLOGY.............................................................................................................10 3.1 QUALITATIVE DATA COLLECTION ............................................................................................ 11 3.1.1 Desk Review .................................................................................................................................................................... 11 3.1.2 Key Informant Interviews............................................................................................................................................. 11 3.2 QUANTITATIVE DATA COLLECTION........................................................................................ 12 3.3 DATA ANALYSIS .................................................................................................................................. 13 3.4 LIMITATIONS ........................................................................................................................................ 13 4 FINDINGS ...........................................................................................................................15 4.1 NETWORK DEVELOPMENT............................................................................................................ 15 4.1.1 New and Strengthened Networks............................................................................................................................. 15 4.1.2 Extent of Network Use ................................................................................................................................................ 18 4.1.3 Examples of Beneficial Networks............................................................................................................................... 20 4.1.4 Opportunities for Network Development ............................................................................................................. 21 4.2 LOCALIZATION................................................................................................................................... 24 4.2.1 Approaches for Localization........................................................................................................................................ 24 4.2.2 Capacity Strengthening for RPs and LPs.................................................................................................................. 26 4.2.3 LPs Determining Priorities and Designing Activities ............................................................................................ 27 4.2.4 Opportunities for Improving Local Engagement.................................................................................................... 30 4.3 ADVOCACY WITH GOVERNMENT AND MULTILATERAL INSTITUTIONS..................33 4.3.1 Approaches for Advocacy with Government and Multilateral Institutions.................................................... 33 4.3.2 Effectiveness of Approaches........................................................................................................................................ 37 4.3.3 Influence on Government and Multilateral Institutions....................................................................................... 39 4.3.4 Opportunities for Advocacy with Government and Multilateral Institutions............................................... 41 4.4 ADVOCACY WITH PRIVATE SECTOR ........................................................................................ 42 4.4.1 Approaches for Advocacy with Private Sector...................................................................................................... 42 4.4.2 Effectiveness of Approaches........................................................................................................................................ 44 4.4.3 Influence on Private Sector.......................................................................................................................................... 46 4.4.4 Opportunities for Advocacy with Private Sector.................................................................................................. 47 4.5 CROSS-CUTTING THEMES............................................................................................................... 48 4.5.1 Digital Security ................................................................................................................................................................ 49 4.5.2 Sustainability and Scaling............................................................................................................................................... 50 4.5.3 Impact of COVID19....................................................................................................................................................... 51 4.5.4 Definitions of terms....................................................................................................................................................... 51 5 RECOMMENDATIONS ...................................................................................................52 5.1 NETWORK DEVELOPMENT............................................................................................................ 52 5.2 LOCALIZATION................................................................................................................................... 53 5.3 ADVOCACY WITH GOVERNMENT AND MULTILATERAL INSTITUTIONS..................53 5.4 ADVOCACY WITH PRIVATE SECTOR ........................................................................................ 53 USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | iii 5.5 CROSS-CUTTING RECOMMENDATIONS FOR FUTURE USAID PROGRAMMING.....54 ANNEX A: EVALUATION STATEMENT OF WORK .....................................................56 ANNEX B: INTERVIEW PARTICIPATION .........................................................................58 ANNEX C: KEY INFORMANT INTERVIEW PROTOCOLS ..........................................60 ANNEX D: SURVEY SAMPLE CHARACTERISTICS .........................................................77 ANNEX E: SURVEY INSTRUMENT......................................................................................79 USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | iv TABLE OF EXHIBITS Exhibit 1 GIF Theory of Change ........................................................................................................................7 Exhibit 2 GIF Regional Partners and Participating Countries......................................................................8 Exhibit 3 GIF Implementation Principles..........................................................................................................9 Exhibit 4 GIF Activities.........................................................................................................................................9 Exhibit 5 Evaluation Timeline (2023) ............................................................................................................. 11 Exhibit 6 Phase I KII Respondents.................................................................................................................. 11 Exhibit 7 Disposition Summary ....................................................................................................................... 13 Exhibit 8 Participation in GIF-Supported Networks.................................................................................. 15 Exhibit 9 Participation in Specific Types of Networking Events and Activities (% of total sample)................................................................................................................................................. 16 Exhibit 10 Networks with USAID Missions or Other USAID-Funded Activities Developed due to GIF........................................................................................................................................... 17 Exhibit 11 Usefulness of Participation in GIF-supported Networks......................................................... 19 Exhibit 12 Beneficial Outcomes of Participation in GIF-supported Networks (% of total sample)................................................................................................................................................. 19 Exhibit 13 Challenges of Participation in GIF-supported Networks (% of total sample).................... 22 Exhibit 14 Activities and Resources Local Partners Benefited from in the Area of Digital Safety and Security (left) and Digital Rights (right) (% of total sample) ............................... 25 Exhibit 15 Engagement in GIF Activity Development and Implementation ............................................ 28 Exhibit 16 Preferred Engagement in GIF Activity Development and Implementation......................... 28 Exhibit 17 Availability of Resources for Undertaking GIF Activities ........................................................ 31 Exhibit 18 Advocacy Involvement with Government and Multilateral Organizations and Multistakeholder Forums Prior to GIF ........................................................................................ 33 Exhibit 19 Organizations Receiving GIF Support for Advocacy with Government and/or Multilateral Institutions .................................................................................................................... 34 Exhibit 20 GIF-supported Advocacy Engagement with Government and Multilateral Organizations and Multistakeholder Forums (% of total sample) ......................................... 35 Exhibit 21 Improved Engagement with Government and Multilateral Organizations and Multistakeholder Forums due to GIF Support (% of total sample)....................................... 37 Exhibit 22 Advocacy with Private Sector on Issues of Technology and Human Rights: Involvement prior to GIF and Receiving GIF Support for Such Advocacy.......................... 42 Exhibit 23 GIF-supported Advocacy Engagement with Private Sector (% of total sample)................ 43 Exhibit 24 Improved Engagement with Private Sector due to GIF Support (% of total sample)....... 45 Exhibit 25 Changes in Private Companies’ Practices due to Advocacy Supported by GIF (% of total sample)....................................................................................................................................... 46 Exhibit 26 Likelihood of Continuing Work on Digital Rights and Security post GIF ........................... 50 USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | v ACRONYMS A19 ARTICLE 19 AI Artificial Intelligence BIRN Balkan Investigative Reporting Network CAR Central African Republic CIPESA Collaboration on International ICT Policy in East and Southern Africa CIPI Civil Internet Policy Initiative Organization COP Chief of Party COR Contracting Officer’s Representative CSO Civil Society Organization DRAPAC Digital Rights Asia Pacific Assembly DRC Democratic Republic of the Congo DRG Democracy, Rights, and Governance DRL Bureau of Democracy, Human Rights and Labor DSLU Digital Security Lab Ukraine EE Europe and Eurasia EQ Evaluation Question ESA East and Southern Africa ET Evaluation Team EU European Union GIF Greater Internet Freedom (USAID Activity) GNI Global Network Initiative ICANN Internet Corporation for Assigned Names and Numbers ICNL International Center for Non-Profit Law IEEE Institute for Electrical and Electronics Engineers IETF Internet Engineering Task Force IF Internet Freedom IGF Internet Governance Forum INGO International Non-Governmental Organization USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | vi IP Implementing Partner ITU International Telecommunication Union KII Key Informant Interview LAC Latin America and the Caribbean LER Learning, Evaluation, and Research LGBTQI+ Lesbian, Gay, Bisexual, Transgender, Queer, Intersex LP Local (Implementing) Partner MEL Monitoring, Evaluation, And Learning MENA The Middle East and North Africa NGO Non-Governmental Organization NORC National Opinion Research Center (NORC at the University of Chicago) OTF Open Technology Fund PE Performance Evaluation PIN Paradigm Initiative RDR Ranking Digital Rights RP Regional (Implementing) Partner SAFETAG Security Auditing Framework and Evaluation Template for Advocacy Groups SEA South and Southeast Asia SIDA Swedish International Development Cooperation Agency SOW Statement of Work TOC Theory of Change UN United Nations USAID United States Agency for International Development WCA West and Central Africa USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 1 EXECUTIVE SUMMARY The worldwide expansion of internet access is accompanied by a worrisome decline in internet freedom (IF), posing a threat to democratic values and institutions. To preserve an open, interoperable, reliable, and secure internet and advance human rights in digital contexts, the United States Agency for International Development (USAID) funded the Greater Internet Freedom Project (GIF), a 3-year, $16.2 million global activity started in August 2020. This global activity is designed to enhance digital security for civil society and media and increase citizen engagement in governing the internet. An Internews-led consortium of over 100 international, regional, and local organizations from over 40 countries implements GIF. EVALUATION AREAS As part of the Democracy, Human Rights, and Governance Learning, Evaluation, and Research (DRG-LER) II Activity, USAID requested NORC to conduct a Performance Evaluation (PE) of the GIF project to identify lessons learned, best practices, and limitations, while focusing on four thematic areas: network development; localization; advocacy with governments and multilateral institutions; and advocacy with the private sector. EVALUATION METHODOLOGY This evaluation employed a mixed methodology of qualitative and quantitative approaches. The qualitative component included a comprehensive desk review and two phases of key informant interviews (KIIs). Phase1 KIIs encompassed 21 individual and group interviews with 29 informants from USAID, Internews, and global and regional implementing partners, while Phase II included 14 individual and group interviews with 18 participants from a sample of local partners (LPs). The quantitative component was based on a web-survey of 48 LPs. The team analyzed qualitative data using a deductive coding scheme in Dedoose software, while for statistical analysis of the web-survey data, the evaluation team (ET) used STATA software. Key limitations of this evaluation are: the focused scope of the evaluation that does not cover the entirety of GIF and its impacts and since the activity is ongoing year 3 is not fully captured; the large scope and complexity of GIF making it infeasible to cover every single activity implemented; indirect beneficiaries not included in the evaluation due to limited timeline; technical issues on respondents’ side when organizing and conducting online KIIs; respondents’ social desirability bias; conflation of GIF-supported activities with activities supported by other donors during KIIs; and survey selection bias within LPs’ organizations. KEY FINDINGS NETWORK DEVELOPMENT Network development is a crucial aspect of GIF, aiming to foster connections and collaboration between local, regional, and global civil society actors working on digital rights, digital security, and internet freedom. GIF supported the creation of new regional networks emerging from connections between organizations in different countries, whereby LPs within the same region formed relationships with each other. This was especially useful in regions with weaker pre-existing networks, such as Central Asia and Central Africa. GIF enhanced existing networks, allowing regional partners (RPs) to expand their footprints and engage with diverse LPs where regional networks already existed, for example in Latin America. GIF also helped establish links between some LPs and USAID missions and other USAID-sponsored activities in countries. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 2 Most respondents found participation in GIF-supported networks to be very useful, as it offered opportunities for promotion, learning, and networking, with concrete examples of collaborative projects and support emerging from these networks. Connections between organizations across countries as part of regional initiatives provided concrete benefits, enhancing the legitimacy and impact of networking events. GIF-supported connections between local and global partners, such as those supported by Article 19 fellowships, were beneficial, enabling local perspectives to be brought into global initiatives. Opportunities for improvement of GIF-supported network development include more inclusive collaboration, more opportunities for LPs to participate in global convenings, more in-person (rather than online) networking events, and the development of clear and flexible communication channels to enhance networking capacities. LOCALIZATION GIF employed various approaches to provide RPs and LPs agency over the activities they implement in their contexts, while ensuring that their needs and priorities are addressed within the parameters set by Internews and global partners. The capacity strengthening efforts of GIF contributed to enhancing the knowledge and expertise of local and regional organizations. LPs benefited from technical advice, trainings, and connections with other organizations, which allowed them to become leading digital rights organizations in their respective countries and enabled them to engage with stakeholders effectively. RPs played an essential role in mediating between all levels of GIF and acted as central connectors for LPs. RPs also underwent capacity strengthening, connecting with global partners and multilateral institutions to gain exposure to international issues while learning more about localized perspectives. GIF’s local focus has provided LPs with some flexibility in designing activities, but there are still concerns about the standardization of activities and the level of autonomy for LPs in decision￾making. The decision-making process consisted of Internews providing a list of activities to RPs, who then worked with LPs to select activities, what these activities would focus on, and who would participate along with LPs. Opportunities for improving GIF’s localized approach include further integrating LPs in decision￾making processes while accounting for differences in LPs’ level of expertise that can be addressed by targeted training; and solving some operational gaps by extending implementation periods for LPs and providing more resources for translations and interpretations. ADVOCACY WITH GOVERNMENTS AND MULTILATERAL INSTITUTIONS GIF supported partners to engage with governments and multilateral institutions by helping them conduct legal analyses through work with ICNL and RDR, supporting their participation in international forums through the Article19 fellowship program, organizing round tables and working meetings with policymakers, as well as providing organizational and technical capacity support to LPs. The effectiveness of approaches to increase dialogue between civil society, governments, and multilaterals was dependent on the country context. In places with a restricted IF landscape, assistance from the GIF network was especially important, as LPs preferred to do advocacy through global partners. Generally, respondents believed meetings and roundtable discussions with governments, engagement with the UN Special Rapporteur, ICNL-led legal analysis, and participation in global forums were effective approaches to support advocacy work. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 3 The evaluation identified several examples of GIF-supported advocacy efforts resulting in positive changes in government policies and practices. Typically, it was achieved through legal analyses of problematic legislative initiatives followed by public campaigns and meetings with legislators, which resulted in problematic legislation not being passed. One of the main barriers to LPs’ successful public advocacy is current political systems in their countries, ranging from the lack of political will to state-level harassment. Other challenges include LPs’ limited technical knowledge and human resources for advocacy work, and GIF’s lengthy administrative processes and limiting timelines. ADVOCACY WITH PRIVATE SECTOR GIF with support from global partners, such as GNI and RDR, connected RPs and LPs with social media and telecommunications companies by engaging them in meetings, workshops, roundtable discussions, and RDR research. GNI hosted trainings and webinars for digital rights activists on corporate accountability and protection of digital rights. RDR led workshops for RPs on engaging with the private sector to advance corporate accountability through research on the RDR Index Methodology. RDR also helped disseminate findings and promote partners’ projects. GIF support for advocacy with the private sector has been especially useful in cases where LPs did not have prior opportunities for this advocacy. The success of the advocacy appeared to vary by region and was dependent on how committed and effective the RPs, LPs, and the private sector representatives were. While some partners thought it was too soon to see the impact of the GIF￾supported advocacy on private companies’ policies and practices, there have been several examples of companies in different regions acting in response to advocacy. A key barrier that prevented local organizations from advocating with the private sector in nearly all regions was the lack of responsiveness or interest on the part of companies to engage with civil society. Also, the lack of specialized skills for advocating with companies limited RPs’ and LPs’ engagement with the private sector. This provides an opportunity for INGOs to facilitate direct access of RPs and LPs to global tech companies and provide more advocacy training when needed. CROSS-CUTTING THEMES The evaluation touched on several additional themes. In the area of digital security, which was one of the two GIF objectives, partners reported receiving GIF support, such as training, assistance in developing digital security policies for NGOs, publishing practical materials on cybersecurity, and conducting digital risk assessment through SAFETAG audits. LPs expressed interest in receiving more technical support, training, access to experts, funding for risk assessments, and access to software to support digital safety. Regarding GIF’s sustainability and scaling, several aspects of GIF are likely to be sustained after the program is over. These are: the research and knowledge products developed with GIF support, including the RDR research adaptations and GIF website; GIF’s capacity strengthening that will continue to benefit partner organizations; and the networks and connections that were created under GIF that will lead to future partnerships outside of GIF. The report also highlighted the varied impact of COVID-19 on project activities, with some hindered but most adapted to remote work and digital platforms, though moving forward, in-person networking is the preferred mode. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 4 KEY RECOMMENDATIONS The ET suggested the following practical recommendations emerging from the evaluation findings. Network Development 1. Strengthen communication and establish persistent channels for internal communications based on partner preference. 2. Continue prioritizing in-person meetings. 3. Prioritize fostering institutional knowledge by taking on board organizations as fellowship members and encouraging multiple individuals within an organization to engage in fellowship activities. Localization 4. Address differing levels of LP’s expertise through targeted trainings and capacity building. 5. Allocate more funding and resources for translation. Advocacy with Government and Multilateral Institutions 6. Establish advocacy leadership to streamline workflows and enhance coordination. 7. Increase support for LPs to attend multistakeholder conferences and forums where policymakers and the private sector are present. Advocacy with Private Sector 8. Scale up work to cultivate relationships with the private sector by fostering direct communication between partners and private companies. 9. Continue encouraging regional and local partner use of RDR research but integrate a more standard review process and adaptation for different local contexts. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 5 1 EVALUATION PURPOSE AND QUESTIONS 1.1 BACKGROUND TO EVALUATION The worldwide expansion of internet access is accompanied by a worrisome decline in internet freedom (IF), posing a threat to democratic values and institutions. According to Freedom House, global IF declined for the 12th consecutive year in 2022, as some countries are moving toward digital authoritarianism by embracing extensive censorship and automated surveillance systems. To preserve an open, interoperable, reliable, and secure Internet and advance human rights in digital contexts, the United States Agency for International Development (USAID) funded the Greater Internet Freedom Project (GIF), a 3-year, $16.2 million global activity started in August 2020. The GIF project is designed to enhance digital security for civil society and media and increase citizen engagement in governing the Internet. GIF does this by conducting digital security support services and civil society advocacy activities that advance rights-respecting legislation and policy frameworks in digital spaces. GIF is implemented by Internews along with a consortium of over 100 international, regional, and local organizations from over 40 countries. As part of the Democracy, Human Rights, and Governance Learning, Evaluation, and Research (DRG-LER) II Activity, USAID has requested NORC to conduct a Performance Evaluation (PE) of the GIF project to identify lessons learned, best practices, and limitations. The evaluation findings will be used to inform (i) the design of a follow-on activity; (ii) the work plan for a potential one-year extension of GIF; and (iii) USAID’s collaboration and coordination work with other donors and stakeholders. 1.2 EVALUATION QUESTIONS The initial Statement of Work (SOW) included evaluation questions reflecting the two domains of the GIF project – digital rights and digital security – and two cross-cutting themes – network building and localization. Upon discussions with NORC and Internews, USAID revised the evaluation questions to focus on four areas: network development; localization; advocacy with governments and multilateral institutions; and advocacy with private sector. The final questions are as follows: 1). Network development a) What networks have been established or strengthened through GIF's programmatic approach, particularly between international, regional, and local civil society actors? b) To what extent are GIF networks used to coordinate and share information on digital rights and security? c) Are there concrete examples where network coordination and information sharing have benefited local members? d) Where do gaps exist in network development that future programming should fill (e.g., stakeholders engaged, regions where network development has been less effective, types of information shared, etc.) 2). Localization a) What approaches did GIF take to support (i) local and (ii) regional organizations? b) To what extent did these approaches strengthen capacity on digital security and digital rights for (i) local and (ii) regional organizations? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 6 c) To what extent were priorities determined and activities designed and led by local actors? d) What opportunities/gaps exist to improve local civil society engagement on issues of technology and human rights? 3). Advocacy: Governments & Multilateral Institutions a) What approaches did GIF take to increase and improve civil society’s engagement with governments and multilateral institutions on issues of technology and human rights? b) How effective were those approaches in increasing constructive dialogue between civil society, governments, and multilateral institutions on these topics? c) To what extent did these approaches influence government and multilateral institutions’ actions, policies, and/or practices? d) What opportunities/gaps exist to advance work between civil society and governments and multilateral institutions on issues of technology and human rights based on GIF’s work? 4). Advocacy: Private Sector a) What approaches did GIF take to increase and improve civil society’s engagement with the private sector on issues of technology and human rights? b) How effective were those approaches in increasing constructive dialogue between civil society and the private sector on these topics? c) To what extent did these approaches influence private sector actions, policies, and/or practices? d) What opportunities/gaps exist to advance work between civil society and the private sector on issues of technology and human rights based on GIF’s work? 2 PROGRAM OVERVIEW: GIF OBJECTIVES AND ACTIVITIES 2.1 GIF OBJECTIVES The GIF activity was designed to strengthen the capacity, data awareness, and activism of civil society organizations (CSOs) and independent media to advance IF in countries where USAID works. This goal is supported by the activity’s two primary areas of focus – (1) Digital Security and (II) Digital rights - and respective objectives, as captured in GIF’s Theory of Change (TOC) depicted in Exhibit 1 (GIF MEL Annual Work Plan Year 1). USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 7 Exhibit 1 GIF Theory of Change 2.2 GIF PARTNERS GIF is a consortium-based activity that brings together global, regional, and local organizations1 to drive IF goals in regions and countries where USAID operates (GIF Annual Workplan Year 1; GIF Annual Workplan Year 2; GIF Internews Quarterly Report FY2023 Q2). Internews, an international media support nonprofit, leads the consortium and plays a three-part role in GIF: 1) it is the prime implementor responsible for overall implementation; 2) it serves as an international non-governmental organization (INGO) partner; and 3) it leads implementation and coordination of activities in regions where no regional organization could be identified as a sufficiently strong implementing partner (IP). In Year 3 of the project, Internews also assumed monitoring, evaluation, and learning (MEL) responsibilities. Several INGO partners provide specialized technical assistance to regional and local partners and support knowledge exchange and strategy development across the consortium. Most INGO partners work on Objective 2 activities. The INGO partners are: Article19 leads GIF efforts related to Internet governance by providing fellowships to interested representative of GIF RPs and LPs. In Year 3, Article19’s Team CommUNITY initiative supported trainings on building regional communities and executing a global community strategy for Objective 1. 1 The GIF Consortium also works with resource partners, organizations in the IF community that provide digital resources and infrastructure to vulnerable beneficiaries under Objective 1. However, they are not in the focus of the current evaluation. Local digital security capacity is strengthened for experts, local CSOs, media outlets, and individuals Local CSOs, media outlets and individuals have better access to digital security and digital hygiene strategies O1 Digital Security Enhanced digital security for civil society and media The IF community builds multi￾stakeholder networks that reach non￾traditional partners Strengthened, more integrated IF advocacy efforts improve relevant legal and regulatory environments O2 Digital rights Increased citizen engagement in internet governance CSOs, media outlets, and individuals in countries where USAID works will be able to advance IF issues relevant to their countries USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 8 Ranking Digital Rights (RDR) mentors RPs and LPs on producing research adaptations of the RDR Corporate Accountability Index2, organizes corporate accountability trainings for RPs and LPs, and helps them to localize existing strategies for engaging tech companies to respond to IF priorities. Global Network Initiative (GNI) connects RPs and LPs with big and small tech companies on issues of human rights through its vast membership, for example by involving GIF partners in private and public multi-stakeholder convenings. GNI also mentors RPs and LPs on building local multi￾stakeholder initiatives. In some regions, GNI supports RPs and LPs on engaging in regulatory reforms. International Center for Non-Profit Law (ICNL) works directly with RPs and LPs to identify regional and country-specific IF legal issues that require interventions through legal analysis, advocacy, legal assessments and consultations, and development of country-level monitoring tools. ICNL also provides training for RPs and LPs on legal and regulatory issues impacting IF and advocacy. Regional Partners (RPs). In the GIF activity, RPs lead the majority of work under the two objectives in their respective geographic regions by directly implementing and informing IF work at the regional and local levels, supporting LPs, and ensuring that local contexts are incorporated into global IF dialogues (GIF Annual Workplan Year 3). Exhibit 2 provides an overview of GIF RPs and countries where they work. Local Partners (LPs). In coordination with the GIF team, RPs identified LPs in 38 target countries and integrated them into the GIF Consortium by connecting them to other partners and opportunities within the activity (GIF Annual Workplan Year 3). Given sensitive contexts in some GIF countries, USAID and Internews decided to redact and replace those country names by codes in all their reporting. Exhibit 2 GIF Regional Partners and Participating Countries Region Regional Partner Organization Countries with Local Partners Latin America and the Caribbean Derechos Digitales Bolivia Brazil Colombia Ecuador LAC_Sensitive-1 LAC_Sensitive-2 Europe and Eurasia Digital Security Lab Ukraine (DSLU) Armenia EE_Sensitive-1 EE_Sensitive-2 Georgia Ukraine The Balkans and Moldova Balkan Investigative Reporting Network (BIRN) Albania Bosnia & Herzegovina Kosovo Moldova3 North Macedonia Serbia Central Asia Internews (O1 and O2) Public Fund Civil Internet Policy Initiative (CIPI)4 (O2) CA_Sensitive-1 Kazakhstan Kyrgyzstan Tajikistan 2 The RDR Index is a standard-setting tool that ranks the policies of largest global internet, mobile ecosystem, and telecommunications companies. The RDR Index encourages companies to abide by international human rights standards regarding freedom of expression and privacy. (https://rankingdigitalrights.org/) 3 Moldova is split between RPs. Digital Security activities are supported by DSLU and digital rights activities are supposed by BIRN. 4 CIPI, previously an LP in Tajikistan, became an RP for Objective 2 in Central Asia in Year 3. The Internews Regional Coordinator will continue to support CIPI to ensure that they are successful in this expanded role. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 9 Region Regional Partner Organization Countries with Local Partners South and Southeast Asia EngageMedia Bangladesh Cambodia Indonesia Maldives Nepal Sri Lanka The Philippines West and Central Africa Co-Creation Hub (O1) Paradigm Initiative (O2) Angola CAR DRC East and Southern Africa DefendDefenders (O1) Collaboration on International ICT Policy in East and Southern Africa (CIPESA) (O2) Lesotho Mozambique Tanzania Uganda Zambia Zimbabwe The Middle East and North Africa Internews Iraq 2.3 GIF PRINCIPLES AND ACTIVITIES As indicated in GIF documentation, the Consortium implements the activity guided by several principles presented in Exhibit 3 (GIF Annual Workplan Year 1; GIF Annual Workplan Year 2). Exhibit 3 GIF Implementation Principles GIF supports locally-led activities through significant sub-grants to RPs, local action funding LPs, and additional sub-grants to technical experts affiliated with INGOs to transfer specialized skills disproportionately held at the global level (Exhibit 4; Source: Overview of GIF Activity Menu). Exhibit 4 GIF Activities Activity Activity Components Objective 1 - Digital Security Digital safety organizational support • Digital safety assessments for organizations and groups through the SAFETAG framework • Connecting digital and physical risks using an integral approach to security • Targeted end user training • Organizational support for digital safety by providing a special dedicated time to each beneficiary to implement safety strategies • Digital resource provision - access to legal software Centering regional and local partners in program design and activities Galvanizing inclusion of a broader, non￾traditional stakeholders in IF activities Uniting digital safety and digital rights communities and concepts at the local, regional, and global level Localizing existing, successful global models for increasing private sector engagement Supporting the entire IF community with stronger tools for measuring effectiveness and impact USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 10 Activity Activity Components Expansion of the in-country expert community Build local ecosystems of trained technologists through capacity-building efforts tailored to each region/country Incident response capacity Design and deploy complete helpdesk solutions and expand the outreach of incident response efforts to more organizations and communities by using the network of in-country trainers and beneficiaries Public outreach and material creation • Creation and localization of digital safety curricula by publishing research reports and blog posts on technical aspects of IF threats • Public digital safety campaigns around digital safety topics Objective 1I - Digital Rights Building collective knowledge trough research publications Research on the following topics: • Global best practices to defend and extend digital rights • Divergent and emerging digital rights issues • Adaptation of the RDR Corporate Accountability Index methodology Campaigns and advocacy • Advocacy campaigns • Awareness-raising and information campaigns • Advocacy toolkits Building capacity through learning events • Training and workshops • Digital rights schools • Lectures at institutions of higher learning Expanding the community through fellowships and mentorship • Article 19 fellowship for RPs and LPs • Regional and local fellowships provided by RPs: o To support the fight against online gender-based violence against women in Southeast Asia o To support new actors in the digital rights space in LAC and Central Asia o To improve knowledge of digital rights among CSOs and journalists in the Balkans and Moldova 3 METHODOLOGY The ET has employed a combination of qualitative and quantitative methods. The team adopted techniques from complexity-aware monitoring, evaluation, and learning methodologies by using an iterative, phased approach to data collection, whereby information collected at one phase informed data collection instruments for subsequent phases. The evaluation began at the end of February 2023 with a launch meeting and consultations among the ET, USAID, and Internews. These discussions resulted in USAID revising initial evaluation questions, with an updated set of questions reflected in the evaluation Concept Note submitted by the ET and approved by USAID. As depicted on Exhibit 5, the team started data collection activities with a comprehensive desk review of program documentation, followed by key informant interviews (KIIs) with USAID, Internews, and global and regional implementing partners (Phase I KIIs). The ET then conducted a web-survey of all available local implementing partners. The final stage of data collection was KIIs with selected local implementing partners (Phase II KIIs). USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 11 Exhibit 5 Evaluation Timeline (2023) 3.1 QUALITATIVE DATA COLLECTION The qualitative approach for this evaluation involved desk review and KIIs with implementing partners (IPs) conducted in two subsequent phases: Phase I included KIIs with USAID, Internews, and global and regional IPs, while Phase II consisted of KIIs with LPs. The mutually reinforcing qualitative strategies provided valuable information regarding contexts and factors that influence the GIF activity and its stakeholders. Preliminary analysis from Phase I KIIs allowed the evaluation team to refine their interview guides for Phase II KIIs to explore emerging findings. 3.1.1 DESK REVIEW At the outset of the evaluation in March and April 2023, the ET has reviewed activity documentation provided by USAID and Internews. This included 10 GIF quarterly reports, three annual work plans, and documentation related to subgrant awards to LPs. 43 documents in total about the GIF activity were reviewed. The ET also reviewed external publications, including the Internet Freedom Evaluation Report (prepared by DevTech Systems, Inc. for U.S. Department of State) and Freedom on the Net 20225 by Freedom House. 3.1.2 KEY INFORMANT INTERVIEWS The ET conducted qualitative data collection via KIIs in two phases. Phase I consisted of 21 individual and group KIIs conducted in April-May 2023 that collected information from 29 informants from USAID, Internews, as well as all global and regional implementing partners (Exhibit 6, see a more detailed table in Annex B). The ET used information from the desk review to develop interview guides for Phase I KIIs ensuring that all four evaluation questions were sufficiently covered (see KII protocols in Annex C). Exhibit 6 Phase I KII Respondents Respondent Category Number of Interviews Number of Respondents USAID staff 2 4 Internews 6 7 INGOs6 4 6 Regional IPs 9 12 Total 21 29 5 https://freedomhouse.org/sites/default/files/2022-10/FOTN2022Digital.pdf 6 In what follows, we use terms “INGOs” and “global partners” interchangeably. Feb March Apr May June July Launch Meeting Concept Phase I KIIs with USAID, Internews, global Web-survey of Data analysis and report Phase II KIIs writing Desk Review USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 12 Conducted in May-June 2023, Phase II included 14 individual and group KIIs totaling 18 participants from a sample of LPs (see Annex B for a list of informants). The selection process was based on purposive sampling to ensure representative geographic coverage was attained7; the two GIF objectives are represented, and informants with a broad range of perspectives are included. Phase II interview guides were informed by findings emerging from the desk review, Phase I KIIs, and a web￾survey of LPs. We then tailored the guides to elicit responses on aspects most relevant to each informant (see KII protocols in Annex C). KIIs had a duration of 50-80 minutes, averaging one hour. Interpretation was provided for interviews in Spanish (2) and Russian (2), and two interviews were conducted in French by a member of the evaluation team who is fluent. All other interviews were conducted in English. 3.2 QUANTITATIVE DATA COLLECTION The ET conducted a web-survey of local GIF partners to uncover their experiences with program participation along the four dimensions captured by the evaluation questions. Survey questions were informed by desk review and qualitative data collected from RPs in Phase I KIIs. USAID reviewed the draft questionnaire prior to translation and fielding. The final survey instrument consisted of 39 questions drafted in English and translated into Spanish, Portuguese, French, and Russian by professional translators (see the English version of the survey instrument in Annex E). The ET programmed and tested two versions of the survey in Qualtrics, a popular online survey platform. One version was for LPs in non-sensitive countries, and the second version developed for partners in sensitive countries used redacted language that avoided direct mentioning of USAID and GIF as well as some other potentially sensitive terms. The ET collected contact information of LPs in non-sensitive countries from GIF’s regional coordinators and partners with support and facilitation from Internews. The team assembled a list of 58 contacts from 478 organizations and independent consultants from seven regions and 29 countries where GIF operates. Individual survey links were sent from Qualtrics platform to all LPs in the list on May 25, 2023. The non-sensitive version of the survey was kept open for response submission until June 20, 2023. To distribute the web-survey to LPs in sensitive countries, the ET sent anonymized Qualtrics survey links along with a suggested explanatory text to RPs in regions where sensitive countries are located, asking them to share this with LPs in those countries using secure communication channels. The RPs sent the survey links to eight LPs in three regions. The non-sensitive version of the survey was open for submitting responses June 6 - 20, 2023. The ET team combined responses obtained from LPs in sensitive and non-sensitive countries in one data set for analysis. Dispositions of attempted and completed responses are in Exhibit 7, showing responses and non-responses for contacts provided and specifying how many of these were from unique organizations, as for some organizations we had more than one contact provided to us. We present the response statistics for Latin America and the Caribbean (LAC), Europe and Eurasia (E&E) and the Balkans, and the Central Asia (CA) in the form of a sum to show numbers separately for non-sensitive and sensitive countries. In total, we received 51 responses from 48 unique 7 We aimed to include at least two participants from each GIF region. 8 By July 2023, GIF has worked with 77 LPs, as reported by Internews. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 13 organizations. Fifteen contacts representing seven unique organizations did not respond to the survey. Thus, the overall response rate (based on unique organizations) is around 87%. Exhibit 7 Disposition Summary Disposition Latin America and the Caribbean Europe and Eurasia and the Balkans Central Asia South and Southeast Asia West and Central Africa East and Southern Africa The Middle East and North Africa Total Responses 11+1 9+1 5+3 7 6 7 1 51 Of them, unique organizations 10+1 9+1 5+3 6 6 6 1 48 No answer 5+1 4+2 0+0 3 0 0 0 15 Of them, unique organizations 0+1 3+2 0+0 1 0 0 0 7 Note: For regions where sensitive countries are located, the table presents a sum of responses from non￾sensitive and sensitive countries. For example, there 11 responses from partners in non-sensitive countries and 1 response from a partner in a sensitive country, presented as 11+1. 3.3 DATA ANALYSIS NORC’s qualitative analysts coded KII transcripts using a deductive coding scheme in Dedoose qualitative analysis software. The coding scheme was developed using an iterative testing process to ensure inter-rater reliability. The coding allowed for the extraction of relevant excerpts by theme and evaluation question. These excerpts were analyzed to find points of convergence and divergence among respondents and to extract information to answer the evaluation questions. The team identified illustrative quotes to demonstrate viewpoints to enrich the findings. NORC analyzed the quantitative data using STATA. Tabulations of data were done based on consultations with the ET members collecting qualitative data, as the body of evidence emerged, and new angles of enquiry appeared. The survey and KII data were used to establish converging lines of evidence triangulated with the reviewed documents to ensure the findings were as robust and accurate as possible. The ET integrated data from this variety of approaches and sources into coherent conclusions and recommendations by aggregating and synthesizing findings. 3.4 LIMITATIONS Despite the multidimensional methodology, there are some limitations inherent to the design and context of this evaluation: Focused scope of evaluation: The ET was tasked with evaluating four aspects of GIF (localization network development, advocacy with governments and multilateral organizations, and advocacy with the private sector). This does not cover the entirety of the GIF program and its impacts. Also, because the evaluation is being undertaken while the project is still ongoing, nor can it reflect the totality of activities being implemented in year three. Large scope of GIF: Since GIF is a large, global program with numerous components, the ET was not able to cover every single activity implemented, since our samples are not comprehensive to all partners. The program is highly contextualized by design so that each partner implements a unique combination of activities dependent on their specific capacities and priorities; therefore, there was no way for the ET to cover each specific activity without sampling 100 percent of partners. Our USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 14 samples were designed to cover as much breadth as possible while going in-depth for the activities most pertinent for the scope of our evaluation. Short evaluation timeline: The evaluation was conducted on a timeline that was shorter than optimal for such a large and complex evaluation. The ET team had to make trade-offs and prioritization in terms of inclusion of respondents (smaller samples), as well as the depth and breadth of data collection, analysis, and reporting. The short timeline also limited rounds of feedback and revisions of the report. Indirect beneficiaries were not included in the evaluation: Due to the large scope of the GIF project and the limited timeline for the evaluations, the ET did not collect data from indirect beneficiaries, i.e. organizations or individuals that did not receive funding from GIF but benefited indirectly from GIF-supported activities. Small sample of local partners included in KIIs: Due to a short evaluation timeline, the ET was able to include only a small sample (18%) of LPs in KIIs (Phase 2). The qualitative sample may not be representative of the entire population of GIF LPs. Organization of online KIIs: Since all KIIs were conducted online and remotely, the ET had some issues with LP connectivity that limited the amount of time and therefore the number of questions that could be asked to partners with bandwidth issues. The ET used a variety of strategies to overcome this, such as switching to different call platforms, not using video and only using audio, and allowing extra time beyond the planned one hour. However, this could have introduced bias as LPs from certain regions such as Central Africa had more connectivity issues than others. In addition, one LP was not able to join their KII due to connectivity issues and we were not able to reschedule and had to remove them from the sample. Social desirability, or halo bias: The extent to which respondents were prepared to reveal their true opinions may also have varied for questions that called upon them to assess GIF-supported activities their organizations depend upon for funding. To mitigate, the ET assured participants of the independent nature of this evaluation; and provided appropriate confidentiality, anonymity, and data protection assurances to all respondents. KII questions were worded to avoid suggesting the most desirable or expected responses. Conflation of GIF-supported activities with activities supported by other donors: In KIIs, the ET noticed that respondents conflated activities supported by GIF with other activities. This was especially the case when their engagement with GIF was limited or when it was highly integrated with their routine work. In such cases, the ET team asked respondents to focus on their work supported by GIF. Survey selection bias: Given that some LP staff members did not participate in the web survey, there is a possibility of selection bias. It is possible that respondents who chose to complete the survey might systematically differ from those who did not in terms of their positions within their organizations, exposure to and experiences with GIF activities, as well as attitudes, perceptions, and socio-demographic characteristics. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 15 4 FINDINGS 4.1 NETWORK DEVELOPMENT Network development was an important part of GIF, as reflected in GIF’s TOC. GIF supported creation of new regional networks emerging from connections between partner organizations in different countries, whereby LPs within the same region formed relationships with each other, which was especially useful in regions with weaker pre-existing networks. GIF also linked LPs with USAID missions and other USAID-funded activities in countries. GIF enhanced existing networks, allowing RPs to expand their footprints and engage with diverse LPs where regional networks already existed. Respondents found participation in GIF-supported networks to be very useful, as it offered opportunities for promotion, learning, and networking, with concrete examples of collaborative projects and support emerging from these networks. Opportunities for improvement of GIF-supported network development include more inclusive collaboration, more opportunities for LPs to participate in global convenings, more in-person networking events, and the development of clear and flexible communication channels. 4.1.1 NEW AND STRENGTHENED NETWORKS EQ: What networks have been established or strengthened through GIF's programmatic approach, particularly between international, regional, and local civil society actors? The web-survey of LPs showed that most respondents (85%, 41 respondents) participated in some networks or networking events supported by GIF, whereas a few respondents did not (13%, six respondents) (Exhibit 8). Exhibit 8 Participation in GIF-Supported Networks Note: N=48. Based on the question “Have you participated in networks or networking events supported by GIF/USAID? Some examples include fellowships and mentorships programs, meetings, conferences, training, lectures, workshops, academic collaborations, online fora and platforms, etc.” The two most common types networking activities were trainings, lectures, and workshops (63%, 30 respondents) and conferences, meetings, and roundtable discussions (58%, 28 respondents) (Exhibit 9). A quarter of respondents participated in advocacy-related collaborations (25%, 12 respondents). Yes 85% No 13% Don't know 2% USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 16 Exhibit 9 Participation in Specific Types of Networking Events and Activities (% of total sample) Note: N=40. Based on the question “Please select networks or networking events supported by GIF/USAID you have participated in? Select all that apply.” New Networks As revealed through desk review and KIIs, most new networks created seemed to be regional networks emerging from connections between organizations in different countries. Different LPs within the same region formed relationships with each other. RPs usually fostered these networks, and once the connections between LPs were established, respondents reported that they plan to continue them past the end of the GIF project. An LP in West and Central Africa (WCA) spoke of new regional networks created under GIF: “We are today able to exchange information, opportunities, activities with the people from the Central African Republic, Angola, and Mozambique without necessarily going through Internews or GIF. So, indeed, thanks to these projects, me, personally, in what concerns me and what concerns my organization, we have the feeling of being part of the networks.” (KII, LP, WCA) In some regions such as LAC, GIF brought LPs into regional networks such as the Observatory for Digital Threats that pre-existed GIF. However, this level of pre-existing organization was only present in some regions; in others, GIF networks were the first of their kind to bring organizations across countries together to collaborate and share information. Some LPs had felt isolated in their countries before GIF. They appreciated becoming part of new networks under the GIF project. An LP in CA explained: “I haven't had any network previously. […] Here in our country there’s one organization to work on digital rights. GIF created some new network for us…I don't know how to say it's creation or they make us part of this network and it was very useful for us.” (KII, LP, CA) For some LPs, GIF helped develop new links with USAID Missions and other USAID-funded activities in their countries, as revealed by the web-survey data (Exhibit 10). Thirty five percent (17 respondents) said involvement with GIF helped them develop such links.9 At the same time, 40% (19 9 Among respondents whom GIF helped to develop links with USAID, most believed those links were very beneficial (29% of the total sample, or 14 out of 17 respondents). 63% 58% 25% 19% 15% 15% 6% 2% Trainings, lectures, or workshops Conferences, meetings or roundtable discussions Collaborations related to advocacy on the regional or global level Online fora or platforms Regional research networks related to implementation of the RDR methodology Other research or academic collaborations Fellowships and mentorships programs Other: collaborations related to Safetag USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 17 respondents) did not develop such links and the remaining quarter (12 respondents) did not know whether such links were created or not. Exhibit 10 Networks with USAID Missions or Other USAID-Funded Activities Developed due to GIF Note: N=48. Based on the question “Did your involvement with GIF help create links and networks with USAID Missions or other relevant USAID funded activities in your country?” Strengthened Networks The RPs seemed best positioned to strengthen their existing networks through GIF. The structure of GIF with global partners, RPs and LPs lent itself to RPs enhancing their networks. They were able to capitalize both on stronger working relationships with global partners and more visibility among organizations in the countries that had LPs, with the latter being most impactful. These regional networks did not only benefit LPs, but also RPs. For example, an RP explained that the diversity of LPs helped them tap into different kinds of expertise: “What is interesting is the diversity of the local partners that we worked with through this project because some of them are more related to the media and disinformation […] but also there are organizations that deal with human rights, some specifically with LGBTQ rights and rights of marginalized groups, and also organizations that deal with the security issues […] This project really helped us broaden our own network of partners that we work with but also helped those local organizations.” (KII, RP) In terms of global and local partners, there was significant variation between the size and strength of partners’ existing networks, which meant that the strengthening of their networks through GIF also varied. A global partner stated that while their existing networks were enhanced through GIF, they weren’t necessarily making new connections: “I don't know that there's any sort of formal networks outside of the project partners that we have become a part of as a direct result of GIF. But we are part of several other networks that sort of potentially have either geographic or subject matter relevance or overlapped with GIF.” (KII, Global Partner) Respondents indicated that there were weaker pre-existing networks in Central Asia and Central Africa (GIF helped build more connections in these regions). The LAC region had particularly strong networks, and GIF helped to further strengthen them. Local and Global Connections Connections between local and global partners were highly beneficial but also limited in scope. Most LPs connected with global partners specifically for standalone activities, such as legal analyses with Yes 35% No 40% Don't know 25% USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 18 ICNL or the RDR exercise, or for webinars and trainings held by global partners. LPs also explained that GIF support allowed them to participate in international advocacy by supporting their attendance at international fora such as RightsCon and the Internet Governance Forum (IGF). An LP from CA explained that GIF allowed them to bring the concerns and priorities of the CA region to the attention of international actors for the first time since they normally don’t have the resources to participate in these fora. Connections between local and global partners also benefited global partners. A global partner explained that their work with GIF allowed them to bring local perspectives to global initiatives: “We always point out within these very high-level spaces that they often disregard the experiences and the evidence of local communities and local networks who are most affected by Internet infrastructure. […] Being able to bring in some of the nuances, which many civil society organizations have, and relaying that back into more global technical standards and policy forums can help shape policies and standards that are more beneficial for those who need it.” (KII, Global Partner) However, there were some LPs who felt disconnected from global partners due to the structure of activities. For example, LPs would participate in trainings or events but then not have access to global partners as they tried to put what they learned into practice. An LP in LAC explained: “I participated in some of the webinars, but I think there still there's a lack of working groups after the webinars […] For example, we’ve done this training on SAFETAG […] but we don't have a clear space later to share our experience or just to have this conversation of what we are doing great or what we are doing wrong.” (KII, LP, LAC) RPs also commented on this disconnect between global and local partners and noted that this is something that could be built into workplans and given more time and resources. 4.1.2 EXTENT OF NETWORK USE EQ: To what extent are GIF networks used to coordinate and share information on digital rights and security? In our web-survey of LPs, the ET asked network participants about the usefulness of participation in GIF-supported networks for their organizations (Exhibit 11). Most respondents (71%, 34 respondents) thought the participation was very useful, while some believed it was moderately useful (10%, five respondents). One respondent (2%) said the participation was little useful. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 19 Exhibit 11 Usefulness of Participation in GIF-supported Networks Note: N=48. Based on the question “Has participation in networks or networking events supported by GIF/USAID been useful for you and your organization?” When the survey asked about specific benefits of participation in GIF-supported networks (Exhibit 12), LPs most frequently selected an opportunity to promote their work on digital rights and digital security among other stakeholders (63%, 30 respondents), learning new information and developing new skills (56%, 27 respondents), and networking with colleagues and other experts from their region (54%, 26 respondents). Benefits that were mentioned less often included those related to engagement in advocacy with governments or multilateral organizations/forums (19%, nine respondents) and with private companies (13%, six respondents). Exhibit 12 Beneficial Outcomes of Participation in GIF-supported Networks (% of total sample) Note: N=39. Based on the question “In what ways has your participation in networks or networking events supported by GIF/USAID been useful for you and your organization? Select all that apply.” As uncovered through KIIs, one of key benefits of GIF networks for LPs has been information sharing across countries within regions. An LP in WCA explained that they were able to establish Little useful 2% Moderately useful 10% Very useful 71% Don't know/Missing 17% 63% 56% 54% 42% 40% 31% 19% 13% Promoted my work on digital rights and security among other stakeholders Learned new information and developed new skills Networked with colleagues and other experts from my region Networked with colleagues and other experts from other regions of the world Can better coordinate my work in the space of digital rights and security with other stakeholders Participated in research activities would not have otherwise been able to be engaged in Engaged in advocacy with governments or multilateral organizations/forums Engaged in advocacy with private companies USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 20 new channels of communication with other organizations in their country and region, through connections supported by their regional partner. While they had focused on their in-country network before, participation in GIF helped them expand to connect with others in the region. 4.1.3 EXAMPLES OF BENEFICIAL NETWORKS EQ: Are there concrete examples where network coordination and information sharing have benefited local members? Partners at all levels provided examples of concrete benefits to the networks that they participated in thanks to GIF. Respondents highlighted access to trainings and fora as key benefits to GIF networks. Beyond these planned activities, partners described benefits that emerged organically from networks. For example, in WCA, there was an example of an LP providing contacts for legal experts to support digital rights work, which allowed an RP to conduct legal analysis. The same LP explained that they had built connections with others in the region and plan to pursue future projects together: “We have initiated exchanges and discussions to see how we are going to implement joint activities, the two of us, our two organizations, because our countries are neighboring and have the same realities […] It is only because we participate in these networks and thanks to what we call sustainability […], we want to continue our networks, our contacts beyond the GIF project .” (KII, LP, WCA) An LP in LAC mentioned that their RP had brought them into a regional network where they joined several groups on applications like WhatsApp. One of their GIF-supported activities was establishing a helpline for technology facilitated gender-based violence, and other organizations were able to help advertise or disseminate the helpline information on their social media platforms. In general, the topic areas that LPs mentioned the most for networking connections was digital security, specifically helping other organizations in the civil society space to implement security measures to protect themselves. An LP in South and Southeast Asia (SEA) explained that they were able to benefit from connections with other LPs in their region, who supported them to provide trainings in other countries by helping them recruit participants, providing venues, and even resources like projectors. A different LP in SEA also highlighted connections between GIF LPs, fostered by attendance of regional and global fora, such as the Asia Pacific Internet Governance Forum, IGF, and the Digital Rights Asia Pacific (DRAPAC) Assembly, where other LPs and RPs were present. Thanks to GIF networks, connections between organizations across countries allowed for regional initiatives. In CA, a partner explained that new connections between organizations in different countries in the region allowed for a regional initiative around a new problematic foreign agent law in Kyrgyzstan. Connections with global partners also provided concrete benefits. An RP explained that their strengthened connections with global partners gave them confidence to invite these partners to events, thus elevating the legitimacy and impact of these events. “If that event happened before GIF or without GIF, I don't think we will have such a good connection to them, to the point that they will be willing to go to our event. But with GIF we were able to confidently invite them.” (KII, RP). Global partners’ convening power and expertise were highly useful, especially for connecting with private sector entities that can be difficult to meet with. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 21 The fellowships and mentorships run by Article 19 (A19) and RPs were an important modality of network building under GIF, making connections between LPs and global initiatives. A respondent from Internews spoke to the role of regional fellowships in supporting local and global connections, especially in regions that lack them: “Our Central Asia fellowship program has created camaraderie and networks among the fellows. But also has enabled them to interact and engage with international networks and different spaces. What we've kept hearing, especially in Central Asia was that they're really great actors, but they're disconnected from the global movement around digital rights and digital security. We've been able to nurture and facilitate that.” (KII, Internews) The A19 fellowship program was beneficial to LPs, as it brought fellows to high-level international networks that they could not easily join otherwise: “We have previously established networks whether it is more formalized working groups within particular standards bodies or more loose networks, coordination kind of networks. And we've introduced fellows into these networks […] Fellows have been able to engage within those groups, being able to monitor some of the developments within these standards and policies.” (KII, Global Partner) An LP from LAC praised the fellowship for sponsoring their participation in the Internet Corporation for Assigned Names and Numbers (ICANN) conference: “We didn't have to pay, because A19 and GIF paid for us. But if you don't have that support, you have to pay as an organization, and the fee is really big. It's not so easy for an organization like us.” (KII, LP, LAC) 4.1.4 OPPORTUNITIES FOR NETWORK DEVELOPMENT EQ: Where do gaps exist in network development that future programming should fill? Challenges and gaps captured in data collected in this evaluation highlight the importance of more inclusive collaboration, effective communication, flexibility, and a balanced approach in addressing digital rights issues and building networks at the local, regional, and global levels. Each of these offer opportunities for further support and enhancement of networking capacities. Findings from our web-survey of LPs speak primarily to specific challenges of participation in networking events. Although 19% of respondents expressed that they had not faced any challenges participating in GIF-supported networking events, other network participants indicated several challenges (Exhibit 13). Nineteen percent (nine respondents) believed that participating in networking events online rather than in-person was challenging. Around 13% (six) of respondents thought that networking events had been too short or not intensive enough; that there had been insufficient number of networking events; and that materials and events had not been offered in local languages. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 22 Exhibit 13 Challenges of Participation in GIF-supported Networks (% of total sample) Note: N=37. Based on the question “What challenges, if any, did your organization face regarding your participation in networks or networking events supported by GIF/USAID? Please select all that apply.” In the open-ended survey question, LPs shared ideas on how future programs for network development in the area of digital rights and security can be more helpful. Multiple respondents from various regions mentioned interest in opportunities for collaboration, networking, and knowledge exchange across regions and with global partners (29%, or 14 respondents); more capacity building and trainings, especially in new technical topics (23%, or 11 respondents); and more in-person rather than online events (15%, 7 respondents). Other responses included the need for more funding; more time for program implementation; more research support; and opportunities to connect with USAID missions. These findings are consistent with KIIs, which addressed broader challenges around network development. Need for More Collaborative Process The need for more collaborative processes and involvement of LPs and communities in the design, planning, and decision-making processes from the outset of the project was reiterated in many ways by a range of partners. One partner explained: “there should be more of a collaborative process with the regional partner, the local partner, and the INGO…sometimes it feels like the regional partner has a lot to do and if this is not one of their priority outputs, then it kind of gets left behind a little bit.” (KII, Global Partner) A respondent from USAID also noted that the structure of GIF is centered around RPs, which has advantages, however, “I do wonder about the extent to which we are not able to potentially forge as many global connections among actors. I don’t think it’s bad that it was regionally focused, but I think there are tradeoffs inherent in that.” (KII, USAID) Fostering more connections between global and local partners could reduce the centralization around RPs and lead to sustainable network connections for LPs. Greater LP Involvement in Global Spaces In KIIs and the web-survey, multiple LPs across different regions shared that they felt that their opportunities to be involved in wider global conversations were limited. An LP explained the challenges for organizations to access spaces where these conversations take place: “We need to be part of these [global] discussions [examples provided: Cyber Crime treaty in New York or Vienna, the global 19% 19% 13% 13% 13% 8% 4% 2% 8% No challenges Networking events were held online instead of in-person Networking events were too short or not intensive enough There were too few networking events Materials and events were not offered in local language(s) I was not given enough opportunities to present or otherwise promote my work Information or training provided at networking events was not relevant to my work Networking events did not offer the opportunity to meet with organizations outside my region Other USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 23 digital Compact in Mexico], because we can say things taking into account the [redacted] context and intercultural context- other kind of perspectives that are not being taken into account in those discussions...but we don't have the opportunity nor the resources…We would really like to be more involved in these global discussions.” (KII, LP, LAC) Internews respondents and respondents from global partners both noted that they see the same people attending global spaces like RightsCon, and it would be good to have more LPs attending and sharing their perspectives. While GIF supported some LPs to participate in these international convenings, there seems to be agreement that this support should be scaled up moving forward. Regional Challenges and Variations Network building has been challenging in certain regions, where the addition of countries or the absence of strong RPs posed difficulties in coordination and integration. The Middle East and North Africa (MENA) region posed a particular challenge due to special vetting requirements that hindered inclusion of other local partner organizations. USAID explained the challenges of working in more closed spaces where this kind of work poses risks, and gaps to address in the future to develop more local partnerships: “In the MENA, it’s been difficult to identify additional partners, particularly local partners. Not only who works on this issue set, but who are able to contribute to the program in a way that would advance the objectives. …Although it was difficult to identify local actors, I think we found several very strong partners and I think…those partners are kind of driving the agenda and setting the priorities.” (KII, USAID) There are challenges inherent in the variable stages of progress among LPs within regions. RPs explained that usually the countries that they have LPs in are not homogeneous, and there needs to be flexibility to address diverse capacities. Capacity Challenges Capacity challenges for smaller organizations present opportunities for greater core support. Limited resources, both financial and human, pose challenges for LPs in dedicating time and personnel to specific project activities. One partner describes their perspective on this: “especially when we are talking about small local organizations, it's very difficult for them to dedicate one or two people to, you know, do something for, say, one month because it would almost jeopardize their core activities.” (KII, RP) Support from GIF goes a long way in helping small organizations participate in long-term advocacy initiatives, especially in international spaces, and helps level out the playing field so that organizations with limited resources can work together with others that have stronger capacities to participate. In-Person Networking Opportunities Many partners reiterated the desire for more in-person networking opportunities that were limited due to COVID during the initial phase of the project. This was reflected in comments from local, regional and global partners, who explained that even though partners were used to working remotely using digital platforms, there is still inherent value in having less formalized in-person spaces to network. One partner summed up the value of in-person gatherings like this: “having those instances in which you can actually share a physical space with someone you've been working with online and having space to actually discuss other things that you cannot do just because of the transactional nature of online video conferences…If that can continue to happen, maybe in a potential Year 4 that we're waiting for, that could be great.” (KII, Global Partner) USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 24 Many LPs mirrored this view in the web-survey and KIIs, for example: “In the future, more in-person regional activities would be valuable for better connection and networking with other partners in the region and globally.” (Web-survey, LP, E&E and the Balkans) Need for Clear Communication Channels Finally, there is the opportunity to strengthen communication channels. Many LPs and RPs mentioned that the lack of a centralized platform or consistent communication channels (email, Slack, Signal, etc.) hampers efficient sharing of information and collaboration. There was, however, no consensus as to what communication platform is best. It would be pertinent for GIF and other similar programs to allow for variation in communication platforms between geographic contexts. As one partner explained: “we are trying the mailing list. I think that may work for some people and I don't think we’ll use Slack. But I know we have a Slack for GIF and I know there's not a lot of movement there…any specific tool is going to be challenging in terms of adoption.” (KII, Global Partner) A regional partner noted that simply being added to communication channels (such as a Signal group) when those channels are not well used by people in their organization is not helpful. Instead, communication channels need to be tailored to the preferences of the partner organizations in their own contexts. 4.2 LOCALIZATION Localization, or the structuring of the project so that RPs and LPs have agency over what kinds of activities they implement in their own contexts, was a key focus of GIF reflected in the project’s TOC. GIF was set up to ensure that partners had their own needs and priorities addressed with GIF funding, within the parameters of activities that Internews, regional and global partners could offer. GIF struck a balance between allowing partners to choose activities and topics of trainings and research while also providing a set menu of options. RPs played an essential role in mediating between all levels of GIF. RPs underwent capacity strengthening, connecting with global partners and multilateral institutions while also learning more about localized perspectives. GIF strengthened LPs’ technical capacity through trainings, lectures and workshops. GIF provided LPs with some flexibility in designing activities, but there are still concerns about the standardization of activities and the level of autonomy for LPs in decision making. Opportunities for improving GIF’s localized approach include further integrating LPs in decision-making processes while accounting for differences in their levels of expertise and solving some operational gaps. 4.2.1 APPROACHES FOR LOCALIZATION EQ: What approaches did GIF take to support (i) local and (ii) regional organizations? Approaches with LPs GIF approaches to localization took several forms. The main avenue of support for LPs was trainings, lectures and workshops provided with technical expertise from the global and regional partners. These activities strengthened capacities of local organizations, and LPs themselves decided on the topics and foci of the trainings, lectures, and workshops. GIF also provided practical knowledge products, such as toolkits and methodology guidance for research, as well as software licenses and support developing policies and protocols around digital security. Results of our web-survey presented in Exhibit 14 highlight the range of GIF activities aimed at supporting LPs. The most common types of activities supported by GIF in both program areas were trainings, lectures, and workshops (digital safety and security: 60%, 29 respondents; and digital USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 25 rights: 44%, 21 respondents) as well as conferences, meetings, and roundtables (digital safety and security: 54%, 26 respondents; and digital rights: 44%, 21 respondents). Other common types of GIF￾supported activities in the area of digital safety and security included security audits and assessments (40%, 19 respondents); development and provision of practical knowledge products such as toolkits (31%, 15 respondents); provision of software, software licensing or hosting (29%, 14 respondents); and development of security and incident policies and protocols (29%, 14 respondents). In the area of digital rights, respondents commonly selected research support (33%, 16 respondents) and support for development of practical knowledge outputs (31%, 15 respondents). Exhibit 14 Activities and Resources Local Partners Benefited from in the Area of Digital Safety and Security (left) and Digital Rights (right) (% of total sample) Note: N1=39 and N2=33. Based on the questions “In the area of digital safety and security, what activities and resources did you benefit from? Select all that apply.” (left) and “In the area of digital rights, what activities and resources did you benefit from? Select all that apply.” (right). Approaches with RPs RPs are a central connector between all the levels of GIF. An RP explained the GIF approach and the role that they have played: “The project until now has been intense, […] it has made us grow a lot in terms of our administrative capacities to manage such a big project with such a diverse group of elements […]. We mediate a lot of different types of relationships and focusing on the national allies, but there are the relationships with the international organizations, other regional partners working within GIF, and the Internews team at the same time.” (KII, RP) 44% 44% 33% 31% 21% 2% 2% Trainings, lectures, or workshops Conferences, meetings, and roundtables Research support Support for development of practical knowledge outputs Support for advocacy campaigns Fellowships and mentorships programs Other 60% 54% 40% 31% 29% 29% 21% 17% 4% Trainings, lectures, and workshops Conferences, meetings, and roundtables Security audits and assessments Development and provision of toolkits and other practical knowledge products Provision of software, software licensing or hosting Development of security and incident policies and protocols Support for incident response and emergent developments Technical support, including web-site creation and updating Other USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 26 RPs support network building with local counterparts, convening LPs together and establishing communication channels for them. This is described in more detail in the section above on networks. RPs also play an important role in handling MEL data and reporting, while Internews provides them with technical support for this task: “We have been focusing [on] making really good templates for regional partners. So [that] they're not overwhelmed with one template for MEL, another for narrative, it [was] merged into one good template, from which we can track all the information - narrative and quantitative. We have also supported our regional partners with how to develop better templates for local partners.” (KII, Internews) This perspective from Internews was also supported by the viewpoints of RPs, who expressed that they were able to comply with monitoring requirements without major issues. 4.2.2 CAPACITY STRENGTHENING FOR RPS AND LPS EQ: To what extent did these approaches strengthen capacity on digital security and digital rights for (i) local and (ii) regional organizations? One of the core aims of GIF is to strengthen further collaborations and partnerships, especially in areas where many LPs are new to the digital space. LPs highlighted the valuable support they received through capacity building activities, including technical advice and connections with other organizations. Trainings, workshops, and materials have enhanced their knowledge and expertise in digital rights and internet governance. The program has also expanded the reach and recognition of local NGOs, allowing them to become leading organizations in their respective fields. The provision of resources and opportunities through GIF has improved their ability to engage with communities and stakeholders. Fellowships have helped bridge the knowledge gap and empower organizations to participate in technical discussions and policymaking. The program has also emphasized the importance of documentation and project planning for impact assessment. RPs play a central role in supporting local organizations, although communication challenges, limited autonomy for LPs, and administrative requirements have been noted. Local Capacity Strengthening USAID described capacity strengthening in support of localization as a spectrum ranging from a lower level of locally-led development wherein local communities are consulted, moving through more substantive forms of engagement with the highest level as locally led development where partners are the ones driving priorities and determining activities. Several LPs said that the most useful support that the GIF project provided for them was capacity strengthening, especially technical advice. Some LPs had not had experience or expertise in specific technical areas related to digital rights and digital security under GIF, as their focus was more on human rights work in general. Thanks to GIF, they were able to build this specific expertise and indeed develop the capacity to host trainings themselves for other NGOs, as mentioned by a LPs in SEA, CA, and ESA. An LP discussed their increased capacity to run trainings and local recognition as a leading organization in the field; trainings have also gone beyond those under GIF: “the demand is quite high, so sometimes we end up with our own off-GIF trainings […] we have trained slightly over 100 people from media, civil society organizations, human rights defenders, and freelance journalists” (KII, LP, ESA) RPs noted that the process of onboarding LPs meant training staff members in technical areas, thus strengthening the technical expertise of partners from the very beginning. GIF also linked LPs to opportunities and connections internationally. As one partner reported: “this is a real way to build the USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 27 capacity of local NGOs, because internationally (we start to get) recognized and are getting a lot of opportunities through this project GIF.” (KII, LP, SEA) LPs also reported that GIF strengthened their internal organizational capacities by linking them to technical resources withing the consortium and by providing financial resources to hire additional staff. An LP explained, “(GIF) has allowed us to consolidate work and complete the research required for different programs and helped us strengthen the dynamics of internal collective work” (KII, LP, LAC) This organizational capacity strengthening included improving LP’s capacities to work with USAID and produce documentation. Multiple RPs spoke of their role in guiding LPs through documentation processes. One RP explained, “you know how young organizations, some organizations are very loose when it comes to the processes from planning a project towards implementation. […] one thing they learned from working with GIF would be how important it is to ensure when you plan an activity or a project that it matches with the commitments in the plan.” (KII, RP) Other partners referenced burdensome administrative requirements of GIF, but in terms of capacity building for larger grant and project management, this is notable to include here. In particular, regional and local partners noted large amounts of paperwork in order to onboard local partners. Regional Capacity Strengthening RPs described several key avenues of capacity strengthening for their organizations. They were able to connect with multilateral institutions and global partners, and due to the length of the grants were able to track issues on international levels over the course of three years. At the same time, working with LPs exposed them to localized perspectives and gave them a window to digital rights and digital security issues ‘on the ground’. An RP described their growth as an organization due to these connections: “We mediate a lot of different types of relationships and focusing on the national allies, but there are the relationships with the international organizations, the other regional partners working within GIF, and the Internews team at the same time. So it has meant a lot of growth for us.” (KII, RP) 4.2.3 LPS DETERMINING PRIORITIES AND DESIGNING ACTIVITIES EQ: To what extent were priorities determined and activities designed and led by local actors? The focus on locally-led development aims to give LPs the flexibility to design activities, agendas, and capacity-building that address the specific needs of their countries or regions. The process involves a range of levels of engagement with LPs to identify issues and propose activities and was noted by some partners as a very positive experience within GIF. However, there are also critiques regarding the standardization of activities and other constraints. The level of involvement of LPs in decision making varies between regions; in most cases the process was more collaborative based on the level of pre-existing relationship between RPs and LPs. In some regions, LPs found there to be too much centralization and control from their RP, whereas another RP would prefer a greater centralized role. Overall, on the level of activity implementation and customization, there were greater roles for LPs in decision making in Years 2 and 3. However, LPs were not involved at the start of GIF, in part by design, and thus far less involved in overall priority and agenda setting. LPs responding to the web-survey suggested that they were very engaged in tasks related to GIF development and implementation, such as GIF activity prioritization in their country (58%, 28 respondents) and dissemination of knowledge products created with GIF support (56%, 27 USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 28 respondents) (Exhibit 15). A sizable proportion of respondents also participated in designing implementation approaches for GIF activities in their country (48%, 23 respondents). Tasks with lower engagement of LPs included: identification of budget needs for GIF activities in respondents’ countries, identification of research needs in respondents’ countries, and identification of indicators to measure outcomes and outputs of GIF-supported activities10 . Exhibit 15 Engagement in GIF Activity Development and Implementation Note: N=48. Based on the question “How engaged were you in the following tasks as part of your engagement with the GIF program?” Most survey respondents (54-67%, 26-32 respondents) wanted to become more engaged across all the listed tasks associated with GIF activities (Exhibit 16). At the same time, a quarter to a third of respondents (23-35%, 11-28 respondents) would prefer to maintain their current level of engagement. Exhibit 16 Preferred Engagement in GIF Activity Development and Implementation Note: N=48. Based on the question “How would you like to change the level of your engagement in the following tasks as part of your work under the GIF program?” 10 The ET notes that the project indicators (outcomes and outputs) had been agreed on at the outset of GIF, before LPs were engaged in the project. 19% 23% 23% 6% 10% 6% 13% 13% 4% 10% 10% 13% 31% 21% 25% 31% 21% 21% 35% 38% 44% 48% 56% 58% 2% 2% 2% 2% 2% 2% Identifying indicators to measure outcomes and outputs Identifying research needs in country Identifying budget needs for activities in country Designing approaches to implement activities in country Disseminating knowledge products created Prioritizing activities to be undertaken in country Not engaged at all A little engaged Somewhat engaged Very engaged Don't know/Missing 4% 4% 2% 2% 2% 33% 35% 29% 27% 27% 23% 54% 58% 60% 63% 65% 67% 8% 6% 6% 8% 6% 8% Identifying indicators to measure outcomes and outputs of GIF-supported activities Disseminating knowledge products created with support from GIF Identifying budget needs for GIF activities in country Prioritizing activities to be undertaken in country under GIF Designing approaches to implement activities supported by GIF in country Identifying research to be supported by GIF in country To be less engaged Stay the same To be more engaged Don’t know/Missing USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 29 The analysis of KII data allowed the ET to examine the process of determining priorities and designing activities from the perspective of global and regional partners and compare it with how LPs view the process. We were also able to delve into limitations of locally-led development in practice. Process of Determining Priorities and Activities from Global and Regional Partners’ Perspective One of the key designs of GIF is a focus on local decision making and locally led activities. This in part emerged from past experience and a recognition that the international community was not focused enough on building the community level- sustainability and capacity amongst local actors. The process of deciding on activities was that Internews provided a menu of options or list of activities to RPs, who then worked with LPs to select activities, what these activities would focus on, and who would participate along with LPs. An RP described their role in the design process: “The rationale is you are in this country, and you know where pain points are. You are also interfacing with local partners and stakeholders at different levels. You are in the best position to know what activity will give the best impact for the region. We were given freedom to choose the activities and the number of potential beneficiaries.” (KII, RP) One RP described their process of involving LPs in developing work plans: “When I'm developing a work plan, I sit with the partners, and we all collectively generally agree on what thematic topics we're going to research. There's some pushback, sometimes unpleasant situations where you have to push back or like accept defeat. So it would happen the same way as we make other decisions collectively as a consortium.” Another RP mentioned the process of co-creation was owed to that fact the LPs had been doing this work for a long time. LPs were not involved in year one work planning since this was before any of them had been identified or onboarded. A respondent from Internews explained that this first year reflected the needs of the RPs, but in year two, RPs were asked to work “very closely” with LPs to select the topics of activities. This was challenging in some contexts where delays in onboarding LPs condensed the timelines for making decisions on which activities to implement. However, the Internews respondent said that in year three, RPs were able to take their time to consult with LPs. RPs described expanding decision making from LPs over time and said that more co-creation processes are expected for activities in year four. Process of Determining Priorities and Activities from LPs’ Perspective LPs had differing perspectives on the process of determining priorities and activities, and this depended on the specific RPs that they worked with. In particular, RPs in LAC were notably praised as working closely with LPs to determine priorities and activities. One LP thought the process was very open: “It was a very open process. We had very good correspondence with them [RP] and we had enough freedom to propose things and to also negotiate if that was even necessary.” (KII, LP, E&E) They went on to note the biggest constraints were the limited time frame and budget, but that the process worked well for them. Another LP explained that they felt their participation in activity decision making was crucial, as they were able to provide contextual information that would not be possible to collect with desk research. The LP explained, “I don't think all the information is possible to find on the internet or remotely. So we gave resources, people, and ideas […] We feel involved…It was not a program that was imposed on us. In fact, we helped make it up.” (KII, LP, WCA) Another LP in CA discussed their own local co-creation practice, wherein they convene focus groups with the local community to help set the topics of the public outreach campaigns they will USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 30 undertake. This was not standard across LPs, but it shows the flexibility and spirit of co-creation that LPs were able to capitalize on due to the structure of GIF. Limitations of Localization in Practice Despite positive experiences with some localization aspects reported above, there was some critique of the process in practice. This area represents the most wide-ranging responses that came up during KIIs. Notably, the experience varied by region. The primary critique from LPs was that they were not able to propose activities or independently determine priority areas. One LP reported: “The first weakness is that we did not leave to the partners the initiative to propose the contents of the project activities. Project activities were already (decided by) Internews for everyone and said you will do such activity and such activity- just adapt it to the local context… It's a bit standardized.” (KII, LP, WCA) Other LPs noted the amount of time spent on or waiting for coordination between Internews and their respective RP was a limiting constraint. The level of local autonomy varied with activities. For some activities, there were budgetary constraints that limited LP’ aspirations by restricting the scale of activities. Another limitation shared was over what subject areas that would fall under GIF, which in some cases differed from local priorities. In particular, some partners wanted to work on specific aspects disinformation, such as fact checking initiatives, but that topic was not included under the scope of GIF.11 4.2.4 OPPORTUNITIES FOR IMPROVING LOCAL ENGAGEMENT EQ: What opportunities/gaps exist to improve local civil society engagement on issues of technology and human rights? Partners discussed various gaps and opportunities for improving local engagement in GIF. They highlighted the need to better include LPs in decision-making processes, while accounting for differences in LPs’ level of expertise that can be addressed by targeted training. Some partners expressed desire for more direct connections between LPs and INGOs. Respondents also identified several operational gaps, including the short time frame of the project, the need for longer implementation periods, and translation needs. Future iterations of GIF should build translation into budgets and workplans, and steps should be taken to reduce delays in onboarding partners so that implementation periods are not shortened; in addition, implementation periods should be mandated to be more than 3-4 months as LPs expressed this timeframe was too short. The web-survey asked LPs about resources (in a broad sense) available to them to implement work under GIF (Exhibit 17). Sufficient time to undertake GIF activities appeared to be more of an issue compared to other resources listed, with 29% (14 respondents) either fully or rather disagreeing that they had enough time for activity implementation. Other two resources indicated as inadequate by 14% (seven respondents) and 12% (six respondents), respectively, were funding from GIF and access to experts from regional and global partners. Resources such as clear understanding of organization's role and tasks in GIF-supported activities; technical knowledge, tools, and equipment; and support for administrative or financial reporting requirements were available to most respondents with only six to eight percent of respondents reporting them as not (fully) available. 11 We note that work on disinformation in terms of broader legal frameworks was included under GIF. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 31 Exhibit 17 Availability of Resources for Undertaking GIF Activities Note: N=48. Based on the question “Did you have the following resources needed to undertake the activities required of you under GIF?” Responses from an open-ended survey question about support LPs need to be able to continue work in the space of digital rights and security beyond the GIF lifetime echo the above findings: financial support and capacity building (including training, access to experts and methodologies) were by far most mentioned (by 60%, 29 respondents, and 42%, 20 respondents, respectively). Other popular opinions included support for networking and building new connections with other actors in the spaces (23%, or 11 respondents) and sufficient time for project implementation (10%, or 5 respondents). More Local Ownership In KIIs, some LPs expressed general sentiment of wanting more control over the process, not just choosing content of activities but choosing the kinds of activities they will implement themselves. One suggestion was that LPs be brought in earlier in the process during year one, so that the very first stages of work planning would include their perspectives. An RP explained this approach: “instead of just giving blank interventions to all the six regions, my approach would be to identify the local partners early enough, bring them on board, let them share their context, what works, what doesn't work… being able to give the local partners an opportunity to come up with the interventions would add value to the consortium.” (KII, RP) Addressing Different Levels of Expertise Several global and regional partners spoke to the issue of addressing different levels of expertise. A global partner recommended that LPs need to identify what areas they need more support in. The respondent went on to describe the need for more upfront information about the level of expertise among partners so they can better tailor their activities and improve workflow: “the knowledge levels vary and also the way companies behave in a different region is completely different. If you try to do a one size fits all workshop it is helpful, but it may not be as useful to everyone. I think that getting more funding to regionalize that work on our side (would be helpful).” (KII, Global Partner) Once partners’ levels of expertise are identified, targeted trainings can be used to address them: “I think training for the implementing partners is a priority at this point…. If the implementing partners do not have enough knowledge to impart on the beneficiaries, it's rolls back, it becomes the same loop. But what I 6% 2% 4% 6% 2% 4% 23% 10% 10% 2% 4% 2% 33% 44% 38% 38% 21% 17% 33% 38% 40% 46% 60% 73% 2% 2% 2% 2% 2% 2% Sufficient time to undertake activities under GIF Access to experts from regional and global partners Adequate funding from GIF Support for administrative or financial reporting requirements Technical knowledge, tools, and equipment Clear understanding of what organization's role and tasks were Fully disagree Rather disagree Somewhat agree Fully agree Don't know/Missing USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 32 found is when the implementing partner has the requisite knowledge, they save costs on getting external facilitators who may not have context of the civil space.” (KII, RP) Strengthening LPs through Greater Global Engagement and Coordination Some RPs stated they were best positioned to connect with LPs and support strengthening their roles. One explained that confusion was generated for local organizations when USAID reached out to them directly. On the other hand, many LPs want a more direct relationship with Internews and opportunities for wider global engagement. A global partner described their wish for more direct strategy building with LPs: “having attended GIF retreats and been on numerous GIF calls, there is quite a bit siloed between the international and regional partners, and the local partners are really only included when it comes to targeted activities.” (KII, Global Partner) Operational Challenges Impacting LPs Among major operational challenges hindering the effectiveness of GIF activities for LPs were delays in approving partners, workplans, and payments for milestones completed. Partners, especially RPs, explained that delays in approvals condensed implementation timelines, which made it very difficult to implement activities to a high standard. An LP noted “our contract is very limited in terms of time, and processes are taking much longer. Sometimes there are also administrative delays, and it takes a very long time to solve some things. Then it's solved and actual work needs to be done really quick.” (KII, LP, CA) Another RP echoed this complaint: “The challenge we had with local partners was the frustration about the slow start to the project because USAID takes long to make their decisions or to come back with issues. I think that was the most challenging part.” (KII, RP) One LP in CA implemented an activity involving a legal analyses and advocacy to prevent a law from passing in their country that threatened freedom of expression, speech, and media. However, because it took six months to approve them as a partner (specifically to verify their budget processes), the law passed before they could complete the legal analysis. This respondent also noted that they experienced months long delays in receiving payments, and it was unclear if there were errors causing the delays that they could fix—there was no open communication. Respondents from USAID commented on these delays in their interviews, noting that the bureaucratic processes involving bilateral operating unit staff caused many delays. There were also some challenges related to language barriers, which came up in interviews with LPs in African francophone and lusophone countries. An RP in WCA explained that they were limited in their choice of LPs because they had to choose organizations that had an English-speaking staff member. The respondent explained: “A lot of our beneficiaries don't have English as their first language […] it will take some time for me to translate the content, or even if I do translate immediately using Google Translate, the context […] is lost because it's a generic translation. I think more efforts can be put into translation as a resource for partners in general.” (KII, RP) Internews respondents noted that it is difficult for them to address language barriers from their role, as they are working in so many different countries. It is easiest to ask RPs and LPs to handle translations and cultural adaptations to their own contexts. However, there must be sufficient guidance, resources, and time built into the project for partners to handle translation work. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 33 Internews respondents also noted that reliance on common regional languages such as Russian in CA tends to skew participation towards organizations that work in capital cities to the detriment of local organizations that work in more rural areas. 4.3 ADVOCACY WITH GOVERNMENT AND MULTILATERAL INSTITUTIONS As part of its work under Objective 2, the GIF consortium supports activities to develop and strengthen partners’ engagement with government and multilateral institutions on issues of IF. This is done through legal research conducted by RPs and LPs, ICNL’s legal and policy interventions, RDR adaptations, GNI consultations, and other activities. Advocacy efforts with government and multilateral institutions is a challenging area, as country contexts greatly the scope of such advocacy. Advocacy with governments and multilaterals was a new experience for many LPs, and advocacy capacities of GIF partners differ substantially. Yet, there are examples of successful advocacy, where LPs supported by GIF were able to review adverse legal initiatives and urge governments to withdraw or substantially revise problematic aspects. There are also barriers for successful public advocacy related to unfavorable political systems, LPs’ limited capacity for advocacy, and GIF’s operational challenges. There was a relatively low baseline of involvement in advocacy with government and multilaterals. Quantitative data from the web-survey of LPs suggested that prior to joining GIF, 40% (19) of respondents had been either barely or not at all involved in government advocacy and 54% (26) of respondents did not have much experience with advocacy with multilateral institutions (Exhibit 18). Exhibit 18 Advocacy Involvement with Government and Multilateral Organizations and Multistakeholder Forums Prior to GIF Note: N=48. Based on the questions “Before becoming part of the GIF project, have you been involved in advocacy work with government on issues of technology and human rights?” and “Before becoming part of the GIF project, have you been involved in advocacy work with multilateral organizations and multistakeholder forums (such as IGF, ICANN, ITU, etc.) on issues of technology and human rights?” 4.3.1 APPROACHES FOR ADVOCACY WITH GOVERNMENT AND MULTILATERAL INSTITUTIONS EQ: What approaches did GIF take to increase and improve civil society’s engagement with governments and multilateral institutions on issues of technology and human rights? The GIF project supported partners to engage with governments and multilateral institutions by helping them conduct legal analyses, supporting their participation in international forums, providing training and technical assistance on how to arrange and conduct meetings with government officials, and supporting dissemination of advocacy materials and legal analyses. According to performance 25% 25% 15% 29% 25% 23% 29% 15% 6% 8% Government Multilateral Organizations and Forums Not involved at all Barely involved Somewhat involved Very involved Don't know/Missing USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 34 monitoring data reported by Internews, GIF supported total of 68 advocacy interventions12 as of FY2023 Q2. The web-survey of LPs showed that 27% (13) of respondents received GIF support for advocacy with both government and/or multilateral organizations and forums, 23% (11) were supported for advocacy with multilateral institutions only, and 6% (three) reported receiving support for advocacy with government only (Exhibit 19).13 We note that 27% (13) of respondents did not receive GIF support for advocacy, and another 17% (8) were either unsure or did not respond. Exhibit 19 Organizations Receiving GIF Support for Advocacy with Government and/or Multilateral Institutions Note: N=48. Based on the question “Has GIF provided you with support to conduct work to improve civil society’s engagement with the types of stakeholders listed below on issues of technology and human rights?” Among survey respondents who received advocacy support from GIF, most common types of support were review, analysis, and provision of input in government’s legislative initiatives (e.g., bills, draft laws, and other regulatory documents); participation in global forums such as IGF or RightsCon; meetings, round tables, and consultations with government stakeholders, policymakers, and multilateral organizations and multistakeholder forums on issues of technology and human rights; and dissemination of research findings supported by GIF directly to government stakeholders and multilateral organizations and multistakeholder forums (for example via mail or email) (Exhibit 20). Each of these options was selected by about 30% (15) of respondents. The least common advocacy support was assistance for liaising with international or regional standard-setting bodies (e.g., International Telecommunication Union (ITU) and Internet Corporation for Assigned Names and Numbers (ICANN)). 12 This statistic is not disaggregated by advocacy with governments and multilateral institutions vs. advocacy with the private sector. 13 The survey data shows that among 13 respondents who suggested that their organization engaged in digital security only, four said they received GIF support for advocacy with both government and multilateral organizations and two received GIF support for advocacy with multilateral organizations. Government 6% Multilateral organizations and forums 23% Both 27% Neither 27% Don't know/Missing 17% USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 35 Exhibit 20 GIF-supported Advocacy Engagement with Government and Multilateral Organizations and Multistakeholder Forums (% of total sample) Note: N=27. Based on the question “What engagement strategies did your advocacy work with government and/or multilateral organizations and multistakeholder forums employ as part of GIF? Select all that apply.” Legal Analysis Global partners provided extensive technical support and coaching for partners to conduct legal analyses. In particular, ICNL focused on legal analyses with LPs. As an example, an RP described their work with ICNL: “ICNL helped us to develop something related to AI and [related] laws, strategies, and policies in this region […] we've been able to map existing solutions […] and then provide some recommendations for the governments and policymakers on how to improve and develop these policies.” (KII, RP) In general, partners greatly appreciated the work with ICNL. It not only furthered partners’ advocacy goals but also strengthened their capacities to engage with governments and multilaterals in the future. While ICNL was the primary partner for legal analyses, as a part of their Research and Advocacy Toolkit, RDR also developed a tool to support jurisdictional analysis and additions to draft laws for LPs. This comprised a document that identified how specific indicators were influenced by local contexts and was meant to serve as a framework to identify narratives in research report that would enable RDR to provide further assistance (GIF Internews Quarterly Report FY1 Q3, Appendix C). Forums A19’s fellowship and mentorship program was instrumental in helping LPs engage with multilateral institutions. The GIF project was not organized to facilitate a lot of direct engagement with multilateral institutions, more so with governments, but the financing of fellows to participate in global forums was one of the activities that did so. It must be noted here that the A19 fellowship was relatively small in scope, with about eight fellows per year, which limited its impacts. However, it gave LPs opportunities to engage with international advocacy in ways that would have been out of their reach otherwise. A global partner spoke to these opportunities: “A lot of these forums occur in 31% 29% 29% 29% 8% Review, analysis, and provision of input in government’s legislative initiatives Participation in global forums such as the IGF or RightsCon Meetings, round tables, and consultations with government stakeholders, policy makers, and multilateral organizations and multistakeholder forums on issues of technology and human rights Dissemination of research findings supported by GIF directly to government stakeholders and multilateral organizations and multistakeholder forums Liaising with international or regional standard-setting bodies (e.g., ITU and ICANN) USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 36 Europe and North America…It's much harder for folks from Latin America and sub-Saharan Africa14 to get visas to go to these places to attend these forums. So doing all those logistics, being to provide financial support like in the form of monthly stipends--because it's a lot of work to be able to engage—[then allowed us to] be able to connect them with different networks.” (KII, Global Partner) Fellowships and mentorships are vital in strengthening the digital rights realm, which is an overall objective of GIF. A19’s provision of paid opportunities for RPs and LPs enabled engagement of fellows at networking events at bodies including the Internet Engineering Taks Force (IETF), Institute of Electrical and Electronics Engineers (IEEE), ICANN, and ITU, to meaningfully improve connections. These fellows came from Bolivia, Lesotho, DRC, and the Philippines, amongst other countries, and worked with their communities on issues including cybersecurity, freedom of expression, and censorship (GIF Internews Quarterly Report FY1 Q3, Appendix A) (GIF Internews Quarterly Report FY2021 Q2, Appendix 3) An RP explained how their participation in these forums had helped them lay the groundwork for more engagement in the future: “During the fellowship I was able to establish my starting relationships with key agencies. […] I guess it's not necessarily a quantifiable impact but it is useful for us […] to have that established familiarity compared to before when we didn't even interface with them.” (KII, RP) Meetings Government advocacy efforts oftentimes start by bringing different parties together to discuss IF issues. KII respondents mentioned round tables and working meetings were organized by Internews and RPs to bring CSOs and policymakers (e.g., parliament members) together. For example, in 2022, GIF organized a panel ("More Money, More Problems") addressing the gap between the funding of digital rights and digital security programs by governments and the increase of threats and new issues. This panel brought together USAID, US Bureau of Democracy, Human Rights, and Labor (DRL), Swedish International Development Cooperation Agency (SIDA), Open Technology Fund (OTF), and Internews to collaboratively inquire about the gap between the funding and the threats that go unaddressed. (GIF Internews FY2022 Q2). While many meetings organized by GIF partners were with representatives from government, ICNL also supported partners to engage with the UN by responding to the UN Special Rapporteur’s calls for submission. (GIF Internews FY2023 Q1). Capacity Support and Dissemination Internews and RPs provided organizational capacity support and technical support to LPs, such as drafting legal amendments, proofreading blog posts, and creating advocacy campaign materials. Thereafter, they helped disseminate advocacy materials and legal analyses. For example, an LP dealing with a repressive government was not able to post blog posts on their own social media for fear of retribution but was able to have their blog posted by RPs instead. In addition, while the GIF website has not officially launched yet, it has a potential to serve as a repository for legal analyses and other advocacy materials to improve future initiatives. This dissemination can be very helpful when it is not possible to meet directly with representatives of governments and multilateral institutions. A local partner in E&E explained, “During our activity Members of Parliament did not participate in our events, but we shared our studies with them, especially our recommendations on information security.” (KI, LP, E&E) 14 GIF partners in Asia also struggled to procure visas, although this was not highlighted by this global partner. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 37 4.3.2 EFFECTIVENESS OF APPROACHES EQ: How effective were those approaches in increasing constructive dialogue between civil society, governments, and multilateral institutions on these topics? Based on the web-survey data, most respondents who received some support from GIF thought it had improved their ability to engage with government, multilateral organizations, and multistakeholder forums by either a lot (29%, 14 respondents) or somewhat (21%, 10 respondents) (Exhibit 21). Six percent (three respondents) believed the improvement was little. Exhibit 21 Improved Engagement with Government and Multilateral Organizations and Multistakeholder Forums due to GIF Support (% of total sample) Note: N=48. Based on the question “To what extent did being a part of GIF improve your ability to engage with government and multilateral organizations and multistakeholder forums?” Despite many LPs finding that their engagement with governments and multilaterals had improved, respondents in KIIs mentioned the effectiveness of approaches to increase dialogue between civil society, governments, and multilaterals depends on the country context. Governments that restrict civil society freedoms are more difficult and dangerous for LPs and RPs to engage with. This is where the broader GIF network has proved instrumental in conducting advocacy. A respondent from Internews described that some LPs preferred to do advocacy through global partners rather than completing this advocacy themselves since in some countries, the IF landscape is restricted, and it may be dangerous to speak openly. LPs echoed this sentiment, appreciating the support from organizations such as EngageMedia and Internews to conduct advocacy in certain contexts: “We have got a lot of support from EngageMedia and Internews in terms of conducting the advocacy work […] because in the context of [redacted, country name] right now the situation has become more intense.” (KII, LP, SEA) Some respondents in KIIs found that the effectiveness of approaches was difficult to discern and needed further investigation, as this was a less developed aspect of GIF programming. Further, it can take years (often beyond the life of USAID-funded projects) to see results of advocacy: “I have a problem with already marking off what we shouldn't do because I don't think that we've done enough and exhausted enough of the different ways to do this because we just haven't worked on advocacy around internet governance policy issues for that long, I mean, we were doing it a little bit in the previous activity that we had before, but just not enough. It wasn't enough years to really see how well that works” (KII, USAID). Further, a respondent from Internews noted that the types of advocacy activities applied were locally led and contingent on what each of the LPs wanted to do, making advocacy less cohesive. 6% 21% 29% 44% A little Somewhat A lot N/A USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 38 Meetings, Roundtables, and Consultations Respondents found the facilitation of roundtables and working meetings by GIF to be useful in improving their relationships with governments and multilateral institutions. One respondent cited the example of a retreat where GIF brought together several civil society members to define strategic plans for their organization and collectively decide upon a plan of action for advocacy initiatives with the government. Another respondent provided an example where roundtable discussions had proven effective in engaging high ranking government actors to combat a government-sponsored initiative to restrict IF: “We provided a big roundtable [discussion]. It was close work, meeting with our partners where we made a strategy on how to struggle with these initiatives. One of the best achievements was that we made an agreement between the President’s administration which was the main initiator of these legislative initiatives and dozens of parliament [members] participated” (KII, LP, CA) RPs and LPs were able to engage with the UN Special Rapporteur by responding to calls for submissions and contributing to a report focused on freedom of expression. Internews brought together GIF partners to provide input into a report the UN Special Rapporteur was creating, resulting in four citations of GIF-funded research. One respondent noted that Internews was able to create a direct relationship between UN bodies and LPs, but that these activities were newer: “We were able to connect them [LPs] with different UN agencies like UN Special Reporter and others in order to report to them if any digital rights violations are happening on local level […] it's crucial to support civil society in these activities to continue that because they are just starting” (KII, Internews). Legal Analysis Many governmental bodies were amenable to discussions on proposed bills, laws, and briefs regarding internet freedom issues when presented with comprehensive research supported by global and regional partners. ICNL was particularly supportive in the legal advice realm, undertaking legal analyses of laws impacting internet freedom in several countries. RPs also developed an IF curriculum to train university staff and students studying political science, journalism, engineering, and law (Internews GIF Quarterly Report FY 2021 Q2). ICNL’s support for rapid turnaround of legal analysis and assessments was critical for enabling civil society to interact with governments. For example, in 2023 ICNL received an urgent request from civil society in Bangladesh to review a newly revised draft Data Protection Act. Government officials stated that they had revised the Act to respond to some of ICNL’s recommendations and requested quick turnaround feedback from civil society. ICNL found that the Act kept in place some of the most problematic aspects of the legislation. ICNL then shared their analysis with civil society (GIF Internews Quarterly Report FY 2023 Q2). Some countries opened up public comment periods where ICNL’s analysis also proved to be helpful in enabling civil society to react to laws that may impact IF. “Given that these issues are new for civil society in the DRC, partners have said that ICNL’s review of the Code will be useful while the public comment period is open.” (GIF Internews Quarterly Report FY 2023 Q2) In 2021, ICNL, Internews, and CIPESA collaborated on a revision to the Cybercrime Law and Online Content Regulation, based on a solicitation for suggested revisions by the Government of Tanzania. CIPESA built upon the existing legal analyses provided by ICNL. Though the final submission was delayed due to COVID-19, RPs strongly appreciated the collaborative approach and the sharing of expertise, stating that it was vital in achieving desired outcomes for LPs “But that [collaboration] shows the power of that analysis once the local organizations know how to react, know how to intervene, and they will have an impact as opposed to if they have no idea how to start engaging in that process.” (KII, RP). USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 39 Global Connections and Dissemination Global forums were useful in bringing together LPs that advocated on similar issues. Advocacy was more effective when there were multiple LPs engaging on a particular issue, therefore, opportunities for networking and dissemination of their work was key to strengthening their efforts. GIF facilitated technical knowledge sharing by bringing together actors to provide resources to civil society with the objective of improving advocacy strategies, proposals, and other material for dissemination. These included meetings with legal professionals to improve the understanding of technical aspects of legislation for civil society members to craft tailored responses and recommendations. Respondents in KIIs also mentioned that advocacy with governments is strengthened by bringing in international organizations and other members of the GIF network: • Leveraging international standard setting bodies can be helpful in increasing RP and LP access to the government. For example, in Georgia, one RP described that leveraging international commitments to the EU was an effective means to convince policymakers to act in a particular way (KII, RP). • Disseminating findings throughout the GIF network was highly valuable for bolstering civil society efforts: “that was the biggest impact of this approach that it was literally all the relevant organizations including media, civil society organizations, think tanks, law related organizations that were actively advocating and reaching out to the organizations and [to] policymakers in Brussels….” (KII, RP) • The private sector can be a valued partner if LPs and RPs are able to establish trust and collaboration. A LP from CA found that policymakers in their region were more likely to pay attention to what the business sector had to say. Barriers Engagement with governments and multilaterals is complicated by many variables including political and regional factors which may impede RPs’ and LPs’ ability to directly engage with their governments, and lead them to further require support from global partners: “Sometimes despite your openness and your willingness to involve different institutions to try to make them accountable for their work, sometimes, there's radio silence…Those institution should be responsible to the citizens, to the people, and the media, but it doesn't work that way in this region and it can be frustrating.” (KII, RP) A respondent from USAID acknowledged that while this advocacy component was highly important, it was also a challenging area with many factors influencing implementation. 4.3.3 INFLUENCE ON GOVERNMENT AND MULTILATERAL INSTITUTIONS EQ: To what extent did these approaches influence government and multilateral institutions’ actions, policies, and/or practices? Data from desk review and KIIs suggested that GIF support did not only strengthen partners’ organizational capacity for advocacy, but also resulted in positive changes in government and multilaterals policies and practices. In several countries, RPs and LPs supported by global partners conducted legal analyses of legislative initiatives that would potentially restrict IF and freedom of speech and make CSOs more vulnerable. The RPs and LPs then reached out directly to legislators to organize working meetings and roundtables to provide their critical input and revisit problematic areas of the initiatives. GIF partners initiated advocacy campaigns in media to support their effort to influence policymakers and USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 40 raise public awareness of negative consequences if these problematic legislative initiatives are passed. The ET identified at least eight countries15 where GIF (ICNL)-supported legal analysis and subsequent advocacy efforts influenced government policy-making processes. One of the success stories recorded in the Internews reporting and a KII was the case of Serbia, where CSOs supported by GIF effectively pushed back a new draft law on internal affairs that included provision for mass video surveillance accompanied biometric facial recognition: “The response from CSOs was loud, swift, and effective – in 16 days, six partner organizations had more than 92 media appearances, covering various media formats The Government of the Republic of Serbia published a statement saying that the Prime Minister of Serbia, in consultation and agreement with the Minister of Internal Affairs, decided to withdraw the Draft Law on Internal Affairs from the adoption procedure.” (GIF Internews Quarterly Report FY2023 Q1) Similarly, LPs with help from ICNL were able to conduct legal analysis and successfully advocate with government ministries to revise a problematic data protection law. Yet, it was not without impediments from policymakers who provided very little time for comments from CSOs to be submitted: “…most recently we did a very quick legal analysis of the third draft of the data protection law. We had to do it very quickly because the government gave CSO partners in Bangladesh just five days before they could meet and provide comments. We did that and then based on our legal analysis, our CSO partners again successfully pushed for the ministry to take that bill off of the legislative agenda…” (KII, Global Partner) Even when achieving impact on government, GIF partners oftentimes remained cautious knowing that advocacy with government is continuous work, since governments may re-try to initiate adverse legislative changes at a later point or through a different legal route: • “An example from Kazakhstan where we supported two legal analyses of different versions of amendments to the cyberbullying [law] that were allegedly aimed at combating cyberbullying by broadly restricting freedom of expression online… We intervened and made some substantive changes to the amendments…. Unfortunately, then the ministry that was responsible for enforcement[…] passed bylaws to those amendments that put back in place some of the negative provisions […]. So there's always like some steps forward, some steps back” (KII, Global Partner) • “For example, in Armenia, we recently had the legislative initiative that was … proposed to give the opportunity to block websites during martial law and it it's not really in line with the international standards…This initiative has been stalled, so now it's not being considered, at least for the time being.” (KII, RP) In the near future, there will likely be more examples of GIF-supported advocacy influencing governments, since it takes time for this type of impact to be realized given how long certain legislative processes and government bureaucracy take. The ET noted several examples of such “work in progress” that may yield positive results, including cooperation with data protection authorities in Georgia on developing amendments to legislation around electronic communications and data protection. In the Central African Republic (CAR), LP supported by GIF is currently advocating with government against a proposed tax on Internet connection that would limit Internet access for many due to prohibitively high cost. 15 These include Kazakhstan, Kyrgyzstan, Georgia, Armenia, Serbia, Iraq, Bangladesh, and CAR. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 41 Regarding influence of GIF-supported advocacy on multilateral institutions, GIF helped RPs and LPs bring regional- and country-level issues around IF to attention of specialized multilateral organizations and multistakeholder forums. For example, one RP mentioned applying together with other experts to raise IF issues in Kazakhstan, Tajikistan, and other regional countries on a global level in forums such as IGF 2022 (KII, RP). 4.3.4 OPPORTUNITIES FOR ADVOCACY WITH GOVERNMENT AND MULTILATERAL INSTITUTIONS EQ: What opportunities/gaps exist to advance work between civil society and governments and multilateral institutions on issues of technology and human rights based on GIF’s work? In an open-ended survey question, LPs shared their views on barriers and challenges that prevented them from advocating with government and multilateral organizations and multistakeholder forums. Over a third of respondents (38%, or 18 respondents) stated the biggest barrier is the current political system and legal regime. This ranges from the lack of political will to state-level harassment and dangers of doing digital rights advocacy. Almost 30% (14 respondents) mentioned the lack of technical or human resources to do this type of advocacy, while 17% (8 respondents) believed insufficient funding for advocacy initiatives was an issue. Findings from KIIs echoed but also complemented the above, as partners discussed (un)responsiveness of governments and multilateral organizations, limited technical knowledge among some LPs, difficult administrative processes, and short project timelines. Mismatched levels of technical knowledge of LPs is a factor limiting effectiveness of information exchange amongst partners and progress in capacity building, which in turn affects the overall quality of advocacy campaigns. Across regions, partners noted that GIF’s administrative processes and limiting timelines (too short implementing periods) created hurdles for effective advocacy, particularly considering that the results from advocacy can take a longer period of time take hold. Streamlining administrative procedures and extending project timelines would create opportunities for more effective efforts: “...The duration of work should be considered [..] for future. Advocacy is a very lengthy process that takes at least a year or longer to really work well. And when you're asked on a quick for timeline, it's just not working as well as it could have been. (KII, LP, E&E) Other opportunities for improving sustainability of existing GIF advocacy include: • Improving institutional backing of civil society actors to support sustainable capacity building and organizational growth. While other sources of support such as funding and networking assistances are appreciated, flexibility in resources provided was highly useful: “…it's not only about buying a ticket and being at the UN, but it's also about all the preparation that we need to do between sessions that we need to do to engage our partners in these processes”. (KII, RP • Facilitating connections between small LPs and civil society actors to form regional coalitions. Respondents suggested that these networks would have localized knowledge that to further support awareness-raising and advocacy activities: “Organizations feel much better if they're working with a similar minded organization locally and in terms of doing advocacy. For instance, it makes more synergies or makes the case stronger.” (KII, RP) USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 42 4.4 ADVOCACY WITH PRIVATE SECTOR Led by global partners RDR and GNI, GIF connected RPs and LPs to social media and telecommunications companies by engaging them in meetings, workshops, and roundtable discussions, and completed RDR research. For some RPs and LPs, advocacy with the private sector is somewhat new, but engagement in GIF-supported activities helped them strengthen their relationships with businesses. As these advocacy activities are starting up in many regions, it may be too soon to tell whether companies are sufficiently responsive to GIF-supported advocacy. Yet, there are a few examples of companies starting to implement enhanced measures to protect users’ digital rights in response to GIF-supported advocacy. There are still many opportunities for furthering this work and testing new methods of collaboration, particularly in countries with a more challenging IF landscapes. There are also opportunities for more capacity building and facilitating direct access of RPs and LPs to global tech companies. 4.4.1 APPROACHES FOR ADVOCACY WITH PRIVATE SECTOR EQ: What approaches did GIF take to increase and improve civil society’s engagement with the private sector on issues of technology and human rights? According to the web-survey of LPs, just over a half of respondents (52%, 25 respondents) had been involved in advocacy work with private companies on issues of technology and human rights prior to GIF (Exhibit 22). GIF provided support for advocacy work with private companies to 50% (24) of respondents, while 27% (13) of respondents did not receive such support. Most of those who obtained GIF support for advocacy with the private sector had already been involved in such work before GIF (16 respondents). Exhibit 22 Advocacy with Private Sector on Issues of Technology and Human Rights: Involvement prior to GIF and Receiving GIF Support for Such Advocacy Note: N=48. Based on the questions “Before becoming part of the GIF project, have you been involved in advocacy work with private companies on issues of technology and human rights?” and “Has GIF provided you with support to conduct work to improve civil society’s engagement with the private sector on issues of technology and human rights?” Similarly, during KIIs, RPs mentioned receiving support to implement private sector advocacy activities, but some mentioned that they had limited previous experience in this type of work. “They [GIF] had these workshops to share their experience with, for example, reaching out to private companies and how you can design your programs in order to engage these new actors, and this is something that we haven't done in our country before…” (KII, RP) Regarding specific activities provided by GIF to support LPs’ advocacy with private companies, training on engagement strategies was the most common activity selected by 23% (11) survey respondents, followed by meeting events with representatives from telecommunication companies 52% 50% 35% 27% 13% 23% Involved prior to GIF Received GIF support Yes No Don’t know/Missing USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 43 and social media businesses, and support for information campaigns with business (each selected by 21%, 10 respondents) (Exhibit 23). Exhibit 23 GIF-supported Advocacy Engagement with Private Sector (% of total sample) Note: N=22. Based on the question “What approaches or strategies did this work [advocacy with private companies on issues of technology and human rights] employ? Select all that apply.” During KIIs and through reporting, partners provided several examples of private sector advocacy activities, including trainings/workshops, meetings, roundtables, advocacy events, and information campaigns, similarly highlighted in the LP survey. Global partner GNI hosted trainings and webinars for digital rights activists on corporate accountability and protection of digital rights by the private sector (Quarterly Report FY 2023 Q2). In 2021-2022, RDR led workshop for RPs on how to engage the private sector to further advance corporate accountability (GIF Internews Quarterly Report FY 2021 Q3). The second workshop was co-hosted by GNI and brought in 10 researchers and advocates from various CSOs (GIF Internews Quarterly Report FY 2022 Q1). One RP described high levels of support in the planning stages of advocacy that then became more limited down the line: “We did have a preliminary meeting with Ranking Digital Rights to discuss what we're thinking and we 23% 21% 21% 21% 13% 10% 10% Training provided by GIF partner organizations to civil society actors on engagement strategies with the private sector Meetings and roundtables, and other advocacy events with representatives of telecommunication companies Meetings, roundtables, workshops, webinars, and other advocacy events with representatives of social media companies Support to organize information campaigns for companies operating in the technological and social media space Facilitating introductions and connections between civil society and the private sector Research related to the RDR methodology Research other than RDR-related research RPs and LPs Use RDR Research RPs and LPs typically use RDR research in a sequence of steps, as described in several KIIs: 1) Adapt the RDR methodology to the country context 2) Conduct RDR research 3) Share findings with the private sector through meetings or roundtables 4) Publish the results in blogs and briefs 5) Periodically follow up and attempt to get companies’ commitment to improve upon identified issues through advocacy campaigns, meetings, workshops, and other engagement methods USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 44 sort of agreed on a strategy…like a pre-engagement meeting…. But generally we had to tap into our other networks. [to complete RDR research]" (KII, RP). GIF hosted online digital rights and IF advocacy camps with private sector actors and civil society to foster cooperation in several regions, including CA, ESA, MENA, and WCA (Appendix 14 GIF Year 1 Activity Completion Status FY 2021). Also, Internews organized roundtable discussions with the business sector to present and promote GIF research and deliver recommendations on IF issues. One RP noted that multiple roundtables and information campaigns would take place in April 2023 in CA. An important line of work is RDR’s corporate accountability research and related training provided to RPs and LPs (GIF Internews Quarterly Report FY 2023 Q1). While surveyed LPs reported engaging in RDR research to a lesser extent, in KIIs, GIF RPs and LPs emphasized the work they had done under RDR. RDR research is centered around the RDR Index Methodology, a standard-setting tool aimed at encouraging companies to abide by international principles and standards safeguarding freedom of expression and privacy.16 The RDR methodology is used as an entry point to facilitate RP and LP advocacy with the private sector including with telecommunications companies, tech companies (e.g., Google, Zoom, and Wikimedia), and social media platforms (e.g., the Meta Oversight Board/Facebook, Twitter, and Telegram). RDR provided training and support to LPs conducting this research: “this was very good experience of working with international experts with their methodology and they were very supportive during this study …for me this experience was very valuable…” (KII, LP, E&E). Once this research was complete, GNI helped partners either engage publicly or privately with the company of focus, while RDR assisted RPs and LPs in developing policy briefs based on research findings and promoted partners’ projects through blog posts on RDR’s website (GIF Internews Annual Work Plan Year 2, 2021). As of April 2023, GIF had reviewed over 70 local companies in more than 30 countries across the Balkans, Central Asia, Latin America and the Caribbean, and South and Southeast Asia (KII, Internews; GIF Internews Quarterly Report FY 2023 Q1). 4.4.2 EFFECTIVENESS OF APPROACHES EQ: How effective were those approaches in increasing constructive dialogue between civil society and the private sector on these topics? GIF saw some gains in relationship building between RPs and LPs and the private sector. Twenty-one percent (10 respondents) of LPs responding to the web-survey reported some improvement in their ability to engage with the private sector due to GIF, whereas 15% (seven respondents) believed GIF improved such an engagement a lot (Exhibit 24). At the same time, eight percent (four respondents) saw little to no improvement in their ability to engage with private companies. 16 Ranking Digital Rights Website, https://rankingdigitalrights.org/methodology-development/ USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 45 Exhibit 24 Improved Engagement with Private Sector due to GIF Support (% of total sample) Note: N=48. Based on the question “To what extent did being a part of GIF improve your ability to engage with the private sector?” GIF support for advocacy with the private sector has been especially useful in cases where LPs did not have prior opportunities for this advocacy. Several respondents described that it was many partners’ first time engaging the private sector. The fact that they are now able to effectively apply the RDR methodology and then use it as a tool for starting a cooperative relationship with the private sector was a huge step forward. RPs and LPs built constructive dialogue with the private sector in a variety of ways. For example, in 2021, GNI hosted a two-day workshop in conjunction with RDR. Participants met with representatives from telecommunications companies, who shared their perspectives and input on working with civil society (GIF Internews Quarterly Report FY2021 Q1). In 2022, RDR hosted a workshop with civil society actors and tech companies. This workshop was a signal of positive relationship building between these actors. A representative from Google offered to put partners in touch with the local policy team in the country to address issues raised during the workshop directly. This was promising because Google tends to be elusive about providing direct lines of contact to CSOs (GIF Internews Quarterly Report FY 2022 Q1). In 2023, an LP in CA engaged with two digital security specialists from the private sector to conduct trainings, digital security assessments, and organizational support to local beneficiaries (GIF Internews Quarterly Report FY 2023 Q1). Using a research-based approach applied through RDR was effective in bringing the private sector to the table, as they could not dispute or ignore data-backed claims made by partners. “But when you have the data […] it becomes incredibly difficult for them to deny so those reports and scorecards seem to be something that has had a positive impact on building the capacity of organizations to advocate with companies.” (KII, Global Partner) KII respondents noted levels of success can differ regionally and found high success in advocating with telecommunication sector in Africa: “Specifically in the [redacted, country in Africa] context for a very long time, the telecommunications companies have tried to distance themselves from the internet governance conversations[...] Because of the GIF project, we've been able to really cultivate a good relationship with the telecom companies in [redacted, country name].” (KII, Global Partner) Some respondents noted that the ability to construct dialogue depended on how committed or effective the RPs and LPs were, as well as the type of private sector the partner is engaging with. 4% 4% 21% 15% 6% 50% Not at all A little Somewhat A lot Don't Know N/A USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 46 Further, one respondent mentioned that applying the RDR methodology could take a lot of guidance and oversight due to its complexity and learning curve: “There's definitely some level of a steep learning curve in terms of using the standards because of the granularity of the questions [...] the explanations and the research guidance, it's pretty extensive…” (KII, Global Partner). 4.4.3 INFLUENCE ON PRIVATE SECTOR EQ: To what extent did these approaches influence private sector actions, policies, and/or practices? In the web-survey of LPs, 19% (nine) of respondents thought it was too soon to see the impact of GIF-supported advocacy on private companies’ policies and practices, whereas 10% (five respondents) said the changes had been observed sometimes (Exhibit 25). Exhibit 25 Changes in Private Companies’ Practices due to Advocacy Supported by GIF (% of total sample) Note: N=48. Based on the question “Thinking of your work supported by GIF to improve civil society’s engagement with the private sector on issues of technology and human rights, did it bring changes in any companies' policies and practices?” One regional partner noted in a KII that RDR work got off the ground “first in Africa, but they're [RDR] starting it up…It's in process in a lot of the other regions and so I don't think we've seen the results of that quite yet” (KII, RP). At the same time, GIF partners provided several anecdotes in KIIs and reports describing the changes they had observed as a result of their advocacy with the private sector: • GIF-supported advocacy led by RPs and LPs with private business brought by progress in Lesotho. Advocacy meetings with Vodafone Group, Vodacom Group, and Vodacom Lesotho resulted in the company developing and publishing a new privacy portal on their website. The company also hired a local specialist to improve the Terms and Conditions in the local language (KII, Global Partner; GIF Internews Quarterly Report FY 2023 Q1). • “We had an investigation on revenge porn shared through different Telegram groups in [redacted, country name]…After the research was published, it was very well received and it was republished widely and as a result, 13 groups out of the 16 were shut down. And it was a breakthrough because I know that some of the other organizations that were in touch with Telegram before on different issues have never received response.” (KII, RP) • “We use the RDR indicators to assess the telcos, two major telcos per country in focus countries in South and Southeast Asia…We were able to engage them as to the results of the research […] 4% 6% 10% 6% 19% 4% 50% Never Rarely Sometimes Most of the time Too soon to tell Don't Know N/A USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 47 and they actually changed their policies. They change specific provisions of their privacy policy, their user policy after the release of the research” (KII, RP) As private sector advocacy activities are newly implemented in many regions, there remains a question as to whether civil society will be able to keep tech companies accountable. While many partners have historically completed advocacy with the government, the private sector is somewhat of an unknown entity with its own norms and procedures. Governments are generally accountable to their constituents and may be more influenced by grassroots advocacy, but the private sector is not necessarily. This could undermine the effect RDR research has on the private sector, despite any GIF-supported advocacy efforts. 4.4.4 OPPORTUNITIES FOR ADVOCACY WITH PRIVATE SECTOR EQ: What opportunities/gaps exist to advance work between civil society and the private sector on issues of technology and human rights based on GIF’s work? While GIF achieved some gains in enhancing interactions between civil society and the private sector and thus saw several examples of advancements in protection of human rights, many partners struggled to collaborate with the private sector. In an open-ended survey question, LPs reflected on barriers and challenges that prevent local organizations from advocating with the private sector on issues of technology and human rights. The most common cited barrier is the lack of responsiveness and/or interest on the part of companies to engage with civil society, and lack of opportunity for engagement, including fear on the part of tech companies to engage in dialogue (42%, or 20 respondents). A couple LPs in different regions also listed a lack of trust they felt with the private sector making it “not worth the effort on unproductive relationships.” The lack of capacity, training and tools, and funding was also cited by LPs across regions. These findings are consistent with KIIs. Findings from KIIs also suggest that willingness of the private sector to engage was an overall challenge experienced across regions and was somewhat perceived to be disingenuous. Some partners believed that the private sector only would interact with them to protect the company’s image and not because they intended to implement any of their recommendations, or transparently share information on what they’re doing to protect digital rights within their platforms. When companies did want to engage, some opted for “closed door” conversations about RDR results, not seeing the benefit to publicizing these results or sharing any steps they are taking to protect digital rights with the broader public: “To be quite honest, we often don't see a lot of willingness from the private tech sector in this area and many of our partners, both in GIF and outside of GIF, find the engagements with the private sector that do happen tend to be rather one sided and fairly disingenuous.” (KII, Global Partner). Partners particularly struggled to collaborate with the private sector in countries with oppressive or corrupt governments. In countries where the government restricts IF, telecommunications companies are often the executor of these acts. For example, telecommunications companies are often responsible for carrying out internet shutdowns. This leads to tension between CSOs and these companies: “Governments were asking telecommunications companies to cut mobile data money automatically in every telephone. If you have five dollars’ worth of mobile data, we take two dollars automatically. Each time you reload the data, we cut you off the day before. It created a lot of demonstrations by the population. It is why companies in the private sector are very hesitant […] and do not receive our pleas easily.” (KII, LP, WCA) USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 48 There is an opportunity for RPs and LPs to benefit from INGO contacts to help counter the unwillingness to transparently interact with the private sector. Relatedly, RPs and LPs struggle with getting direct access to global tech companies which could also be facilitated by INGOs. “It would be also helpful if we can get a more direct access and communication with representatives of the big tech companies and platforms and that could be facilitated by USAID or Internews.” (KII, RP) One USAID respondent noted a possibility to engage with a wider diversity of tech companies, including ones that are small and emerging since they may have a greater stake in IF issues regionally and nationally: “And also small and emerging tech companies and tech startups […] that care about what happens in a national context or regional context […] I think could be mobilized and engaged much more effectively and I think there's more of an opportunity to see dividends.” (KII, USAID) It was many LPs’ first time interacting with private sector companies in such a direct manner, and some struggled to determine the best way to engage with these kinds of organizations. This creates several opportunities to further explore how partners can more effectively interact with these entities, as described below: • There is an opportunity to delve deeper into testing various approaches to engaging the private sector to determine which of these are most effective in achieving results in each context. One respondent described addressing this gap by planning more round table discussions: “In the past these civil society organizations used to do advocacy at the government level. They have very limited understanding and very limited experience how to engage with business sector […] therefore we are planning to do a series of round table discussions in each country and to invite business sector…” (KII, Internews). • There is also an opportunity to identify the right spaces for civil society to engage with the private sector and to ensure LPs are represented in the spaces where companies interact: “One keeps wondering about the specific spaces where CSOs […] should be engaging with the tech sector. … And I think identifying those places and then understanding […] who needs to be there from the local or regional partners.” (KII, Global Partner) • Further, respondents noted that engagement with the private sector required a specialized set of skills that needed to be learned to advocate effectively, partially due to the unique internal politics that must be navigated. This warrants an opportunity for further training and learning for LPs. 4.5 CROSS-CUTTING THEMES The ET collected useful data on aspects of GIF that do directly fit within the four focus areas of this evaluation. Partners reported receiving GIF support for digital security (GIF’s Objective I), including training, assistance in developing digital security polices for organizations, publishing practical information materials on cybersecurity, and conducting SAFETAG audits. LPs expressed interest in receiving more such support. The ET also noted several aspects of GIF that are likely to be sustained past the activity lifetime, including the research and knowledge products developed with GIF support, GIF-supported capacity strengthening that will continue to benefit partner organizations, and the networks connections that will lead to future partnerships outside of GIF. Our findings also highlight the varied impact of COVID19 on GIF activities, with some hindered but most adapted to remote work and digital platforms, though moving forward, in-person networking is preferred by LPs. Regarding terminology of digital rights, digital security, and IF, most respondents were comfortable using it. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 49 4.5.1 DIGITAL SECURITY Digital security, the focus of GIF’s Objective 1, was not one of the four thematic areas of this evaluation. However, data collected in the evaluation through discussions around network development and localization incidentally revealed some useful insights on digital security. Useful Support for Digital Security Many LPs expressed their appreciation for the digital security initiatives under GIF. While some LPs praised GIF support in this area in general terms, for example an LP from the WCA region commented in the web-survey: “It's through GIF that my country's digital security history began”. Other LPs provided specific examples of useful support for digital security, such as assistance in developing NGOs’ digital security policy approach, publishing a cybersecurity handbook for CSOs, and a cyber security training for CSOs delivered by an external IT expert. Many LPs mentioned the usefulness of the SAFETAG audits as an effective model to help organizations consider digital risks and mitigation strategies. Some RPs wanted to do more such audits. One RP appreciated SAFETAG audit tools being adjustable to country contexts: “I think that this is one of the best available tools, and we use it and it works for our region… It gives you flexibility to adjust it to the needs of the local organizations you work for.” (KII, RP). Another RP who had been using the tool previously, mentioned the benefit of GIF affording the opportunity to implement the audit in other countries. Finally, LPs operating in countries with hostile political environment and repressive governments, where digital security work puts partners at risk, appreciated GIF responsiveness in providing emergency support, noting that Internews was very accommodating of their rapidly changing digital security and safety needs. Gaps and Opportunities In open-ended questions of the web-survey, a number of LPs mentioned their desire for more technical support and knowledge sharing in the area of digital security, as well as the need for more training of technicians from civil society organizations on this issue. One LP stated they would benefit from more practical guidance and actionable steps to help their constituents improve their experience in securing online practices and utilizing privacy-enhancing tools. They went on to suggest a creative approach for doing so: “Incorporating interactive elements within the program, such as quizzes, simulations, or tutorials, would enable users to actively engage and enhance their understanding of digital rights and security. This interactive approach can empower users to develop necessary skills and knowledge to navigate the digital landscape more effectively” (Web-survey, LP, E&E and the Balkans). The ET discussed the need for more learning opportunities in greater detail in the Network Development section, but this issue also came up in relation to digital security: “We would recommend more interaction and connections with technologists and security experts, as there is a lack of such communities in (our country).” (Web-survey, LP, SEA). A number of RPs and LPs mentioned the need for more trainings in advanced technical topics (VPNs, encryption, data protection, and online privacy) for non-technical staff around, as well as trainings on advanced topics for those already with levels of expertise (on AI, malware, computer forensics, and technical aspects of cyber security). More such trainings would help nurture more specialists at the local level. Some LPs expressed a need for support for internal risk assessments on digital security and a need for more significant support in digitally securing local organizations: “More attention to the need for technical infrastructure to serve the organizations we work with” (Web-survey, LP, LAC). The requests USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 50 were also made for more financial support to carry out audits in organizations that could not otherwise afford them and for open access to software to better support digital safety and to regularly update security protocols.17 4.5.2 SUSTAINABILITY AND SCALING Most LPs we surveyed believed it was very likely (79%, 38 respondents) or likely (13%, six respondents) that their organizations would continue work on digital rights and security after the end of the GIF project, building on progress achieved with the GIF support (Exhibit 26). Exhibit 26 Likelihood of Continuing Work on Digital Rights and Security post GIF Note: N=48. Based on the question “How likely is it that your organization will be able to continue work on digital rights and security using capacity, networks, and advocacy systems developed or supported by GIF after the GIF project is over?” There are several aspects of GIF that will be sustained after the program is over. Firstly, the research and knowledge products developed with GIF support will be sustained and will be useful for many, such as example reports from the RDR exercise. A global partner explained, “The RDR research adaptations are super valuable or will be super valuable. I think a lot haven't been published yet. That's a mechanism that can help companies do things that change policies, especially at the regional level… Some of the outputs like the toolkits, if they're well thought out and not just a checklist item, they'll be good resources that will stay well beyond the project.” (KII, Global Partner) The GIF website has the potential to support the sustainability of knowledge products by hosting them in a repository. If the website can be fully operationalized in the remaining time of the project, it can serve as a lasting resource. In regions where there is not already an established body of research on digital rights, digital security and IF, having contextualized and translated research products can move the field forward beyond the end of the GIF project. This is also a vector of the project that would be pertinent for scaling: building a repository of tools and research products that can support further work in contexts that do not have an established body of knowledge. Secondly, the capacity strengthening of GIF will continue to benefit partner organizations and the field as a whole after the project is over. The fellowships, mentorships, trainings, and technical 17 We note that GIF aims to support access to software licenses and associated targeted end-user training in year three (GIF AWP Year 3). Very unlikely 2% Likely 13% Very likely 79% Don't know/Missing 6% USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 51 support from global partners have improved expertise and skills for LPs and RPs. An RP spoke to this: “if we have more professionals in the field […], that means the work doesn't just end at our engagement, it means that someone, somewhere in Angola, or in Democratic Republic of Congo is able to take up cybersecurity as a career path and they are able to provide those services in the long run to the civil societies in their countries.” (KII, Regional Partner) An LP from LAC explained in the same vein that now that they know how to apply the SAFETAG audit methodology, they can obtain additional external funding to use the method in the future. Thirdly, the networks that were created under GIF will be at least partly sustained, as there is evidence of connections between organizations that will lead to future partnerships outside of GIF. A respondent from Internews spoke to this: “In terms of the sustainability, I think the networks, especially amongst local partners and the regional partners, I think those are connections that probably wouldn't have happened outside of GIF and now people have others that they can call and reach out to.” (KII, Internews) However, it is important that permanent communication channels be established so that network connections can be sustained without external funding. This is especially important in regions like WCA and CA where there are not already well-established networks in place around digital rights, digital security, and IF. 4.5.3 IMPACT OF COVID19 The impacts of COVID on the GIF project were varied: for some activities, the pandemic was a hindrance, but for most activities it seemed that organizations were used to working remotely and using digital platforms for their work. An LP stated, “most of the things we do nowadays are through the Internet. I wouldn't say that there were any major hiccups in this regard.” (KII, LP, E&E). An LP in LAC did note that some of the SAFETAG audit components were difficult to implement remotely and would have been easier with in-person office visits. The largest impact of COVID19 was on the ability to convene partners in-person in the beginning of the project. One RP stated that networking activities had limited impacts at the start of GIF because they could not be in-person: “We started in 2020 when there was a pandemic, so the borders were closed…As I see it, international conferences where you can attend in-person and actually communicate are much better in this network building. We were not able to do that. We only had these in-person meetings in the last year of the project.” (KII, RP) However, this respondent went on to say that partners became acclimated to digital platforms and remote ways of working, and eventually they were able to implement activities such as conducting interviews and trainings at the same intensity as before. 4.5.4 DEFINITIONS OF TERMS USAID requested the ET to additionally inquire as to the relevance of current terminology in the field, broadly. While in general partners interviewed said that they were comfortable with the terminology of digital rights, digital security and IF, there was also some criticism of these terms as being either too broad or even giving off connotations that undermine the work. Several LPs indicated that they use these terms because they are vehicles of funding and are widely used in international discourse. However, LPs also use more operational and descriptive terms that reflect how people actually experience these issues within their communities. Partners in non-English speaking countries noted that they give longer explanations when translating these terms in their own languages and aim to distinguish between human rights and digital rights. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 52 While longer explanations can work in the context of trainings, a global partner stated that the lack of clarity around their role due to the broadness of terms made it difficult to plan their work without a more targeted approach. A local partner in CA also noted that the term “digital rights” is so broad that it makes it difficult to find funding because there are so many organizations that are working on topics that fall under its umbrella. This respondent said that the term ‘digital rights’ can be useful, especially when aligning with UN definitions of rights, but that sometimes funding might go to the ‘wrong’ organizations because of its vagueness. Several LPs noted that they use term “cybersecurity” rather than “digital security”. A local partner explained that ‘digital rights’ and ‘digital security’ can cause suspicion with their government: “When we are doing digital rights we cannot call it that, but we can say ‘cyber security’. It's like a secret code that we all agree [to]. Some governments are more paranoid than others and when they hear ‘digital security’, they are thinking illegal things.” (KII, LP, ESA) 5 RECOMMENDATIONS This section presents practical recommendations for USAID and implementing partners that emerged from the findings of this evaluation. 5.1 NETWORK DEVELOPMENT • Strengthen communication and establish persistent channels for internal communications based on partner preference. GIF needs a comprehensive communication strategy to cater to diverse partner needs. Internews can designate a liaison role to support GIF communication and devise a communication plan to keep alums connected after the project. USAID should expedite approval of the GIF website, as this is an important tool that should be used for regular updates. Since partners have varied preferences, Internews should conduct a web-survey to identify preferred modalities with awareness that multiple channels might be necessary given the scope and range of partners (WhatsApp, Slack, email, etc.) and establish clear channels for post-GIF communication. GIF should allocate resources to connect alums through updates, conversations, or a quarterly newsletter featuring Quarterly Report extracts. • Continue prioritizing in-person meetings. Although partners indicated that they are comfortable with remote modalities, there was strong consensus that networking opportunities for in-person meetings are invaluable and not replaceable by digital forums. While there should be opportunities for virtual participation especially for cases of visa difficulties, in-person meetings are crucial for strengthening networks. In-person meetings should be used to connect partners from different regions for cross-regional learning, as well as to foster direct interaction between LPs and INGOs. • Prioritize fostering institutional knowledge. Fellowship and mentorship activities at both global and regional levels could take on board organizations as fellows instead of individuals and encourage multiple individuals within an organization to engage in the program, allowing knowledge to be shared across the team. This will ensure that if one person leaves, the organization can sustain its participation and retain valuable expertise, thus building institutional knowledge. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 53 5.2 LOCALIZATION • Address differing levels of LP’s expertise through targeted trainings and capacity building. GIF should assess partners’ skills and knowledge in advance to tailor trainings and materials and provide more targeted trainings on a range of areas to be determined by local partner needs, particularly technical knowledge needs on emerging issues. Strengthening the training component is especially important in two areas. The first is the digital security objective, where more training for LPs is needed on topics data protection, privacy, VPNs, and cloud computing. This investment will help create a sustainable pool of professionals who can assist local NGOs and vulnerable communities with their digital security needs. Another area where more training for LPs is recommended is advocacy, focusing on topics such as investigative skills, human rights approaches, UN guiding principles, and effective messaging strategies. • Allocate more funding and resources for translation. This includes provision of more resources for translation of resources and live translation during events and forums. GIF should encourage RPs to allocate more resources and funding to translation, particularly for regions that do not share a common language such as WCA and LAC. Further, GIF could explore relevant indigenous languages for publicly disseminated reports to reach a more inclusive range of local stakeholders. 5.3 ADVOCACY WITH GOVERNMENT AND MULTILATERAL INSTITUTIONS • Establish advocacy leadership to streamline workflows and enhance coordination. GIF should consider appointing a dedicated individual or team to lead advocacy efforts across partners. This leadership can provide valuable support and guidance to partners in their advocacy initiatives, including support for the development of advocacy tools for grassroots mobilization led by LPs. GIF should provide clear workflows for advocacy activities to ensure efforts are not merely treated as checkboxes but are strategically planned and executed. Foster better coordination between partners, including regional collaborations, to address common challenges and amplify advocacy effort and impacts. • Increase support for LPs to attend multistakeholder conferences, forums where policymakers and the private sector are present. LPs would benefit from more direct interactions in the multistakeholder conferences and forums with policymakers and the private sector to build more in-person informal connections that could increase trust and familiarity between both parties. This would also help to create a more direct line of communication between LPs, RPs, and governments. Continue supporting RPs participation in spaces such as IGF and RightsCon, and increase support for LPs to attend, including sufficient funding and time allocated for visa applications as necessary. 5.4 ADVOCACY WITH PRIVATE SECTOR • Scale up work to cultivate relationships with private sector representatives by fostering direct communication between partners and private companies. GIF should facilitate direct access and communication between LPs and representatives of tech companies using its connections in the tech sector by providing further opportunities for meetings and exchanges of experiences. This engagement should expand beyond large tech companies and include more smaller and emerging tech companies, tech startups, and private sector companies that have a stake in national and/or regional contexts, with the aim of engaging with them in advocacy initiatives. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 54 • Continue encouraging regional and local partner use of RDR research but integrate a more standard review process and adaptation for different local contexts. RPs and LPs found that a research-based approach created an effective foundation for interactions with the private sector. Some LPs need targeted training on implementation of research methodologies, particularly the RDR methodology. Some struggled to apply the methodology due to its complexity and the steep learning curve. GIF should consider adding greater support to apply this methodology, such as including expert review of results and quality control, to improve consistent and rigorous application by partners. 5.5 CROSS-CUTTING RECOMMENDATIONS FOR FUTURE USAID PROGRAMMING GIF partners provided a wide range of recommendations for consideration in future iterations of work in the area of digital rights and digital security. These broad recommendations came up in the majority of KIIs and open-ended survey questions. They include: • More time. While GIF itself is a 3-year project, by the time LPs were selected and brought on board, many reported having short time frames and limited scope for their activities. The request for more implementation time also came from a number of RPs. The ET recommends USAID to consider the impact of current time frames on deliverables and project impacts. A few RPs specifically asked for added time and funds to focus on network building, citing the struggle to stay atop workplans as well as coalition building. • More funding. While partners appreciated the range of activities they were able to implement or participate in, they wanted to further expand and enrich the set of GIF￾supported activities, especially around advocacy, digital safety, research, trainings, and fellowship programs. • More flexibility and local control. Locally-led development is a key tenant of GIF and was noted by many partners as a demonstrable strength. However, some LPs requested greater decentralization and local decision-making. Future work should involve LPs in earlier stages of planning, agenda setting, and decision making around activities and priorities, which should in turn contribute towards project sustainability. • More streamlining of administrative processes. Although many bureaucratic challenges are due to internal USAID processes and requirements and U.S. Government procurement regulations, many partners viewed administrative delays as one of the major challenges of GIF implementation. USAID and Internews should jointly explore options for resolving administrative limitations and burdens within the parameters of existing regulations and requirements. This could include altering approval processes, extending subaward implementation periods, increasing amounts of subawards, or other changes to current administrative practices. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 55 ANNEXES USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 56 ANNEX A: EVALUATION STATEMENT OF WORK USAID DRG-LER II Tasking Request Tasking N087: GIF Performance Evaluation Date of Request: 1/10/23 Type of Task: Developing Learning Agenda Case Study Indicator Selection X Performance Evaluation Country Data Portrait Impact Evaluation Evidence Gap Map Metaketa Literature Review Assessment Systematic/Evidence Review Auxiliary Study (Other Research) Public Opinion Survey Dissemination and/or Utilization Purpose/Utilization Plan: The purpose of the evaluation is to assess the approaches and activities of the Greater Internet Freedom (GIF) global activity to identify lessons learned, best practices, and successful lines of work. GIF deploys a wide range of approaches to supporting civil society on digital rights and digital security issues, some of which are quite innovative for this sub-sector. USAID wants to determine which of these lines of effort have been particularly successful or promising, which have been less promising, and the reasons approaches/activities have “worked” or “not worked.” The evaluation findings will be used to inform: (1) the design of a follow-on activity; (2) the work plan for a potential one-year extension of the GIF program; and (3) USAID’s collaboration and coordination work with other donors and stakeholders working on digital rights issues. Statement of Work (SOW): This is a performance evaluation of the global Greater Internet Freedom (GIF) activity. GIF is a 3- year, $16.2 million award that strengthens the capacity of civil society organizations, independent media, and civic actors to address digital repression. GIF does this by conducting digital security support services and civil society advocacy activities that advance rights-respecting legislation and policy frameworks in digital spaces. GIF operates in 38 countries. The activity is implemented by a coalition of 90 international, regional, and organizations led by Internews. This performance evaluation will focus on four domains: digital security, digital rights advocacy, network building, and localization. A set of draft research questions is attached as a separate document. The contractor will conduct a desk review of GIF deliverables and solicit input from key stakeholders based on the evaluation questions. The contractor will conduct a fieldwork outbrief and present the draft report to core stakeholders for input and feedback. This feedback will inform a final report. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 57 Deliverable(s): ● One presentation of the draft report to core stakeholders ● One final report addressing the evaluation questions Dates of performance and timeline: Early January, 2023: USAID and the contractor agree on the tasking, including the SOW and evaluation questions Mid-January - March: Desk review, interviews, and field work Early April: Fieldwork outbrief presentation April: Draft the report End of April: Presentation of draft report to core stakeholders End of May: Submit final report Dissemination and utilization options: X Fieldwork outbrief presentation to core stakeholders X Presentation of draft report to core stakeholders Utilization workshop to present and discuss recommendations with core stakeholders (select this or the next option) Utilization workshop to co-create recommendations with core stakeholders (select this or the previous option) Dissemination event beyond core stakeholders Two-pager/infographic and dissemination Translation of two-pager/infographic (language(s): XXXX, YYYY) Other: Note that in addition to these options, every tasking will be required to include the following dissemination and utilization tools: ● Draft post-evaluation/learning action plan ● Utilization follow-up USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 58 ANNEX B: INTERVIEW PARTICIPATION Phase I Key Informant Interviews Respondent Category Position/ Organization Number of Interviews Number of Respondents USAID staff AOR 1 1 Project staff 1 3 Internews COP 1 1 Regional coordinators for Central Asia 1 2 Regional coordinator for Africa 1 1 Technical Advisors 2 2 MEL specialist 1 1 Global IPs Article 19 1 1 ICNL 1 2 RDR 1 1 GNI 1 2 Regional IPs CIPESA 1 2 CIPI 1 2 DSLU 1 1 BIRN 1 1 Defend Defenders 1 1 Engage Media 1 1 CcHub 1 1 Derechos Digitales 1 1 Paradigm Initiative 1 2 Total 21 29 USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 59 Phase II Key Informant Interviews Region Country Organization Number of Respondents Latin America and the Caribbean Bolivia Internet Bolivia 2 Sensitive 1 - 1 Europe and Eurasia and the Balkans Georgia Institute for Development of Freedom of Information 1 Serbia SHARE Foundation 1 Kosovo Kosovar Centre for Security Studies 1 Central Asia Kyrgyzstan Legal Clinic Adilet 1 Kazakhstan Human Rights Consulting Center 1 South and Southeast Asia Maldives Society for Peace and Democracy 2 Cambodia Cambodian Center for Independent Media 2 West and Central Africa Democratic Republic of the Congo Institut Congolais des Droits de l'Homme 1 CAR Association of Central African Bloggers 1 East and Southern Africa Zimbabwe Digital Society 2 Zambia Zambian Cyber Security Initiative Foundation 1 The Middle East and North Africa Iraq Tech 4 Peace 1 Total 18 USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 60 ANNEX C: KEY INFORMANT INTERVIEW PROTOCOLS Phase I KII Guide: Respondent Category: USAID Informed Consent: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of the Greater Internet Freedom activity. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future programming. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Anna Solovyeva, the project director for this study at solovyeva￾anna@norc.org. [PUT ANNA EMAIL IN CHAT]. If you have any other concerns about your rights as a research participant that have not been answered by the investigators, you may contact April Baker, NORC’s Senior Institutional Review Board Manager, at irb@norc.org. [PUT IRB EMAIL IN CHAT.] With your permission, I’d like to audio-record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with your colleagues at USAID. It will be kept within this research team and destroyed at the end of this study. If you are not comfortable being audio-recorded we can still proceed with the interview and we will take detailed notes. Do you consent to be recorded? Do you consent to continue with this interview? [If respondent consents to being audio-recorded:] Now that I have started the recording, please state one more time for me that you consent to be recorded. Introduction: To begin, could you please tell me a bit about your involvement with the Greater Internet Freedom Project? Networks: To start things off, we would like to ask you about networks that GIF has established or strengthened. 1. What kinds of networks has GIF created? Are there any categorizations of networks that we should consider outside of global, regional, and local? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 61 a. [Probe: Networks around shared areas of focus for orgs, or similar mandates?] How has Internews branded these networks? 2. How effective are these networks in terms of coordinating between members and sharing information? [Probe about each type of network described in Q1] 3. If the project is scaled up or extended, what kinds of activities do you recommend to strengthen networks for digital rights advocacy? a. Are there any you would recommend not pursuing? Advocacy: Governments & Multilateral Institutions: Next, I will ask you about advocacy with governments and multilateral institutions. 4. What do you think were the most important GIF activities around advocacy with governments and multilateral institutions? 5. Which activities do you think were most effective? Least effective? a. Did this vary by region? How so? 6. Do you have any recommendations for how Internews can improve their activities around advocacy with governments and multilateral institutions moving forward? Advocacy: Private Sector Now I’ll ask you about advocacy with the private sector. 7. What do you think were the most important GIF activities around advocacy with the private sector? 8. Which activities do you think were most effective? Least effective? a. Did this vary by region? How so? 9. Do you have any recommendations for how Internews can improve their activities around advocacy with the private sector moving forward? Localization: The next set of questions will focus on working with local partners and GIF’s localization approach. 10. What do you think were the most effective components of Internews' localization strategy? a. Were there any components that you think were less effective? 11. Could you tell me about any successes that Internews has had strengthening the capacity of local organizations? a. What were the main challenges for this work? 12. From your perspective, to what extent were priorities determined and activities designed and led by local actors? 13. Do you have any recommendations for Internews to improve its work with local partners moving forward? Miscellaneous: I just have a few more questions for you, about things that have come up during GIF implementation. 14. Were there planned strategies and approaches for government and private sector advocacy that were not implemented? 15. Please tell me about any GIF activities that worked specifically with multilateral institutions. Do you have any recommendations for how GIF can increase engagement with this group? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 62 16. Were there any changes to your approach or pivots in strategy during implementation? Please tell me about them. Closing: Thank you very much for the information that you’ve shared with us. Do you have any other comments that you would like to make about GIF? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 63 KII Guide: Respondent Category: Internews Informed Consent: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of the Greater Internet Freedom activity funded by the United States Agency for International Development. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on digit rights and internet freedom. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are completely neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Anna Solovyeva, the project director for this study at solovyeva￾anna@norc.org. [PUT ANNA EMAIL IN CHAT]. If you have any other concerns about your rights as a research participant that have not been answered by the investigators, you may contact April Baker, NORC’s Senior Institutional Review Board Manager, at irb@norc.org. [PUT IRB EMAIL IN CHAT.] With your permission, I’d like to audio-record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. If you are not comfortable being audio-recorded we can still proceed with the interview and we will take detailed notes. Do you consent to be recorded? Do you consent to continue with this interview? [If respondent consents to being audio-recorded:] Now that I have started the recording, please state one more time for me that you consent to be recorded. Introduction: To begin, could you please tell me a bit about your role(s) in the Greater Internet Freedom project? How do you support GIF? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 64 Networks: To start things off, we would like to ask you about networks that GIF has established or strengthened. 1. How does Internews brand networks created within the GIF program? a. Would local implementers and end-users (i.e. beneficiaries) be aware that networks they are contributing to and/or participating in are supported by GIF/USAID? 2. Where there opportunities created for regional and local partners to meet with global partners? 3. How are GIF networks coordinated? What role does Internews play in this coordination? 4. Can you provide examples of where network coordination and information sharing benefited local partners? Regional partners? a. What role did Internews play in supporting this coordination and sharing? 5. What recommendations do you have for how future programming can support network development? Advocacy: Governments & Multilateral Institutions: Next, I will ask you about advocacy with governments and multilateral institutions. 6. How were the priorities to implement civil society engagement with governments and multilateral institutions on issues of technology and human rights decided? 7. What resources did Internews provide partners for engaging in policy advocacy? 8. Which components of these activities do you think were most effective? Least effective? 9. Can you tell me about any specific examples in which GIF activities led to changes in policy or practice for any governments or multilateral institutions? 10. Are there any activities around advocacy with governments and multilateral institutions that were not implemented under GIF, that you think should be implemented in the future? Advocacy: Private Sector Now I’ll ask you about advocacy with the private sector. How were the priorities to implement activities with the private sector on issues of technology and human rights decided? 11. What resources did Internews provide partners for engaging in policy advocacy with the private sector? 12. What components of these do you think were the most effective? Least effective? 13. Do you think that GIF activities have influenced private sector practices? a. [If yes:] How so? [If no:] Why not? 14. Are there any activities around advocacy with the private sector that were not implemented under GIF, that you think should be implemented in the future? 15. Were there regions where these activities were more or less successful then others? a. What were some of the factors as to why? Localization: The next set of questions will focus on working with local partners and GIF’s localization approach. 16. What was your process for selecting regional partners? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 65 a. How involved is Internews in selecting local partners? 17. What resources did Internews provide to help regional partners select local partners? 18. How involved is Internews in the activities taken place on the local level? a. Or in the network development between local and regional partners? 19. How were priorities and activities undertaken by GIF determined? a. To what extent were priorities determined and activities designed and led by local actors? i. What are some of the ways Internews supported this? 20. What tools did Internews provide to help improve civil society engagement on issues of technology and human rights? a. How did Internews monitor or keep track of these developments? Miscellaneous: I just have a few more questions for you, about things that have come up during GIF implementation. 21. How did Internews facilitate the involvement of partners in contributing the GIF website? Or other outreach methods of information sharing? 22. Were there planned strategies and approaches for government and private sector advocacy that were not implemented? 23. How were the priorities set to implement advocacy activities with governments? With the private sector? What resources did Internews provide partners for engaging in policy advocacy? 24. How did the COVID-19 pandemic impact your activities? Do you have any lessons learned around adapting to COVID restrictions or switching to online formats for activities? 25. Which GIF activities do you think are most sustainable? Least sustainable? What are the primary barriers to sustainability? 26. There was some staff turnover at Internews during the project. How did that impact implementation? How were these changes communicated to partners? 27. In your experience on the project, what might some of the ways GIF could be scale in the future? 28. Who decided on the research priorities for GIF? Internews, global, regional, or local partners? How is Internews supporting dissemination of research? Are these activities being coordinated on global or regional level? 29. Can you speak to the reasons why Chile was removed as a country in year one? 30. What mechanisms did Internews put in place to best keep partners safe? 31. How was the process for developing the fellowship and mentorship program determined? Any lessons learned that you could share around running a fellowship/mentorship program around digital rights advocacy? 32. Can you speak about how the war in Ukraine impacted your work? 33. Can you say more about the decision making behind the SAFETAG audits? Are these tools you would want to implement or expand usage of in the future? 34. How were decisions made around what digital security tools, trainings, and software were part of GIF? How involved were local and regional partners in making these decisions? Closing: Thank you very much for the information that you’ve shared with us. Do you have any other comments that you would like to make about GIF? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 66 KII Guide: Respondent Category: Global Partners Informed Consent: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of the Greater Internet Freedom activity funded by the United States Agency for International Development. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on digit rights and internet freedom. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are completely neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Anna Solovyeva, the project director for this study at solovyeva￾anna@norc.org. [PUT ANNA EMAIL IN CHAT]. If you have any other concerns about your rights as a research participant that have not been answered by the investigators, you may contact April Baker, NORC’s Senior Institutional Review Board Manager, at irb@norc.org. [PUT IRB EMAIL IN CHAT.] With your permission, I’d like to audio-record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. If you are not comfortable being audio-recorded we can still proceed with the interview and we will take detailed notes. Do you consent to be recorded? Do you consent to continue with this interview? [If respondent consents to being audio-recorded:] Now that I have started the recording, please state one more time for me that you consent to be recorded. Introduction: To begin, could you please tell me a bit about your involvement with the Greater Internet Freedom Project? Please briefly describe the activities that you have implemented under GIF. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 67 Networks: To start things off, we would like to ask you about networks that GIF has established or strengthened. ▢ What kinds of networks are you a part of thanks to GIF, and how are they organized? How are the networks branded by Internews? ▢ How effective are these networks in terms of coordinating between members and sharing information? ▢ Do you have any concrete examples that you could share where network coordination and information sharing have benefited local members? ▢ If the project is scaled up or extended, what kinds of activities do you recommend to strengthen networks for digital rights advocacy? Are there any you would recommend not pursuing? Advocacy: Governments & Multilateral Institutions: Next, I will ask you about advocacy with governments and multilateral institutions. ▢ Were you involved in GIF activities to improve advocacy with governments and multilateral institutions? What activities did you participate in? a) [If yes:] Which components of these activities do you think were most effective? Least effective? ▢ Do you know of any examples in which GIF activities led to changes in policy or practice for any governments or multilateral institutions? Please tell me about them. ▢ Are there any activities around advocacy with governments and multilateral institutions that were not implemented under GIF, that you think should be implemented in the future? ▢ Do you have any recommendations for Internews and USAID around working with the government and multilateral institutions? Advocacy: Private Sector Now I’ll ask you about advocacy with the private sector. ▢ Were you involved in GIF activities to improve advocacy with the private sector? What activities did you participate in? a) [If yes:] Which components of these activities do you think were most effective? Least effective? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 68 ▢ Do you think that GIF activities have influenced private sector practices? a) [If yes:] How so? [If no:] Why not? ▢ Are there any activities around advocacy with the private sector that were not implemented under GIF, that you think should be implemented in the future? ▢ Do you have any recommendations for Internews and USAID around influencing the private sector? Localization: The next set of questions will focus on working with local partners and GIF’s localization approach. ▢ Did you work directly with local level partners? a) What kinds of activities did you do with them? ▢ Do you think that any of the activities that you did with local partners strengthened their organizational capacities? How so? ▢ Who came up with the ideas to do these activities? a) [Probe: Were they co-created together? What did that co-creation process look like? Was the local partner leading? Were you leading them through the process?] ▢ Do you have any recommendations for USAID and Internews that could help them improve their work with local partners? a) [Probe: How their activities with local partners are designed? New activities that could help improve local capacities to advocate for digital rights?] Miscellaneous: I just have a few more questions for you, about things that have come up during GIF implementation. ▢ Did you contribute to the GIF website? a) [If yes:] How did you participate in its development? [If no:] Have you used it? Do you have any recommendations to improve it? ▢ We understand that there was a major shift in the approach to Monitoring and Evaluation part-way through the project. Were you part of the project at that point? [If yes:] Could you please comment on it? Did the change improve your experience with MEL, or make it more difficult? Do you agree with the changes that they made to the MEL approach? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 69 ▢ Please tell me about any GIF activities that worked specifically with multilateral institutions. Do you have any recommendations for how GIF can increase engagement with this group? ▢ How did the COVID-19 pandemic impact your activities? a) Do you have any lessons learned around adapting to COVID restrictions or switching to online formats for activities? ▢ Which GIF activities do you think are most sustainable? Least sustainable? What are the primary barriers to sustainability? ▢ Have you conducted any research for GIF? a) [If yes:] Please tell me about your research work, and what its status is. i) Was it in partnership with RDR ? ii) Was your organization already involved in research activities? If so, did GIF bring new opportunities for research collaboration? b) If the program is scaled up or extended, what kinds of evidence should GIF focus on generating to move the field forward? c) Do you have any recommendations for Internews and USAID around the research components of GIF? ▢ Did you have any challenges with safety and security while working under GIF? a) [If yes:] How did you address them? Do you have any lessons learned that you could share? ▢ Were there any changes to your approach or pivots in strategy during implementation? Please tell me about them. ▢ [Only for partners that we already know ran fellowships and mentorship programs] Could you please tell me about some of the successes that you saw in the GIF fellowship/mentorship program? a) How about challenges? b) Any lessons learned that you could share around running a fellowship/mentorship program around digital rights advocacy? Closing: Thank you very much for the information that you’ve shared with us. Do you have any other comments that you would like to make about GIF? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 70 KII Guide: Respondent Category: Regional Implementing Partners Informed Consent: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of the Greater Internet Freedom activity funded by the United States Agency for International Development. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on digit rights and internet freedom. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are completely neutral on all the issues we will be talking about, and we’re just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Anna Solovyeva, the project director for this study at solovyeva￾anna@norc.org. [PUT ANNA EMAIL IN CHAT]. If you have any other concerns about your rights as a research participant that have not been answered by the investigators, you may contact April Baker, NORC’s Senior Institutional Review Board Manager, at irb@norc.org. [PUT IRB EMAIL IN CHAT.] With your permission, I’d like to audio-record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. If you are not comfortable being audio-recorded we can still proceed with the interview and we will take detailed notes. Do you consent to be recorded? Do you consent to continue with this interview? [If respondent consents to being audio-recorded:] Now that I have started the recording, please state one more time for me that you consent to be recorded. Introduction: To begin, could you please tell me a bit about your involvement with the Greater Internet Freedom Project? Please briefly describe the activities that you have implemented under GIF. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 71 Networks: To start things off, we would like to ask you about networks that GIF has established or strengthened. 1. Are there networks of stakeholders in your region supported by the GIF program? 2. How do you contribute to this network development as part of your engagement with GIF? 3. Do you (and how) brand networks created with the support of the GIF program? a. Are local implementers and end-users (i.e. beneficiaries) aware that networks they are participating in are supported by GIF/USAID? 4. How effective are these networks in terms of coordinating between members and sharing information on digital rights and security? Please comment first on regional and then country level networks. 5. Can you provide any examples of how networks supported by GIF/USAID benefited stakeholders in your region? 6. What do you think can be improved in how networks supported by GIF are organized and function to make these networks more useful for stakeholders? a. In your opinion, are there stakeholders who are currently not engaged in networks who may benefit from participation? Advocacy: Governments & Multilateral Institutions: Next, I will ask you about advocacy with governments and multilateral institutions. 7. Have you with been involved in GIF funded work to improve civil society’s engagement with government or multilateral organizations on issues of technology and human rights? Can you please describe that work? a. Who were they key parties involved and what was the question/issue in focus? b. What government bodies you engaged with? c. What multilateral organizations you engaged with? i. [if yes] Which components of your work have been most effective in increasing engagement/dialogue between civil society and governments? ii. How about between civil society and multilateral institutions? iii. Are there any components that have not been effective? 8. Do you know of any examples in which GIF activities led to changes in policy or practice for any governments or multilateral institutions? Please tell me about them. 9. Are there any activities focused on advocacy with governments and multilateral institutions that were not implemented under GIF, that you think should be implemented in the future? 10.Do you have any recommendations for Internews and USAID to improve the way civil society works with the government? a. What about with multilaterial institutions? Advocacy: Private Sector Now I’ll ask you about advocacy with the private sector. 11.Were you involved in GIF activities to improve advocacy with the private sector? What activities did you participate in? a. [If yes:] Which components of these activities do you think were most effective? Least effective? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 72 12.Do you think that GIF activities have influenced private sector practices? a. [If yes:] How so? [If no:] Why not? 13.Are there any activities around advocacy with the private sector that were not implemented under GIF, that you think should be implemented in the future? 14.Do you have any recommendations for Internews and USAID around influencing the private sector? Localization: The next set of questions will focus on working with local partners and GIF’s localization approach. 15.Did you participate in any activities to support local partners under GIF? Can you describe what activities your organizations implemented? a. [if yes to supporting local partners] Did you observe any improved capacity of the local partners you supported? In what ways? Can you give a few examples? i. Which approaches would you say were the most helpful to local partners? Why do you think these approaches were more beneficial than others? ii. [can probe about SAFETAG audits - they supported Local Partners to implement these] 16.Did you receive any support from GIF? What kind of support did you receive? b. [if yes to receiving support] Did any of the support you received from GIF help to strengthen your capacity? How? c. Which of these were the most impactful? What made these the most impactful? 17.Who came up with the ideas to do these activities? d. [Probe: Did you work with global partners and/or local partners to design activities? What about with Internews?] e. What did the process of determining activities and priorities look like? 18.Do you have any recommendations for USAID and Internews that could help them improve their work with local partners? f. [Probe: How their activities with local partners are designed? New activities that could help improve local capacities to advocate for digital rights?] Miscellaneous: I just have a few more questions for you, about things that have come up during GIF implementation. 19.How did you come to work with Internews? 20.How did the process of selection local subgrantees work? 21.How has the shifting context within your region affected your work? What kinds of adjustments have you had to make to your activities? How has this process worked in coordination with Internews? Closing: Thank you very much for the information that you’ve shared with us. Do you have any other comments that you would like to make about GIF? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 73 PHASE II KII Guide: Respondent Category: Phase II – Local Partners Informed Consent: Hello. My name is ______ and I work for NORC at the University of Chicago. I’ll be leading today’s interview. I will let my colleague(s) introduce themselves. I want to thank you for coming and participating in this interview, which is part of an evaluation of the Greater Internet Freedom activity funded by the United States Agency for International Development. NORC has been contracted as an external, independent organization to collect data for USAID that will inform current and future USAID-funded programming focused on digit rights and internet freedom. While NORC does a lot of work WITH USAID, we do not work FOR USAID. We are just here to learn about your perspective and experiences. That means you don’t need to worry about making us happy or hurting our feelings. Please be candid in your answers. Today’s interview is planned for 60 minutes. If we need to go over by a few minutes, would that be okay? Your participation is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful in helping in informing current and future USAID-funded programming. Your responses will be kept confidential and anonymous. The information you provide will not identify you as a participant of this interview/discussion. Do you have any questions before we get started? [ANSWER QUESTIONS] If you have any questions later, please e-mail Anna Solovyeva, the project director for this study at solovyeva￾anna@norc.org. [PUT ANNA EMAIL IN CHAT]. If you have any other concerns about your rights as a research participant that have not been answered by the investigators, you may contact April Baker, NORC’s Senior Institutional Review Board Manager, at irb@norc.org. [PUT IRB EMAIL IN CHAT.] With your permission, I’d like to audio-record today’s interview. This will enable us to go back and substantiate our notes. The recording will never be shared with USAID. It will be kept within this research team and destroyed at the end of this study. If you are not comfortable being audio-recorded we can still proceed with the interview and we will take detailed notes. Do you consent to be recorded? Do you consent to continue with this interview? [If respondent consents to being audio-recorded:] Now that I have started the recording, please state one more time for me that you consent to be recorded. Introduction: To begin, could you please tell me a bit about your involvement with the Greater Internet Freedom Project? Please briefly describe the activities that you have implemented under GIF. USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 74 Networks: To start things off, we would like to ask you about networks that GIF has established or strengthened. 1. Do you feel part of any networks as a result of your involvement with GIF? Was this a new network(s) or existing network(s)? a. Please describe the network-who have you connected with. b. Do you especially feel part of a global network as a result of your involvement with GIF? c. Regional network? 2. How effective are these networks in terms of coordinating between members and sharing information on digital rights and security? 3. Can you provide examples of how networks supported by GIF/Internews benefited stakeholders in your country? 4. Did you participate in any global forums like the IGF or RightsCon? Was your participation supported by GIF in any way? 5. What do you think can be improved in how networks supported by GIF are organized and function to make these networks more useful for stakeholders? What else? 6. In your opinion, are there stakeholders who are currently not engaged in networks who may benefit from participation? 7. If the project is scaled up or extended, what kinds of activities do you recommend to strengthen networks for digital rights advocacy? Are there any you would recommend not pursuing? Localization: The next set of questions will focus on working with local partners and GIF’s localization approach. 1. What has been the most useful kinds of support that GIF has provided for your organization? 2. Are there other kinds of support from Internews or [relevant regional org] that would make it easier or more efficient to do your work? 3. Has your work with the GIF project strengthened your organization's capacities? a. [Probe: advocacy capacities? Research capacities? Administrative capacities? Other capacities?] How so? 4. How were decisions made about what priority areas you would be working on? (check examples for each region ex. Latin America digital id, etc.) 5. Who decided what activities you would be working on? Were you or your organization involved in deciding what activities? Who did you work with to design activities? a. [Probe: Internews? Global partners? Relevant regional partner?] 6. What did the process of determining activities and priorities look like? a. Who was 'sitting around the table' in design sessions? 7. Is your organization involved in any of the following GIF activities? a. Did you participate in the fellowship and mentorship program? With Article 19? Other? Any lessons learned that you could share about that? How were fellows selected? Did you have any say in the process or issues? b. Did you receive legal support or advice from any of the global partners such as GNI? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 75 c. Did you participate in any research activities as part of GIF? With RDR? Other? Please describe. Were you already involve din research before GIF? How were research priorities and activities determined? Did you have a say in them? d. What recommendations do you have for improving any of these activities? Recommendations for how local partners could be more involved in the process of setting the agenda for activities and issue areas? 8. Do you have any recommendations for USAID and Internews that could help them improve their work with local partners? a. [Probe: How their activities with local partners are designed? New activities that could help improve local capacities to advocate for digital rights?] Advocacy: Governments & Multilateral Institutions: Next, I will ask you about advocacy with governments and multilateral institutions. 8. Were you involved in GIF activities to improve advocacy with governments and multilateral institutions and multistakeholder forums? What activities did you participate in? a. [If yes:] Which components of these activities do you think were most effective? What would you change or do differently? b. What government bodies you engaged with? c. What multilateral organizations you engaged with? 9. Do you know of any examples in which GIF activities led to changes in policy or practice for any governments or multilateral institutions? Please tell me about them. 10. In your opinion, what are some of the factors that make it difficult to do this work? (ie hinder dialogue between civil society and government on issues of technology and human rights?) 11. Are there any activities around advocacy with governments and multilateral institutions that were not implemented under GIF, that you think should be implemented in the future? 12. Do you have any recommendations for Internews and USAID around working with the government and multilateral institutions? Advocacy: Private Sector Now I’ll ask you about advocacy with the private sector. 9. Were you involved in GIF activities to improve advocacy with the private sector? What activities did you participate in? a. Was any of this work undertaken with the support or input from any of the global partners? (esp RDR? GNI?) b. [If yes] which components of these activities do you think were most effective? What would you do differently or change? 10. Do you think that GIF activities have influenced private sector practices? a. [If yes:] How so? [If no:] Why not? b. Did your work effect dialogue between civil society and the private sector? 11. Are there any activities around advocacy with the private sector that were not implemented under GIF, that you think should be implemented in the future? 12. Do you have any recommendations for Internews and USAID around influencing the private sector? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 76 Miscellaneous: I just have a few more questions for you, about things that have come up during GIF implementation. 13. Did you contribute to the GIF website? [If yes:] How did you participate in its development? [If no:] Have you used it? Do you have any recommendations to improve it? 14. How did the COVID-19 pandemic impact your activities? 15. We understand some activities were online, in part due to COVID and in part for other reasons. Did you find online activities useful? Do you have any recommendations for online formats for the future? 16. If the program is scaled up or extended, what kinds of evidence should GIF focus on generating to move the field forward? 17. Did you have any challenges with safety and security while working under GIF? a. [If yes:] How did you address them? Do you have any lessons learned that you could share? 18. How has the shifting context (political, economic, etc) within your country and region affected your work? a. What kinds of adjustments have you had to make to your activities? b. How has this process worked in coordination with Internews? Closing: Thank you very much for the information that you’ve shared with us. Do you have any other comments that you would like to make about GIF? USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 77 ANNEX D: SURVEY SAMPLE CHARACTERISTICS Figure A presents the geographic distribution of respondents, showing that the largest number of respondents are from the LAC (11), followed by E&E and the Balkans (10), and the CA (8). South and Southeast Asia, West and Central Africa, and East and Southern Africa had six respondents each. GIF’s only local partner in the Middle East and North Africa also took part in the survey. Figure A Distribution of Respondents by Region Note: N=48 The majority of respondents’ organizations (26 respondents) are involved in both thematic areas of GIF, while 13 respondents focus only on digital security, and the remaining 7 respondents work only on digital rights (Figure B). Figure B Local Partners’ Engagement in GIF Thematic Areas Note: N=48. Based on the question “What are thematic areas of GIF-supported work that you have engaged in or benefited from?” Regarding respondents’ roles or positions in their organizations (Figure C), half of our sample (24) is represented by high-level senior staff, such as founders, directors, or deputy directors. A quarter of respondents (12) are technical staff (specialist, researchers, or experts). The remaining respondents are program coordinators (11). 11 10 8 6 6 6 1 Latin America and the Caribbean Europe and Eurasia and the Balkans Central Asia South and Southeast Asia West and Central Africa East and Southern Africa The Middle East and North Africa Digital security 13 27% Digital rights 7 15% Both 26 54% Missing 2 4% USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 78 Figure C Distribution of Respondents by Role/Position in Organization Note: N=48 As shown in Figure D, there are more male respondents than females (32 vs. 14), and the largest age group (23) consists of 30–40-year-olds, followed by 41-50-year-olds (12), and those under 30 (10). Figure D Distribution of Respondents by Gender and by Age Group Note: N=47 Note: N=48 24 50% 12 25% 11 23% 1 2% Founder/Director/Deputy Director Technical specialist/Researcher/Expert Program coordinator Other: lawyer Male 32 68% Female 14 30% Gender diverse 1 2% Male Female Gender diverse 10 23 12 3 Under 30 30-40 41-50 Over 50 USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 79 ANNEX E: SURVEY INSTRUMENT Q1 NORC at the University of Chicago, a non-partisan research institution based in the United States, is inviting you to participate in a performance evaluation of the Greater Internet Freedom activity funded by the United States Agency for International Development (USAID). The purpose of this study is to inform current and future USAID-funded programming focused on digital rights and internet freedom. We will be asking you questions in four areas of GIF: networking, localization, advocacy with government and multilateral organizations, and advocacy with the private sector. Your participation is important to help learn more about the implementation of the GIF activities. This survey is voluntary. You can start the survey and then return to complete it at a later point using the same survey link. You have the right not to answer any question you do not wish to, as well as withdraw completely from the study at any point during the process without any negative consequences for you. Additionally, you have the right to request that that we not use any of your information or survey responses in the study. With all research we conduct, we follow strict procedures to protect your information and use your answers only for statistical analyses. Your answers will never be connected to you personally and will only be viewed in aggregate – which means they will be grouped together and counted along with the rest of the responses to this survey. Your participation does not involve any risks other than what you would encounter in daily life. The survey will take about 20 minutes to complete. If you have any further questions about the study, please contact Anna Solovyeva, the project director for this study at solovyeva-anna@norc.org. If you have any other concerns about your rights as a research participant, you may contact April Baker, NORC’s Senior Institutional Review Board Manager, at irb@norc.org. Do you agree to participate in this survey? o Yes o No Skip To: End of Survey If NORC at the University of Chicago, a non-partisan research institution based in the United States... = No Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 80 End of Block: Introduction Start of Block: Background Q2 What is your gender? o Male oFemale o Gender diverse oPrefer not to say Q3 What is your age? o Under 30 o30-40 o41-50 o Over 50 oPrefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 81 Q4 What is the geographic region where your organization operates? o West & Central Africa oEast & Southern Africa oLatin America & the Caribbean oEurope and Eurasia and the Balkans oSouth & Southeast Asia o Central Asia o The Middle East & North Africa oPrefer not to say Q6 What is your position or primary role in your organization? oFounder/Director/Deputy Director o Technical specialist/Researcher/Expert oProgram coordinator o Other, please specify __________________________________________________ oPrefer not to say End of Block: Background Start of Block: Network Development USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 82 Q7 Have you participated in networks or networking events supported by GIF/USAID? Some examples include fellowships and mentorships programs, meetings, conferences, training, lectures, workshops, academic collaborations, online fora and platforms, etc. o Yes o No o Don't know/not sure oPrefer not to say Skip To: Q11a If Have you participated in networks or networking events supported by GIF/USAID? Some examples incl... = No Skip To: Q11a If Have you participated in networks or networking events supported by GIF/USAID? Some examples incl... = Don't know/not sure Skip To: Q11a If Have you participated in networks or networking events supported by GIF/USAID? Some examples incl... = Prefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 83 Q8 Please select networks or networking events supported by GIF/USAID you have participated in? Select all that apply. ▢ Fellowships and mentorships programs ▢ Conferences, meetings or roundtable discussions (virtual or in-person) ▢ Trainings, lectures, or workshops (virtual or in-person) ▢ Regional research networks related to implementation of the Ranking Digital Rights methodology ▢ Collaborations related to advocacy on the regional or global level ▢ Online fora or platforms ▢ Other research or academic collaborations ▢ Other, please specify __________________________________________________ ▢ ⊗Prefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 84 Q9 Has participation in networks or networking events supported by GIF/USAID been useful for you and your organization? o Not at all useful oLittle useful o Moderately useful o Very useful o Don't know/ not sure oPrefer not to say Skip To: Q11 If Has participation in networks or networking events supported by GIF/USAID been useful for you and... = Not at all useful Skip To: Q11 If Has participation in networks or networking events supported by GIF/USAID been useful for you and... = Little useful Skip To: Q11 If Has participation in networks or networking events supported by GIF/USAID been useful for you and... = Don't know/ not sure Skip To: Q11 If Has participation in networks or networking events supported by GIF/USAID been useful for you and... = Prefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 85 Q10 In what ways has your participation in networks or networking events supported by GIF/USAID been useful for you and your organization? Select all that apply. ▢ I learned new information and developed new skills ▢ I participated in research activities I would not have otherwise been able to be engaged in ▢ I networked with colleagues and other experts from my region ▢ I networked with colleagues and other experts from other regions of the world ▢ I promoted my work on digital rights and security among other stakeholders ▢ I engaged in advocacy with governments and/or multilateral organizations and/or in multistakeholder forums ▢ I engaged in advocacy with private companies ▢ I can now better coordinate my work in the space of digital rights and security with other stakeholders ▢ Other, please specify: __________________________________________________ ▢ ⊗Don't know/not sure ▢ ⊗Prefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 86 Q11 What challenges, if any, did your organization face regarding your participation in networks or networking events supported by GIF/USAID? Please select all that apply. ▢ Networking events were too short or not intensive enough ▢ There were too few networking events ▢ Networking events did not offer the opportunity to meet with organizations outside my region ▢ Materials and events were not offered in local language(s) ▢ Information or training provided at networking events was too complicated to understand ▢ Information or training provided at networking events was not relevant to my work ▢ I was not given enough opportunities to present or otherwise promote my work ▢ Networking events were held online instead of in-person, which limited opportunities for connecting with other participants ▢ ⊗No challenges ▢ Other, please specify: __________________________________________________ ▢ ⊗Don't know/not sure ▢ ⊗Prefer not to say USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 87 Q11a Did your involvement with GIF help create links and networks with USAID Missions or other relevant USAID funded activities in your country? o Yes o No o Don't know/not sure oPrefer not to say Skip To: Q12 If Did your involvement with GIF help create links and networks with USAID Missions or other relevan... = Don't know/not sure Skip To: Q12 If Did your involvement with GIF help create links and networks with USAID Missions or other relevan... = Prefer not to say Skip To: Q12 If Did your involvement with GIF help create links and networks with USAID Missions or other relevan... = No Page Break Q11b Were these links and networks with USAID Missions or other relevant USAID funded activities in your country beneficial to your work on digital rights and security? o Not beneficial at all oLittle beneficial o Moderately beneficial o Very beneficial o Don't know/not sure oPrefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 88 Q12 How can future programs supporting networks and network development in the area of digital rights and security be more helpful for your work? What can be improved? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ End of Block: Network Development Start of Block: Localization Q13 Did you have the following resources needed to undertake the activities required of you under GIF? Fully disagree Rather disagree Somewhat agree Fully agree Don't know/ not sure Clear understanding of what your organization's role and tasks were o o o o o Technical knowledge, tools, and equipment o o o o o Access to experts from regional and global partners o o o o o Support for administrative or financial reporting requirements o o o o o Sufficient time to undertake activities under GIF o o o o o Adequate funding from GIF o o o o o Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 89 Q14 How engaged were you in the following tasks as part of your engagement with the GIF program? Not engaged at all A little engaged Somewhat engaged Very engaged Don’t know/ not sure Prioritizing activities to be undertaken in your country under GIF o o o o o Designing approaches to implement activities supported by GIF in your country o o o o o Identifying budget needs for activities in your country o o o o o Identifying research to be supported by GIF in your country o o o o o Identifying indicators to measure outcomes and outputs of GIF￾supported activities o o o o o Disseminating knowledge products (e.g., reports, briefs, etc.) created with support from GIF o o o o o Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 90 Q15 How would you like to change the level of your engagement in the following tasks as part of your work under the GIF program? To be less engaged Stay the same To be more engaged Don’t know/ not sure Prioritizing which activities would be undertaken in your country under GIF o o o o Designing approaches to how GIF supported activities are implemented in your country o o o o Identifying budget needs for specific activities in your country o o o o Identifying research to be supported by GIF in your country o o o o Identifying indicators to measure outcomes and outputs of GIF￾supported activities o o o o Disseminating knowledge products (e.g., reports, briefs, etc.) created with support from GIF o o o o Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 91 Q16 What are thematic areas of GIF-supported work that you have engaged in or benefited from? o Digital safety and security o Digital rights oBoth o Don’t know/not sure oPrefer not to say Skip To: Q19 If What are thematic areas of GIF-supported work that you have engaged in or benefited from? = Digital rights Skip To: End of Block If What are thematic areas of GIF-supported work that you have engaged in or benefited from? = Don’t know/not sure Skip To: End of Block If What are thematic areas of GIF-supported work that you have engaged in or benefited from? = Prefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 92 Q17 Regarding GIF-supported work in the area of digital safety and security, what activities and resources did you benefit from? Select all that apply. ▢ Provision of software, software licensing or hosting ▢ Development of security and incident policies and protocols ▢ Support for incident response and emergent developments ▢ Technical support, including web-site creation and updating ▢ Security audits and assessments ▢ Development and provision of toolkits and other practical knowledge products ▢ Trainings, lectures, and/or workshops ▢ Conferences, meetings, and roundtables ▢ Other __________________________________________________ ▢ ⊗Don’t know/not sure ▢ ⊗Prefer not to say Skip To: Q19 If Regarding GIF-supported work in the area of digital safety and security, what activities and reso... = Don’t know/not sure Skip To: Q19 If Regarding GIF-supported work in the area of digital safety and security, what activities and reso... = Prefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 93 Carry Forward Selected Choices - Entered Text from "Regarding GIF-supported work in the area of digital safety and security, what activities and resources did you benefit from? Select all that apply." Q18 How useful was this support/activities provided by GIF for your work on digital safety and security? Not at all useful Little useful Moderately useful Very useful Don't know/ not sure Provision of software, software licensing or hosting o o o o o Development of security and incident policies and protocols o o o o o Support for incident response and emergent developments o o o o o Technical support, including web-site creation and updating o o o o o Security audits and assessments o o o o o Development and provision of toolkits and other practical knowledge products o o o o o Trainings, lectures, and/or workshops o o o o o Conferences, meetings, and roundtables o o o o o Other o o o o o ⊗Don’t know/not sure o o o o o ⊗Prefer not to say o o o o o Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 94 Q19 Regarding GIF-supported work in the area of digital rights, what activities and resources did you benefit from? Select all that apply. ▢ Fellowships and mentorships programs ▢ Training, lectures, or workshops ▢ Conferences, meetings, and roundtables ▢ Support for development of practical knowledge outputs (policy briefs, blogposts, etc.) ▢ Research support ▢ Support for advocacy campaigns ▢ Other __________________________________________________ ▢ ⊗Don’t know/not sure ▢ ⊗Prefer not to say Skip To: End of Block If Regarding GIF-supported work in the area of digital rights, what activities and resources did you... = Don’t know/not sure Skip To: End of Block If Regarding GIF-supported work in the area of digital rights, what activities and resources did you... = Prefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 95 Carry Forward Selected Choices - Entered Text from "Regarding GIF-supported work in the area of digital rights, what activities and resources did you benefit from? Select all that apply." Q20 How useful was this support provided by GIF for your work on digital rights? Not at all useful Little useful Moderately useful Very useful Don't know/ not sure Fellowships and mentorships programs o o o o o Training, lectures, or workshops o o o o o Conferences, meetings, and roundtables o o o o o Support for development of practical knowledge outputs (policy briefs, blogposts, etc.) o o o o o Research support o o o o o Support for advocacy campaigns o o o o o Other o o o o o ⊗Don’t know/not sure o o o o o ⊗Prefer not to say o o o o o End of Block: Localization Start of Block: Advocacy with Governments, Multilateral Institutions and Multistakeholder Forums USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 96 Q21 Before becoming part of the GIF project, have you been involved in advocacy work with government on issues of technology and human rights? o Not involved at all oBarely involved oSomewhat involved o Very involved o Don't know/not sure oPrefer not to say Page Break Q22 Before becoming part of the GIF project, have you been involved in advocacy work with multilateral organizations and multistakeholder forums (such as IGF, ICANN, ITU, etc.) on issues of technology and human rights? o Not involved at all oBarely involved oSomewhat involved o Very involved o Don't know/not sure oPrefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 97 Q23 Has GIF provided you with support to conduct work to improve civil society’s engagement with the types of stakeholders listed below on issues of technology and human rights? o GIF provided support for advocacy with government o GIF provided support for advocacy with multilateral organizations and multistakeholder forums o GIF provided support for advocacy with BOTH government AND multilateral organizations and multistakeholder forums o No, GIF has not provided support for advocacy with government or multilateral organizations and multistakeholder forums o Don’t know/not sure o Prefer not to say Skip To: Q27 If Has GIF provided you with support to conduct work to improve civil society’s engagement with the... = No, GIF has not provided support for advocacy with government or multilateral organizations and multistakeholder forums Skip To: Q27 If Has GIF provided you with support to conduct work to improve civil society’s engagement with the... = Don’t know/not sure Skip To: Q27 If Has GIF provided you with support to conduct work to improve civil society’s engagement with the... = Prefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 98 Q24 What engagement strategies did your advocacy work with government and/or multilateral organizations and multistakeholder forums employ as part of GIF? Select all that apply. ▢ Review, analysis, and provision of input in government’s legislative initiatives (e.g., bills, draft laws, and other regulatory documents) ▢ Liaising with international or regional standard-setting bodies (e.g., ITU and ICANN) ▢ Participation in global forums such as the Internet Governance Forum (IFG) or RightsCon ▢ Meetings, round tables, and consultations with government stakeholders, policy makers, and multilateral organizations and multistakeholder forums on issues of technology and human rights ▢ Dissemination of research findings supported by GIF directly to government stakeholders and multilateral organizations and multistakeholder forums, for example via mail or email ▢ Other, please specify __________________________________________________ ▢ ⊗Don’t know/not sure ▢ ⊗Prefer not to say USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 99 Q25 To what extent did being a part of GIF improve your ability to engage with government and multilateral organizations and multistakeholder forums? o Not at all o A little oSomewhat o A lot o Don’t know/not sure oPrefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 100 Display This Question: If Has GIF provided you with support to conduct work to improve civil society’s engagement with the... = GIF provided support for advocacy with BOTH government AND multilateral organizations and multistakeholder forums And Has GIF provided you with support to conduct work to improve civil society’s engagement with the... = GIF provided support for advocacy with government Q26 Thinking of your work supported by GIF to improve civil society’s engagement with government on issues of technology and human rights, did it bring changes in government's actions, policies, and practices? o Never o Rarely oSometimes o Most of the time o Too soon to tell o Don’t know/not sure oPrefer not to say Page Break Q27 What are the barriers and challenges that prevent local organizations like yours from advocating with (a) government in your country and (b) multilateral organizations and multistakeholder forums? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 101 Q28 Do you have any recommendations for how GIF can help to improve civil society’s engagement with (a) government in your country and (b) multilateral organizations and multistakeholder forums? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ End of Block: Advocacy with Governments, Multilateral Institutions and Multistakeholder Forums Start of Block: Advocacy with the Private Sector Q29 Before becoming part of the GIF project, have you been involved in advocacy work with private companies on issues of technology and human rights? o Yes o No o Don't know/not sure oPrefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 102 Q30 Has GIF provided you with support to conduct work to improve civil society’s engagement with the private sector on issues of technology and human rights? o Yes o No o Don't know/not sure oPrefer not to say Skip To: Q34 If Has GIF provided you with support to conduct work to improve civil society’s engagement with the... = No Skip To: Q34 If Has GIF provided you with support to conduct work to improve civil society’s engagement with the... = Don't know/not sure Skip To: Q34 If Has GIF provided you with support to conduct work to improve civil society’s engagement with the... = Prefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 103 Q31 What approaches or strategies did this work employ? Select all that apply. ▢ Facilitating introductions and connections between civil society and the private sector ▢ Training provided by GIF partner organizations to civil society actors on engagement strategies with the private sector ▢ Research related to the Ranking Digital Rights (RDR) methodology ▢ Meetings and roundtables, and other advocacy events with representatives of telecommunication companies ▢ Meetings, roundtables, workshops, webinars, and other advocacy events with representatives of social media companies ▢ Support to organize information campaigns for companies operating in the technological and social media space ▢ Research other than Ranking Digital Rights (RDR)-related research ▢ Other, please specify __________________________________________________ ▢ ⊗Don’t know/not sure ▢ ⊗Prefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 104 Q32 To what extent did being a part of GIF improve your ability to engage with the private sector? o Not at all o A little oSomewhat o A lot o Don’t know/not sure oPrefer not to say Page Break Q33 Thinking of your work supported by GIF to improve civil society’s engagement with the private sector on issues of technology and human rights, did it bring changes in any companies' policies and practices? o Never o Rarely oSometimes o Most of the time o Too soon to tell o Don’t know/not sure oPrefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 105 Q34 What are the barriers and challenges that prevent local organizations like yours from advocating with the private sector on issues of technology and human rights in your country? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ Page Break Q35 Do you have any recommendations for how GIF can help improve civil society’s engagement with the private sector on issues of technology and human rights in your country? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ End of Block: Advocacy with the Private Sector Start of Block: General Comments and Recommendations Q36 How likely is it that your organization will be able to continue work on digital rights and security using capacity, networks, and advocacy systems developed or supported by GIF after the GIF project is over? o Very unlikely o Rather unlikely oLikely o Very likely o Don't know/not sure oPrefer not to say Page Break USAID.GOV GREATER INTERNET FREEDOM PERFORMANCE EVALUATION | 106 Q37 What support do you need for your organization to be able to continue work in the space of digital rights and security after the GIF project is over? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ Page Break Q38 Are there any barriers and challenges that you have not yet mentioned that prevent local organizations like yours from fully benefiting from GIF support? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ Page Break Q39 Are there any additional recommendations you can share on how to improve support for local organizations like yours in your work on digital rights and security? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ Page Break Q40 Is there anything else you would like to add about your experience being part of GIF? ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ End of Block: General Comments and Recommendations